Skip to main content
The log_file_schema_dataset_links table is designed to establish and manage relationships between log file schemas and datasets. This table plays a crucial role in the system by facilitating the mapping of structured data schemas to various datasets, enabling flexible and versioned data integration.

​
Table Structure

The log_file_schema_dataset_links table is structured to maintain the connections between schemas and datasets while providing additional metadata. Here’s a detailed breakdown of its columns:
Column NameData TypeConstraintsDescription
iduuidprimary keyUnique identifier for each link
company_iduuidforeign key (companies)Reference to the company owning the link
schema_iduuidforeign key (log_file_schemas)Reference to the associated schema
dataset_iduuidforeign key (datasets)Reference to the associated dataset
path_keyjsonbnot nullJSON representation of the data mapping path
created_attimestamptznot null, default now()Timestamp of link creation
is_activebooleannot null, default trueIndicates whether the link is currently active
schema_revisionintegernot null, default 1Revision number of the associated schema

​
Usage and Functionality

The log_file_schema_dataset_links table is designed to be flexible and support versioned schema-to-dataset mappings. Here are some key points about its usage:
  1. Schema-Dataset Association: The table allows for many-to-many relationships between schemas and datasets, enabling reuse of schemas across multiple datasets and vice versa.
  2. Versioning Support: The schema_revision column facilitates schema versioning, allowing the system to maintain multiple versions of schema-dataset mappings over time.
  3. Active Link Management: The is_active column provides a mechanism to enable or disable specific schema-dataset links without deleting them, offering flexibility in managing data integrations.
  4. Data Mapping: The path_key column stores a JSON representation of the mapping between schema elements and dataset fields, allowing for complex and nested data structures.

​
Notes

  • Example usage in TypeScript: 
    export const generateBlankSchemaDatasetLink = (
  schemaDatasetLink: Partial<LogFileSchemaDatasetLink>
): LogFileSchemaDatasetLink => {
  const newSchemaDataset: LogFileSchemaDatasetLink = {
    id: uuidv4(),
    company_id: "",
    schema_id: "",
    schema_revision: 0,
    dataset_id: "",
    path_key: [],
    is_active: true,
    created_at: new Date().toISOString(),
    ...schemaDatasetLink,
  };
  return newSchemaDataset;
};
  • The table works in conjunction with the log_file_schemas table, which stores the actual schema definitions, including sample file data and metadata.
By leveraging the log_file_schema_dataset_links table, the Serial application can maintain a flexible and versioned mapping between data schemas and datasets, enabling robust data integration capabilities and supporting complex data processing workflows across different components of the system.