Composite Data Source (`COMPOSITE`) v25.1+ PREVIEW

The Composite Data Source is a proxy data source that allows you to create a schema by composing multiple other schemas.
Unlike traditional database sources, it does not maintain direct connections with remote data sources. Instead, it intercepts incoming commands, modifies them as needed, and routes them to the appropriate data source.

When to Use Composite

Unifying Multiple Sources into One Schema

The primary use case is when you want to avoid configuring clients (such as applications or AI agents) to interact with multiple schemas. Instead, you can define a composite schema, which acts as a single logical unit, but internally routes queries to the appropriate underlying schemas on a per-table basis.

Maintaining a Consistent Data Model

In complex scenarios—such as designing a data model for an IT landscape—you may need to integrate dozens of remote data sources.
Instead of dealing with multiple scattered schemas, Composite Data Source allows you to consolidate everything into one unified schema, making the entire data model easier to understand and manage.

Configuration

The Composite Data Source can be configured in two ways:

Direct Import – Similar to passthrough mode, where schemas are cloned from other data sources.
Custom Schema (DDL) – Allows renaming tables, adding fields, or applying transformations.

Direct Import

The composite schema is automatically generated by cloning schemas from other data sources.

- name: "composite_import"
  dataSourceType: "COMPOSITE"
  configObject:
    dataSourceName: "composite"
    imports:
      - dataSource: "redis"
        tableNamePattern: ".+"
      - dataSource: "psql"
        tableNamePattern: "\\bITOPS_\\w+\\b"
  ...

⚠️

Important Considerations:

Table name conflicts: Composite does not allow importing tables with the same name (as schemas must have unique table names).
Choosing the right approach: If the data sources you are trying to compose share the same schema structure, you likely need ROUTING instead of COMPOSITE.

Custom Schema (DDL)

If you need to rename tables, add fields, or apply transformations, you can define a custom schema using DDL.
In this case, each TABLE must specify its parent table using the fully qualified object name via the composite_parent directive.

- name: "composite_import"
  dataSourceType: "COMPOSITE"
  configObject:    
    ...
  schema:
    type: "PHYSICAL"
    cacheDefaultStrategy: "NO_CACHE"
    ddl: |
      CREATE FOREIGN TABLE INTERNAL_APPS (
          name string,
          version string,
          team string,
          ...
      )
      OPTIONS(updatable true,
              composite_parent 'psql.ITOPS_APPS');

Specific `TABLE` Directives

Directive	Type	Options	Description
`composite_parent`	String	Any	Specifies the fully qualified parent `TABLE` in another schema that this `TABLE` maps to. The format must be `[schema_name].[table_name]`. This directive is required when defining a composite schema using custom DDL.

Kubling Routing

Composite Data Source (COMPOSITE) v25.1+ PREVIEW