Kubling Documentation

Portable Local Data Source (`PORTABLE_LOCAL`) v24.5.7+

This database data source is based on the H2 database engine. It creates a single-file, locally-stored database.

The primary goal of this data source is to create lightweight, portable databases for storing small amounts of information that:

Must persist across Kubling engine shutdowns.
Can be shared with other Kubling instances or tools.

H2 was chosen over Kubling’s internal small database engine (used for metadata storage) because it provides greater flexibility:

You can open H2 databases using a wide variety of third-party tools.
Accessing data does not depend on Kubling, making it easier to extract or view stored information independently.
It allows you to treat this data source as any other regular remote data source.

Configuration

Portable Source configuration

type: "object"
id: "schema:kubling:dbvirt:model:vdb:sources:PortableLocalSourceConfig"
properties:
  dataSourceName:
    type: "string"
    description: "Internal Engine data source name."
  serverAddress:
    type: "string"
    description: "Address of the remote data source."
  serverPort:
    type: "integer"
    description: "Port where the remote data source is listening for connections."
  databaseName:
    type: "string"
    description: "Remote Data Base name."
  userName:
    type: "string"
    description: "Username used to establish the connection."
  token:
    type: "string"
    description: "Token or password used to establish the connection."
  maxConnectionPoolSize:
    type: "integer"
    description: "Maximum number of connections the Engine will open. Default: 5"
  minConnectionPoolSize:
    type: "integer"
    description: "Minimum number of connections the Engine will keep open. Default:\
      \ 1"
  maxConnectionIdleMilliseconds:
    type: "integer"
    description: "Maximum time in milliseconds a connection can live in the pool without\
      \ being used. Once reached, the connection is closed and removed from the pool.Default:\
      \ 7200000ms (2 hours)"
  maxConnectionLifeTimeMilliseconds:
    type: "integer"
    description: "Maximum time in milliseconds a connection can live in the pool.\
      \ Once reached, the connection is closed and removed from the pool. In case\
      \ of pool size is less than minConnectionPoolSize after the removal, a new fresh\
      \ connection is created.Default: 14400000ms (4 hours)"
  cache:
    type: "object"
    id: "schema:kubling:dbvirt:translation:model:CacheDataSourceConfig"
    properties:
      enabled:
        type: "boolean"
        description: "Specifies whether the cache is enabled for this Data Source.\
          \ Default is false."
      ttlSeconds:
        type: "integer"
        description: "The time-to-live (TTL) for cache entries, in seconds. Default\
          \ is 43,200 seconds (12 hours)."
  allowBruteForceOperations:
    type: "boolean"
    description: "Allows the engine to perform Update and Delete operations using\
      \ all fields as a fallback when no other way to identify rows is available."
  enableSoftTransactions:
    type: "boolean"
  contributesToHealth:
    type: "boolean"
    description: "Indicates whether this data source contributes to the engine's overall\
      \ health status. When set to true, if this data source is not healthy, the engine\
      \ will be marked as unhealthy. Otherwise, the health status of this data source\
      \ is ignored in the overall assessment."
  filePath:
    type: "string"
    description: "Path to file where to store the database."
  filePassword:
    type: "string"
  ddl:
    type: "string"
    description: "Allows to run a DDL in the REMOTE data source once the connection\
      \ is established."
  ddlFilePaths:
    type: "array"
    description: "Allows to run multiple DDL, loaded from external file paths, in\
      \ the REMOTE data source once the connection is established."
    items:
      type: "string"
  migrationDir:
    type: "string"
    description: "Specifies a directory containing migration scripts. When set, the\
      \ ddlFilePaths configuration option is ignored."

Sample configuration in VDB file:

- name: mydb
  dataSourceType: PORTABLE_LOCAL
  configObject:
      dataSourceName: mydb-portable
      databaseName: mydb-portable
      userName: sa
      token: [REDACTED]
      maxConnectionPoolSize: 5
      minConnectionPoolSize: 1
      maxConnectionIdleMilliseconds: 7200000
      maxConnectionLifeTimeMilliseconds: 14400000
      cache:
        enabled: false
        ttlSeconds: 43200
      allowBruteForceOperations: false
      filePath: /some/path
      filePassword: [REDACTED]
      DDL: |
        CREATE TABLE IF NOT EXISTS MyTable...
  schema:
      type: PHYSICAL
      properties:
        importer.schemaName: PUBLIC
        importer.useFullSchemaName: "false"
        importer.useCatalogName: "false"
        cacheDefaultStrategy: CACHE

Limitations

As this data source is based on H2, some limitations apply. For more information, check H2’s official documentation.

⚠️

Based on our internal testing, and to maintain optimal performance, we recommend keeping this data source under 1 GB in size when operating under memory constraints. Complex queries may require memory that is also used by the engine itself.

If you plan to work with larger database files, we strongly suggest enabling local disk buffering (via the buffer.useDisk option in the application configuration). This allows Kubling to minimize memory usage and ensures more resources are available for the embedded H2 engine.

Schema migrations v25.5+

Local H2 database schemas can be migrated using an embedded mechanism based on the battle-tested Flyway.

When migrationDir is specified, the engine performs a Flyway-based migration using the SQL files located in the given directory. For example:

bundle-info.yaml

You can then reference the directory like this: migrationDir: "bundle:migration".

Audit Table

Flyway requires a table to track migration state. For this purpose, Kubling automatically creates a table named KUBLING_MIGRATION_AUDIT, which must not be modified externally.

Note that ddlFilePaths is ignored when migrationDir is used — the two mechanisms are mutually exclusive.

For stateful, non-ephemeral Kubling instances that rely on a local database, we strongly recommend using migrations instead of embedding schema creation logic in a single H2 DDL file.

Recommendations

Here are some best practices and useful notes when using Flyway-based migrations in Kubling:

Use Versioned Files with Clear Naming

Follow Flyway’s naming convention: V<version_number>__<description>.sql

Examples:

V1__create_tables.sql
V2__add_indexes.sql
V3__insert_initial_data.sql

This helps track schema evolution clearly and consistently.

One Logical Change per Migration

Keep each migration file focused on a single change (e.g., creating a table, adding a column).
This improves readability and makes it easier to troubleshoot if a migration fails.

What Happens If a Migration Fails?

Kubling will abort the engine initialization process and exit with an error message.
Flyway stops the process immediately at the failing script.
Successful migrations remain applied.
The failing migration is not marked as executed and will be retried on the next engine startup.
There is no automatic rollback. If rollback is required, it must be defined inside the SQL script using transactional logic.

No Manual Schema Changes

Avoid modifying the database schema outside of migrations.
Manual changes may cause checksum errors or future migrations to fail.

Do Not Modify the Audit Table

The table KUBLING_MIGRATION_AUDIT is managed internally and must not be edited or cleared manually.

Retrying a Failed Migration

To retry:

Fix the faulty migration file.
Restart the engine.
Flyway will reapply the failed script and continue with the sequence.

Do not rename or remove the failing file, as this will break integrity checks.

Test Before Production

If running a stateful or non-ephemeral Kubling instance, always test migrations in a staging environment to avoid irreversible schema changes.

Version Control Your Migrations

Migration files are part of your application’s state. Always include them in version control to keep environments consistent.

Embedded Kubling

Portable Local Data Source (PORTABLE_LOCAL) v24.5.7+