Embedded RocksDB v26.7+ PREVIEW

The ROCKSDB Data Source stores relational tables in a local filesystem directory owned by Kubling. It is intended for persistent local state that needs declared keys and indexes without running a separate database or provider.

Which RocksDB feature is this?

Kubling uses RocksDB in three independent places:

The directories are not interchangeable. Do not point these features at the same path.

Configure the Data Source

Declare one physical schema and provide its DDL:

dataSources:
  - name: "local_state"
    dataSourceType: "ROCKSDB"
    configObject:
      storagePath: "{{ ROCKSDB_STORAGE_PATH }}"
      fullScanAllowed: false
      schemaManagement: "VALIDATE"
    schema:
      type: "PHYSICAL"
      ddlFilePaths:
        - "bundle:local-state.ddl"

Use a separate storagePath for every RocksDB Data Source. The mounted filesystem defines the capacity and durability boundary.

Define tables and indexes

The DDL is the authoritative relational catalog:

CREATE FOREIGN TABLE DEVICE_STATE (
    DEVICE_ID string(64) NOT NULL,
    SITE string(64) NOT NULL,
    STATUS string(32) NOT NULL,
    UPDATED_AT timestamp NOT NULL,
    PRIMARY KEY (DEVICE_ID),
    CONSTRAINT IX_DEVICE_SITE INDEX (SITE),
    CONSTRAINT IX_DEVICE_STATUS INDEX (STATUS)
)
OPTIONS (UPDATABLE TRUE);

A writable table must declare a primary key. A table intended only for reads can omit it when declared with OPTIONS (UPDATABLE FALSE).

Named unique keys and ordinary indexes provide equality access paths. The preview supports scalar strings, booleans, integer and decimal numbers, floating-point numbers, date/time values and varbinary. Procedures, functions, foreign keys, views and other non-table schema objects are not supported by this Data Source.

Indexed access and full scans

With the default fullScanAllowed: false, a query, UPDATE or DELETE must provide equality values for every column of a primary key or one declared index. For example, the index above makes this access bounded:

SELECT DEVICE_ID, STATUS, UPDATED_AT
FROM local_state.DEVICE_STATE
WHERE SITE = 'madrid';

A filter that does not satisfy a complete key fails instead of scanning the table. Enable fullScanAllowed only after evaluating the maximum table size, query concurrency and latency impact.

Joins and aggregations remain Kubling operations. Each physical table access must still satisfy the configured scan policy.

Mutation and transaction boundary

By default, INSERT, UPDATE and DELETE statements are committed synchronously and atomically as individual SQL statements.

Explicit transactions v26.7.1+

Explicit multi-statement transactions require Soft Transactions to be enabled both in the application configuration and for this Data Source:

# Application configuration
softTransactions:
  enabled: true

# ROCKSDB Data Source configObject
softTransactions:
  enabled: true
  strategy: "DEFER_OPERATION"
  mvcc:
    enabled: true
    dbBasePath: "/var/lib/kubling/mvcc/local-state"

With DEFER_OPERATION, mutations are applied at commit. Simulated MVCC is optional and provides transaction-local read-your-writes visibility before that commit. See Soft Transactions and Simulated MVCC for the complete behavior and limitations.

Reopen an existing store

The configured DDL is checked every time Kubling opens storagePath.

VALIDATE

VALIDATE is the default. Any difference between the requested DDL and the persisted catalog prevents the Data Source from becoming available. Use it when schema changes must be applied through an application-managed migration.

MIGRATE_SAFE

MIGRATE_SAFE applies only changes that can preserve the existing rows:

Index work is checkpointed and resumes after an interrupted startup. The new catalog becomes active only after required indexes are complete. A validation or uniqueness failure leaves the previous catalog active and prevents the requested schema from being published.

Changes outside this matrix require a new storagePath or an application-managed data migration.

Deploy and operate

• Mount storagePath on storage sized for the expected rows and indexes.
• Keep the path stable across restarts.
• Do not let two running Kubling instances open the same path for writing.
• Treat a storage-lock error as evidence of another owner or an unclean deployment boundary; do not bypass the lock by sharing the directory.
• Test DDL changes against a copy of the store before using MIGRATE_SAFE on important data.
• Define an application-level backup procedure before relying on the preview for state that cannot be recreated.

Troubleshooting

The generated field reference is available under RocksDB Data Source Configuration.