NoSQL Datastore Schema Management

Celerity makes NoSQL datastore schemas a first-class concern, bridging the gap between development and data teams. With SQL databases, data teams can introspect information_schema to discover what exists. With NoSQL databases (DynamoDB, Cosmos DB, Firestore), there is no database-level schema to introspect. The "schema" exists only in application code, scattered across handler functions and tribal knowledge, and data teams building pipelines on top of a NoSQL table are flying blind.

Celerity solves this by providing a declarative schema definition that serves as a single source of truth for the intended structure of your data, with tooling for validation, type generation, contracts, data-shape drift detection and exports.

For SQL database schema management, see the SQL Database Schema Management guide.

How It Works

Unlike SQL schema management, there are no migrations and no DDL. NoSQL databases are schemaless by design, so field-level changes do not require database operations. Instead, Celerity's schema management for NoSQL focuses on visibility, validation, contracts and data team tooling.

Schema YAML (desired state) ──► Schema Manager ──► Diff + Contracts ──► State Update
                                      ▲
                              Current state from
                              previous deployment

The schema manager computes the difference between the current schema (from the previous deployment) and the desired schema (from the YAML file). It does not connect to the database or execute any operations against it. All infrastructure changes (indexes, TTL, keys) are handled by the Bluelink deploy engine separately.

How NoSQL Differs from SQL

Bluelink / Celerity Split

The split between Bluelink and Celerity is different from SQL:

• Bluelink provider handles: Table creation, keys, secondary indexes, TTL config and capacity settings, the infrastructure concerns defined in the blueprint spec
• Celerity handles: Schema definition (the field-level YAML), SDK validation, data team contracts, type codegen, schema state tracking, exports

Schema Definition Format

Project File Structure

✅ Available in v0

A typical Celerity project with NoSQL schema management follows this structure:

my-app/
├── .celerity/                       # Generated only (merged blueprint, compose, logs)
├── app.bp                           # Main blueprint — references schema files (.bp, .yaml, or .jsonc)
├── app.deploy.jsonc                  # Deploy target config
├── config/
│   ├── local/                       # Plaintext app config for local development
│   └── test/                        # Plaintext app config for testing
├── secrets/
│   ├── local/                       # Secrets for local development
│   └── test/                        # Secrets for testing
├── seed/
│   ├── local/                       # Seed data for local development
│   └── test/                        # Seed data for testing
├── schema-contracts.yaml            # Data team dependency contracts (optional)
├── schemas/
│   └── user-store.yaml              # Schema for userStore resource
├── scripts/
│   └── user-store/                  # Escape hatch data scripts (per datastore)
│       ├── V001__backfill_tier_field.js
│       └── V002__migrate_legacy_roles.py
├── src/
│   └── ...
└── generated/                       # Optional: codegen output
    └── ...

The blueprint references schema files via the schemaPath field on celerity/datastore resources:

Blueprint Language

version "2025-11-02"
transform "celerity-2026-02-27-draft"

resource userStore: celerity/datastore {
    metadata {
        displayName = "User Store"
        labels = {
            application = "users"
        }
    }

    spec {
        name = "users"
        keys = {
            partitionKey = "id",
            sortKey = "createdAt"
        }
        schemaPath = "./schemas/user-store.yaml"
        scriptsPath = "./scripts/user-store"
        indexes = [
            { name = "emailIndex", fields = ["email"] },
            { name = "tierCreatedIndex", fields = ["tier", "createdAt"] }
        ]
        timeToLive = {
            fieldName = "expiresAt",
            enabled = true
        }
    }
}

YAML

version: 2025-11-02
transform: celerity-2026-02-27-draft

resources:
  userStore:
    type: "celerity/datastore"
    metadata:
      displayName: "User Store"
      labels:
        application: "users"
    spec:
      name: "users"
      keys:
        partitionKey: "id"
        sortKey: "createdAt"
      schemaPath: "./schemas/user-store.yaml"
      scriptsPath: "./scripts/user-store"
      indexes:
        - name: "emailIndex"
          fields: ["email"]
        - name: "tierCreatedIndex"
          fields: ["tier", "createdAt"]
      timeToLive:
        fieldName: "expiresAt"
        enabled: true

Note that keys, indexes and timeToLive remain in the blueprint spec. They are infrastructure concerns that the Bluelink provider needs to create the table and its secondary indexes on the deploy target (e.g. a DynamoDB table and GSIs on AWS). The schemaPath points to the field-level schema that describes the data, which is a Celerity concern.

What Goes Where

Schema YAML Format

✅ Available in v0

Schema files define the intended structure of items in a datastore using Celerity-native field types. One schema file per celerity/datastore resource.

# schemas/user-store.yaml
description: "User accounts and profiles. Partition key: id, sort key: createdAt."
owner: "platform-team"
tags: ["pii", "core-entity"]

required: ["id", "createdAt", "email", "name"]

fields:
  id:
    type: "string"
    description: "UUID. Partition key."
    classification: "internal-id"

  createdAt:
    type: "number"
    description: "Unix timestamp (ms). Sort key. Set at account creation, never updated."

  email:
    type: "string"
    description: "Primary email address. Unique per user (enforced in application)."
    classification: "pii"

  name:
    type: "string"
    description: "Display name."
    classification: "pii"

  tier:
    type: "string"
    description: "Subscription tier: free, pro, enterprise. Added 2025-03. Items before this date may not have this field."
    default: "free"
    tags: ["business-metric"]

  isActive:
    type: "boolean"
    description: "Whether the account is active. Inactive accounts are soft-deleted."
    nullable: true

  roles:
    type: "array"
    description: "Authorization roles assigned to the user."
    items:
      type: "string"
      description: "Role identifier (e.g., admin, editor, viewer)"

  profile:
    type: "object"
    description: "User profile details. Optional — may not exist for accounts created via API."
    nullable: true
    fields:
      bio:
        type: "string"
        nullable: true
        description: "Free-text biography"
        classification: "pii"
      avatarUrl:
        type: "string"
        nullable: true
        description: "URL to avatar image"
      preferences:
        type: "object"
        description: "User-configurable preferences"
        fields:
          theme:
            type: "string"
            description: "UI theme: light, dark, system"
            default: "system"
          notifications:
            type: "boolean"
            description: "Whether email notifications are enabled"
            default: true

  lastLogin:
    type: "number"
    nullable: true
    description: "Unix timestamp (ms) of last login. Null if user has never logged in."
    tags: ["engagement-metric"]

  expiresAt:
    type: "number"
    nullable: true
    description: "Unix timestamp (s) for TTL. Set for temporary/trial accounts."

Design choices:

• Field types are Celerity-native: string, number, boolean, object and array map naturally to DynamoDB, Cosmos DB and Firestore types. Unlike SQL schemas which use engine-native types (varchar, jsonb, etc.), NoSQL field types are portable across providers.
• required list at the top level: Required means "every item MUST have this field" from the application's perspective.
• nullable: A field can exist but be null. Different from "not required" (field may not exist at all).
• default: The value the application should write if none is provided. Not enforced by the database; used for codegen and SDK validation.
• Nested objects and arrays: Recursive structure supporting deeply nested documents.
• Data evolution notes in descriptions: e.g., "Added 2025-03. Items before this date may not have this field." This is a NoSQL reality: old items don't automatically get new fields.

Field Types

Rich Metadata

The following fields have no effect on validation. They exist for documentation, data governance and tooling:

These fields make the schema YAML self-documenting for data team consumption. They are included in schema exports and used by schema contracts for dependency tracking.

Multiple Entities (Single-Table Design)

✅ Available in v0

Storing multiple entity types in a single table is a common NoSQL pattern, particularly with DynamoDB. The flat schema format above describes a single item shape; for single-table designs, a schema file can instead declare a discriminator field and an entities map:

# schemas/app-store.yaml
description: "Application data. Single-table design with users and sessions."
owner: "platform-team"

discriminator: "entityType"          # Field whose value selects the entity

fields:                              # Shared fields, present on every entity (keys live here)
  pk:
    type: "string"
    description: "Partition key"
  sk:
    type: "string"
    description: "Sort key"
  entityType:
    type: "string"
    description: "Entity discriminator: user | session"

entities:
  user:
    description: "User account"
    required: ["pk", "sk", "entityType", "email"]
    fields:
      email:
        type: "string"
        classification: "pii"
      name:
        type: "string"
        nullable: true
        classification: "pii"

  session:
    description: "Login session"
    required: ["pk", "sk", "entityType", "expiresAt"]
    fields:
      expiresAt:
        type: "number"
        description: "Unix timestamp (s) for TTL"
      device:
        type: "string"
        nullable: true

• The flat format (top-level fields/required only) remains valid and is the common case; it is an implicit single entity
• Shared top-level fields apply to every entity; each entity adds its own fields and required list
• Type generation emits one item type per entity plus a discriminated union (AppStoreItem = UserItem | SessionItem)
• Contracts can scope a datastore dependency to specific entities
• Drift detection buckets sampled items by discriminator value and reports per entity, flagging items with unknown discriminator values

Entities are supported in external schema files (schemaPath) only; the inline schema field on the resource keeps the flat format.

Schema Evolution

NoSQL doesn't have migrations in the SQL sense. Field-level changes don't require database operations. Instead, schema management for NoSQL is about tracking evolution, validating conformance, and maintaining contracts.

What Changes and How

Escape Hatch Data Scripts

✅ Available in v0

Versioned scripts for data operations that cannot be expressed in the schema YAML. These live in the directory specified by scriptsPath on the celerity/datastore resource.

File Naming Convention

V<number>__<description>.<ext>

Examples:

• V001__backfill_tier_field.js
• V002__migrate_legacy_roles.py
• V003__cleanup_deprecated_fields.ts

Scripts can be written in any language, as long as they are executable and accept connection config via environment variables.

Execution Model

1. Scripts are tracked by name + content_hash in schema state
2. celerity schema diff shows pending scripts
3. Scripts are executed via celerity schema scripts run, which lists pending scripts, runs them with connection config injected as environment variables, and records each completion in schema state
4. Scripts executed out-of-band (e.g. through a batch system) can be recorded without running via celerity schema scripts run --mark-only

For local development: scripts run automatically during celerity dev run since local data is ephemeral.

Use Cases

• Backfilling new fields on existing items (UPDATE equivalent for NoSQL)
• Migrating data from one field structure to another
• Cleaning up deprecated fields
• Transforming field values (e.g., string-to-number conversion on existing items)

Evolving a Schema Safely

The core NoSQL evolution workflow is tightening a schema over data that predates it. The safe sequence for adding a required field:

Step 1: Add the field as optional. Update the schema YAML with the new field (not in required, or nullable: true). Deploy. New writes can include the field; old items are unaffected, and nothing enforces its presence yet.

Step 2: Backfill existing items. Write a data script (e.g. V004__backfill_tier.js) that scans and updates items missing the field, then run it:

celerity schema scripts run userStore --env production

Step 3: Verify with a drift scan. Confirm the backfill reached everything before tightening:

celerity schema drift userStore --env production --sample 10000

The per-field conformance report shows whether any sampled items still lack the field.

Step 4: Make the field required. Update required in the schema YAML and deploy. Under the default warn validation mode this is safe even if stragglers remain: non-conforming writes are logged, not rejected. Teams running strict mode should only take this step once the drift scan reports full conformance, since read-modify-write operations on non-conforming items would otherwise fail.

Schema Validation Strategy

✅ Available in v0

Since the database doesn't enforce schema, Celerity provides a layered validation approach that catches issues at multiple stages.

Runtime Validation (SDK Abstraction)

The Celerity datastore SDK provides a cloud-agnostic abstraction (putItem, batchWriteItems, query, scan, etc.). When a schema is defined for a datastore, the SDK validates items against it on writes:

• Type mismatches: Writing a number where the schema expects a string
• Missing required fields: Omitting a field that's in the required list
• Unexpected nulls: Writing null to a field that isn't marked nullable
• Nested structure: Validating nested objects and arrays recursively

How the Schema Reaches the Runtime

The schema is bound to the deployment at build time, in two parts:

• The transformer embeds a schema hash into the datastore resource spec, so a schema change is a spec change: the deploy engine sees the diff and deployment state records exactly which schema version shipped
• The CLI validates and normalises the schema (YAML to JSON) during the build and includes it as a generated asset in the build artifacts (Lambda packages, container images), alongside the resource configuration the CLI already generates. The SDK loads it from the artifact when the datastore client is initialised

Handlers therefore always validate against the schema they shipped with: there is no runtime schema fetch, no version skew between handlers and schema state, and celerity schema apply never changes the behavior of running handlers. Schema updates reach handlers on their next deployment. A malformed schema fails the build rather than surfacing as a startup error in production, and the SDKs consume pre-validated JSON with no YAML parsing at runtime.

Validation Modes

The write-validation behavior is controlled by the celerity.datastore.validationMode annotation on the resource:

The warn default is deliberate: tightening a schema (e.g. making a field required) can never break production writes, because existing items that predate the change would otherwise fail read-modify-write operations. Teams that want hard enforcement opt into strict once a drift scan confirms conformance. See Evolving a Schema Safely for the workflow.

Because the annotation is set per resource, off also serves as an escape hatch for extreme high-throughput write paths: a single hot-path datastore can skip write validation while every other datastore keeps it, and drift scans still observe that datastore's data shape from the outside.

Developers who use the Celerity SDK get runtime validation automatically in warn mode, with no additional setup needed.

Build-Time Validation (Generated Types)

Generated types from celerity schema codegen shift schema enforcement to compile/type-check time:

• TypeScript: Generated interfaces make the compiler catch type mismatches, missing required fields and incorrect nested structures, regardless of whether the developer uses the Celerity SDK or the native provider SDK.
• Python: Generated Pydantic models provide runtime validation, IDE autocomplete and type checking via mypy/pyright.

See Type Generation for details and examples.

CI Validation

celerity schema validate --check-codegen in CI ensures generated types are always in sync with the schema YAML. If someone changes the schema but doesn't regenerate types, validation fails.

Validation Layers

Drift Detection

✅ Available in v0

Since the database doesn't enforce schema, the actual shape of stored data can silently diverge from the schema YAML: items written before a field existed, other services writing to the same table, or writes bypassing validation entirely. For NoSQL, this data-shape drift is the central risk, and it's what keeps the "single source of truth" claim honest.

celerity schema drift <datastore> --env <env> performs a point-in-time sampled scan of the live datastore (default 1,000 items, configurable with --sample) and reports drift in both directions:

1. Conformance drift (schema → data): sampled items that don't conform to the schema, with per-field conformance percentages: missing required fields, type mismatches, unexpected nulls
2. Unknown-field drift (data → schema): fields present in sampled items but absent from the schema YAML, the undocumented shape changes introduced by other writers

When entities are defined, items are bucketed by discriminator value and reported per entity; items with an unknown discriminator value are flagged.

$ celerity schema drift userStore --env production --sample 10000

userStore (datastore: users) — sampled 10,000 of ~1.2M items:

  Conformance drift:
    tier         87.3%  ⚠ 12.7% of items missing (required field; likely pre-2025-03 items)
    roles        78.4%  ⚠ 21.6% missing (field added later)
    tier types   99.8%  ⚠ 0.2% number instead of string (12 items; legacy tier IDs)

  Unknown fields (in data, not in schema):
    ~ legacySource — present on 4.1% of sampled items

  Contracts affected:
    ⛔ user-analytics-pipeline — userStore shape drifted (blocking)

  Resolution:
    - Backfill missing fields: create a data script, then celerity schema scripts run
    - Adopt unknown fields: add them to schemas/user-store.yaml
    - Legacy type mismatches: transform via a data script, then re-scan

Drift output is contract-aware: drifted datastores are matched against schema-contracts.yaml so the report shows which downstream consumers are affected. The command exits non-zero when drift is found, so a scheduled CI job gives continuous shape tracking.

Because this is a sampled read-only scan, it needs no runtime plumbing: it observes what's actually in the datastore regardless of which SDK or service wrote it, making it the complement to runtime validation, which only sees new writes made through the Celerity SDK.

Scheduled Drift Monitoring

Deploy Pipeline Integration

✅ Available in v0

Schema management is integrated into the celerity deploy pipeline:

celerity deploy
  │
  ├─ Phase 1: Infrastructure
  │   Transformer converts celerity/datastore → provider-specific table (e.g. AWS DynamoDB)
  │   Provider creates/updates table, secondary indexes, TTL
  │   Stabilise (table + all indexes active)
  │
  ├─ Phase 2: Schema State Update
  │   Read desired schema from YAML
  │   Read current schema from previous deployment state
  │   Compute diff, update schema state
  │   Log any pending escape hatch scripts that should be run
  │   (No database connection — field changes don't need one)
  │
  └─ Phase 3: Application
      Deploy handler resources with datastore connection configuration
      Build artifacts include the normalised schema (hash tracked in the
      resource spec); handlers start with SDK validation in the
      configured mode (warn by default)

Local Development

✅ Available in v0

Running Locally

When celerity dev run starts an application with datastore resources:

1. A local emulator is started based on the deploy target (DynamoDB Local for AWS targets)
2. Tables are created with keys and indexes from the blueprint spec
3. Escape hatch data scripts are executed automatically (local data is ephemeral)
4. Connection environment variables are injected, pointing to the local emulator
5. The runtime starts with SDK schema validation in the configured mode (warn by default)

Testing

When celerity dev test runs an application with datastore resources:

1. An isolated emulator instance is created per test suite
2. Tables are created with keys and indexes
3. Test fixtures are loaded
4. Tests run against the fully-configured datastore
5. The instance is torn down after tests complete

CLI Commands

✅ Available in v0

The celerity schema command group works for both SQL databases and NoSQL datastores. For full command reference including all flags and configuration options, see the CLI Reference: schema.

Example: celerity schema diff

The diff output for NoSQL datastores shows field-level changes and data conformance warnings. Unlike the SQL diff, no migration file is generated, since field changes do not require database operations.

$ celerity schema diff

userStore (datastore: users):
  Schema changes:
    [+] Field 'tier' added (string, default: "free")
        ⚠ Existing items will NOT have this field. Consider a backfill script.
    [~] Field 'roles' changed: was nullable, now required
        ⚠ Existing items with null 'roles' will not conform. Consider a backfill script.
    [-] Field 'legacyId' removed from schema
        ⚠ Existing items may still contain this field.

  Infrastructure changes (handled by Bluelink):
    [+] Index 'tierCreatedIndex' added (fields: [tier, createdAt])
        This will be created as a secondary index by the deploy target's provider.

  Data scripts (pending):
    V003__cleanup_deprecated_fields.js (not yet run)

  Contracts:
    ⚠ user-reporting — userStore schema changed (notify)

  Apply with: celerity schema apply

Type Generation

✅ Available in v0 (TypeScript, Python)

Generate types from the schema YAML: TypeScript interfaces or Python Pydantic models representing datastore items, plus field name and index constants. No ORM coupling. Combine with whatever library you prefer.

For SDK usage examples with generated types, see the Node.js SDK - Datastore and Python SDK - Datastore documentation.

TypeScript

celerity schema codegen --lang typescript --out ./src/generated/

Generated output:

// generated/user-store.ts — auto-generated, do not edit

/** User store item */
export type UserStoreItem = {
  /** UUID. Partition key. */
  id: string;
  /** Unix timestamp (ms). Sort key. */
  createdAt: number;
  /** Primary email address. */
  email: string;
  /** Display name. */
  name: string;
  /** Subscription tier: free, pro, enterprise. */
  tier?: string;
  /** Whether the account is active. */
  isActive?: boolean | null;
  /** Authorization roles. */
  roles?: string[];
  /** User profile details. */
  profile?: UserStoreProfile | null;
  /** Unix timestamp (ms) of last login. */
  lastLogin?: number | null;
  /** Unix timestamp (s) for TTL. */
  expiresAt?: number | null;
}

export type UserStoreProfile = {
  bio?: string | null;
  avatarUrl?: string | null;
  preferences?: UserStoreProfilePreferences;
}

export type UserStoreProfilePreferences = {
  theme?: string;
  notifications?: boolean;
}

/** Field name constants */
export const UserStoreFields = {
  id: "id",
  createdAt: "createdAt",
  email: "email",
  name: "name",
  tier: "tier",
  isActive: "isActive",
  roles: "roles",
  profile: "profile",
  lastLogin: "lastLogin",
  expiresAt: "expiresAt",
} as const;

/** Index names for query operations */
export const UserStoreIndexes = {
  emailIndex: "emailIndex",
  tierCreatedIndex: "tierCreatedIndex",
} as const;

/** Key schema */
export const UserStoreKeys = {
  partitionKey: "id",
  sortKey: "createdAt",
} as const;

Note the generated types reflect NoSQL realities:

• Non-required fields are optional (?), since items may not have them
• Nullable fields are | null, since items may have the field set to null
• Required fields are always present
• Index names and key schema are exported as constants for use with query operations

Python

celerity schema codegen --lang python --out ./src/generated/

Generated output:

# generated/user_store.py — auto-generated, do not edit
from pydantic import BaseModel, Field
from typing import Optional

class UserStoreProfilePreferences(BaseModel):
    theme: Optional[str] = None
    notifications: Optional[bool] = None

class UserStoreProfile(BaseModel):
    bio: Optional[str] = None
    avatar_url: Optional[str] = None
    preferences: Optional[UserStoreProfilePreferences] = None

class UserStoreItem(BaseModel):
    # Required fields
    id: str
    created_at: float
    email: str
    name: str
    # Optional fields
    tier: Optional[str] = "free"
    is_active: Optional[bool] = None
    roles: Optional[list[str]] = None
    profile: Optional[UserStoreProfile] = None
    last_login: Optional[float] = None
    expires_at: Optional[float] = None

Defaults

Schema default values are applied at item construction time by generated code and are never silently injected by the SDK on write. Python surfaces them directly as Pydantic field defaults (tier: Optional[str] = "free" above). TypeScript types cannot carry values, so defaults are exported as a constant for explicit use:

/** Schema defaults */
export const UserStoreDefaults = {
  tier: "free",
} as const;

A field that is absent stays absent across languages: optional fields without an explicit default generate as None/? rather than materialising empty values, so an item round-tripped through one language's generated types has the same shape in the other.

Entities

When a schema defines entities, codegen emits one item type per entity plus a discriminated union, so AppStoreItem = UserItem | SessionItem in TypeScript and a Pydantic discriminated union (on the discriminator field) in Python, along with per-entity field constants.

Go

🚀 Planned for v1 - Go type generation (struct types) is planned for a future release.

Java

🚀 Planned for v1 - Java type generation (record classes) is planned for a future release.

C#

🚀 Planned for v1 - C# type generation (record types) is planned for a future release.

Schema Contracts

Schema contracts allow data teams to declare which datastores they depend on, so they are automatically informed when schema changes affect them. Any structural change to a watched datastore's schema triggers the contract's policy.

The schema manager already knows exactly what changed (fields added, removed, type changes, etc.), so contracts don't need to duplicate that information. They simply declare: "I care about this datastore. Tell me when its schema changes."

Contracts work the same way for SQL databases. See SQL Database Schema Contracts.

Contracts File Format

✅ Available in v0

Data teams maintain a contracts file in the repository alongside the blueprint. The same file can include contracts for both SQL databases and NoSQL datastores:

# schema-contracts.yaml
contracts:
  - name: "user-analytics-pipeline"
    owner: "data-team"
    dependencies:
      - datastore: "userStore"
        policy: "blocking"       # non-zero exit code if this datastore's schema changes

  - name: "engagement-reporting"
    owner: "data-team"
    dependencies:
      - datastore: "userStore"
        policy: "notify"         # warning output, zero exit code

For single-table datastores, scoping a dependency to specific entities keeps contracts quiet when unrelated entities change:

contracts:
  - name: "session-analytics"
    owner: "data-team"
    dependencies:
      - datastore: "appStore"
        entities: ["session"]    # Only fires when the session entity's schema changes
        policy: "blocking"

When a schema change touches a datastore listed in a contract, the diff output includes the full details of what changed: fields added, removed, type changes, etc. The contract itself just identifies which datastores matter.

Validation and CI Integration

✅ Available in v0

Contract checking is built into celerity schema validate. When a schema-contracts.yaml file exists and deployed state is available, validate evaluates contracts as part of its checks:

• blocking contracts affected: validate exits with a non-zero exit code, failing your CI pipeline
• notify contracts affected: validate prints warnings but does not affect the exit code

This means a single celerity schema validate step in CI covers everything: schema correctness, field type validation, and contract impact:

# Example CI step (GitHub Actions)
- name: Validate schema
  run: celerity schema validate

When a blocking contract fires, the CI output shows exactly what changed, giving the data team the information they need to review the PR. Use your platform's existing notification mechanisms (CODEOWNERS, required reviewers, Slack integrations on CI failure) to alert the right people.

Contract impact is also shown in celerity schema diff output, giving visibility into downstream effects during local development.

Deploy-Time Enforcement and Webhook Notifications

Programmatic Schema API

Data Catalog Integrations

For Data Teams

The schema YAML files serve as always-accurate documentation because they are what Celerity uses for validation, type generation and contract evaluation. For NoSQL datastores, this is even more critical than for SQL databases because there is no database-level schema to introspect. The schema YAML is the only reliable source of truth for what fields exist, their types, and who owns them.

Data teams benefit from:

• description, owner, tags, classification fields on the schema and every field make datastores self-documenting
• Schema exports (celerity schema export --format markdown) generate human-readable documentation
• Git history of schema files (git log schemas/user-store.yaml) provides full change history with PR review
• Contract definitions in schema-contracts.yaml let data teams declare and protect their dependencies
• Drift scans (celerity schema drift) report per-field conformance of live data against the schema, with affected contracts called out
• Machine-readable exports (celerity schema export --format json-schema) integrate with pipeline tools