Schema Registry v2.0

Know your data.
Enforce its rules.

Ledger is the schema registry and data obligation manager for distributed systems. Track every field, classify every tier, propagate every obligation — across databases, caches, queues, object stores, and external APIs.

schemas/users.yaml
schemas: - backend_id: users_db unit: users unit_type: table fields: - name: email type: varchar(255) classification: PII annotations: - gdpr_erasable - indexed - name: payment_token type: varchar(512) classification: FINANCIAL annotations: - encrypted_at_rest - tokenized - name: created_at type: timestamptz classification: PUBLIC annotations: - audit_field - immutable
Terminal
$ ledger schema add schemas/users.yaml Registered 3 fields in users_db.users $ ledger schema validate All schemas valid. 0 conflicts, 0 warnings. $ ledger export --format arbiter Wrote classification rules for 3 fields → ledger_classifications.yaml $ ledger migrate plan user_svc 002.sql DROP COLUMN created_at BLOCKED: cannot remove audit_field column $ ledger mock users_db users --count 3 Generated 3 records (PII: faked, FINANCIAL: tokenized)
Capabilities

What Ledger Does

One registry for every backend. One set of annotations for every obligation. One source of truth for the entire stack.

■

Schema Registry

Register schemas for 12 backend types through a unified unit/unit_type abstraction. Postgres tables, Kafka topics, Redis key patterns, S3 object patterns — all stored verbatim, all queryable.

▲

Migration Gating

Parse SQL, Alembic, CQL, and Kafka schema evolution. Compute blast radius. Return BLOCKED, HUMAN_GATE, or AUTO_PROCEED based on annotation impact. No audit field ever dropped by accident.

◆

Annotation Propagation

Data-driven rules engine. Annotate a field as gdpr_erasable and Pact gets contract assertions, Arbiter gets classification rules, Sentinel gets severity mappings. No code changes for custom annotations.

●

Mock Generation

Schema-aware test data. PII fields get realistic fakes. Encrypted fields get token-shaped values. Canary fields get fingerprinted values that Arbiter tracks through the system. Deterministic with seed.

⬤

Conflict Validation

Catch contradictory obligations at schema time. Immutable + erasable? Hard error. Audit + deletable? Hard error. REQUIRES satisfaction checked for every field. All violations reported, not just the first.

◼

Multi-Format Export

Export to Pact (contract assertions), Arbiter (classification rules), Baton (egress node config), Sentinel (severity mappings), and retention (infrastructure config hints). YAML out, machine-readable.

Universal Model

12 Backend Types, One Abstraction

Every backend maps its native concept to unit / unit_type. Annotations and propagation rules work identically across all of them.

PostgreSQL
Relational
MySQL
Relational
SQLite
Relational
MongoDB
Document
Redis
Key-Value
Cassandra
Wide-Column
Kafka
Streaming
RabbitMQ
Message Queue
SQS
Managed Queue
S3 / R2
Object Storage
Stripe
External API
HTTP APIs
Generic External
PUBLIC < PII < FINANCIAL < AUTH < COMPLIANCE

CLASSIFICATION SEVERITY →

Propagation

Annotate Once, Enforce Everywhere

Define obligations on fields. Ledger propagates them into every downstream tool through a data-driven rules engine. No code changes for new annotations.

Schema Field
email : PII
annotated with →
Annotations
gdpr_erasable, indexed
rules engine →
Ledger
Propagation
Pact
Erasure handler test
Query filter assertion
Arbiter
PII classification rule
Audit event on access
Baton
Canary-eligible field
PII tier in egress config
Sentinel
HIGH severity minimum
on field-level errors
Retention
Infrastructure config
Lifecycle policy hints
Safety

Three Gate Levels

Every migration is analyzed against the annotation registry. The result is unambiguous.

Blocked

Redesign Required

Cannot proceed under any circumstances. An engineer must change the migration. No approval path exists.

DROP COLUMN created_at
→ audit_field: unconditionally blocked

ALTER COLUMN id TYPE bigint
→ immutable: cannot modify
Human Gate

Approval Required

Can proceed after explicit human approval with documented rationale. Review reference and justification recorded in the audit trail.

ALTER COLUMN token DROP ENCRYPTION
→ encrypted_at_rest: security downgrade

ADD COLUMN ssn varchar(11) -- PII
→ tier not in component data_access
Auto Proceed

Ship It

Safe to deploy automatically through the normal pipeline. No annotations affected, or only PUBLIC-tier changes to a declared component.

ADD COLUMN page_views integer
→ PUBLIC field, component declared: ok

CREATE INDEX idx_url ON pages(url)
→ operational only: no gate required
Ecosystem

Part of the Stack

Ledger and Arbiter are peers — Ledger owns "what does data look like?" while Arbiter owns "who can touch it?" Together with Pact, Baton, Sentinel, and Constrain, they form a complete governance layer.

Constrain
Specification
→
Pact
Contracts
→
Arbiter
Trust
↔
Ledger
Schema
→
Baton
Orchestration
→
Sentinel
Attribution