LoginSign Up
← Back to all sparks
O

OpenHouse

ANALYTICS
Velocity5.0

Control plane for managing tables in open-source data lakehouse deployments

www.openhousedb.org ↗

OpenHouse is hardening the seams where table policies and jobs quietly fail.

icebergdata governancetable policiesobservabilityfeature toggles
◆Current state
OpenHouse ships continuously — five releases in the twelve days covered here — with each tag carrying a single merged pull request. The substantive recent work sits in two areas: table governance, where CREATE OR REPLACE AS SELECT was silently dropping retention, replication, history and PII column tags, and operability, where the DataLoader gained a typed exception hierarchy with per-request IDs and the scheduler gained targeted log tokens. Table feature toggles also picked up self-service overrides that let table owners opt in to a feature the server has not ramped.
◆Where it's heading
The project is at the stage where correctness at the edges matters more than new surface: policies surviving a replace, auth failures not being retried as if they were transient, scheduler decisions being greppable in production logs. The observability work is explicitly phased, with OTEL gauges and DLQ counters deferred to a later step, so instrumentation is being staged rather than dropped in at once. The pattern of one PR per release tag means the feed reads as a commit log and the meaningful changes have to be picked out of dependency bumps.
◆Prediction
Phase 2 of the jobs observability plan — OTEL gauges, a heartbeat sampler, and dead-letter-queue counters — is named in the notes as deferred and is the most likely next substantive change.

◆Recent moves

  1. 7d ago

    Metrics for misconfigured HCR tables

    A one-line addition emitting metrics for tables with misconfigured HCR. Operator-facing instrumentation, consistent with the observability thread running through recent releases, with no change for table users.

    View source ↗
  2. 15d ago

    CREATE OR REPLACE no longer silently drops table policies

    A replace that omitted policies was wiping the table's retention, sharing, PII column tags, replication and history, because the replace path rebuilt the policies property purely from the incoming request while ordinary user properties survived. Policies are now merged plane by plane, with omitted planes carried forward and provided ones overriding. Silent loss of PII tags and retention is a governance failure, not a bug in the ordinary sense, which makes this the most consequential fix in the window.

    View source ↗
  3. 16d ago

    Bump iceberg-core to 1.2.0.20

    An automated dependency bump whose own upstream changelog says there were no notable improvements and no referenced pull requests. Nothing ships here.

    View source ↗
  4. 16d ago

    Scheduler log tokens for jobs observability

    About thirty-five lines across three files adding stable log tokens to the scheduler so eligible counts, parallelism settings and queued-timeout causes can be distinguished in production logs. Explicitly phase 1.5, with the metrics and counters work deferred.

    View source ↗
  5. 18d ago

    DataLoader gains request IDs and typed catalog exceptions

    Every outbound catalog request now carries a unique X-Request-ID surfaced on a typed exception hierarchy, replacing timestamp-based correlation between DataLoader errors and Tables Service requests. The behavioral fix underneath matters more: authentication failures were being represented as generic OSErrors and retried as transient I/O, and 401s and 403s are now classified as non-transient.

    View source ↗
  6. 18d ago

    Table owners can self-serve onto server-gated features

    Feature toggles gain a table-property override so an owner can opt in to a feature the server has not ramped, with the explicit property winning and absence delegating to the server rule. The design note is careful about the safety boundary — the override-honoring method is deliberately named differently rather than overloaded, so authorization gates keep using the server-only form.

    View source ↗