LoginSign Up
← Back to all sparks
D

Delta Lake

ANALYTICS
Velocity5.0

Open table format bringing ACID transactions to data lakes

delta.io ↗

Delta Lake's public releases are patch work while Databricks kernel builds fill the feed.

lakehousetransaction-logdelta-sharingkernelpatch-releasesuniform
◆Current state
Two kinds of entry dominate: numbered patch releases on the 3.3 and 4.3 lines, and near-daily Databricks kernel build tags that carry a single commit message each. The patch releases are targeted correctness work — a metadata cleanup that could delete transaction log files still needed to reconstruct versions inside the retention window, a Delta Sharing cache refresh that dropped deletion-vector URL mappings, an OAuth key-casing bug in the REST Catalog, and S3A fast-listing through FilterFileSystem wrappers. The 3.3.3 release also declines to publish delta-iceberg, leaving UniForm users on the prior patch.
◆Where it's heading
The project is stabilising two supported lines in parallel rather than moving the format forward in these entries, and the recurring theme is metadata and log durability — the parts of Delta that silently break time travel and CDF when they are wrong. Kernel work continues in the build tags, most visibly treating _last_checkpoint as opaque JSON. The unpublished UniForm artifact is the one open thread with a stated follow-up plan.
◆Prediction
A follow-up patch that restores the delta-iceberg artifact for the 3.3 line is the clearest next step; otherwise expect the kernel build cadence to continue between numbered releases.

◆Recent moves

  1. 6d ago

    Log-retention and Delta Sharing cache fixes; UniForm jar not published

    A 3.3 patch fixing log cleanup that could delete transaction log files still needed inside the retention window, plus a Delta Sharing deletion-vector cache bug and Spark 3.5.6 TRUNCATE TABLE compatibility. It also wires up the previously declared randomizeFilePrefixes properties on the write path, and notably ships without the delta-iceberg artifact.

    View source ↗
  2. 19d ago

    Databricks kernel build tag (2026-07-30)

    A Databricks kernel build tag setting an internal version string. Build-pipeline bookkeeping visible in the release feed rather than a release.

    View source ↗
  3. 1mo ago

    Kernel build tag: _last_checkpoint captured as opaque JSON

    A kernel build tag capturing _last_checkpoint as opaque JSON for SnapshotHint. One of the few build tags carrying an actual code change, though still a single internal commit.

    View source ↗
  4. 1mo ago

    Delta Lake 4.3.1

    A 4.3 patch fixing OAuth config keys being lowercased by CaseInsensitiveStringMap, which broke Delta REST Catalog authentication, along with S3A fast listing through FilterFileSystem wrappers and UC managed-table metadata handling. Same targeted-correctness shape as the 3.3 line.

    View source ↗
  5. 1mo ago

    Databricks kernel build tag (2026-07-07)

    A dated Databricks kernel build with no described change. Internal release-pipeline output.

    View source ↗
  6. 1mo ago

    Databricks kernel build tag, DBI variant (2026-07-06)

    The DBI counterpart to the same-day DBR tag, carrying the identical version bump. The duplication is an artefact of Databricks publishing two build streams from one commit.

    View source ↗