LoginSign Up
← Back to all sparks
Z

zarr-python

DEVOPS
Velocity6.3

Chunked compressed N-dimensional arrays

github.com/zarr-developers/zarr-python ↗

Zarr's monorepo split keeps spinning out standalone packages, each with its own release cadence.

package splitarray storagetype systemrelease toolinglazy indexing
◆Current state
Zarr-python has decomposed into independently versioned packages — zarr-metadata, zarr-indexing, zarr-http-server — each shipping on its own clock inside one repository. The http server was the point where the split stopped being a refactor and produced a capability Zarr did not have. The most recent releases are the boring half of that work: docs builds, changelog tooling, and type-alias widening that only matters to downstream annotators.
◆Where it's heading
The split is being taken seriously as a distribution decision, not just a directory layout: each package gets its own docs site, its own towncrier changelog, and its own release notes discipline. That implies more packages will follow, and that the core zarr-python distribution is heading toward being a thin composition over them. Type-level changes in zarr-metadata are already being versioned as minor releases because they change what consumers can annotate against.
◆Prediction
Expect further subpackages carved out of zarr-python along the same pattern, and zarr-indexing's LazyArray to gain the transform surface that TensorStore already exposes. A 3.2.0 final following the rc is the other open thread.

◆Recent moves

  1. 5d ago

    zarr-metadata 0.5.0: JSONValue widened to Sequence, old constant spellings removed

    Cut as a minor rather than a patch for two reasons that matter to consumers: JSONValue's array arm widens to the covariant Sequence, changing what every downstream annotation against it accepts, and the old version-last constant spellings are removed outright. The rest is release-notes plumbing — the sdist allowlist got a news fragment, and towncrier's misc category was redefined to actually show its content instead of rendering a bare PR link.

    View source ↗
  2. 7d ago

    Subpackage docs now build from the package directory

    Read the Docs was invoking mkdocs from the repo root, so settings resolved relative to the working directory pointed at the wrong tree and zarr-indexing's snippet includes failed while local builds passed. Building from the package directory makes all three invocations identical. Infrastructure correctness for the package split, invisible to users.

    View source ↗
  3. 7d ago

    zarr-indexing 0.2.0: LazyArray brings lazy indexing to the array API

    LazyArray generalizes the index transforms shipped in 0.1.0 into lazy indexing over any array-API-compatible backend. It moves zarr-indexing from a translation utility toward something usable as a general indexing layer, which is what makes the package worth having outside Zarr itself.

    View source ↗
  4. 7d ago

    zarr_http_server-v0.1.0: HTTP server that exposes stores, arrays, groups (#3732)

    ⚡ SPARK

    The first release of a standalone zarr-http-server package, exposing stores, arrays, and groups over HTTP — the point where the package-split arc stops being a refactor and produces a surface Zarr did not have. The same tag corrects byte-order handling for structured dtypes in the bytes codec, which had been silently corrupting data whose field byte order differed from the stored layout.

    View source ↗
  5. 19d ago

    zarr-indexing 0.1.0: TensorStore-style index transforms as a standalone package

    Carves index transforms out into their own package, borrowing TensorStore's model. The first of the split packages to add a concept rather than relocate one, and the base that 0.2.0's LazyArray builds on.

    View source ↗
  6. 21d ago

    zarr-metadata 0.4.0: model-layer changes and a standalone docs site

    Consumes the model-layer feature and removal fragments and adds a standalone documentation site and justfile. The docs site is the signal here: each split package is being given the trappings of an independent project, not treated as an internal module.

    View source ↗