LoginSign Up
← Back to all sparks
P

pyjanitor

DEVOPS
Velocity2.5

Clean APIs for data cleaning in Python

github.com/pyjanitor-devs/pyjanitor ↗

pyjanitor breaks its pandas 2.x floor and returns from a four-month quiet spell.

pandasdata-cleaninggroupbyperformancepolarsbreaking-change
◆Current state
After a stretch of dependency-only releases through spring, v0.32.24 is the first substantive release since March. It carries a 5.9x speedup in find_replace by swapping .apply() for .map(), two new options on the cleaning verbs (strip_whitespace on clean_names, drop_first on expand_column), a cheaper polars expand path, and a hard requirement of pandas 3.0 and Python 3.11. The releases before it were the groupby migration arc — by methods moved onto groupby objects, an assign method added there, and pd.col column references supported.
◆Where it's heading
Two arcs are converging. The API arc keeps folding pyjanitor's verbs into pandas' own grouping and column-reference idioms rather than maintaining a parallel vocabulary, with mutate formally deprecated along the way. The maintenance arc has now committed to pandas 3.0 as the floor, which closes off the 2.x user base but frees the library to use the new implementation instead of working around two majors at once. The polars work continues quietly beside both.
◆Prediction
With pandas 3.0 established as the baseline, expect the next releases to lean on it directly — retiring compatibility shims and continuing the deprecation of the older standalone verbs in favor of the groupby-attached forms.

◆Recent moves

  1. 1d ago

    find_replace 5.9x faster; pandas 3.0 and Python 3.11 now required

    The first release with real content since March: find_replace gets up to a 5.9x speedup on large datasets while preserving unmapped values, clean_names and expand_column each gain a parameter, and the polars expand path avoids costly schema computation. The same release enforces pandas >=3.0.0 and Python >=3.11, a hard floor that cuts off anyone still on pandas 2.x.

    View source ↗
  2. 4mo ago

    Dependency bumps only; no functional changes

    A maintenance release with no features, fixes, or deprecations — only CI action and dependency bumps. Representative of the quiet stretch that preceded v0.32.24.

    View source ↗
  3. 4mo ago

    Codecov GitHub Action bumped to v6

    A single CI change bumping the Codecov action from v5 to v6. Nothing user-visible.

    View source ↗
  4. 4mo ago

    pivot_longer refactored for speed on pandas

    pivot_longer is refactored for the pandas path, using indexing instead of concatenation and cheaper data checks. Part of the recurring pattern of revisiting individual verbs for performance rather than adding new ones.

    View source ↗
  5. 6mo ago

    by methods migrate to groupby objects, old forms deprecated

    The by methods move onto groupby objects with deprecation warnings on the older forms, a direct step in aligning pyjanitor's API with pandas' own grouping model.

    View source ↗
  6. 6mo ago

    pd.col column references supported in DataFrame operations

    Adds support for pd.col references in DataFrame operations where possible, letting users write against pandas' own column-reference idiom instead of a pyjanitor-specific one.

    View source ↗