LoginSign Up
← Back to all sparks
S

Slurm

DEVOPS
Velocity2.5

Workload manager and job scheduler for HPC clusters

slurm.schedmd.com ↗

Two supported branches, both absorbing the aftershocks of the 26.05 identity change.

hpc-schedulerstabilityregression-fixesparallel-branchescgroup-identity
◆Current state
Slurm is maintaining the 26.05 and 25.11 lines in parallel, and both are running fix-only. The 26.05 point releases are still cleaning up after the major release that made SLUID the default identity and added an in-tree TLS path: 26.05.1 repaired upgrade and mixed-version job handling, 26.05.2 restored the option to use job IDs in cgroup paths, and 26.05.3 clears srun completion hangs, a QOS-preemption segfault, and several slurmctld crash paths.
◆Where it's heading
The pattern across four consecutive releases is consolidation, not expansion — crash fixes, race conditions during shutdown and reconfigure, and regressions traced back to 25.05 and 26.05. New surface is limited to small operational corrections like sinfo state filters and license matching. Sites on 25.11 are getting the same stability work backported, which suggests the older branch is still carrying real production load.
◆Prediction
Expect the 26.05.x line to keep issuing fix-only point releases until the SLUID and TLS changes stop generating regressions; nothing in these entries points to new features landing on either branch first.

◆Recent moves

  1. 6d ago

    26.05.3 clears srun completion hangs and a QOS preemption segfault

    Another fix-only point release on the 26.05 line: srun no longer hangs intermittently at job or step completion, slurmctld stops segfaulting on QOS-based suspend preemption with OverSubscribe, and async srun steps now honor --kill-on-bad-exit. Small behavioral corrections to sinfo state filters and license fuzzy matching round it out.

    View source ↗
  2. 1mo ago

    26.05.2 adds a cgroup path escape hatch back to job IDs

    Gives sites an option to put the job ID back in the cgroup path instead of SLUID, with SLUID remaining the default. A direct concession to the tooling breakage the 26.05 identity change caused, alongside fixes for stale assoc_mgr pointers and PMIx startup failures.

    View source ↗
  3. 1mo ago

    25.11.7 fixes requeued jobs losing their license reservations

    Repairs a 25.05 regression that let requeued jobs run without holding sufficient licenses — a correctness problem for sites that schedule against licensed software. Ships with slurmctld crash and deadlock fixes shared with the 26.05 branch.

    View source ↗
  4. 2mo ago

    26.05.1 repairs mixed-version and post-upgrade job handling

    The first point release after 26.05, focused on sites mid-upgrade: steps launched from mismatched releases, jobs that survived the jump from 25.05, and SLUID-based step completion lookups in the database. Confirms the major release landed with rough edges around version skew.

    View source ↗
  5. 2mo ago

    Slurm 26.05 ships SLUID identity and an in-tree TLS path

    ⚡ SPARK

    The major release that sets up everything the point releases have been repairing since: SLUID becomes the default identity used for cgroup paths and step lookup, TLS lands through an s2n plugin with certificate generation, and slurmdbd storage can now be reached over a Unix socket. The fix-heavy releases that follow are the cost of those two changes.

    View source ↗
  6. 3mo ago

    25.11.6 clears reconfigure and shutdown hangs in slurmctld

    Resolves slurmctld hangs during reconfigure and shutdown caused by unenforced client I/O timeouts, plus jobs stuck in COMPLETING when PrologFlags=RunInJob is set. Maintenance on the older branch that the 26.05 line later inherits.

    View source ↗