LoginSign Up
← Back to all sparks
S

sdcMicro

INFRA · APIS
Velocity0.0

Statistical Disclosure Control Methods for Anonymization of Data and Risk Estimation

github.com/sdctools/sdcmicro ↗

A 20-year anonymization toolbox now has a language model inside its refinement loop.

disclosure-controlanonymizationllm-assisted-workflowsk-anonymityofficial-statisticsr-packages
◆Current state
sdcMicro is the reference R implementation of statistical disclosure control — k-anonymity, local suppression, PRAM, microaggregation, record swapping — used by national statistical offices, with a Shiny GUI (sdcApp) as its second face. The feed shows a long GUI-maintenance era through 2018-2022 and then a gap, and the package that reappears in 5.8.2 has an AI_applyAnonymization() workflow and a query_llm() helper that the older entries know nothing about. The July release tunes that loop rather than introducing it.
◆Where it's heading
Two threads run in parallel. The visible one is the LLM-assisted anonymization path maturing: 5.8.2 gives its refinement loop early stopping via tol and patience so it stops when the combined utility score plateaus instead of burning all max_iter rounds, and teaches query_llm() to drop the temperature parameter for reasoning models that reject it. The other is unglamorous statistical correctness — a distinct l-diversity computation fixed for NAs in key variables, with the C++ simplified and tests added. The release also ships reproducibility scripts for a SoftwareX paper, which suggests the AI path is being written up rather than quietly trialled.
◆Prediction
The provider-compatibility fix is reactive — a parameter dropped because one model family rejected it — so expect more of the same as query_llm() meets other backends. Given tol and patience were added to stop wasted iterations, cost or runtime of the refinement loop is the live concern, and further controls on it are the likeliest next move.

◆Recent moves

  1. 1mo ago

    Early stopping for the AI anonymization loop, plus GPT-5.x support

    The LLM-assisted path is already in the package; this release makes it cheaper and less brittle. Early stopping ends refinement once the utility score stops improving by more than tol for patience rounds, and query_llm() now omits temperature for model families that reject it. The l-diversity NA bugfix is the statistical half of the same release.

    View source ↗
  2. 4y ago

    recordSwap() adds targeted record swapping

    Adds targeted record swapping as a new anonymization method alongside the existing suppression and perturbation methods — real new capability, but accretion within a toolbox designed to accumulate methods. The rest is a fix in extractManipData() and a configurable undo threshold.

    View source ↗
  3. 5y ago

    Two new information-loss measures and a run of sdcApp fixes

    Typical of the package's GUI-maintenance era: buttons that stopped rendering in sdcApp restored, plus IL_correl() and IL_variables() added to the data-utility measures. The measures are the part users of the R interface would notice.

    View source ↗
  4. 5y ago

    Compatibility fixes for R 4.1 sorting and rsconnect deployment

    Maintenance only — adapting to a base R sort/order change, writing rmarkdown intermediates to a temp directory so report() deploys under rsconnect, and bugfixes in pram() and dRisk(). No change to what the package can do.

    View source ↗
  5. 8y ago

    Minor release, mostly GUI work

    A one-line release note claiming GUI improvements without naming any. Nothing in the entry identifies a user-visible change, which fits the stretch where sdcApp absorbed most of the maintenance effort.

    View source ↗
  6. 8y ago

    dUtility() corrected so IL1 and IL1s can be distinguished

    Mostly GUI bugfixes and easier Shiny Server deployment, but the dUtility() fix changes a reported number: IL1 and IL1s become distinguishable methods rather than collapsing together. Anyone comparing information-loss figures across versions gets different output.

    View source ↗