Delta Lake
A 4.4.0 tag appears, but the feed carries only its release plumbing
Comparison · Analytics
distributions3 vs sparklyr
A side-by-side editorial comparison of distributions3 and sparklyr — release velocity, themes, recent moves, and the top alternatives to consider.
distributions3 vs sparklyr: at a glance
What is distributions3?
distributions3 0.3.0 adds sample-based distributions and likelihood derivatives
An R package giving probability distributions a consistent object interface - d/p/q/r functions, moments, and prodist() methods that pull a fitted distribution out of a regression object. Version 0.3.0 is the first substantive release under Achim Zeileis's maintenance, and it widens what a distribution is allowed to be: Empirical() represents a distribution by a random sample rather than by parameters, and numerical fallbacks now fill in cdf(), pdf(), quantile(), random() and the moments for any object that implements only some of them. New score() and hessian() generics compute first and second derivatives of the log-likelihood with respect to the parameters, analytically for a few distributions and numerically for the rest.
What is sparklyr?
sparklyr now spends its releases absorbing dbplyr changes and feeding pysparklyr
sparklyr connects R to Spark, and almost nothing in this window originates inside the package. Releases restore compatibility after dbplyr changes its SQL generation, adapt to Spark 4.0 and to R 4.4's version-comparison changes, and convert functions into S3 methods so pysparklyr can supply its own implementations.
distributions3 vs sparklyr: editorial side-by-side
distributions3
6.3ANALYTICS
distributions3 0.3.0 adds sample-based distributions and likelihood derivatives
◆ Current state
An R package giving probability distributions a consistent object interface - d/p/q/r functions, moments, and prodist() methods that pull a fitted distribution out of a regression object. Version 0.3.0 is the first substantive release under Achim Zeileis's maintenance, and it widens what a distribution is allowed to be: Empirical() represents a distribution by a random sample rather than by parameters, and numerical fallbacks now fill in cdf(), pdf(), quantile(), random() and the moments for any object that implements only some of them. New score() and hessian() generics compute first and second derivatives of the log-likelihood with respect to the parameters, analytically for a few distributions and numerically for the rest.
◆ Where it's heading
Growth used to arrive as new distribution families contributed from outside - the extreme-value set, Erlang, later the Poisson binomial. This release changes the axis: alongside two new distributions it adds an inference layer (score, hessian) and a forecast-evaluation one (crps() methods against scoringRules), which are capabilities about distributions rather than more of them. Dependency weight is being cut at the same time, with ggplot2 demoted to Suggests and glue replaced by base R sprintf().
◆ Prediction
With numeric fallbacks and the derivative generics in place, expect analytic score() and hessian() methods to be filled in across more of the distribution catalogue. The constructor-default change is the likeliest source of follow-up fixes, since calls like Poisson() now return a length-zero distribution where they previously errored.
sparklyr
0.0ANALYTICS
sparklyr now spends its releases absorbing dbplyr changes and feeding pysparklyr
◆ Current state
sparklyr connects R to Spark, and almost nothing in this window originates inside the package. Releases restore compatibility after dbplyr changes its SQL generation, adapt to Spark 4.0 and to R 4.4's version-comparison changes, and convert functions into S3 methods so pysparklyr can supply its own implementations.
◆ Where it's heading
Two dependencies set the agenda. dbplyr repeatedly changes identifier quoting and lazy-table internals, and each change costs sparklyr a release. Meanwhile the package is being hollowed into a backend: ml_fit(), spark_apply(), spark_write_delta() and now tune_grid_spark() exist as methods so that pysparklyr, the Databricks Connect path, can override them. Dependency removal - tibble, rappdirs, digest - runs alongside as the package slims down.
◆ Prediction
Expect the next releases to continue tracking dbplyr and Spark versions, and more functions to be converted to methods as functionality shifts toward pysparklyr; new capability arriving in sparklyr itself looks unlikely.
Alternatives to distributions3 and sparklyr
Other Analytics products tracked by Sparkpulse, ranked by recent ship velocity. Each card links to a full editorial trajectory and lets you pivot into a head-to-head comparison with either distributions3 or sparklyr.
Basedash
Basedash keeps pushing its data out of the workspace — now to people without accounts
RStudio
RStudio ships through release branches, and the notes are commit messages
dbt Core
dbt Fusion's second beta is adapter work: ClickHouse gets materializations, indexes, and catalogs
Fulcrum
Fulcrum is betting its whole map stack on Esri, with a hard Google Maps cutoff on September 1.
Holistics
Holistics keeps fencing in the AI layer it spent the summer building.
See all distributions3 alternatives → · See all sparklyr alternatives →
Recent activity from distributions3 and sparklyr
Latest ship moves from both products, interleaved chronologically. ⚡ = editorial spark.
- 3h agodistributions3Empirical distributions, plus score and hessian generics ⚡
- 29d agodistributions3Maintenance moves to Achim Zeileis; moment calculations corrected
- 1mo agosparklyrRestores compatibility after dbplyr changed Hive quoting
- 4mo agosparklyrAdds tune_grid_spark() for pysparklyr to implement
- 10mo agodistributions3ggplot2 compatibility for the plotting functions
- 10mo agosparklyrFixes lazy-table field lookup and a name collision
- 1y agosparklyrCatches up with released Spark 4.0; ml_load() reads via Spark
- 1y agodistributions3Poisson binomial distribution, with a normal-approximation fallback
- 2y agosparklyrDatabricks autoloader streaming ingestion; R 4.4 fixes
- 2y agosparklyrDrops tibble and rappdirs; retires Spark 2.3 JARs
- 3y agodistributions3is_discrete and is_continuous generics, plus elementwise type-safety
- 4y agodistributions3Extreme-value family, Erlang, and a plotting generic
Frequently asked questions
What is the difference between distributions3 and sparklyr?
They serve adjacent needs but don't currently overlap on shipped themes. distributions3 is currently shipping more aggressively (velocity 6.3 vs 0.0), with 1 editorial sparks in the last 30 days against 0. See the at-a-glance table above for a side-by-side breakdown of velocity, recent sparks, and editorial themes.
Is distributions3 better than sparklyr?
Sparkpulse doesn't pick a winner — we score release velocity, not feature parity. distributions3 is currently shipping more aggressively (velocity 6.3 vs 0.0), with 1 editorial sparks in the last 30 days against 0. For your specific use case, the alternatives sections above list other Analytics products to evaluate alongside.
What are the best alternatives to distributions3?
Top distributions3 alternatives in Analytics are ranked by recent ship velocity. Browse the "distributions3 alternatives" section above for the current picks, or visit /alternatives/distributions3-r for the full list with editorial commentary on each.
What are the best alternatives to sparklyr?
Top sparklyr alternatives in Analytics are ranked by recent ship velocity. Browse the "sparklyr alternatives" section above for the current picks, or visit /alternatives/sparklyr for the full list with editorial commentary on each.