provisdom.solvers

A comprehensive Clojure library for numerical optimization, root-finding, interpolation, regression, and clustering. Built on Hipparchus, OjAlgo, and jcobyla, with much of the library implemented in pure Clojure.

Namespaces

Root-Finding

• roots.continuous — Univariate root-finding with multiple algorithms: Brent-Dekker, modified Newton-Raphson, Muller, plus Hipparchus methods (bisection, Brent, Illinois, Pegasus, Ridders, secant, Regula-Falsi). Includes quadratic equation solver.

• roots.integer — Bisection algorithm for strictly increasing discrete functions. Returns the minimum integer with function value ≥ 0.

• roots.plateau — Root-finding for monotonic functions that return plateau values. Supports univariate and multivariate cases.

• roots.polynomial — Polynomial root-finding using Laguerre's method via Hipparchus.

Optimization

• optimize.continuous-univariate — Brent optimizer for 1D minimization/maximization over bounded intervals.

• optimize.integer-univariate — Integer optimizer using exponential search or ternary search. Handles unimodal functions over integer domains.

• optimize.linear — Two-phase Simplex Method. Minimizes/maximizes linear objectives subject to linear constraints (equality, ≤, ≥).

• optimize.mixed-integer — Mixed Integer Programming (MIP) using OjAlgo. Extends linear programming to support integer and binary variable constraints.

• optimize.quadratic — Minimizes (1/2)x^T P x + q^T x subject to equality and inequality constraints using OjAlgo.

• optimize.nlp-constrained — Constrained nonlinear optimization using COBYLA. Minimizes objectives subject to nonlinear inequality constraints (≥ 0).

• optimize.nlp-unbounded — Unbounded nonlinear optimization with multiple pure-Clojure algorithms:

  • Powell: derivative-free direction-set method
  • Nelder-Mead: simplex-based derivative-free optimization
  • L-BFGS: limited-memory quasi-Newton with optional gradient
  • Gauss-Newton: nonlinear least-squares for residual minimization
  • Auto-selecting orchestrator tries multiple algorithms

• optimize.nlp-bounded — Bounded nonlinear optimization with box constraints:

  • COBYLA: linear-approximation constrained optimization
  • BOBYQA: quadratic-model bound-constrained optimization
  • CMA-ES: evolutionary strategy for non-smooth/non-convex objectives

Systems (Equation Solving)

Solvers for systems of equations without an objective function to optimize:

• systems.linear — Iterative solvers for symmetric linear systems (A × y = b): SYMMLQ and Conjugate Gradient methods. For small/dense systems, use direct least squares from provisdom.math.linear-algebra instead.

• systems.nonlinear — Finds variable values that satisfy nonlinear constraints:

  • Nonlinear Least Squares (Levenberg-Marquardt, Gauss-Newton)
  • Nonlinear Ordered Constraints (prioritized constraint satisfaction)

Interpolation

Interpolation passes through all data points exactly. Use for clean data, lookup tables, or when exact fidelity matters.

• interpolation.univariate — 1D interpolation methods:

  • Methods: cubic, cubic-hermite, cubic-closed, Akima, PCHIP (monotone-preserving), barycentric-rational, linear, polynomial, step
  • Specialized: quadratic with slope, cubic-clamped with 2nd derivatives, B-spline with knots
  • Extras: extrapolation modes (error/flat/linear), batch evaluation, spline integration, auto-selection
  • Slope interpolation: compute derivatives at query points

• interpolation.grid-2d — 2D grid-based interpolation (x-vals × y-vals → f-matrix):

  • Methods: polynomial, bicubic, bicubic-Hermite, bilinear

• interpolation.grid-3d — 3D grid-based interpolation (x-vals × y-vals × z-vals → f-tensor):

  • Methods: tricubic

• interpolation.scatter-nd — N-D scattered (non-grid) data interpolation:

  • Methods: microsphere projection, RBF (radial basis functions), natural neighbor (2D only)

Choosing a 1D Interpolation Method

Situation	Recommended Method	Why
Smoothest curve through clean data	:cubic (default)	C2 continuity for maximum smoothness
Monotonic data (CDFs, dose-response)	:pchip	Preserves monotonicity, prevents overshoots
Sharp corners or discontinuities	:akima	C1 smooth without oscillations near features
Need extrapolation outside range	:cubic-hermite, :pchip, :barycentric-rational	Native extrapolation support
Many equally-spaced points	:barycentric-rational	Avoids Runge oscillations at boundaries

Fitting (Smoothing & Curve Fitting)

Fitting methods do not pass through all points—they find a curve that balances fidelity to data against smoothness or a specified functional form. Use for noisy data or when you want a simpler representation.

• fitting.smoothing-spline — Smoothing cubic splines using eigendecomposition. Minimizes curvature penalty while fitting data. Supports weighted observations, effective degrees of freedom control, GCV-based automatic smoothing, derivatives, and variance estimation.

• fitting.p-spline — Penalized B-splines (P-splines). Uses B-spline basis with difference penalty on coefficients. More flexible knot placement than smoothing splines, good for non-uniform data. Supports weighted observations, GCV, derivatives, and multiple extrapolation modes.

• fitting.b-spline — B-spline curve fitting with user-specified knot placement. Provides direct control over curve flexibility via knot positions. No smoothing penalty—knot count determines flexibility.

• fitting.loess — Local polynomial regression (LOESS/LOWESS). Fits local polynomials in sliding windows. Good for discovering trends without assuming global structure.

• fitting.curve — Parametric curve fitting via nonlinear least squares (Levenberg-Marquardt):

  • Built-in: Gaussian, Harmonic, Polynomial
  • Custom: Arbitrary parametric functions
  • Diagnostics: R², MSE, residuals

Choosing a Fitting Method

Situation	Recommended Method	Why
Noisy data, want smooth trend	smoothing-spline or p-spline	Penalized splines balance fit vs smoothness
Non-uniform x spacing	p-spline	B-spline basis handles irregular grids well
Need derivatives of fitted curve	smoothing-spline or p-spline	Both provide first and second derivative functions
Want automatic smoothing selection	smoothing-spline or p-spline with ::auto-lambda?	GCV finds optimal smoothing
Specify degrees of freedom directly	smoothing-spline with ::effective-df	Newton's method finds λ for target EDF
Control knot positions exactly	b-spline	Direct knot specification, no penalty
Discover local trends	loess	Local regression adapts to changing patterns
Known functional form (Gaussian, etc.)	curve/fit-*	Levenberg-Marquardt finds best parameters
Polynomial of specific degree	curve/fit-polynomial	Fits degree-n polynomial (n+1 coefficients)

Interpolation vs Fitting

Aspect	Interpolation	Fitting
Passes through all points?	Yes	No (minimizes error)
Use case	Exact data, lookup tables	Noisy data, smoothing
You specify	Interpolation method	Smoothing level or functional form
Returns	Function (fn [x] -> y)	Function + diagnostics (R², residuals, etc.)

Example: Given 10 noisy data points:

• (interpolation-1D ...) with :cubic creates a wiggly curve through all 10 points
• (fit-smoothing-spline ...) creates a smooth curve that approximates the trend
• (fit-polynomial ...) with degree 2 fits a parabola (3 coefficients)

Curve Fitting vs Regression

Aspect	Curve Fitting	Regression
Goal	Find parameters of a known functional form	Model statistical relationships between X and y
Functional form	Arbitrary: a·e^(-((x-b)/c)²), a·cos(ωx+φ)	Linear in parameters: y = Xβ (possibly transformed)
Optimization	Nonlinear least squares (Levenberg-Marquardt)	Closed-form (OLS/Ridge) or IRLS (GLMs)
Assumptions	Just minimize residuals	Error distribution, link function
Output	Parameters of the specific function	Coefficients + diagnostics (R², MSE, precision)

Curve fitting answers: "Given this formula, what parameters fit best?"

Regression answers: "What's the statistical relationship between predictors X and response y?"

Logistic and beta regression aren't just curve fitting—they model the distribution of y given X (Bernoulli, Beta) with appropriate link functions, using Iteratively Reweighted Least Squares (IRLS).

Regression

All regression methods are in the provisdom.solvers.regression namespace hierarchy:

• regression.ols — Ordinary least squares with optional regularization:

  • OLS — Standard least squares via QR decomposition
  • Ridge (L2) — Closed-form solution with λI penalty
  • LASSO (L1) — Coordinate descent for sparse solutions
  • Elastic Net — Combined L1+L2 via coordinate descent

• regression.ols-streaming — Streaming/online regression with incremental QR decomposition. Supports real-time coefficient updates as data arrives, sliding windows via downdate, forgetting factors for weighting recent data, and parallel processing via state merging.

• regression.logistic — Binary logistic regression using IRLS with optional ridge regularization.

• regression.multinomial-logistic — Multi-class logistic regression (one-vs-all approach).

• regression.beta — Beta regression for responses bounded in (0, 1) using IRLS.

• regression.kernel-grnn — Generalized Regression Neural Network (Nadaraya-Watson kernel regression) with automatic spread optimization.

• regression.linear-basis — Linear basis fitting using closed-form OLS. Fits y = Σ wᵢ·bᵢ(x) for custom basis functions. Returns full OLS diagnostics.

• regression.stepwise — Stepwise variable selection:

  • Forward — Start empty, iteratively add best predictor
  • Backward — Start full, iteratively remove worst predictor
  • Both — Combine forward and backward steps
  • Supports AIC/BIC scoring with ordinary, logistic, or beta regression

IRLS (Iteratively Reweighted Least Squares)

Logistic and beta regression use IRLS, which fits generalized linear models by iteratively solving weighted least squares problems:

1. Compute working weights W based on current predictions
2. Compute working response z (adjusted dependent variable)
3. Solve weighted OLS: β = (X'WX)⁻¹ X'Wz
4. Repeat until convergence

For logistic regression: W = diag(μ(1-μ)), z = η + (y-μ)/(μ(1-μ))

For beta regression: W = diag(φ·μ(1-μ)), z = η + (y-μ)/(μ(1-μ))

Clustering

• clustering.k-means — K-Means clustering with K-Means++ initialization. Partitions n points into k clusters by minimizing within-cluster variance. Includes predict for assigning new points and silhouette-score for quality evaluation.

• clustering.dbscan — Density-based clustering algorithms that find clusters of arbitrary shape:

  • DBSCAN — Classic density-based clustering with epsilon neighborhood radius
  • OPTICS — Ordering-based algorithm; extract clusters at multiple epsilon thresholds from a single run
  • HDBSCAN — Hierarchical DBSCAN with automatic epsilon selection; best when you don't know your data's density structure
  • Helpers: k-distances (epsilon selection), silhouette-score, davies-bouldin-index (cluster validation)

Choosing a Clustering Algorithm

Situation	Recommended Method	Why
Know number of clusters	k-means	Fast, simple, well-understood
Arbitrary cluster shapes	dbscan or hdbscan	Density-based methods find non-convex clusters
Don't know epsilon or k	hdbscan	Automatic parameter selection
Want to explore density thresholds	optics	Single run, extract at multiple epsilons
Need to tune epsilon	dbscan with k-distances	Plot k-distances to find the "elbow"

Usage

(require '[provisdom.solvers.roots.continuous :as roots])
(require '[provisdom.solvers.optimize.nlp-bounded :as nlp])
(require '[provisdom.solvers.interpolation.univariate :as interp])

;; Find root of f(x) = x³ - 3x
(roots/root-solver
  {::roots/univariate-f (fn [x] (- (* x x x) (* 3 x)))
   ::roots/guess 2.0
   ::roots/interval [-5.0 5.0]})

;; Minimize f(x,y) = x² + y² subject to bounds
(nlp/bounded-nlp-without-evolutionary
  {::nlp/objective (fn [da] (let [[x y] da] (+ (* x x) (* y y))))
   ::nlp/var-intervals [[-10.0 10.0] [-10.0 10.0]]
   ::nlp/vars-guess [1.0 1.0]})

;; Cubic spline interpolation
(let [f (interp/interpolation-1D
          {::interp/f-vals [0.0 1.0 4.0 9.0]
           ::interp/x-vals [0.0 1.0 2.0 3.0]})]
  (f 1.5))

;; K-Means clustering
(require '[provisdom.solvers.clustering.k-means :as k-means])
(k-means/k-means!
  {::k-means/clustering-points [[0.0 0.0] [1.0 1.0] [10.0 10.0] [11.0 11.0]]
   ::k-means/k 2})

;; DBSCAN clustering
(require '[provisdom.solvers.clustering.dbscan :as dbscan])
(dbscan/dbscan
  {::dbscan/clustering-points [[0.0 0.0] [0.1 0.1] [10.0 10.0] [10.1 10.1]]
   ::dbscan/epsilon 0.5
   ::dbscan/min-points 2})

Dependencies

• Hipparchus — root-finding, optimization, interpolation, least squares
• OjAlgo — linear, quadratic, and mixed-integer programming
• jcobyla — constrained nonlinear optimization (COBYLA algorithm)

Many namespaces (regression, fitting, clustering, some interpolation) are pure Clojure with no external dependencies beyond provisdom.math.

License

Copyright © 2018-2026 Provisdom Corp.

Distributed under the GNU Lesser General Public License version 3.0.