Loss Functions¶

torchregress provides a comprehensive library of loss functions for regression — from simple point predictions to full distributional models with uncertainty quantification. This page is the catalogue: every loss family with its formula, use case, and link to the full guide.

From likelihood to loss¶

Almost every loss here is the negative log-likelihood of a conditional model. Given inputs \(x\), a target \(y\), and a model that predicts distribution parameters \(\theta(x)\), training minimises

The choice of \(p_\theta\) is the choice of loss: a fixed-variance Gaussian gives squared error, a Laplace gives absolute error, a Gaussian whose variance is also predicted gives GaussianNLLLoss, a mixture gives MixtureDensityLoss, and so on. Several losses are instead scoring rules: pinball is a consistent score for a quantile, while CRPS, the energy score and the Gaussian Wasserstein bound are proper scores for the whole predictive distribution even when no parametric likelihood is assumed. That distinction matters when reading the tables below: likelihood losses are tied to a distributional family, scoring rules are not.

All losses share the same call contract and the same reduction semantics:

loss_fn(y_pred, target, mask=None, weights=None)   # mask: True = valid, False = missing

mask removes missing entries from the denominator and weights scales samples; reduction="mean" is always the weighted mean over valid entries (see Base Classes for the exact zero-fill rule).

At a Glance¶

graph LR
    A["Your Data"] --> B{"What do you need?"}
    B -->|Point prediction| C["WeightedMSELoss / WeightedHuberLoss"]
    B -->|Uncertainty| D["GaussianNLLLoss / BetaNLLLoss"]
    B -->|Robustness| E["CauchyLoss / TukeyBiweightLoss"]
    B -->|Intervals| F["QuantileLoss + CQR"]
    B -->|Full distribution| G["MixtureDensityLoss / NormalizingFlowLoss"]
    B -->|Count data| H["PoissonDevianceLoss / TweedieLoss"]

Standard Losses¶

The building blocks — these wrap PyTorch losses with unified support for masks, sample weights, and consistent reduction semantics.

torchregress vs PyTorch native — why both exist¶

Several torchregress losses share names with torch.nn primitives. They are not redundant — each adds mask/weight support that native losses lack:

Every other loss in torchregress has no PyTorch-native equivalent — they are domain-specific formulations for uncertainty quantification, robust regression, censored/ordinal targets, error-in-variables, conformal prediction, etc.

See Base Classes for foundations.

Gaussian Losses¶

Parametric losses for the Gaussian family — supporting heteroscedastic (input-dependent) variance.

Read the full Gaussian losses guide for deep dives on diagonal, full-covariance, and low-rank variants. Also see Beta-NLL for stabilized training, Faithful Gaussian & CRPS for decoupled mean/variance training and a proper-score alternative, and Wasserstein bound for covariance supervision.

Robust Losses¶

Bounded-influence losses for data with outliers or heavy-tailed noise.

Read the full Robust losses guide — influence functions, redescending behavior, Barron family, CVaR.

Quantile & Expectile Losses¶

Distribution-free models for specific aspects of the conditional distribution.

Read the full Quantile & Expectile guide — crossover penalties, multi-level prediction intervals, expectile-to-quantile conversion. For minimum-volume prediction sets with a learned level-set boundary, see Super-Level-Set (SLS) regression.

Ordinal Losses¶

For ordered categorical targets where class distance matters.

Read the full Ordinal losses guide — cumulative link, CORAL, cross-entropy baselines.

Censored Regression Losses¶

For partially observed outcomes — right/left/interval censoring.

Read the full Censored losses guide — Gaussian censored NLL, AFT survival models, quantile censoring.

Poisson, Tweedie & Count Losses¶

For count data and targets with specific mean-variance relationships.

Read the Poisson & Tweedie guide for count models and Poisson-Gaussian for shot noise plus read noise (PoissonGaussianLoss).

Evidential Regression¶

Single-forward-pass aleatoric + epistemic decomposition via a Normal-Inverse-Gamma prior:

Read the full Evidential regression guide — Normal-Inverse-Gamma prior, single-pass decomposition, calibration caveats.

Mixture Density Networks (MDN)¶

For multimodal conditional distributions:

Read the full MDN guide — component selection, label switching, mixture-of-mixtures ensembles.

Normalizing Flows¶

For arbitrarily complex conditional distributions:

Read the full Normalizing flows guide — NSF/cNSF, cINN/RealNVP, BPF, contrastive flow variant, inference sampling — and the neural ratio estimation guide for the classifier-based likelihood-ratio estimator, posterior-weight utilities and SBI calibration diagnostics. The flow design notes record why exact conditional likelihood is the default for low-dimensional tabular regression and when flow matching or diffusion is the better fit.

Grid & Basis Densities¶

For heads that output a whole distribution on a fixed 1D support (binned or spline-basis):

Read the Grid & basis densities guide.

Error-in-Variables Losses¶

For regression when inputs have measurement uncertainty:

Read the full EIV losses guide for loss-based correction, split-measurement and corrected-score losses for attenuation-free objectives, or see the algorithm pages for Regression Calibration, SIMEX and forward generative inversion.

Imbalanced Regression¶

For targets with significant distribution imbalance:

Read the full Imbalanced regression guide — density weighting, Focal-R, LDS, propensity weighting — and PRIME & ConR for representation-level methods, plus Balanced MSE (BMC) and RankSim. Group-robust training over target bins is covered by Continuous Group DRO.

Noisy Labels & Uncertain Ground Truth¶

Read the Noisy labels guide, the noise-aware density guide, for target-noise-aware NLL/densities and consistency losses, or the Uncertain GT guide for pseudo-label workflows.

Transform Losses¶

Read the full Transform losses guide — log, Box-Cox, sqrt, Yeo-Johnson transforms, and the generic TransformedTargetLoss wrapper.

Flexible-Shape Families (losses.families)¶

Parametric NLLs for non-Gaussian conditional distributions — skew, heavy-tailed, bounded, and extreme-value families. See the full Families guide for formulas, usage, and the unconstrained_inputs pattern.

All NLL families (except SQRLoss) expose unconstrained_inputs=True (default): positivity-constrained parameters are mapped internally as softplus(raw)+eps. Set unconstrained_inputs=False when your head already outputs positive values — the loss applies the inverse softplus transform internally to avoid softplus(softplus(x)).

Read the full Families guide — skew/t, beta, Johnson SU, sinh-arcsinh, GEV, asymmetric Laplace, SQR, and the unconstrained_inputs contract.

Conformal Prediction Wrappers¶

Conformal prediction is a post-hoc layer, not a likelihood: it turns any point or quantile predictor into intervals with finite-sample coverage. The predictors are exported from torchregress.losses for convenience, but the methodology (split, CQR, CV+, weighted/non-exchangeable, distributional) is documented in Conformal Prediction and the methods section.

Loss Selection Guide¶

For a generic scalar binned-CDF training objective, use CumulativeCDFLoss or cumulative_cdf_loss. The astronomy-specific recipe/model composites and stellar-spectrum examples are removed from the core library; applications can compose the generic losses and FTTransformer in their own training pipelines.