Practical Usage Guide¶

This guide maps real-world data pathologies to specific torchregress methods, with quantitative tradeoffs and mathematical justification for each choice.

For the principled workflow (baseline → probabilistic → robust), see Best Practices. For the full capability matrix, see Method Selection Matrix.

Loss Selection by Data Pathology¶

Heteroscedastic Noise (variance depends on \(x\))¶

The canonical approach is to predict both \(\mu(x)\) and \(\log\sigma^2(x)\), optimising the Gaussian negative log-likelihood:

This is a proper scoring rule — it is uniquely minimised by the true conditional distribution \(p(y \mid x)\) (Gneiting & Raftery 2007).

• Use GaussianNLLLoss for unimodal, heteroscedastic data.
• Use BetaNLLLoss when Gaussian NLL exhibits variance collapse early in training (Skafte et al. 2019).
• For heteroscedasticity with outliers, combine a robust mean term with a variance head: see Robust losses.

Outliers & Heavy Tails¶

The choice of robust loss is governed by the influence function \(\psi(r) = \rho'(r)\), which measures how much a single residual \(r = y - \hat{y}\) affects the parameter estimate (Huber 1964):

# Rule of thumb:
# <10% outliers  → WeightedHuberLoss(delta=1.0)
# 10–25%         → CauchyLoss(c=1.0)
# >25%           → TukeyBiweightLoss(c=4.685)
# unknown        → AdaptiveRobustLoss()  # learn alpha

For a complete treatment with influence-function plots, see the Robust losses guide.

Count Data & Positive Targets¶

The mean–variance relationship determines the appropriate loss:

See Poisson & Tweedie guide for details.

Censored Targets¶

When the target is only partially observed — right-censored (sensor clipped), left-censored (detection limit), or interval-censored — the likelihood is:

where \(\Phi\) is the standard Gaussian CDF (Tobin 1958).

• Use CensoredGaussianNLLLoss for Gaussian censored regression.
• Use CensoredQuantileLoss for a non-parametric alternative.
• Use AFTLoss for survival / time-to-event modelling (Cox 1972).

Multimodal Targets¶

When a given \(x\) can produce multiple distinct \(y\) values, Gaussian losses are misspecified (they average modes). Use mixture models:

• MixtureDensityLoss (Mixture Density Network) for known \(K\) (Bishop 1994).
• NormalizingFlowLoss for flexible, non-parametric densities (Rezende & Mohamed 2015).

See Multimodal targets guide.

Imbalanced Targets¶

When the target distribution is heavily skewed, loss reweighting methods target specific tail-region tradeoffs (Ren et al. 2019):

See Imbalanced regression guide.

Measurement Error in Inputs¶

When \(x\) is observed with noise \(x_{\text{obs}} = x_{\text{true}} + \epsilon\), standard regression produces attenuated estimates (Carroll et al. 2006):

See EIV losses, RC, SIMEX.

Prediction Intervals: Distributional vs. Distribution-Free¶

Recommendation: use quantile regression + CQR for robust, adaptive intervals with finite-sample coverage. Use Gaussian NLL when you need a full density (not just intervals) and trust the Gaussian assumption.

Evaluation Checklist¶

Before trusting a model's uncertainty estimates, verify:

• RMSE / NLL / CRPS all improve on holdout (not just one).
• Reliability diagram: 90% intervals contain \(\approx 90\%\) of holdout data.
• PIT histogram: uniform for well-calibrated densities.
• OOD sensitivity: uncertainty increases on out-of-distribution inputs.
• No variance collapse: \(\sigma(x)\) is not near zero everywhere.
• Residuals: no obvious structure (heteroscedasticity, curvature).

For implementation, see Visualization Methods and Calibration Metrics.

References¶