Skip to content

User Guide

This guide is organized by what you want to achieve, not by module. Each section below maps a regression problem to the right tool, with code snippets and links to deeper references. If you know your problem, jump straight to it. If you're exploring, read top-to-bottom — the sections build from simple (heteroscedastic regression) to advanced (measurement error, shift adaptation).

New here?

Start with Core Concepts for the vocabulary (aleatoric vs. epistemic, proper scoring rules, conformal coverage), then come back here to find the right tool for your task.

Predicting with Uncertainty

Most regression models produce a single point prediction \(\hat{y}\). But in practice you also need to know how confident the model is. The sections below walk you from simplest (heteroscedastic head) to most comprehensive (full ensemble decomposition).

Heteroscedastic variance (simplest)

Predict mean and variance per sample using GaussianNLLLoss:

from torchregress.losses import GaussianNLLLoss

loss_fn = GaussianNLLLoss()

# Model outputs 2× target dim: [mean, log_var]
out = model(x)          # (batch, 2)
loss = loss_fn(out, y)  # trains both mean and variance

The predicted variance \(\sigma^2(x)\) captures aleatoric uncertainty — irreducible noise that varies with input.

When is this enough?

If you only need to know "how noisy is this region of input space", GaussianNLLLoss is sufficient and cheap. For model uncertainty (epistemic), add an ensemble.

Train \(M\) independent models and examine disagreement:

from torchregress.ensemble import HeteroscedasticEnsembleModel

ensemble = HeteroscedasticEnsembleModel(base_model=MyModel, ensemble_size=5)
result = ensemble.predict(x_test)
aleatoric = result["aleatoric_variance"]   # data noise
epistemic = result["epistemic_variance"]   # model ignorance

→ See Ensembles for Uncertainty for full details. For terminology and edge cases such as quantile ensembles, see Uncertainty Decomposition.

Single-pass decomposition

If you can't afford an ensemble, evidential regression provides both uncertainty types in one forward pass — see Evidential Regression. This page continues with other common tasks below.

Handling Outliers

Standard MSE is highly sensitive to outliers — a single extreme point can dominate the gradient.

Robustness Loss When to use
⭐⭐ WeightedHuberLoss Quick fix, moderate outliers
⭐⭐⭐ CauchyLoss Logarithmic suppression
⭐⭐⭐⭐ TukeyBiweightLoss Complete rejection beyond threshold

→ See Robust Losses for the full family.

Prediction Intervals

Quantile regression (training time)

from torchregress.losses import MultiQuantileLoss

loss_fn = MultiQuantileLoss(quantiles=[0.05, 0.5, 0.95])
# Model outputs 3 values per target

→ See MultiQuantileLoss API.

Conformal prediction (post-hoc, guaranteed coverage)

Wrap any trained model with a calibration procedure:

from torchregress.losses import CQR

cqr = CQR(alpha=0.1)  # target 90% coverage
cqr.calibrate(y_pred_cal, y_cal)
lower, upper = cqr.predict_interval(y_pred_test)

→ See CQR API.

Finite-sample guarantee

\[P(Y_{n+1} \in [\text{lower}, \text{upper}]) \geq 1 - \alpha\]

→ See Conformal Prediction.

Dealing with Imbalanced Targets

When some target ranges are under-represented (e.g., rare extreme values):

from torchregress.losses import DensityWeightedLoss, FocalRLoss

# Inverse-density reweighting
loss_fn = DensityWeightedLoss(base_loss=GaussianNLLLoss())

# Focus on hard (rare) examples
loss_fn = FocalRLoss(gamma=2.0)

→ See Imbalanced Regression.

Noisy, Censored, or Ordinal Data

Problem Loss Link
Known target noise NoisyTargetGaussianNLL Noisy Labels
Censored / survival CensoredGaussianNLLLoss Censored
Ordered discrete CumulativeLinkLoss Ordinal

Measurement Error in Inputs

When features \(X\) are measured with known noise \(\sigma_u\):

from torchregress.algorithms import RegressionCalibration

rc = RegressionCalibration(sigma_u=0.3)
X_clean = rc.fit_transform(X_noisy)

from torchregress.algorithms import SIMEX

simex = SIMEX(model_factory, train_func, sigma_u=0.3)
simex.fit(X_noisy, y)

from torchregress.losses import StructuralEIVLoss

# sigma_x, sigma_y: noise covariances; sigma_xy: cross-covariance (see EIV Losses)
loss_fn = StructuralEIVLoss(model=model, sigma_x=cov_x, sigma_y=cov_y, sigma_xy=cov_xy)

→ See RC · SIMEX · EIV Losses

Multi-Target Regression

When you need to predict multiple continuous outcomes \(\mathbf{y} \in \mathbb{R}^K\) jointly:

→ See Multi-Target Regression.

Post-Hoc Calibration

After training, calibrate your model's uncertainty estimates on a held-out set:

from torchregress.calibration import VarianceTemperatureScaler

scaler = VarianceTemperatureScaler()
scaler.fit(pred_mean_cal, pred_var_cal, y_cal)
calibrated_var = scaler.transform(pred_var_test)

→ See Post-Hoc Calibration.

Common Acronyms

Acronym Meaning
NLL Negative Log-Likelihood
CRPS Continuous Ranked Probability Score
OOD Out-of-Distribution
ECE Expected Calibration Error
MPIW Mean Prediction Interval Width
PICP Prediction Interval Coverage Probability
RMSE / MAE Root Mean Squared Error / Mean Absolute Error
MDN Mixture Density Network
SWAG Stochastic Weight Averaging — Gaussian
EIV Error-in-Variables
BNN Bayesian Neural Network

What Next?

If you've read this far top-to-bottom, you've covered the most common regression tasks. The remaining guide pages dive deeper into: