Uncertainty Decomposition¶

← Choosing by Constraint | Guide Overview →

Uncertainty language is easy to overstate. The vocabulary used in torchregress follows a strict four-contract taxonomy, with each contract backed by a different mathematical construction. This page formalises the taxonomy, states the law that underpins the variance decomposition, and gives method-by-method semantics so that you can audit which contract a given API actually delivers.

The guiding principle: the answer to "what kind of uncertainty does this method give me?" determines which metric, which diagnostic, and which downstream decision is appropriate. Conflating the contracts is the most common source of miscommunicated uncertainty claims.

1. The four contracts¶

The contracts are distinct and not interchangeable. A method that satisfies the predictive spread contract does not satisfy the epistemic signal contract. A method that satisfies the coverage guarantee does not satisfy the decomposition contract.

2. The Law of Total Variance (the identity)¶

For a set of \(M\) probabilistic predictors, each producing a conditional distribution \(p_m(y \mid x) = \mathcal{N}(\mu_m(x), \sigma_m^2(x))\) (the identity extends to non-Gaussian members with the appropriate variance definitions):

with \(\bar\mu(x) = \frac{1}{M} \sum_m \mu_m(x)\).

This is a direct application of the Law of Total Variance to the mixture distribution \(p(y \mid x) = \frac{1}{M} \sum_m p_m(y \mid x)\):

The first term is the expected per-model noise (aleatoric); the second is the variance of the per-model means (epistemic).

In torchregress, compute the decomposition with uncertainty_decomposition(means, variances) (where means is [M, B] and variances is [M, B]) or with ensemble_variance_decomposition(...) for richer inputs (e.g. multivariate).

2.1 The multivariate identity¶

For multi-target heads, the Law of Total Covariance gives

with \(\bar{\boldsymbol\mu} = \frac{1}{M} \sum_m \boldsymbol\mu_m\). The first term captures the expected per-member noise; the second captures the disagreement of the per-member means.

3. The taxonomy: method-by-method semantics¶

The table below is the canonical reference for which uncertainty contract each method delivers. Read it column-by-column: the "Epistemic" column says whether the method models model ignorance via disagreement or sampling; the "Aleatoric / spread" column says whether the method models data noise or only the predictive band; the "Decomposition status" column says whether the API returns a split into the two components.

For constructor signatures and scoring-rule definitions, see the Losses API, Metrics API, and Ensemble API.

The "no" entries are not failures — they are scope statements. A single-model GaussianNLLLoss is a perfectly correct aleatoric model; it just doesn't claim anything about model ignorance.

4. Quantile ensembles (a special case)¶

Yes, you can deep-ensemble quantile regressors. An ensemble of quantile heads gives useful epistemic information through disagreement among the predicted quantile functions. For example, if independently trained models disagree strongly about the \(0.9\) quantile, that is a model-uncertainty signal.

But this is not a clean variance decomposition. Quantile regression predicts conditional quantiles, not per-member Gaussian variances. You can:

• Summarise ensemble disagreement across quantiles (epistemic signal, well-defined).
• Estimate interval width from the quantile band (predictive spread, well-defined).
• Not call those two numbers epistemic_variance and aleatoric_variance without extra modelling choices.

Use this framing:

• Single quantile model: calibrated / non-Gaussian intervals, no epistemic split.
• Quantile ensemble: epistemic-style disagreement across quantile functions.
• Quantile ensemble + conformal calibration: stronger interval coverage story.
• Quantile ensemble + explicit distributional / variance model: only then consider formal variance decomposition.

5. What "epistemic" actually means¶

Three distinct mathematical objects share the epistemic label, and the choice of object affects downstream decisions:

1. Weight-space variance. The posterior over weights \(p(\boldsymbol\theta \mid \mathcal{D})\) induces a distribution over functions. Methods: BayesianNeuralNetwork, IVON, HeteroscedasticLaplaceRegressor. Cheapest in inference (one forward pass per weight sample); requires a weight-perturbation sampling step.
2. Function-space disagreement. A finite sample of models \(\{f_m\}_{m=1}^M\) produces a sample of predictions. The empirical variance of the sample is the epistemic estimate. Methods: BaseEnsembleModel, MCDropoutWrapper (with \(N\) dropout masks as the sample), SWAG (with \(N\) weight samples).
3. Single-pass prior. A prior over the predictive distribution parameters (e.g. Normal-Inverse-Gamma) is updated to a posterior in one pass. Methods: EvidentialRegressionLoss. The "epistemic" component is the variance of the posterior over the likelihood parameters, not over the weights or the predictions.

These three objects are not interchangeable. A weight-space variance is small when the posterior over weights is concentrated; a function-space disagreement is small when the sampled models agree on the prediction; a single-pass prior's epistemic term is small when the posterior over likelihood parameters is concentrated.

In all three cases, the epistemic term should vanish as the training set size grows — but it does so at different rates, and the calibration diagnostics should be run on each separately.

6. The catalog contract¶

When updating method_catalog.py, use conservative capability labels:

• yes: the component is explicitly modelled and returned, or directly computed by a tested helper.
• partial: the component exists only for a specific head, sampling mode, ensemble construction, or modelling assumption.
• no: the method may produce intervals or spread, but not that uncertainty component.

Avoid listing single-model MDN, flow, or quantile losses as epistemic + aleatoric decomposition methods. List their ensemble variants when the intended claim is model-disagreement plus predictive spread.

The method_catalog API exposes list_methods(capability_filters=...) and a per-row task_tags field. Use these for capability-flag-based shortlisting:

import torchregress as tr

# Methods that explicitly deliver the full variance decomposition
rows = tr.method_catalog.list_methods(
    capability_filters={"decomposition": "yes"},
    maturity=("Core", "Strong", "Available"),
)
for row in rows:
    print(row["name"], row["family"], row["maturity"], row["task_tags"])

7. Diagnostics by contract¶

A single diagnostic is not enough to validate any contract. Combine at least one primary and one secondary diagnostic from above.

8. Decision rules¶

References¶

Next steps¶

• Method Selection Matrix — task-first capability matrix.
• Choosing by Constraint — latency / coverage / decomposition tradeoffs.
• Multi-Target Regression — joint multi-target modelling and decomposition.
• Practical Usage — concrete loss recipes.