Evidential Regression¶

← Poisson-Gaussian | Mixture Density Networks →

Evidential regression provides aleatoric + epistemic uncertainty from a single forward pass, without ensembles or Monte Carlo sampling. It does this by placing a higher-order prior (Normal-Inverse-Gamma) over the Gaussian likelihood parameters.

See the EvidentialRegressionLoss API for constructor parameters and the NIG loss definition. For ensemble-based decomposition, compare with uncertainty_decomposition.

Mathematical Background¶

The Normal-Inverse-Gamma Prior¶

The model places a NIG prior on the mean and variance of the observation:

which factors as:

Uncertainty Decomposition¶

Loss Function¶

The total loss combines NIG negative log-likelihood with a regulariser that penalises overconfident wrong predictions:

with the NIG marginal predictive (Amini et al. 2020, Eq. 8)

so the predictive is a Student-\(t\) with \(2\alpha\) degrees of freedom, location \(\gamma\) and squared scale \(\Omega/(2\alpha\nu)\) — the quantity predict_interval inverts. Using \(2\beta\) in place of \(\Omega\) gives the Student-\(t\) of the mean \(\mu\), not the predictive of \(y\), and under-covers; 0.3.0 fixed that. Matches EvidentialRegressionLoss with regularisation coefficient coeff_nig.

Usage¶

The model must output 4 values per target feature: \([\gamma, \nu, \alpha, \beta]\).

import torch
import torch.nn as nn
import torch.nn.functional as F
from torchregress.losses import EvidentialRegressionLoss

class EvidentialModel(nn.Module):
    def __init__(self, in_dim, out_dim=1):
        super().__init__()
        self.net = nn.Sequential(
            nn.Linear(in_dim, 64), nn.ReLU(),
            nn.Linear(64, 64), nn.ReLU(),
            nn.Linear(64, 4 * out_dim),  # γ, ν, α, β per target
        )

    def forward(self, x):
        out = self.net(x)
        # Enforce constraints: ν > 0, α > 1, β > 0
        gamma = out[..., 0:1]
        nu    = F.softplus(out[..., 1:2]) + 1e-2
        alpha = F.softplus(out[..., 2:3]) + 1.01
        beta  = F.softplus(out[..., 3:4]) + 1e-2
        return torch.cat([gamma, nu, alpha, beta], dim=-1)

model = EvidentialModel(in_dim=10)
loss_fn = EvidentialRegressionLoss(coeff_nig=0.01)

Inference Methods¶

predict_with_uncertainty¶

Decompose predictions into mean + aleatoric + epistemic:

model.eval()
with torch.no_grad():
    params = model(x_test)
    mean, aleatoric, epistemic = loss_fn.predict_with_uncertainty(params)

    total_unc = aleatoric + epistemic
    lower = mean - 1.96 * torch.sqrt(total_unc)
    upper = mean + 1.96 * torch.sqrt(total_unc)

predict_interval¶

Get proper prediction intervals using the Student-t predictive distribution:

lower, upper = loss_fn.predict_interval(params, confidence=0.95)

sample_predictions¶

Sample from the NIG prior for Monte Carlo analysis:

samples = loss_fn.sample_predictions(params, n_samples=100)
# samples shape: [n_samples, batch, n_features]

When to Use¶

Recommendations¶

1. Start with coeff_nig=0.01 and tune in log-space over {0.001, 0.01, 0.1, 1.0} on a validation set. Monitor epistemic uncertainty — if it collapses below 1% of aleatoric, increase \(\lambda\); if the model underfits (train loss plateaus high), decrease \(\lambda\).
2. Verify \(\alpha > 1\) consistently: \(\alpha\) (the NIG shape) must stay above 1 for finite variance. Enforce with F.softplus(...) + 1.01. If \(\alpha\) drifts near 1 during training, the Student-t predictive distribution has infinite variance — increase coeff_nig to penalise overconfident wrong predictions harder.
3. Use predict_interval over predict_interval_gaussian: The true predictive is Student-t with \(2\alpha\) degrees of freedom. The Gaussian approximation is only reliable when \(\alpha \gg 1\) (typically \(\alpha > 5\)). For safety, default to the exact Student-t method.
4. Batch size \(\geq 32\): The per-sample NIG regulariser is noisy at small batch sizes. Below 16, gradient variance destabilises the \(\nu\) and \(\alpha\) parameters. Use gradient accumulation if memory-constrained.
5. Monitor OOD calibration, not just in-distribution error: Evidential models can achieve low in-distribution RMSE while producing severely overconfident epistemic estimates on shifted data. Validate on covariate-shifted or worst-slice hold-out sets.
6. Compare with ensembles for critical applications: Evidential single-pass uncertainty is fast but less reliable than BaseEnsembleModel disagreement for epistemic decomposition. When safety-critical, use both and flag disagreements.

Next steps¶

• Ensemble methods — more reliable (but costlier) epistemic uncertainty
• Gaussian losses — standard heteroscedastic regression (aleatoric only)
• Uncertainty decomposition — taxonomy of aleatoric vs. epistemic contracts
• Evidential example — runnable comparison with ensembles

References¶