Regression Calibration (RC)¶

← SIMEX | LatentNN →

Regression Calibration is a classical method for correcting measurement error in inputs (errors-in-variables). When the observed input \(W\) is a noisy version of the true input \(X\), naive regression on \(W\) produces attenuated (biased-toward-zero) estimates.

The Attenuation Problem¶

When inputs have additive Gaussian measurement error:

the naive OLS estimate of \(\beta\) is biased:

Every entry of the reliability matrix \(\Lambda\) satisfies \(0 \leq \Lambda_{ij} \leq 1\), so coefficients are shrunk toward zero.

The RC Correction¶

RC estimates calibrated inputs via the conditional expectation:

where:

Usage¶

import torch
from torchregress.algorithms import RegressionCalibration

# Known measurement error (scalar, vector, or covariance matrix)
rc = RegressionCalibration(sigma_u=0.5)

# Fit on observed noisy inputs, then transform
X_calibrated = rc.fit_transform(X_observed)

# Or separate fit / transform steps for new data
rc.fit(X_observed)
X_cal_new = rc.transform(X_new)

rc.posterior(X, sigma_u=...) accepts the same formats (plus per-sample (N, D) standard deviations) and uses the given sigma_u instead of the one stored at fit.

When to Use¶

Limitations¶

1. Linear correction only: RC applies a linear shrinkage \(\Lambda \cdot (W - \bar{W})\). For nonlinear \(f(X)\), the debiasing is approximate at best. Use SIMEX or EIV losses for nonlinear models.
2. PSD clamping: If measurement noise exceeds the observed signal (\(\Sigma_u > \Sigma_w\)), the estimated signal covariance \(\hat\Sigma_x = \Sigma_w - \Sigma_u\) becomes negative. RC clamps to a small positive value but cannot recover when noise dominates signal.
3. Homoscedastic noise only: RC assumes a single \(\Sigma_u\) for all samples. Heteroscedastic measurement error is not supported.
4. Gaussian noise assumption: The conditional expectation \(\mathbb{E}[X \mid W]\) is exact only under joint Gaussian \((X, W)\). For non-Gaussian noise, RC is a best-linear predictor, not the true conditional expectation.
5. Known \(\Sigma_u\) required: RC requires the measurement error covariance as a known input. It does not estimate \(\Sigma_u\) from data.

Recommendations¶

• Default for linear models: RC is the fastest, simplest EIV correction. Apply it first, then validate on held-out data before considering more complex methods.
• For nonlinear models: If RC's linear correction is insufficient (validate on test set), upgrade to SIMEX (flexible but expensive) or LatentNN (joint optimisation).
• Validate the correction: Compare RC-corrected predictions against a naive baseline on clean held-out data (if available) or via simulation with known ground truth.
• Combine with robust losses: RC corrects input noise. Pair with Robust losses to handle outlier targets simultaneously.

Next steps¶

• SIMEX — simulation-extrapolation for nonlinear models where RC's linear correction is insufficient
• LatentNN — joint optimisation of latent clean inputs and model parameters
• EIV losses — functional and structural error-in-variables losses at the loss level
• Error-Aware Encoding — quality-gated feature engineering from known measurement noise

References¶