WALDO: Wald Confidence Regions¶

← Calibration | Conformal Prediction →

WALDO (Masserano et al., AISTATS 2023) constructs confidence regions for a parameter from a conditional mean and covariance estimator. It replaces the classical Wald statistic's MLE and variance with predictive moments:

The confidence set for the observed \(x\) is the Neyman inversion

where \(C(y_0)\) is a critical value estimated by quantile regression in the original WALDO simulator setting. A constant \(C\) is the split-conformal special case.

WaldoConformal¶

from torchregress.losses import WaldoConformal

cp = WaldoConformal(alpha=0.1, critical_bins=10)
cp.calibrate(mean_cal, y_cal, variance_cal)       # diagonal variances
lo, hi = cp.predict_interval(mean_test, std_test, y_min=-3, y_max=3)
inside = cp.contains(mean_test, y_test, variance_test)

• Scalar targets: intervals from Neyman inversion on a grid.
• Multi-target targets: use contains or predict_region for the joint Wald ellipsoid; the class deliberately does not manufacture per-axis intervals that would promise joint coverage they do not have.
• Critical values: critical_bins estimates a target-binned quantile of \(\tau\); pass critical_value_fn for an externally fitted quantile regression. A constant threshold (no bins) is the safest split-conformal fallback.
• Covariance type: "diagonal" or "full".

Posterior/sample mode¶

WaldoDensityConformal is the simulator-free conditional-density wrapper: pass a sampler sample_fn(context, n), calibrate on held-out targets, and use contains/predict_region/predict_interval. It estimates conditional moments from the density itself, so no simulator or parameter prior is required. sample_fn is called on row chunks of context (keyword-only chunk_size, default 2**20 // n_samples rows), so at most [chunk_size, n_samples, D] samples exist at once. With covariance_type="diagonal" the per-target sample variances are used. When the split fits in one chunk, a fixed seed gives the same draws as a single call; with several chunks, a sampler that draws one [n_samples, N, D] noise block (e.g. flow(c).sample((n,))) uses the generator in a different order, so thresholds differ by Monte Carlo error. Deterministic or per-row-seeded samplers give identical results for any chunk_size.

WaldoPosteriorCalibrator estimates the conditional mean/covariance from predictive samples (e.g. a flow):

from torchregress.losses import WaldoPosteriorCalibrator

cp = WaldoPosteriorCalibrator(alpha=0.1, covariance_type="full")
cp.calibrate(samples_cal, y_cal)          # samples_cal: [N, S, D]
inside = cp.contains(samples_test, y_test)
region = cp.predict_region(samples_test, candidate_grid)

Moments and Wald scores are computed in row chunks (chunk_size, same default as above), so no second [N, S, D] temporary is created; the samples tensor itself must still fit in memory. To stream the samples too, use WaldoDensityConformal with a sampler.

This is the WALDO "posterior estimator" mode. It is a calibration layer: a bad posterior can still produce a valid but wide region; validate empirical coverage by parameter region.

Critical-value regression without a simulator¶

The original WALDO critical-value function is fitted by quantile regression on simulator draws. When only observed calibration data are available, WaldoCriticalValueRegressor fits a local quantile curve instead:

1. reduce the calibration features (usually the candidate target) to a scalar key;
2. split the key into quantile bins;
3. estimate the \((1-\alpha)\) quantile of the Wald statistic in each bin;
4. interpolate between bin centers, falling back to the global split-conformal quantile for sparse bins.

from torchregress.losses import WaldoConformal, WaldoCriticalValueRegressor

reg = WaldoCriticalValueRegressor(alpha=0.1, n_bins=8, min_samples_per_bin=10)
cp = WaldoConformal(alpha=0.1, critical_value_regressor=reg)
cp.calibrate(mean_cal, y_cal, variance_cal)
inside = cp.contains(mean_test, y_test, variance_test)

This is a simulator-free local approximation. It can tighten regions where the Wald statistic is heteroscedastic, but it does not recover WALDO's original simulator-based conditional guarantee; validate coverage by target region.

References¶