Uncertain Ground Truth¶

← Noisy Labels | Transform Losses →

These losses target settings where labels are noisy, partially trusted, weakly supervised, or augmented with pseudo labels.

Available losses:

• NoisyTargetGaussianNLL
• ConsistencyRegLoss
• PseudoLabelNLL
• PseudoLabelConsistencyLoss

Related utilities:

• torchregress.utils.generate_pseudo_labels
• torchregress.utils.update_ema_teacher_

When To Use Which¶

Objectives¶

Noise-aware Gaussian NLL¶

If the observed target has known variance \(\sigma_y^2\) and the model predicts \((\mu, \sigma_{pred}^2)\):

Consistency regularization¶

Semi-supervised composite loss¶

PseudoLabelConsistencyLoss is intentionally narrow: it is the loss for the common student-teacher semi-supervised regression pattern where you have:

1. a labeled subset with real targets,
2. an unlabeled subset with pseudo targets,
3. an optional teacher prediction used for consistency regularization.

It combines:

with label_mask controlling which samples contribute to the supervised term and pseudo_confidence controlling how strongly unlabeled pseudo labels contribute.

API Examples¶

Known target uncertainty¶

import torch
from torchregress.losses import NoisyTargetGaussianNLL

loss_fn = NoisyTargetGaussianNLL()

mean = torch.randn(64, 1)
log_var = torch.randn(64, 1)
y_obs = torch.randn(64, 1)
y_obs_var = torch.rand(64, 1) * 0.05

loss = loss_fn((mean, log_var), y_obs, target_variance=y_obs_var)

Pseudo-label Gaussian training¶

from torchregress.losses import PseudoLabelNLL

loss_fn = PseudoLabelNLL(pseudo_weight=0.8)
loss = loss_fn(
    (mean, log_var),
    target,
    pseudo_target=pseudo_target,
    pseudo_confidence=pseudo_confidence,
    label_mask=label_mask,
)

Point-regression semi-supervision¶

from torchregress.losses import PseudoLabelConsistencyLoss

loss_fn = PseudoLabelConsistencyLoss(
    pseudo_weight=0.8,
    consistency_weight=0.25,
    confidence_threshold=0.35,
)
loss = loss_fn(
    student_pred,
    target_with_placeholders,
    pseudo_target=pseudo_target,
    pseudo_confidence=pseudo_confidence,
    teacher_pred=teacher_pred,
    label_mask=label_mask,
)

Unlabelled entries of target_with_placeholders may hold any placeholder, including NaN: entries outside label_mask are replaced (via torch.where) before the supervised term is evaluated, so they affect neither the loss nor its gradient. The same holds for PseudoLabelNLL.

Pseudo-label generation and EMA teacher update¶

from torchregress.utils import generate_pseudo_labels, update_ema_teacher_

pseudo_target, pseudo_confidence, accepted = generate_pseudo_labels(
    teacher_mean,
    log_variance=teacher_log_var,
    confidence_threshold=0.35,
)
update_ema_teacher_(ema_teacher, student, momentum=0.95)

Selection Bias / Propensity Support¶

NoisyTargetGaussianNLL, PseudoLabelNLL, and PseudoLabelConsistencyLoss support optional propensity_scores or propensity_weights so weak supervision can be combined with covariate-dependent label-observation correction.

Comparison Examples¶

• Semi-Supervised Regression Comparison
• Uncertain-GT + Density Conformal Comparison (synthetic and --dataset diabetes)

Companion metrics (torchregress.metrics)¶

Evaluation for the same workflow lives in torchregress.metrics.uncertain (→ See Distribution metrics):

from torchregress.metrics import uncertain_gt_metrics_report

report = uncertain_gt_metrics_report(
    pred_mean=mu,
    pred_variance=var,
    target=y_noisy,
    target_variance=y_var,
    teacher_pred=teacher_mu,      # optional
    pseudo_confidence=conf,       # optional
)  # {"NoisyTargetNLL": …, "ConsistencyMSE": …, "PseudoAcceptanceRate": …}

Limitations¶

1. Confirmation bias in pseudo-labeling: Pseudo-labeling is prone to confirmation bias — the model repeatedly fits its own incorrect but confident predictions, amplifying errors over epochs. Use an EMA teacher (update_ema_teacher_), apply threshold gating on pseudo_confidence via generate_pseudo_labels, and never include pseudo-labeled samples in held-out validation/test splits.
2. Hyperparameter sensitivity: confidence_threshold gates which pseudo-labels are trusted — too low admits noise, too high rejects most unlabeled data. Tune on a small held-out labeled set, not the unlabeled data.
3. Target variance quality (NoisyTargetGaussianNLL): If target_variance is poorly estimated (e.g., guessed rather than measured), performance can be worse than standard GaussianNLLLoss. Only use when variance estimates are trusted more than the model's ability to learn them.
4. PseudoLabelConsistencyLoss is intentionally narrow: It covers one common pattern (labeled subset + pseudo labels + teacher consistency). For richer semi-supervised workflows with multiple views, augmentation, or density-based weighting, use TeacherStudentTrainer from Semi-supervised API.

Practical Notes¶

Next steps¶

• Noisy labels — when noise variance is unknown
• Semi-supervised comparison — benchmark all approaches
• Gaussian losses — standard heteroscedastic regression
• Conformal + Uncertain GT — density-based conformal on noisy labels

References¶