Base Loss Classes¶

← Loss Functions | Gaussian →

The foundation of all torchregress loss functions — providing unified masking, sample weighting, and consistent reduction semantics.

Every loss in the library is a scalar functional of a forecast and a target. The two families differ only in what the forecast represents:

RegressionLoss covers the first form; DistributionLoss covers the second, where the model's raw output is parsed into distribution parameters (\(\mu\), \(\log\sigma^2\), mixture weights, quantile levels, …). Both inherit the same mask/weight/reduction pipeline, so switching from a point loss to a probabilistic one never changes how missing data or sample weights are handled.

Class Hierarchy¶

graph TD
    A["BaseLoss (nn.Module)"] --> B["RegressionLoss"]
    A --> C["DistributionLoss"]
    D["WeightedLossWrapper"] -. "wraps any nn.Module" .-> E["nn.Module (any PyTorch loss)"]
    B --> F["WeightedHuberLoss, WeightedMSELoss, WeightedL1Loss"]
    C --> G["GaussianNLLLoss, MixtureDensityLoss, ..."]

BaseLoss¶

Root class extending nn.Module (see BaseLoss API). All subclasses inherit:

• reduction: "mean" (default), "sum", or "none"
• _reduce(loss, mask, weights): applies reduction with masking and weighting

All reductions follow one unified zero-fill policy. Let \(\ell_i\) be the per-sample (elementwise) loss, \(m_i \in \{0,1\}\) the validity mask, and \(w_i \ge 0\) the sample weights. With a combined influence factor \(c_i = m_i\,w_i\) an element influences the loss if and only if \(c_i > 0\); masked-out positions contribute exactly zero in every mode:

so 'mean' is the weighted mean over valid samples (dividing by the number of unmasked elements when no weights are given, by the sum of weights otherwise) — never divided by the full batch size. A mask with more dims than the loss collapses with all over the trailing dims (a partially masked row drops); weights broadcast and average over trailing dims.

class CustomLoss(BaseLoss):
    def forward(self, y_pred, y_true, mask=None, weights=None, **kwargs):
        loss = (y_pred - y_true) ** 2
        return self._reduce(loss, mask, weights)

RegressionLoss¶

For losses that operate on point predictions \(\hat{y}\) (see RegressionLoss API):

from torchregress.losses import WeightedMSELoss, WeightedL1Loss, WeightedHuberLoss

loss_fn = WeightedMSELoss()
loss = loss_fn(y_pred, y_true, mask=valid_mask, weights=sample_weights)

All RegressionLoss subclasses accept:

DistributionLoss¶

For losses that model full probability distributions (see DistributionLoss API):

from torchregress.losses import GaussianNLLLoss

loss_fn = GaussianNLLLoss()

# Model outputs (mean, log_var) — either as tuple or concatenated
mean = torch.randn(64, 1)
log_var = torch.randn(64, 1)
loss = loss_fn((mean, log_var), y_true)

For the full API contract, see GaussianNLLLoss API.

Key methods:

WeightedLossWrapper¶

Adapts any PyTorch loss to the torchregress interface (mask + weights support):

from torchregress.losses import WeightedLossWrapper
import torch.nn as nn

# Wrap standard PyTorch loss
wrapped = WeightedLossWrapper(nn.MSELoss)
loss = wrapped(y_pred, y_true, mask=mask, weights=weights)

Pre-Defined Weighted Losses¶

Implementing Custom Losses¶

import torch
from torchregress.losses.base import RegressionLoss

class AsymmetricLoss(RegressionLoss):
    """Penalise under-prediction more than over-prediction."""

    def __init__(self, alpha=0.7, reduction="mean"):
        super().__init__(reduction=reduction)
        self.alpha = alpha

    def forward(self, y_pred, y_true, mask=None, weights=None, **kwargs):
        error = y_true - y_pred
        loss = torch.where(
            error > 0,
            self.alpha * error ** 2,        # under-prediction
            (1 - self.alpha) * error ** 2,   # over-prediction
        )
        return self._reduce(loss, mask, weights)

Limitations¶

1. Weight semantics: The weights parameter is NOT the same as mask. mask excludes samples entirely (binary). weights scales the contribution of valid samples. Both can be used together but have different semantics: mask=False means "this sample should not contribute to the loss at all"; weight=0.0 on a masked-in sample means "this is a valid sample but contributes nothing."
2. Reduction behaviour: reduction="none" returns per-sample losses before masking. Apply mask and weights manually if you need per-sample values with masking.
3. Custom loss contract: If you implement a custom loss inheriting from RegressionLoss or DistributionLoss, you must call self._reduce() or self._reduce_with_mask() — manual reduction will bypass the mask and weights contract.

Recommendations¶

• Start from RegressionLoss for point-prediction losses (MSE, MAE, Huber variants). Start from DistributionLoss for NLL-based losses.
• Always call self._reduce(): Do not manually compute loss.mean() — use the base class reduction to ensure mask and weight support.
• Accept **kwargs in forward() for forward-compatibility with the loss registry and factory functions.
• Document the optimisation objective: Include the mathematical formula (LaTeX) in the docstring. See existing losses for examples.

References¶

Next steps¶

• Loss Functions index — every loss family with formulas and use cases
• Losses API reference — complete symbol table with signatures
• Method Selection Matrix — task-first guidance