Flexible-Shape Distributional Families¶

← Losses Catalogue | API Reference →

Parametric NLL losses for non-Gaussian conditional distributions — skew, heavy-tailed, bounded, and extreme-value families. Each loss consumes the raw output matrix of a flexible-shape head (y_pred[..., k]) and routes the elementwise NLL through the unified BaseLoss._reduce pipeline (mask, sample weights, reduction).

All families share the same contract:

loss_fn(y_pred, target, mask=None, weights=None) -> Tensor

where y_pred columns map to distribution parameters and positivity-constrained parameters are handled internally via softplus(raw) + eps.

Catalogue¶

Functional forms are also exported (skew_normal_nll, skew_t_nll, beta_regression_nll, johnson_su_nll, sinh_arcsinh_nll, gev_nll, asymmetric_laplace_nll, sqr_loss).

Each minimises the elementwise negative log-likelihood of its density family:

for positivity-constrained parameters, with \(\theta_k = \text{raw}_k\) for unconstrained ones (\(\xi\), \(\alpha\), \(\gamma\), \(\epsilon\), \(\mu\)). The elementwise NLLs are:

• Skew-normal (Azzalini 1985), \(z = (y-\xi)/\omega\):

• Skew-t (Azzalini & Capitanio 2003), degrees of freedom \(\nu > 0\):

• Beta regression (Ferrari & Cribari-Neto 2004), for \(y \in (0,1)\) with mean \(\mu = \sigma(\text{logit})\) and precision \(\phi > 0\):

• Johnson SU, with \(u = (y - \xi)/\lambda\) and the standardized variate \(Z = \gamma + \delta\,\operatorname{asinh}(u)\):

• Sinh-arcsinh (Jones & Pewsey 2009), with \(z = (y-\mu)/\sigma\) and \(s = \sinh\big((\operatorname{asinh}(z) - \epsilon)/\delta\big)\) — the inverse of the transformation \(y = \mu + \sigma\sinh(\delta\,\operatorname{asinh}(Z)+\epsilon)\) for \(Z \sim \mathcal{N}(0,1)\):

At \(\epsilon=0,\ \delta=1\) this reduces exactly to Gaussian NLL.

• GEV (Coles 2001 parameterization), \(z = (y-\mu)/\sigma\); observations outside the support \(1 + \xi z > 0\) receive infinite NLL:

The analytic Gumbel limit is used for \(|\xi|\) below the implementation's numerical threshold, avoiding the \(0/0\) limit form.

• Asymmetric Laplace, \(u = y-\mu\):

The NLL reduces to its normalisation constant plus \(\dfrac{\sqrt{2}(1+\kappa^2)}{\sigma\kappa}\,\rho_\tau(y-\mu)\) where \(\rho_\tau\) is the pinball (check) loss at the level \(\tau = 1/(1+\kappa^2)\) — the exact \(\kappa\)–\(\tau\) correspondence, matching the QuantileLoss family.

Positivity and unconstrained_inputs¶

Several parameters must stay strictly positive (omega, phi, lambda, delta, sigma, kappa, nu). By default each family assumes unconstrained network outputs and maps them with:

positive = softplus(raw) + eps   # eps defaults to 1e-6

Set unconstrained_inputs=False when your head already enforces positivity (e.g. a softplus or exp activation). In that mode the loss avoids softplus(softplus(x)) by applying the inverse first:

# raw head — default
loss_fn = BetaRegressionNLLLoss(unconstrained_inputs=True)  # expects raw phi
loss = loss_fn(torch.randn(8, 2), targets_in_01)

# head already outputs positive phi/sigma/etc.
loss_fn = BetaRegressionNLLLoss(unconstrained_inputs=False)  # expects phi>0
loss = loss_fn(torch.cat([logits, phi_positive], dim=-1), targets_in_01)

Internally (unconstrained_inputs=False):

raw = softplus_inverse((positive - eps).clamp(min=1e-6)) = s + log(-expm1(-s))
positive_recovered = softplus(raw) + eps  # == positive

This keeps the elementwise NLL numerically identical between modes while letting you compose heads that already satisfy constraints. All seven NLL families (SkewNormalNLLLoss, SkewTLoss, BetaRegressionNLLLoss, JohnsonSUNLLLoss, SinhArcsinhNLLLoss, GEVNLLLoss, AsymmetricLaplaceNLLLoss) expose unconstrained_inputs with the same semantics; SQRLoss has no positivity constraint and does not use the flag.

Usage¶

Skew-normal / Skew-t¶

from torchregress.losses import SkewNormalNLLLoss, SkewTLoss

# Skew-normal: [xi, omega_raw, alpha]
sn = SkewNormalNLLLoss(unconstrained_inputs=True)
loss = sn(y_pred_sn, y)  # y_pred_sn: [batch, 3]

# Skew-t: [xi, omega_raw, alpha, nu_raw]
st = SkewTLoss(unconstrained_inputs=True)
loss = st(y_pred_st, y)  # y_pred_st: [batch, 4]

The skew-normal reduces exactly to the Gaussian NLL at alpha=0; the skew-t reduces to Student-t at alpha=0.

A skewness head can be initialised at alpha=0 (symmetric start) for both families: the gradient in \(\alpha\) there is non-zero. For the skew-t,

which SkewTLoss reproduces (before 0.3.0 its Student-t CDF term had an exactly zero derivative at \(\alpha = 0\), so a zero-initialised head never learned skew).

Beta regression¶

from torchregress.losses import BetaRegressionNLLLoss

loss_fn = BetaRegressionNLLLoss(unconstrained_inputs=True)
# y in (0, 1); y_pred: [logit(mu), phi_raw]
loss = loss_fn(y_pred, y)

Targets must lie strictly in (0, 1); 0/1 raise ValueError.

Johnson SU / Sinh-arcsinh¶

from torchregress.losses import JohnsonSUNLLLoss, SinhArcsinhNLLLoss

jsu = JohnsonSUNLLLoss(unconstrained_inputs=True)   # [xi, lam_raw, gamma, delta_raw]
sas = SinhArcsinhNLLLoss(unconstrained_inputs=True) # [mu, sigma_raw, eps, delta_raw]

Both families are Gaussian at the identity (gamma=0, delta scaled, or epsilon=0, delta=1) and support flexible skew/kurtosis via their shape parameters.

GEV (extreme values)¶

from torchregress.losses import GEVNLLLoss

gev = GEVNLLLoss(unconstrained_inputs=True)  # [mu, sigma_raw, xi]
loss = gev(y_pred, y)

For |xi| < 1e-6 the analytic Gumbel limit is used; observations outside 1 + xi*z > 0 receive infinite NLL.

Asymmetric Laplace / pinball correspondence¶

from torchregress.losses import AsymmetricLaplaceNLLLoss

ald = AsymmetricLaplaceNLLLoss(unconstrained_inputs=True)  # [mu, sigma_raw, kappa_raw]
# kappa=1 is symmetric Laplace; tau = 1/(1+kappa^2) gives the equivalent quantile

SQR (sorted quantile regression)¶

from torchregress.losses import SQRLoss

sqr = SQRLoss(n_levels=32)
# y_pred: [batch, L] quantile levels; sorted internally via cummax
loss = sqr(y_pred, y)

Recommendations¶

1. Start with the identity: verify that SkewNormal(alpha=0) matches Gaussian NLL and SinhArcsinh(epsilon=0, delta=1) matches Gaussian NLL on your data pipeline before adding skew/tail learning.
2. Bounded targets → Beta: if y ∈ (0, 1), prefer BetaRegressionNLLLoss over transforming y to ℝ; it preserves the (0, 1) support and calibrates near the boundaries.
3. Extremes → GEV: use only for block-maxima or threshold-exceedance targets; validate support (1 + xi*z > 0) and monitor for infinite NLLs that indicate out-of-support predictions.
4. Check positivity wiring: if you set unconstrained_inputs=False, ensure every positivity-constrained column is already > eps. Passing raw logits with False double-constrains and biases gradients.

References¶