Loss-library test coverage map¶

This doc cross-references every public symbol exported from torchregress.losses with the dedicated test class that locks its behavior and the discriminator invariant the test guards.

When you change a loss internals formula, factory, or protocol, the table below tells you which tests/losses/test_*.py file (and which Test* class inside it) is the right entry point. The invariants listed are the mathematical or structural promises the test enforces — failing any of them is the canonical way a contract break shows up in CI.

Conventions¶

• DEDICATED — the symbol has a Test<S> class (or top-level test_* function) that locks in its contract.
• DIRECT-LIGHT — covered indirectly via the per-symbol regression suite (test_loss_fixes.py, test_sls_internals.py, test_eiv_internals.py, test_functional_wrappers.py, test_indirect_utilities.py), each one introduced to fill gaps in the per-class suites.
• INDIRECT — exercised only by other tests' incidental usage. Marked so refactors catch them on the next review pass.

Master matrix¶

Note: Test* class names are best-effort. If a refactor renames or splits a class, run grep '^class Test' tests/losses/<file>.py for the current names rather than trusting this table verbatim. The test-file column is more stable -- use that as the canonical entry point.

Supplementary symbol coverage¶

The matrix above covers the core loss surface; these additional public symbols are exercised by the tests listed here. Status follows the same conventions: DEDICATED when a Test* class targets the symbol, DIRECT-LIGHT when a focused test function exists in the per-symbol suites, and INDIRECT otherwise (a refactor should add a direct test).

Internal helpers in utils/gaussian_output.py¶

The three helpers below are not in __all__ — they remain internal contract helpers outside the public losses API. They are exercised indirectly by their BetaNLLLoss/GaussianNLLLoss/FaithfulGaussianLoss callers but have no dedicated test class. Treat any change as a cross-module refactor that should land alongside a direct test in tests/losses/test_indirect_utilities.py.

• parse_heteroscedastic_output — tuple/dict/concat-2D routing
• split_mean_log_variance — tuple/dict/concat routing; ValueError on odd dims
• variance_from_logvar — clamp + exp signing

Per-class test surface (consistency / cross-family)¶

Discriminator invariants: where to look when refactoring¶

INDIRECT-only symbols (refactor risk)¶

The authoritative list is the Coverage column of the supplementary symbol coverage table: symbols marked DIRECT-LIGHT or INDIRECT have no dedicated Test* class. The zuko-dependent flow factories (create_flow_model, create_flow_loss, create_contrastive_flow_loss) remain the canonical example: optional dependency, covered indirectly.

Any non-trivial change to these symbols should land alongside a small direct test in tests/losses/test_indirect_utilities.py (or specific file) so their behaviour doesn't drift silently.

Per-class test surfaces¶

For matrix rows citing (per-class suite), the dedicated Test* classes live in the file under Test file. Use grep '^class Test' <test_file> to list the exact class names — the names change frequently and the matrix does not attempt to enumerate them. Cross-consistency coverage is documented per file in test_gaussian_consistency.py and test_non_gaussian_consistency.py.

Known silent regressions¶

The following invariants are documented. Pre-fix code raises the listed error under older PyTorch but is silently absorbed under modern PyTorch's version-counter autograd; the listed regression test now discriminates both states at runtime, so the original caveat no longer applies:

• MixtureDensityLoss H2 in-place scatter. Post-fix code composes the full-cov Cholesky factor functionally via torch.where(diag_mask, L_diag, L_offdiag), yielding a brand-new L_matrices tensor with _version == 0 at forward-return time. Pre-fix code derived L_matrices via two in-place __setitem__ writes onto L_offdiag, bumping its version to >= 2 by the time it was returned.The discriminator tests/losses/test_loss_fixes.py::TestMixtureDensityLossFullCovBackward::test_full_covariance_L_version_zero wraps _extract_distribution_parameters to capture L_matrices._version immediately after forward and asserts it equals 0, surviving PyTorch's silent absorption of the in-place pattern.

Coverage invariants: what test authors MUST pin¶

Future test authors must follow this discipline when introducing any ad-hoc reference to a covariance, sigma, stddev, or Cholesky factor in a test fixture. Ad-hoc torch.eye(...) / torch.diag(...) / torch.diag_embed(...) literals MUST pin device and dtype so the fixture does not silently rely on the loss module handling dtype/device of input fixtures internally. Two accepted forms depending on whether the literal is on the result side or the input side of the comparison:

• Result comparison (the literal is the reference for a tensor returned by a loss module). Pin the literal to the result's metadata:

  ref = torch.eye(n_features, device=cov.device, dtype=cov.dtype) * 0.25
  self.assertTrue(torch.allclose(cov, ref))
  

• Input builder (the literal is part of a fixture fed into a loss module — e.g. A @ A.T + torch.eye(dim) * jitter). Pin to the co-built anchor tensor that's already on the right device/dtype:

  A = torch.randn(dim, dim)
  base = A @ A.T + torch.eye(dim, device=A.device, dtype=A.dtype) * 1e-3
  

The legacy .to(cov.device, dtype=cov.dtype) cast on the literal is accepted as a fallback when the torch.eye(...) / torch.diag(...) form would change the literal's mathematical meaning (e.g. when the literal is a hand-built third-party reference like torch.tensor([...]) the test materials outside an obvious anchor).

Why this discipline matters¶

torch.allclose and torch.testing.assert_close are dtype/device-tied at the framework level. torch.eye(3) silently defaults to float32/CPU; if a future loss module grows a device-dispatching code path (e.g. CUDA NLL, autograd-double NLL, the to("cuda") migration in test_gaussian.py's setUp) the test would silently pass under old builds while passing against mismatched metadata under new builds — a covered-by-follow-up symptom of implicit fixture metadata rather than a single-step failure flagged by CI.

Concretely: under a dtype/device mismatch, torch.allclose raises a RuntimeError rather than returning the comparison result. CI fails with an unrelated-looking error message (e.g. Found dtype Float but expected Double), masking the real loss-graph regression behind a metadata mismatch — the developer fixes the missing dtype pin rather than the actual loss bug the test was guarding.

Canonical examples for reference while authoring new tests¶

• tests/losses/test_eiv_internals.py::TestBaseEIVLossInternals::test_prepare_covariance_* — the canonical result-comparison form. Pin the reference to cov after _prepare_covariance_from_sigma: torch.eye(n_features, device=cov.device, dtype=cov.dtype) * 0.25.
• tests/losses/test_functional_wrappers.py::TestGaussianWassersteinBoundWrapper.setUp — the canonical input-builder form. Pin the SPD builder's eye to the co-built A: A @ A.T + torch.eye(self.d, device=A.device, dtype=A.dtype).
• All _make_spd_cov-style helpers in tests/losses/test_gaussian_consistency.py and the _psd_matrix helper in tests/losses/test_indirect_utilities.py reproduce the input-builder form on the helper's anchor A.
• tests/losses/test_gaussian.py::TestGaussianLosses.setUp — adds a dtype= pin on top of an existing device= pin for the SPD jitter term.

Cross-cutting rule for code review¶

When reviewing or extending any test in tests/losses/, tests/, notebooks/, or examples/, re-grep torch.eye and torch.diag / torch.diag_embed. Any unpinned site that - constructs an input fixture (an A @ A.T + eye(dim) * jitter builder, an eye-in-a-list literal passed as a sigma parameter, etc.), or - compares against a result tensor in torch.allclose / torch.testing.assert_close,

is a coverage-invariance violation under this rule. Fix before merging, exactly per the patterns above.

Mechanical enforcement (TOR001 + TOR002 + TOR003)¶

Pin-rollout drift doctor¶

A second local hook, scripts/check_pin_rollout_drift.py, complements the enforcer above by auditing every# noqa: TORxxxopt-out in thetests/,examples/,notebooks/, andsrc/ trees. Three severity classes are reported:

• [error] — the noqa is orphaned (no torch.eye / torch.diag / torch.diag_embed call on that line). The opt-out is attached to nothing; remove it.
• [warning] — the noqa is redundant: the call on that line already pins device= / dtype= (or chains .to(device=..., dtype=...)), so the noqa silences nothing. This typically follows a successful pin rollout -- remove the now-defunct opt-out to keep the source clean.
• [hint] -- advisory only (does NOT cause a non-zero exit): the cited rule ID does not match the file's path-resolved rule, or the rule ID is not in _RULE_PREFIXES (typo). The enforcer ignores cross-rule noqa already; the hint surfaces the stale scope assumption so a contributor can verify which rule the file actually lives in.

.pre-commit-config.yaml registers the doctor as a separate repos: local hook entry (pin-rollout-drift-doctor) so its diagnostics are distinct from the enforcer's TORxx output in pre-commit logs.

Mechanical enforcement (TOR001 + TOR002 + TOR003)¶

A local pre-commit hook at scripts/check_test_fixture_pin_discipline.py AST-scans every staged tests/**.py / examples/**.py / notebooks/**.py / src/**.py file for un-pinned torch.eye(...) / torch.diag(...) / torch.diag_embed(...) literals. The rule ID is resolved per file path via POSIX path-component matching (_rule_id_for_path in the script), so a file like examples/tests/helper.py lands under TOR002 (the examples directory component is the first match), not TOR001:

• TOR001 — files under tests/ (the original tests-only scope).
• TOR002 — files under examples/ or notebooks/.
• TOR003 — files under src/ (production loss fixtures plus the algorithms/, metrics/, ensemble/, utils/, losses/ helpers that emit the same fixtures internally; .pre-commit-config.yaml registers a third hook entry with files: '^src/.*\.py\$').

# noqa precedence contract: a # noqa: TOR00x comment on a line silences only the matching rule ID — # noqa: TOR001 does NOT silence a violation routed to TOR002 or TOR003. The order in _RULE_PREFIXES matters: the script iterates path components in order and returns the first component whose name matches (so a relative path examples/tests/helper.py lands under TOR002, not via nested tests substring). An absolute path with a pre-project-root src/ segment (/home/user/src/torchregress/tests/foo.py) WILL route to TOR003 because src is matched first by the ABS path-component walk; callers should normalize absolute paths to project-relative before relying on the rule ID.

.pre-commit-config.yaml registers three repos: local: hook entries, each with its own rule ID, so pre-commit output surfaces which scope each violation belongs to. The script emits filename:lineno: torch.<func>() missing device= or dtype= kwarg violations to stderr and exits with status 1; the message body embeds the resolved rule ID so # noqa opt-outs must match (# noqa: TOR001, # noqa: TOR002, or # noqa: TOR003).

Recognised canonical pin patterns: - Direct kwarg (works for torch.eye, which natively supports the kwargs): torch.eye(N, device=A.device, dtype=A.dtype) - Chained .to(...) cast (the documented fallback for torch.diag and torch.diag_embed, which do NOT accept those kwargs natively): torch.diag(matrix).to(device=matrix.device, dtype=matrix.dtype)

Inline opt-out: a trailing # noqa: TOR001 (under tests/) or # noqa: TOR002 (under examples/ or notebooks/) comment on the offending line. Signature scope is intentionally narrow — only direct torch.<func>(...) attribute access is scanned. Aliases (t.eye(...)), imports (from torch import eye; eye(...)), and **kwargs splats are flagged as unverifiable; opt-out with the appropriate # noqa: TORxxx for the splat case if it provably carries the kwargs.

Why ast rather than a custom ruff plugin: ruff is a Rust CLI and is not importable as a Python module for plugin code; AST-based analysis covers the rule completely without spawning an extra Rust tool. Unit-tested in tests/utils/test_check_test_fixture_pin_discipline.py (24 cases covering direct-kwarg pin, chained .to(), scope filters, walk depth, and rule-ID stability).

TL;DR for code review¶

When reviewing a loss change, look for:

1. The matrix row for the changed symbol — confirm the test class actually enforces the changed invariant (not just instantiates the loss).
2. If you can't find a row, add a dedicated test before merging; the INDIRECT-only section flags where the gap lives.
3. Re-running torchregress/scripts/revert_verify.py after a change gives empirical evidence the test discriminates the bug class the change risks.