Representation Geometry and Frozen Readouts¶

A learned encoder can rank differently under different regression readouts. The companion methods paper therefore reports three descriptive summaries on frozen features before fitting a readout:

These diagnostics answer a question that is upstream of predictive accuracy: does the representation organise itself by the target? If a \(k\)-nearest-neighbour readout on frozen features already predicts well, the encoder has preserved the target-relevant geometry; if not, no amount of head tuning will recover it. They are reported per encoder, at equal capacity and training budget, precisely because a comparison of end-to-end RMSE conflates the encoder with the readout.

These capture different geometry: neighbourhood homogeneity and distance ordering can move in opposite directions. Report both rather than a single "representation quality" score.

from torchregress.metrics import (
    frozen_readout_metrics,
    neighbour_target_gap,
    representation_geometry_report,
    target_ordering,
)

features = encoder(x)
report = representation_geometry_report(features, z, k=10)
# {'neighbour_target_gap': ..., 'target_ordering': ...,
#  'ols_RMSE': ..., 'knn_NMAD': ...}

nmad(y_pred, y_true) is the normalized median absolute deviation \(1.482602218505602\,\mathrm{median}|r|\) and is used in the readout metrics.

References¶