
LeJEPA replaces the heuristic machinery of modern self-supervised learning (asymmetric encoders, stop-gradients, momentum teachers, schedulers) with a single isotropic-Gaussian regulariser, SIGReg. Carrying SIGReg to sequence models exposes a documented hazard: a naive pooled application can drive every sequence to a constant vector along time, a time-axis collapse the objective fails to penalise. We build a heuristics-free time-series representation-learning library and compare three placements of SIGReg relative to the time axis (pooled, dual, structured). The collapse is real and placement-controlled: dual raises across-time variance by roughly an order of magnitude. Our central result is negative and clarifying: through a factorial crossing encoder architecture with placement and probe feature, the across-time variance collapse does not measure downstream order availability and is anticorrelated with it (Spearman rho = -0.64). What determines temporal-order retention is the encoder's positional structure, an axis orthogonal to the placement that controls the collapse: a position-free encoder gives a permutation-invariant pooled feature at exactly chance (0.500) on a content-matched probe over two datasets, while a positional transformer recovers order at 0.97-1.00 even when maximally collapsed. The dual placement is nonetheless valuable where the task needs the richer representation it produces: on UCI HAR it beats pooled by 8-11 accuracy points, and an eight-seed paired-significance forecasting study finds it significantly better on ETTh2 and ETTm1, worse on PEMS08, and indistinguishable on ETTh1. We propose a two-axis account: preventing the collapse does not change order availability (set by positional structure) but increases representation richness (effective rank), which helps tasks whose targets are structured rather than near-constant.
positional encoding, effective rank, self supervised learning, SIGReg, dimensional collapse, time-series representation learning, LeJEPA, transformers, joint-embedding predictive architectures, isotropic-Gaussian regularisation
positional encoding, effective rank, self supervised learning, SIGReg, dimensional collapse, time-series representation learning, LeJEPA, transformers, joint-embedding predictive architectures, isotropic-Gaussian regularisation
| selected citations These citations are derived from selected sources. This is an alternative to the "Influence" indicator, which also reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically). | 0 | |
| popularity This indicator reflects the "current" impact/attention (the "hype") of an article in the research community at large, based on the underlying citation network. | Average | |
| influence This indicator reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically). | Average | |
| impulse This indicator reflects the initial momentum of an article directly after its publication, based on the underlying citation network. | Average |
