Let $\mathbf{A}$ be an $n\times n$-matrix over $\mathbb{F}_2$ whose every entry equals $1$ with probability $d/n$ independently for a fixed $d>0$. Draw a vector $\mathbf{y}$ randomly from the column space of $\mathbf{A}$. It is a simple observation that the entries of a random solution $\mathbf{x}$ to $\mathbf{A} x=\mathbf{y}$ are asymptotically pairwise independent, i.e., $\sum_{i\mathrm{e}$ the overlap concentrates on a single value once we condition on the matrix $\mathbf{A}$, while over the probability space of $\mathbf{A}$ its conditional expectation vacillates between two different values $α_*(d)<α^*(d)$, either of which occurs with probability $1/2+o(1)$. This bifurcated non-concentration result provides an instructive contribution to both the theory of random constraint satisfaction problems and of inference problems on random structures.