In several sensory pathways, input stimuli project to sparsely active downstream populations that have more neurons than incoming axons. Here, we address the computational benefits of expansion and sparseness for clustered inputs, where different clusters represent behaviorally distinct stimuli and intracluster variability represents sensory or neuronal noise. Through analytical calculations and numerical simulations, we show that expansion implemented by feed-forward random synaptic weights amplifies variability in the incoming stimuli, and this noise enhancement increases with sparseness of the expanded representation. In addition, the low dimensionality of the input layer generates overlaps between the induced representations of different stimuli, limiting the benefit of expansion. Highly sparse expansive representations obtained through synapses that encode the clustered structure of the input reduce both intrastimulus variability and the excess overlaps between stimuli, enhancing the ability of downstream neurons to perform classification and recognition tasks. Implications for olfactory, cerebellar, and visual processing are discussed.