ABSTRACT Evolutionary adaptation occurs when individuals vary in access to fitness‐relevant resources and these differences in ‘material wealth’ are heritable. It is typically assumed that the inheritance of material wealth reflects heritable variation in the phenotypic abilities needed to acquire material wealth. We scrutinise this assumption by investigating additional mechanisms underlying the inheritance of material wealth in collared flycatchers. A genome‐wide association analysis reveals a high genomic heritability ( h 2 = 0.405 ± 0.08) of access to caterpillar larvae, a fitness‐relevant resource, in the birds' breeding territories. However, we find little evidence for heritable variation in phenotypic abilities needed to acquire this material wealth. Instead, combined evidence from simulations, experimental and long‐term monitoring data indicate that inheritance of material wealth is largely explained by philopatry causing a within‐population genetic structure across a heterogeneous landscape. Therefore, allelic variants associated with high material wealth may spread in the population without having causal connections to traits promoting local adaptation.