ABSTRACT Dust destruction by supernovae is one of the main processes removing dust from the interstellar medium (ISM). Estimates of the efficiency of this process, both theoretical and observational, typically assume a shock propagating into a homogeneous medium, whereas the ISM possesses significant substructure in reality. We self-consistently model the dust and gas properties of the shocked ISM in three supernova remnants (SNRs), using X-ray and infrared (IR) data combined with corresponding emission models. Collisional heating by gas with properties derived from X-ray observations produces dust temperatures too high to fit the far-IR fluxes from each SNR. An additional colder dust component is required, which has a minimum mass several orders of magnitude larger than that of the warm dust heated by the X-ray emitting gas. Dust-to-gas mass ratios indicate that the majority of the dust in the X-ray emitting material has been destroyed, while the fraction of surviving dust in the cold component is plausibly close to unity. As the cold component makes up virtually all the total dust mass, destruction time-scales based on homogeneous models, which cannot account for multiple phases of shocked gas and dust, may be significantly overestimating actual dust destruction efficiencies, and subsequently underestimating grain lifetimes.