Abstract The strength of quick-converting railway tracks is expected to be lower due to abraded fine particles attached to the surface of ballast aggregates caused by gravel friction under repetitive train loading. The effects of abraded fine particle content on the failure process of quick-hardening concrete were investigated using digital image correlation (DIC) analysis during the testing of modeled cored aggregate concretes (MCACs). It was revealed that the failure strength of the MCACs was significantly affected by the amount of abraded fine particles attached to the surface of aggregates because the interfacial properties between aggregates and injected quick-hardening mortar were clearly affected by the particles. As the amount of particles increased from 0.0 to 0.6 wt%, the compressive strength of the MCACs decreased from 46.2 to 38.1 MPa owing to the deteriorated interfacial properties; however, the reduction was negligible until the abraded fine particle content reached 0.02 wt%. A micromechanical analysis was also carried out to highlight the effects of the interfacial strength on the mechanical response of quick-hardening concrete.