Reclaimed Asphalt Pavement (RAP) as recycled aggregates is a relatively new construction process of rigid pavements due to the scarcity and degradation of natural aggregates. This study aims at the sequential characterization of RAP aggregate to obtain optimized proportions for strength. For this purpose, RAP aggregates were used for the replacement of natural aggregates (NA) in the concrete mix which was achieved by varying from 0–100%. Furthermore, zirconia silica fume (ZSF) was used as a partial replacement of the cement in the concrete mix, replacing Ordinary Portland Cement (OPC). Experimental studies have shown that the incorporation of washed RAP (WRAP) slightly reduces the compressive strength of concrete by 2.7–37.35% as compared to the reference control concrete mix. Although the 7-days, 28-days and 56-day compressive strength of WRAP recycled aggregate-based concrete is slightly better than the 7-days, 28-days and 56-day compressive strength of dirty RAP (DRAP) recycled aggregate-based concrete. A similar trend was observed in the flexural strength and split tensile strength of WRAP recycled aggregate-based. Overall, the results show that 40% WRAP recycled aggregates with 10% ZSF as a replacement for cement outperform DRAP aggregates in concrete mixes. According to the ANOVA results, the combination of ZSF and WRAP aggregates met the cement concrete pavement strength standard, thereby contributing to sustainable development. Reclaimed Asphalt Concrete Pavements (RACP) are now seen as a potential and long-term answer to the present environmental and economic crisis.