Abstract A heterogeneous asphalt mixture is regarded as a multiphase material comprised of coarse aggregates, air voids, and asphalt mastic; namely fine aggregates, fines, and asphalt binder in this study. An indirect tensile test (IDT) model of asphalt mixture with a given gradation was developed. The parallel-bond model (PBM), whose sets of microparameters are obtained by comparing laboratory measures with numerical simulation results, was used to characterize internal contact behavior of asphalt mixture. After that, digital samples were employed to implement a series of numerical tests to investigate the influence factors, such as coarse aggregate size, sample scale, air voids content, and loading rate on the IDT microstructure and macroscopic mechanical behaviors of asphalt mixture, qualitatively or quantitatively. The results indicated that the coarse aggregate size has a significant effect on material IDT behaviors, and the 4.75–9.5 mm aggregate particles are critical to forming the skeleton structure. Increasing sample scale, air voids and loading have a negative impact on asphalt mixture fracture behaviors, especially on the tensile strength. Moreover, increasing scale revealed that the presented IDT model can be used to simulate crack growth, and the DEM provides an auxiliary tool to further study anti-cracking behaviors of asphalt mixture.