Abstract This paper presents a comparison of behavior and energy absorption of neat Kevlar fabric and polymer matrix composites under high velocity impact loading . Two types of matrices including rubber and thermoset (epoxy) matrices were used in order to study the effect of a hard and brittle matrix compared with the soft and flexible matrix on energy absorption of the composite. Moreover, two types of rubber matrix with high hardness (HH) and low hardness (LH) were used in this study to investigate the effect of rubber matrix formulation on impact resistance of composites. Ballistic impact tests were performed by firing a 10 mm hemispherical projectile onto neat fabric and composites in a velocity range of 30–150 m/s for two- and four-layer samples. Results show that the matrix affects the ballistic performance of composites significantly. Rubber matrix enhances the energy absorption of the fabric by keeping composite flexibility. Increase the number of layers for Kevlar/rubber composite results in better ballistic performance. On the contrary, the thermoset matrix leads to an inflexible composite that restricts the fabric deformation and has a negative effect on the fabric’s ballistic performance. Finally, damage mechanisms were discussed in detail for each sample.