The research has been initiated to develop the asphalt mixtures which are suitable for the surface of asphalt concrete directly fastened track (ADFT) system and evaluate the performance of the asphalt mixture. Three aggregate gradations which are upper (finer), medium, and below (coarser). The nominal maximum aggregate size of asphalt mixture was 10 mm. Asphalt mixture design was conducted at 3 percent air voids using Marshall mix design method. To make impermeable asphalt mixture surface, the laboratory permeability test was conducted for asphalt mixtures of three different aggregate gradations using asphalt mixture permeability tester. Moisture susceptibility test was conducted based on AASHTO T 283. The stripping percentage of asphalt mixtures was measured using a digital camera and analyzed based on image analysis techniques. Based on the limited research results, the finer aggregate gradation is the most suitable for asphalt mixture for ADFT system with the high TSR value and the low stripping percentage and permeable coefficient. Flow number and beam fatigue tests for finer aggregate asphalt mixture were conducted to characterize the performance of asphalt mixtures containing two modified asphalt binders: STE-10 which is styrene-butadiene-styrene (SBS) polymer and ARMA which is Crum rubber modified asphalt. The performance tests indicate that the STE-10 shows the higher rutting life and fatigue life.