Hydroxyl-terminated polyurethane (HTPU) prepolymer and crystalline polymer particles were used to modify the toughness of diglycidyl ether of bisphenol-A (DGEBA) epoxy cured with diaminodiphenyl sulphone (DDS). For the crystalline polymers, poly(butylene terephthalate) (PBT) and poly(hexamethylene adipamide) (Nylon 6,6), with particle size under 401 μm were employed. The PU-modified epoxies exhibit a significantly improved fracture toughness. The β-relaxation of cured epoxy resin shows a more clear two-phase separation as the polyurethane content increases. Besides, the addition of crystalline polymer particles could further enhance the toughness of PU-modified epoxy at low particle content. Scanning electron microscopy (SEM) shows that PU-modified epoxy has a two-phase structure. The fracture properties of PBT particle filled epoxy are found to be better than those of Nylon 6,6 particle filled epoxy. Nevertheless, the toughening effect of these crystalline polymer particles is much less efficient than that of PU modification.