Charpy V-notch impact testing serves as a fundamental method for assessing the toughness of conventional steels, but its applicability becomes challenging for duplex (δ-ferrite + γ-austenite) lightweight steels due to their complex microstructures and varied deformation mechanisms. In this study, Charpy absorbed energies of hot-rolled duplex lightweight steel plates, incorporating both transformation-induced plasticity (TRIP) and twinning-induced plasticity (TWIP) mechanisms, were investigated to understand their fracture behavior comprehensively. The microstructural analysis revealed that the D4 (0.4C–15Mn–6Al (wt.%)) steel exhibited lower δ-ferrite volume fraction and superior Charpy absorbed energy than the D2 (0.2C–15Mn–6Al) steel, attributed to its dominance of TWIP mechanism. Conversely, the D2 steel, with many ferrite/austenite interfaces, showed lower strain hardening and absorbed energy. At lower temperatures, both TWIP and TRIP mechanisms operated, but dominant TRIP mechanism along with higher δ-ferrite volume fraction in the D2 steel accelerated crack propagation, further reducing absorbed energy. The superior toughness of the D4 steel, driven by the dominant TWIP mechanism at lower temperature, highlighted its potential for practical utilization of thick hot-rolled lightweight steel plates in automotive, shipbuilding, military, and construction industries.