Some liquids, if cooled rapidly enough to avoid crystallization, can be frozen into a nonergodic glassy state. The tendency for a material to form a glass when quenched is called “glass-forming ability,” and it is of key significance both fundamentally and for materials science applications. Here, we consider liquids with competing orderings, where an increase in the glass-forming ability is signaled by a depression of the melting temperature towards its minimum at triple or eutectic points. With simulations of two model systems where glass-forming ability can be tuned by an external parameter, we are able to interpolate between crystal-forming and glass-forming behavior. We find that the enhancement of the glass-forming ability is caused by an increase in the structural difference between liquid and crystal: stronger competition in orderings towards the melting point minimum makes a liquid structure more disordered (more complex). This increase in the liquid-crystal structure difference can be described by a single adimensional parameter, i.e., the interface energy cost scaled by the thermal energy, which we call the “thermodynamic interface penalty.” Our finding may provide a general physical principle for not only controlling the glass-forming ability but also the emergence of glassy behavior of various systems with competing orderings, including orderings of structural, magnetic, electronic, charge, and dipolar origin.