This paper reports the effects of chemical composition on the hardness of the heat affected zone of re-austenitized and water quenched steels. Heat affected zone peak temperatures in the range 300–1350 °C were simulated using a Gleeble 3800 simulator using thermal cycles appropriate to welds with cooling times between 800 and 500 °C of 12s. The maximum softening relative to the base material occurred in the intercritical and subcritical heat affected zones at the peak temperatures of 700 or 800 °C. Usually softening was greatest when the peak temperature was 700 °C. Linear regression analysis showed that carbon, and to some extent manganese and nickel, are detrimental at the peak temperature of 700 °C, but beneficial at the peak temperature of 800 °C in respect to softening relative to the base material, whereas niobium and especially molybdenum are beneficial at both temperatures. The beneficial effects of molybdenum alloying are seen down to peak temperatures of 400 °C whereas the effect of niobium microalloying is not statistically significant at peak temperatures lower than 700 °C. The softening in the intercritical, fine-grained and coarse grained heat affected zones are discussed and the effects of the alloying elements on the hardness of the subcritical heat affected zone are compared with their known effects on martensite hardness during conventional tempering.