The optical excitation of metals initially creates short-lived non-Fermi distributions of the electrons. The electrons and holes excited far above and below the Fermi level quickly relax to hot Fermi-distributions that subsequently cool via electron-phonon scattering. Here, we show that such non-thermal charge carriers beyond the Fermi-distribution speed up the prototypical first-order antiferromagnetic-to-ferromagnetic phase transition in FeRh. In ultrafast x-ray diffraction experiments, we vary the maximum electron temperature by increasing the pump pulse duration up to 10ps. For direct optical excitation of FeRh, ferromagnetic domains nucleate within 8ps as soon as the successively deposited energy surpasses the site-specific threshold energy. In contrast, suppressing the direct optical excitation by an optically opaque Pt layer leads to a nucleation on a 50ps timescale driven by the near-equilibrium heat transport. These findings unambiguously identify the photo-excitation of non-thermal electrons and not electron-phonon non-equilibria to enable the rapid phase transition in FeRh. This dataset contains all raw data, data evaluation and plot scripts used in the linked publication. The data.rar archive contains 3 folders, linked to the measurements, the simulations and the publication figures. The data analysis scripts as well as the modelling of the laser-induced lattice dynamics are written in Python.