The application of machine learning models in chemistry has made remarkable strides in recent years. While the field of analytical chemistry has also received considerable interest from machine learning practitioners, very few models have been adopted into everyday use. Among the analytical instruments available to chemists, Infrared (IR) spectroscopy is one of the cheapest, easiest and most accessible. So far the use of IR has been limited to the identification of a select few functional groups with well-known vibrational frequencies with the interpretation of most peaks lying outside of human capabilities. We present a novel machine learning model that enables chemists to leverage the complete information contained within an IR spectrum to directly predict the molecular structure. To achieve this, we developed a transformer model trained on IR spectra that predicts the molecular structure as a SMILES string. To cover a vast portion of chemical space, we generated a training set of 634,585 simulated IR spectra using molecular dynamics. Our approach achieved a top-1 accuracy of 45.33% and a top-10 accuracy of 78.5% on a test set sampled from PubChem with a heavy atom count ranging from 6 to 13. The model is useful also in cases where an incorrect structure is predicted, as it is capable of predicting the correct scaffold in 77.01% of cases as the top-1 prediction and in 91.54% in the top-10 predictions. In addition, the model outperforms other models solely trained to predict the functional group from the IR spectrum.