The metabolomics field is under rapid development. In particular, biomarker identification and pathway analysis are growing, as untargeted metabolomics is usable for discovery research. Frequently, new processing and statistical strategies are proposed to accommodate the increasing demand for robust and standardized data. One such algorithm is XCMS, which processes raw data into integrated peaks. Multiple studies have tried to assess the effect of optimizing XCMS parameters, but it is challenging to quantify the quality of the XCMS output. In this study, we investigate the effect of two automated optimization tools (Autotuner and isotopologue parameter optimization (IPO)) using the prediction power of machine learning as a proxy for the quality of the data set. We show that optimized parameters outperform default XCMS settings and that manually chosen parameters by liquid chromatography-mass spectrometry (LC-MS) experts remain the best. Finally, the machine-learning approach of quality assessment is proposed for future evaluations of newly developed optimization methods because its performance directly measures the retained signal upon preprocessing.