ABSTRACT In the design of molecules, materials and processes, outliers or outlier samples can be included in a dataset when performing machine learning or regression analysis. Although outlier samples with high prediction errors in regression analysis have been divided into bad leverage points and vertical outliers (good leverage points have low prediction errors), this study classifies the outlier samples into consistent outliers (CO) and inconsistent outliers (ICO) for a detailed discussion of outlier samples and their effective utilisation. The relationship between the explanatory variables ( x ) and dependent variables ( y ) is consistent with the other samples for CO but not for ICO. Furthermore, an index of ICO‐likeness based on triple cross‐validation and the mean absolute error is proposed, and a method to determine whether an outlier sample is an ICO or a CO is developed. Data analysis using numerical simulation datasets and a compound dataset with boiling points confirms that the proposed method can appropriately discriminate between ICO and CO. When an outlier sample is determined to be an ICO, the errors in x and y should be checked first for the sample. If no errors exist in x and y , a new x should be added to explain y of the ICO. When an outlier sample is determined to be CO, it is expected that exploring the extrapolation from CO in x will further improve the y values using a model that includes CO.