This paper aims to provide a deep neural network (DNN) considering the statistical properties of data for robust one-class classification. To achieve that, we take advantage of the properties of Wavelet Scattering Transform (WST) to guide the DNN. WST is a translation-invariant image representation that retains high-frequency information for classification while being stable to rotation. The resulting stable and low-variance features make the clustering of data easier for DNN. The importance of WST in guiding the DNN for the classification of highly textured images is evaluated in terms of accuracy gain and robustness to outlier pollution. Superior robustness to both translation and rotation is also demonstrated. The method is not only evaluated in a standard computer vision dataset (CIFAR10), but the use of largely invariant features allows for coping with the more challenging case of satellite imagery (EuroSAT).