Robust Autoencoders separate the input image into a Signal(L) and a Noise(S) part which, intuitively speaking, roughly corresponds to a more stable background scene (L) and an undesired anomaly (or defect) (S). This property of the method provides a convenient theoretical basis for divorcing intermittent anomalies that happen to clutter a relatively consistent background image. In this paper, we illustrate the use of Robust Deep Convolutional Autoencoders (RDCAE) for defect detection, via a pseudo-supervised training process. Our method introduces synthetic simulated defects (or structured noise) to the training process, that alleviates the scarcity of true (real-life) anomalous samples. As such, we offer a pseudo-supervised training process to devise a well-defined mechanism for deciding that the defect-normal discrimination capability of the autoencoders has reached to an acceptable point at training time. The experiment results illustrate that pseudo supervised Robust Deep Convolutional Autoencoders are very effective in identifying surface defects in an efficient way, compared to state of the art anomaly detection methods.