We develop a novel embedded vision system for online monitoring of arc welding with thermal imaging. The thermal images are able to provide clear information of the melt pool and surrounding areas during the welding process. We propose a deep learning processing pipeline with a CNNLSTM architecture for the detection and classification of defects based on video sequences. The experimental results show that the CNN-LSTM architecture is able to model the complex dynamics of the welding process and detect and classify defects with high accuracy. In addition, the embedded vision system implements an OPC-UA server, enabling an easy vertical and horizontal integration in Industry 4.0.