The Coordinated Network of Cameras tool is based on a Deep-Non-linear Auto-Regressive Moving Average (Deep-NARMA) filter that leverages the representational capabilities of Convolutional Neural Networks (CNN), properly modified to cope with the autoregressive nature of a tapped delay line, and transforms the inputs in an efficient non-linear feature map. The proposed classifier achieves an effective feature representation of the heterogeneous inputs but it simultaneously introduces an input-and output memory. We also propose a novel data fusion processing of different modalities to improve the detection accuracy of unusual behavior. Specifically, alongside the normal RGB surveillance, we can also leverage thermal imaging.