A critical role in monitoring and understanding human activities at sea is held by the detection of moving vessels, a challenging task that can be accomplished, in specific conditions, by inspecting their long wakes left in the sea. To solve the ship wake detection problem, the traditional methodology based its research on domain transformation from lines to points, such as Radon or the Hough transform. Assuming wakes as linear features, such as class of algorithms is not capable of capturing irregular or curved wakes and shows poor generalization. Nevertheless, the current digital era is dominated by Deep Learning (DL) techniques thanks to their capability of abstract feature extraction. Representation learning has proven to tackle the increasing speed and breadth of”Big Data”, outperforming humans on a variety of challenging tasks. Convolutional Neural Networks (CNNs) can glean relevant patterns from remotely sensed images and represents the core of the paper which intends to realize an automatic wake recognition system from spaceborne optical images. Several state-of-the-art DL-based approaches are benchmarked with model baselines including both object detection and instance segmentation architectures, including one- and multi-stage methods. The usage of ResNet backbones as the main feature extractor is motivated by their effectiveness on many computer vision datasets. Feature Pyramid Network (FPN), used as a neck of the backbone, grants for multi-size detection. To perform supervised learning, a novel dataset is built and proposed in this paper. The Multi-Spectral Ship Wake Dataset (MSSWD) is represented by multi-spectral chips extracted from the European Sentinel-2 mission, selected for its publicly available data policy. Chips are extracted from Level-2A ortho-images. MSSWD is composed of 1059 ship wakes gathered from 50 multi-spectral granules. Data variety was curated by selecting wakes in multiple dimensions and orientations while data veracity is assured by the corresponding AIS (Automatic Identification System) information. The multi-band side of the MSSWD has been analyzed by covering 4 bands, i.e. B2 (blue), B3 (green), B4 (red), and B8 (infrared) bands, all characterized by the same spatial resolution. The analysis of the results proves that this class of algorithms is capable of detecting the vast majority of the wakes with high confidence scores, very low probability of false alarms, and fast processing speed. In particular, Cascade Mask R-CNN, Mask R-CNN, and RetinaNet have shown the best results in terms of Average Precision (AP), being able to correctly detect the of the test dataset wakes, reporting only 3 false alarms consisting of aircraft wakes. Moreover, all bands produced the same results in terms of detection performance. However, the multi-band feature of the MSSWD could still be of use to detect the false alarms, on the basis of the temporal offset in the acquisition time of each band.