The cold storage time of salmon has a significant impact on its freshness, which is an important factor for consumers to evaluate the quality of salmon. The efficient, accurate, and convenient protocol is urgent to appraise the freshness for quality checking. In this paper, the ability of visible/near-infrared (VIS/NIR) spectroscopy was evaluated to predict the cold storage time of salmon meat and skin, which were stored at low-temperature box for 0~12 days. Meanwhile, a double-layer stacked denoising autoencoder neural network (SDAE-NN) algorithm was introduced to establish the prediction model without spectral pre-preprocessing. The results showed that, compared with the common methods such as partial least squares regression (PLSR) and back propagation neural network (BP-NN), the SDAE-NN method had a better performance due to its high efficiency in decreasing noise and optimizing the initial weights. The determination coefficient of test sets (R2test) and root mean square error of test sets (RMSEP) have been calculated based on SDAE-NN, for the salmon meat (skin), the R2test can reach 0.98 (0.92), and the RMSEP can reach 0.93 (1.75), respectively. It is highlighted that the algorithm is efficient and accurate and that the salmon meat would be more suitable for predicting freshness than the salmon skin. VIS/NIR spectroscopy combined with the SDAE-NN algorithm can be widely used to predict the freshness of various agricultural products.