In spite of achieving promising results in hyperspectral image (HSI) restoration, deep-learning-based methodologies still face the problem of spectral or spatial information loss due to neglecting the inner correlation of HSI. To address this issue, we propose an innovative deep recurrent convolution neural network (DnRCNN) model for HSI destriping. To the best of our knowledge, this is the first study on HSI destriping from the perspective of inner band and interband correlation explorations with the recurrent convolution neural network. In the novel DnRCNN, a selective recurrent memory unit (SRMU) is designed to respectively extract the correlative features involved in spectral and spatial domains. Moreover, an innovative recurrent fusion (RF) strategy incorporated with group concatenation is further proposed to remove strip noise and preserve scene details using the complementary features from SRMU. Experimental results on extensive HSI datasets validated that the proposed method achieves a new state-of-the-art (SOTA) HSI destriping performance.