Polar codes are closer to the Shannon limit with lower complexity in coding and decoding. As traditional decoding techniques suffer from high latency and low throughput, with the development of deep learning technology, some deep learning-based decoding methods have been proposed to solve these problems. Usually, the deep neural network is treated as a black box and learns to map the polar codes with noise to the original information code directly. In fact, it is difficult for the network to distinguish between valid and interfering information, which leads to limited BER performance. In this paper, a deep residual network based on information refinement (DIR-NET) is proposed for decoding polar-coded short packets. The proposed method works to fully distinguish the effective and interference information in the codewords, thus obtaining a lower bit error rate. To achieve this goal, we design a two-stage decoding network, including a denoising subnetwork and decoding subnetwork. This structure can further improve the accuracy of the decoding method. Furthermore, we construct the whole network solely on the basis of the attention mechanism. It has a stronger information extraction ability than the traditional neural network structure. Benefiting from cascaded attention modules, information can be filtered and refined step-by-step, thus obtaining a low bit error rate. The simulation results show that DIR-Net outperforms existing decoding methods in terms of BER performance under both AWGN channels and flat fading channels.