In this paper, an adaptive interpolation algorithm is presented based on the Newton Polynomial to improve the limitation of the traditional algorithm for image resizing. The second-order difference of adjacent pixels gray values shows the relativity among the pixels. Accordingly, the adaptive function for image interpolation is deduced according to both this relativity and the classical Newton polynomial. Then the efficiency of our method is compared with that of the traditional algorithm for image resizing in Matlab. Furthermore, the implementation circuit architecture is devised by three stage paralleling pipelines for the adaptive image resizing algorithm and is verified in FPGA (field programmable gate array). The experimental results show that our proposed algorithm excels the bicubic interpolation in visual effect, and has a lower complexity. Therefore, the algorithm adapts to real-time image resizing.