Since the proposal of differential cryptanalysis and linear cryptanalysis in 1991 and 1993, respectively, the resistance to these cryptanalyses have been studied for many cryptosystems. Moreover, some block ciphers with provable security against differential and linear cryptanalysis have been proposed. One of them is the KN cipher proposed by Knudsen and Nyberg. The KN cipher is a prototype cipher with provable security against ordinary differential cryptanalysis, and has been proved to be secure against linear cryptanalysis, too. Recently a new method of attacking block ciphers, the higher order differential attack, was proposed, and Jakobsen and Knudsen showed that the KN cipher can be attacked by this method in FSE4. In this paper, we improve this attack to reduce both of the required chosen plaintexts and running time, and apply it to the cryptanalysis of the KN cipher. We show that, for the attacking of the KN cipher with 6 rounds, the number of required chosen plaintexts can be reduced by half and running time reduced from 241 to 214, and that all round keys can be derived in only 0.02 seconds on a Sun Ultra 1 (UltraSPARC 170MHz).