The longitudinal testing of student achievement requires the solution of several new problem areas. In this article, several small groups of medical students at the University of Limburg Medical School in Maastricht, The Netherlands, are compared with respect to their performances. The results indicate, that, despite the possession of more knowledge at entrance, students with a low rate of growth of knowledge in the first year demonstrate a lower level of knowledge after the second academic year and continue to do so throughout the academic program when compared to students who show a higher rate of growth of knowledge in the first year. The analysis has been carried out using a distribution‐free version of a longitudinal IRT‐model suggested by Albers, Does, Imbos, and Janssen (1989). Furthermore, growth of knowledge has been described by means of a general regression model. Statistical inferences are possible using a randomization design extended to the situation where the observations are time‐dependent proportions of correct answers.