The main purpose of this article is to examine some questions of the Invalsi test on the Italian language, which is carried out by Italian students at three different stages of their school career, i.e. at the years 2 (age 6-7) and 5 (age 9-10) of the primary school, at year 3 of the secondary school (age 12-13), and at year 2 of the high school (age 15-16). I will focus on the type of grammatical competence pupils are expected to possess to perform successfully the test. To do that, I will start from the considerations given as a foreword to the test by the Istituto Invalsi; I will then examine some revealing questions of the test and reflect on the (often low) percentages of correct answers. I will finally draw some conclusions on the gap between the elaborate linguistic competence the Invalsi test aims to investigate and the mechanical knowledge of grammar Italian students actually possess.