The goal of this paper is to present and discuss the difficulties in defining and measuring innovation. The large number of existing definitions is yet failing to make the necessary distinction between innovation and other related concepts. There is no agreement about the most relevant dimensions of innovation and, without a proper definition of innovation, measurement is more difficult. Several methods and attempts of measurement– Awards and Surveys – are identifying different meanings and characteristics of innovation. A more in-depth analysis of a pilot exercise in measuring innovation, namely the European Public Sector Innovation Scoreboard 2013, is done in order to examine its methodology and the validity of its findings for the specific case of Romania. The need for an innovative public sector is generally recognized, and present measurement attempts are actually overestimating the level of innovation. In order to assess this correctly a clearer and narrower definition is needed, the framework of the innovative process needs to be further investigated and research designs have to be improved