Abstract Background It is generally accepted that evidence-informed decision making contributes to better health system performance and health outcomes, yet we are lacking benchmarks to monitor the impact of national health information systems (HIS) in policy and practice. Hence in this study, we have aimed to identify criteria for monitoring Knowledge Translation (KT) capacity within countries. Methods We conducted a web-based Delphi with over 120 public health professionals from 45 countries to reach agreement on criteria to monitor KT at the level of national HIS. Public health professionals participated in three survey rounds, in which they ranked 85 preselected criteria and could suggest additional criteria. Results Experts working in national (public) health agencies and statistical offices, as well as in health policy and care agreed on 29 criteria which constitute the Health Information (HI)-Impact Index. The criteria cover four essential domains of evaluation: the production of high-quality evidence, broad access and dissemination, stakeholder engagement and knowledge integration across sectors and in civil society. The HI-Impact Index was pretested by officials working in ministries of health and public health agencies in eight countries; they found the tool acceptable and user-friendly. Conclusions The HI-Impact Index provides benchmarks to monitor KT so that countries can assess whether high-quality evidence can be easily accessed and used by the relevant stakeholders in health policy and practice, by civil society and across sectors. Next steps include further refining the procedure for conducting the assessment in routine, and sharing experiences from HIS evaluations using the HI-Impact Index.