
Digital health interventions (DHIs) have the potential to improve health care and health promotion. However, there is a lack of guidance in the literature for the development, refinement, and prioritization of key performance indicators (KPIs) for the evaluation of DHIs. This paper presents a 4-stage process used in the Gravitate Health project based on stakeholder consultation and consensus for this purpose. The Gravitate Health consortium, which comprises private and public partners from across Europe and the United States, is developing innovative digital health solutions in the form of Federated Open-Source Platform and G-lens to present users with individualized digital information about their medicines. The first stage of this was the consultative process for the development of KPIs involving stakeholder (Gravitate Health project leads) consultations at the planning stages of the project. This resulted in the formation of an extensive list of KPIs organized into 7 categories. The second stage was conducting a scoping review, which confirmed the need for extensive stakeholder consultation in all stages of the KPI development, refinement, and prioritization process. The third stage was a period of further consultation with all consortium members, which resulted in the elimination of 1 category of KPIs. The fourth stage involved using the Delphi technique for refining and prioritizing the remaining 6 categories of KPIs. It is unusual to use this methodology in a nonresearch exercise, but it provided a clear consultative framework and structure that facilitated the achievement of consensus within a large consortium of 250 members on a substantial list of KPIs for the project. Consortium members ranked the relevance and importance of each KPI. The final list of KPIs provides substantial indicators sensitive to the needs of a broad group of stakeholders that are being used to capture real-world data in developing and evaluating DHIs.
digital health interventions, Consensus, Computer applications to medicine. Medical informatics, R858-859.7, key performance indicators, Telemedicine, United States, consensus methodology, Delphi technique, drug-related side effects and adverse reactions, consultation, Tutorial, Humans, Digital Health, referral, Public aspects of medicine, RA1-1270, Quality Indicators, Health Care
digital health interventions, Consensus, Computer applications to medicine. Medical informatics, R858-859.7, key performance indicators, Telemedicine, United States, consensus methodology, Delphi technique, drug-related side effects and adverse reactions, consultation, Tutorial, Humans, Digital Health, referral, Public aspects of medicine, RA1-1270, Quality Indicators, Health Care
| selected citations These citations are derived from selected sources. This is an alternative to the "Influence" indicator, which also reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically). | 0 | |
| popularity This indicator reflects the "current" impact/attention (the "hype") of an article in the research community at large, based on the underlying citation network. | Average | |
| influence This indicator reflects the overall/total impact of an article in the research community at large, based on the underlying citation network (diachronically). | Average | |
| impulse This indicator reflects the initial momentum of an article directly after its publication, based on the underlying citation network. | Average |
