26.C. Workshop: How to evaluate digital health: three examples based on the model of Continuous eHealth Evaluation

Abstract

Abstract Despite the common advantages of digital health, such as the increase of quality of care and the decrease in healthcare costs, the implementation of these innovations into public health are lacking. A positive impact in daily clinical practice is important for the successful implementation of digital health [1, 2]. Still it is difficult to assess the true impact of digital health in a real-life setting as proper evaluation studies are challenging [3-5]. Within a clinical setting the focus of these evaluations are mainly on clinical effectiveness or quality of care. But these kinds of evaluations are only feasible when the technology is mature and no further development is needed. This is also indicated by the most popular method for the evaluation of digital health, based on the Health Technology Assessment (HTA) model. This method, the Model for Assessment of Telemedicine (MAST) [6-8], is only applicable for fully mature technologies. Unfortunately, MAST does not specify when a technology is mature enough. Resulting in evaluations of immature technologies in clinical trials. In our opinion, the maturity of the technology can be assessed based on the Technology Readiness Levels (TRLs). With TRLs we can clearly communicate whether a technology, such as a digital health intervention, is ready for use in daily clinical practice. An understanding of both the performance and the intended use of the technology is required to establish TRL and should be the starting point for every evaluation. The objective of this workshop is to present our model for continuous eHealth evaluation [9] and to demonstrate it based on three evaluation studies of new developed digital health innovations for public health. All three innovations are still under development and have different TRLs. This will help other researchers to plan their evaluation studies with health technologies in an early phase of development. In our opinion the evaluation of digital health should be a continuous and iterative process, aligned to the progress in the development process itself. The presentations will each focus on the evaluation of a new digital innovation with real end-users and in a real-life setting. These evaluations are based on our model for continuous eHealth evaluation and will be presented as three separate cases. We will conduct a 60-minutes interactive workshop, which starts with an overview presentation (15 minutes) of our model for continuous eHealth evaluation. Next, three presentations (3x 10 minutes) on the evaluation of the three new digital health innovation will follow. The workshop ends with a plenary discussion (15 minutes) to discuss our view on the evaluation of digital health (model for continuous eHealth evaluation) and the development of the digital health innovations with the attendees. Key messages The evaluation of digital health is challenging but the maturity of the technology must be the starting point of every digital health evaluation. The evaluation of digital health should be a continuous and iterative process, aligned to the progress in the development process itself.