Six years of lessons learned in monitoring and evaluating online discussion forums

Megan Avila ,Catherine Richey ,Suresh Kannan Balasingam ,Lisa Mwaikambo

doi:10.34105/j.kmel.2011.03.041

Abstract

This paper presents the plan for evaluating virtual discussion forums held on the Implementing Best Practices in Reproductive Health (IBP) Knowledge Gateway, and its evolution over six years. Since 2005, the World Health Organization Department of Reproductive Health and Research (WHO/RHR), the Knowledge for Health (K4Health) Project based at Johns Hopkins Bloomberg School of Public Health’s Center for Communication Programs (JHU∙CCP), and partners of the IBP Initiative have supported more than 50 virtual discussion forums on the IBP Knowledge Gateway. These discussions have provided global health practitioners with a platform to exchange evidence-based information and knowledge with colleagues working around the world. In this paper, the authors discuss challenges related to evaluating virtual discussions and present their evaluation plan for virtual discussions. The evaluation plan included the following three stages: (I) determining value of the discussion forums, (II) in-depth exploration of the data, and (III) reflection and next steps and was guided by the “Conceptual Framework for Monitoring and Evaluating Health Information Products and Services” which was published as part of the Guide to Monitoring and Evaluation of Health Information Products and Services. An analysis of data from 26 forums is presented and discussed in light of this framework. The paper also includes next steps for improving the evaluation of future virtual discussions.

Highlights

The urgent need to accelerate progress towards meeting the Millennium Development Goals, coupled with the expansion in Internet connectivity around the world presents a timely opportunity to virtually connect those working in the health sector to address challenges and to share effective practices and lessons learned
Smaller numbers worked in government ministry (8%), reproductive health/family planning service provider (7%), private sector (5%), United States Agency for International Development (USAID) (3%), United Nations agency (3%), faith-based organization (1%), and news media (1%; see Figure 3)
Starting in 2009, WHO/RHR and JHU∙CCP strengthened the focus of the evaluation on intermediate outcomes, including three of the five types of intermediate outcomes listed in the conceptual framework – environment informed, programs and practices enhanced, and collaboration and capacity building facilitated

Summary

Introduction

As Meessen and colleagues suggest, “the community of practice strategy could be a real breakthrough for managing knowledge on implementation challenges" (2011; p 1007). These activities can "link colleagues in remote locations and across time zones, amplify the voice of people in remote locations, and facilitate communication and the generation of new knowledge" Since 2002, partners of the Implementing Best Practices in Reproductive Health (IBP) Initiative have collaborated on knowledge management strategies for virtual information exchange, primarily through developing the capacity, technology, and a methodology to support online communities of practice and discussion forums.

Objectives

Results

Discussion

Conclusion