Finding a Healthy Match: A Discussion on Applying the Appropriate Level of Analysis for Global Health Engagement Activities.

Abstract

Matching is part of our everyday life, whether it is matching debits to credits, finding the right match for a residency program or, in some instances, using match.com. The bottom line is that finding the right match is important. Yet when applying assessment techniques to Global Health Engagements (GHEs), the Department of Defense (DoD) sometimes engenders a mismatch by using measures of effectiveness (MOEs) that are ill-suited to the objectives sought by program and project stakeholders. In order to right this mismatch, it must first be recognized that a policy and operationally relevant analysis of mission impact(s) cannot be effectively undertaken without first considering whether the “level of analysis” at which the assessment is undertaken is suitably matched to the goals of the missions at hand. In certain cases, collecting MOE data on localized effects (e.g., for a particular clinic’s catchment area or in a specific village or town) is sufficient for determining mission “success” and we consider some of those special cases below. For the moment, however, it is important to note that it is often necessary to collect data at the country level in order to measure engagements’ strategic efficacy when considering their contribution to the Geographic Combatant Command’s (GCC’s) Theater Campaign Plans and country plans. For example, consider a notional immunization engagement. Ideally, the GHE immunizes several hundred patients (a measure of performance [MOP]) and thereby substantially lowers disease prevalence within the treated area (a MOE). But was this GHE successful? If the question was whether it had a locally favorable impact, the assessment should focus on these local measures of success. But if the question was whether it supported the GCC’s country plan, the MOEs used for the assessment must reflect this broader focus. If the country plan’s goals were to, say, improve citizens’ perception of the host nation government or reduce political instability in the country, health-specific and localized effects may be insufficient to demonstrate success. Analyzing the contribution of these engagements to broader GCC goals will thus require that data are collected and analyzed at the national, rather than the village, level. Measuring the national impact of a single project, such as in our immunization example, may be problematic however. This is because it is unlikely that any “single” local mission will have a sufficiently large impact to affect national level MOEs, no matter how well it was executed. Aggregating local missions at the national level in an effort to demonstrate missions’ combined strategic effectiveness may help to resolve this problem. Of course, even aggregated GHE activities, may still be insufficient to measurably alter outcomes countrywide because changing any national outcomes (e.g., security, health, economy, etc.) is difficult and often requires great effort. But the larger and more intense the effort, the greater the chance it will have to make a positive impact, underscoring the points made by Lt Gen Robb at the 2014 AMSUS Meeting, in which he argued that GHEs have become too scattered and haphazard and that the military health community must reduce its one-off projects and focus resources and attention on critical partners and key programs. None of this is to say that individual or local activities are unimportant or that their impact is impossible to measure. As noted above, data collection and analysis of local GHEs are critical to the overall assessment system in two distinct ways: individual missions are essential sources of data and measuring local effects can be important when DoD is a supporting agency. Assessments of aggregated activities would not be possible without high-quality data, which allows assessors to measure the intensity of effort. Without measuring the degree to which GHE immunization activities increased/decreased in a host nation, it would be impossible to determine whether better national health outcomes were associated with higher levels of DoD activity. It is therefore essential that “downrange” personnel rigorously capture MOPs so that they can then be aggregated at the GCC level. *Center for Global Health Engagement, Uniformed Services University of the Health Sciences, 11300 Rockville Pike, Rockville, MD 20852. †The Center for Disaster and Humanitarian Assistance Medicine, 11300 Rockville Pike, Rockville, MD 20852. The views expressed are those of the authors and do not reflect the official policy or position of the Uniformed Services University of the Health Sciences, the U.S. Navy, the Department of Defense, or the U.S. Government. doi: 10.7205/MILMED-D-15-00403