Health Metrics Research Consortium Research Articles

Validation studies of verbal autopsy methods: a systematic review

BackgroundVerbal autopsy (VA) has emerged as an increasingly popular technique to assign cause of death in parts of the world where the majority of deaths occur without proper medical certification. The purpose of this study was to examine the key characteristics of studies that have attempted to validate VA cause of death against an established cause of death.MethodsA systematic review was conducted by searching the MEDLINE, EMBASE, Cochrane-library, and Scopus electronic databases. Included studies contained 1) a VA component, 2) a validation component, and 3) original analysis or re-analysis. Characteristics of VA studies were extracted. A total of 527 studies were assessed, and 481 studies screened to give 66 studies selected for data extraction.ResultsSixty-six studies were included from multiple countries. Ten studies used an existing database. Sixteen studies used the World Health Organization VA questionnaire and 5 studies used the Population Health Metrics Research Consortium VA questionnaire. Physician certification was used in 36 studies and computer coded methods were used in 14 studies. Thirty-seven studies used high level comparator data with detailed laboratory investigations.ConclusionMost studies found VA to be an effective cause of death assignment method and compared VA cause of death to a high-quality established cause of death. Nonetheless, there were inconsistencies in the methodologies of the validation studies, and many used poor quality comparison cause of death data. Future VA validation studies should adhere to consistent methodological criteria so that policymakers can easily interpret the findings to select the most appropriate VA method.Prospero RegistrationCRD42020186886.

Estimating causes of out-of-hospital deaths in China: application of SmartVA methods

BackgroundMost deaths in China occur at home, making it difficult to collect reliable cause of death (CoD) information. Verbal autopsy (VA) was applied using the SmartVA tool to a sample of home deaths in China to explore its feasibility as a means of improving the quality of CoD data.MethodsThe study was carried out in 22 districts in 9 provinces, located in north-east, central, and western areas of China during 2017 and 2018. Trained interviewers selected suitable respondents in each household to collect information using the Population Health Metrics Research Consortium (PHMRC) shortened and validated electronic VA questionnaire on tablets. The CoD was diagnosed from the interview data using the SmartVA-Analyze 2.0 software (Tariff 2.0).ResultsNon-communicable diseases (NCDs) dominated the leading causes of death in all age groups and for both sexes. After redistribution of undetermined causes, stroke (24%), ischemic heart diseases (IHD) (21%), chronic respiratory diseases (11%), and lung cancer (6%) were the leading causes of death. The cause fractions for level-one cause categories and ranking of specific causes were similar between SmartVA and results from the Global Burden of Disease (GBD) study.ConclusionEvidence from this large pilot study suggests that SmartVA is a feasible and plausible tool and could be a valuable tool to improve the quality and standardization of CoD information across China.

How advanced is the epidemiological transition in Papua New Guinea? New evidence from verbal autopsy.

BackgroundReliable cause of death (COD) data are not available for the majority of deaths in Papua New Guinea (PNG), despite their critical policy value. Automated verbal autopsy (VA) methods, involving an interview and automated analysis to diagnose causes of community deaths, have recently been trialled in PNG. Here, we report VA results from three sites and highlight the utility of these methods to generate information about the leading CODs in the country.MethodsVA methods were introduced in one district in each of three provinces: Alotau in Milne Bay; Tambul-Nebilyer in Western Highlands; and Talasea in West New Britain. VA interviews were conducted using the Population Health Metrics Research Consortium (PHMRC) shortened questionnaire and analysed using the SmartVA automated diagnostic algorithm.ResultsA total of 1655 VAs were collected between June 2018 and November 2019, 87.0% of which related to deaths at age 12 years and over. Our findings suggest a continuing high proportion of deaths due to infectious diseases (27.0%) and a lower proportion of deaths due to non-communicable diseases (NCDs) (50.8%) than estimated by the Global Burden of Disease Study (GBD) 2017: 16.5% infectious diseases and 70.5% NCDs. The proportion of injury deaths was also high compared with GBD: 22.5% versus 13.0%.ConclusionsHealth policy in PNG needs to address a ‘triple burden’ of high infectious mortality, rising NCDs and a high fraction of deaths due to injuries. This study demonstrates the potential of automated VA methods to generate timely, reliable and policy-relevant data on COD patterns in hard-to-reach populations in PNG.

BAYESIAN FACTOR MODELS FOR PROBABILISTIC CAUSE OF DEATH ASSESSMENT WITH VERBAL AUTOPSIES.

The distribution of deaths by cause provides crucial information for public health planning, response and evaluation. About 60% of deaths globally are not registered or given a cause, limiting our ability to understand disease epidemiology. Verbal autopsy (VA) surveys are increasingly used in such settings to collect information on the signs, symptoms and medical history of people who have recently died. This article develops a novel Bayesian method for estimation of population distributions of deaths by cause using verbal autopsy data. The proposed approach is based on a multivariate probit model where associations among items in questionnaires are flexibly induced by latent factors. Using the Population Health Metrics Research Consortium labeled data that include both VA and medically certified causes of death, we assess performance of the proposed method. Further, we estimate important questionnaire items that are highly associated with causes of death. This framework provides insights that will simplify future data.

Robustness of the Tariff method for diagnosing verbal autopsies: impact of additional site data on the relationship between symptom and cause

BackgroundVerbal autopsy (VA) is increasingly being considered as a cost-effective method to improve cause of death information in countries with low quality vital registration. VA algorithms that use empirical data have an advantage over expert derived algorithms in that they use responses to the VA instrument as a reference instead of physician opinion. It is unclear how stable these data driven algorithms, such as the Tariff 2.0 method, are to cultural and epidemiological variations in populations where they might be employed.MethodsVAs were conducted in three sites as part of the Improving Methods to Measure Comparable Mortality by Cause (IMMCMC) study: Bohol, Philippines; Chandpur and Comila Districts, Bangladesh; and Central and Eastern Highlands Provinces, Papua New Guinea. Similar diagnostic criteria and cause lists as the Population Health Metrics Research Consortium (PHMRC) study were used to identify gold standard (GS) deaths. We assessed changes in Tariffs by examining the proportion of Tariffs that changed significantly after the addition of the IMMCMC dataset to the PHMRC dataset.ResultsThe IMMCMC study added 3512 deaths to the GS VA database (2491 adults, 320 children, and 701 neonates). Chance-corrected cause specific mortality fractions for Tariff improved with the addition of the IMMCMC dataset for adults (+ 5.0%), children (+ 5.8%), and neonates (+ 1.5%). 97.2% of Tariffs did not change significantly after the addition of the IMMCMC dataset.ConclusionsTariffs generally remained consistent after adding the IMMCMC dataset. Population level performance of the Tariff method for diagnosing VAs improved marginally for all age groups in the combined dataset. These findings suggest that cause-symptom relationships of Tariff 2.0 might well be robust across different population settings in developing countries. Increasing the total number of GS deaths improves the validity of Tariff and provides a foundation for the validation of other empirical algorithms.

Estimating Causes of Out-of-Hospital Deaths in China: Application of SmartVA Methods

Background: Most deaths in China occur at home, making it difficult to collect reliable cause of death (CoD) information. Verbal autopsy (VA) was applied using the SmartVA tool to a sample of home deaths in China to explore its feasibility as a means of improving the quality of CoD data. Methods: The study was carried out in 22 districts in 9 provinces, located in north-east, central, and western areas of China during 2017 and 2018. Trained interviewers selected suitable respondents in each household to collect information using the Population Health Metrics Research Consortium (PHMRC) shortened and validated electronic VA questionnaire on tablets. The CoD was diagnosed from the interview data using the SmartVA-Analyze 2.0 software (Tariff 2.0). Findings: Non-communicable diseases (NCDs) dominated the leading causes of death in all age groups and for both sexes. After redistribution of undetermined causes, stroke (24%), ischemic heart diseases (IHD) (21%), chronic respiratory diseases (11%), and lung cancer (6%) were the leading causes of death. The cause fractions for level-one cause categories and ranking of specific causes were similar between SmartVA and results from the Global Burden of Disease (GBD) study. Interpretation: Evidence from this large pilot study suggests that SmartVA can be a valuable tool to improve the quality and standardization of CoD information across China. Funding Statement: Bloomberg Data for Health Initiative. Declaration of Interests: The authors declare no competing interests. Ethics Approval Statement: The ethics committee from the National Center for Chronic Non-Communicable Disease Control and Prevention (NCNCD) of the Chinese Center for Disease Control and Prevention approved the study. Verbal informed consent was obtained and recorded for all participants.

Analysis of causes of death using verbal autopsies and vital registration in Hidalgo, Mexico.

IntroductionVerbal autopsy (VA) is a useful tool for evaluating causes of death, especially in places with limited or no vital registration systems. The Population Health Metrics Research Consortium (PHMRC) developed a validated questionnaire and a set of automated methods to determine the cause of death from a VA. However, the application of these methods needs to be tested in a community environment.ObjectiveTo estimate cause-specific mortality fractions (CSMFs) using VAs and compare them against those obtained in the vital statistics of the state of Hidalgo, Mexico.MethodsA random sample of deaths occurred in 2009 was selected from vital statistics in the state of Hidalgo. The full PHMRC validated VA instrument was applied to the relatives of the deceased, and the cause of death was determined using Tariff's automated method. The causes of death were grouped into 34 causes for adults, 21 for children and 6 for newborns. Results were compared with cause of death on death certificates for all deaths.ResultsA total of 1,198 VAs were analyzed. The Tariff method was not able to assign a cause of death in only 9% of adults, 2% of children and 7% of neonatal deaths. The CSMFs obtained from the Tariff method were similar in some cases to those of vital statistics (e.g. cirrhosis), but different in others (e.g. sepsis).ConclusionThe application of VAs in a community sample, analyzed with the Tariff method, allowed assigning a cause of death to most of the cases, with results similar to those of vital statistics for most conditions. This tool can be useful to strengthen the quality of vital statistics.

Comparing tariff and medical assistant assigned causes of death from verbal autopsy interviews in Matlab, Bangladesh: implications for a health and demographic surveillance system

BackgroundDeaths in developing countries often occur outside health facilities, making it extremely difficult to gather reliable cause of death (COD) information. Automated COD assignment using a verbal autopsy instrument (VAI) has been proposed as a reliable and cost-effective alternative to traditional physician-certified verbal autopsy, but its performance is still being evaluated. The purpose of this study was to compare the similarity of diagnosis by Medical Assistants (MA) in the Matlab Health and Demographic Surveillance System (HDSS) with the SmartVA Analyze 1.2 (Tariff 2.0) diagnosis.MethodsThis study took place between January 2011 and April 2014 in Matlab, Bangladesh. MA with 3 years of medical training assigned COD to Matlab residents by reviewing the information collected using the Population Health Metrics Research Consortium (PHMRC) long-form VAI. Smart VA Analyze 1.2 automatically assigned COD using the same questionnaire. COD agreement and cause-specific mortality fractions (CSMFs) were compared for MA and Tariff.ResultsOf the 4969 verbal autopsy cases reviewed, 4328 were adults, 296 were children, and 345 were neonates. Cohen’s kappa was 0.38 (0.36, 0.40) for adults, 0.43 (0.38, 0.49) for children, and 0.27 (0.22, 0.33) for neonates. For adults, the top two COD for MA were stroke (29.6%) and ischemic heart diseases (IHD) (14.2%) and for Tariff these were stroke (32.0%) and IHD (14.0%). For children, the top two COD for MA were drowning (33.5%) and pneumonia (13.2%) and for Tariff these were also drowning (36.8%) and pneumonia (12.4%). For neonates, the top two COD for MA were birth asphyxia (41.2%) and meningitis/sepsis (22.3%) and for Tariff these were birth asphyxia (37.0%) and preterm delivery (30.9%).ConclusionThe CSMFs for Tariff and MA showed very close agreement across all age categories but some differences were observed for neonate preterm delivery and meningitis/sepsis. Given the known advantages of automated methods over physician certified verbal autopsy, the SmartVA software, incorporating the shortened VAI questionnaire and Tariff 2.0, could serve as a cost-effective alternative to Matlab MA to routinely collect and analyze verbal autopsy data in a HDSS to generate essential population level COD data for planning.

A de-identified database of 11,979 verbal autopsy open-ended responses

As part of the Gates Grand Challenge 13, the Population Health Metrics Research Consortium (PHMRC) collected data to enable the development and validation of methods that measure cause-specific mortality in populations with incomplete or inadequate cause of death coding. This work yielded 11,979 verbal autopsy interviews (VAIs). In each, a field interviewer spoke with an individual familiar with the deceased and their final illness, and used a semi-structured questionnaire to collect information about the symptoms of the deceased in their final illness. The VAI collected demographic characteristics, possible risk factors (such as tobacco use), and other potentially contributing characteristics. It also included the open-ended question, “Could you please summarize, or tell us in your own words, any additional information about the illness and/or death of your loved one?” (open narrative). The VAI data were released in a de-identified format in September 2013 through the Global Health Data Exchange, in files that contain verbal autopsies that were collected at six sites in four countries (India, Mexico, Tanzania, and the Philippines). Due to research interest, we have now created redacted versions of the open narratives from the open-ended question of the questionnaire. We hope that this database will be the source of innovations that increase our knowledge about the causes of ill health and, through this knowledge, produce improvements in health for individuals and populations.

Risk profile for drowning deaths in children in the Indian state of Bihar: results from a population-based study

BackgroundWe report on incidence of drowning deaths and related contextual factors in children from a population-based study in the Indian state of Bihar which estimated the causes of death using...

A de-identified database of 11,979 verbal autopsy open-ended responses.

As part of the Gates Grand Challenge 13, the Population Health Metrics Research Consortium (PHMRC) collected data to enable the development and validation of methods that measure cause-specific mortality in populations with incomplete or inadequate cause of death coding. This work yielded 11,979 verbal autopsy interviews (VAIs). In each, a field interviewer spoke with an individual familiar with the deceased and their final illness, and used a semi-structured questionnaire to collect information about the symptoms of the deceased in their final illness. The VAI collected demographic characteristics, possible risk factors (such as tobacco use), and other potentially contributing characteristics. It also included the open-ended question, "Could you please summarize, or tell us in your own words, any additional information about the illness and/or death of your loved one?" (open narrative). The VAI data were released in a de-identified format in September 2013 through the Global Health Data Exchange, in files that contain verbal autopsies that were collected at six sites in four countries (India, Mexico, Tanzania, and the Philippines). Due to research interest, we have now created redacted versions of the open narratives from the open-ended question of the questionnaire. We hope that this database will be the source of innovations that increase our knowledge about the causes of ill health and, through this knowledge, produce improvements in health for individuals and populations.

Identification of factors associated with stillbirth in the Indian state of Bihar using verbal autopsy: A population-based study.

BackgroundIndia was estimated to have the largest numbers of stillbirths globally in 2015, and the Indian government has adopted a target of <10 stillbirths per 1,000 births by 2030 through the India Newborn Action Plan (INAP). The objective of this study was to use verbal autopsy interviews to examine factors associated with stillbirth in the Indian state of Bihar and make recommendations for the INAP to better inform the setting of priorities and actions to reduce stillbirths.Methods and findingsVerbal autopsy interviews were conducted for deaths including stillbirths that occurred from January 2011 to March 2014 in a sample of 109,689 households (87.1% participation) in 1,017 clusters representative of the state of Bihar. The Population Health Metrics Research Consortium shortened verbal autopsy questionnaire was used for each interview, and cause of death was assigned using the SmartVA automated algorithm. A stillbirth was defined as a foetal death with a gestation period of ≥28 weeks wherein the foetus did not show any sign of life. We report on the stillbirth epidemiology and present case studies from the qualitative data on the health provider interface that can be used to improve success of improved, skilled care at birth and delivery interventions. The annualised stillbirth incidence was 21.2 (95% CI 19.7 to 22.6) per 1,000 births, with it being higher in the rural areas. A total of 1,132 stillbirths were identified; 686 (62.2%) were boys, 327 (29.7%) were firstborn, and 760 (68.9%) were delivered at a health facility. Of all the stillbirths, 54.5% were estimated to be antepartum. Only 6,161 (55.9%) of the women reported at least 1 antenatal care visit, and 33% of the women reported not consuming the iron folic acid tablets during pregnancy. Significant differences were seen in delivery-related variables and associated maternal conditions based on the place of delivery and type of stillbirth. Only 6.1% of the women reported having undergone a test to rule out syphilis. For 34.2% of the stillbirths, the possible risk factor for stillbirth was unexplained. For the remaining 65.8% of the women who reported at least 1 complication during the last 3 months of pregnancy, maternal conditions including anaemia, fever during labour, and hypertension accounted for most of the complications. Of importance to note is that the maternal conditions overlapped quite significantly with the other possible underlying risk factors for stillbirth. Obstetrics complications and excessive bleeding during delivery contributed to nearly 30% of the cases as a possible risk factor for stillbirth, highlighting the need for better skilled care during delivery. Of the 5 major themes identified in open narratives, 3 were related to healthcare providers—lack of timely attention, poor skills (knowledge or implementation), and reluctance to deliver a dead baby. The case studies document the circumstances that highlight breakdowns in clinical care around the delivery or missed opportunities that can be used for improving the provision of quality skilled care. The main limitation of these data is that stillbirth is defined based on the gestation period and not based on birth weight; however, this is done in several studies from developing country settings in which birthweight is either not available or accurate.ConclusionsTo our knowledge, this study is among the few large, population-based assessments of stillbirths using verbal autopsy at the state level in India. These findings provide detailed insight into investigating the possible risk factors for stillbirths, as well as insight into the ground-level changes that are needed within the health system to design and implement effective preventive and intervention policies to reduce the burden of stillbirths. As most of the stillbirths are preventable with high-quality, evidence-based interventions delivered before and during pregnancy and during labour and childbirth, it is imperative that with INAP in place, India aspires to document stillbirths in a systematic and standardised manner to bridge the knowledge gap for appropriate actions to reduce stillbirths. We have made several recommendations based on our study that could further strengthen the INAP approach to improve the quality and quantity of stillbirth data to avoid this needless loss of lives.

The paradox of verbal autopsy in cause of death assignment: symptom question unreliability but predictive accuracy.

BackgroundWe believe that it is important that governments understand the reliability of the mortality data which they have at their disposable to guide policy debates. In many instances, verbal autopsy (VA) will be the only source of mortality data for populations, yet little is known about how the accuracy of VA diagnoses is affected by the reliability of the symptom responses. We previously described the effect of the duration of time between death and VA administration on VA validity. In this paper, using the same dataset, we assess the relationship between the reliability and completeness of symptom responses and the reliability and accuracy of cause of death (COD) prediction.MethodsThe study was based on VAs in the Population Health Metrics Research Consortium (PHMRC) VA Validation Dataset from study sites in Bohol and Manila, Philippines and Andhra Pradesh, India. The initial interview was repeated within 3–52 months of death. Question responses were assessed for reliability and completeness between the two survey rounds. COD was predicted by Tariff Method.ResultsA sample of 4226 VAs was collected for 2113 decedents, including 1394 adults, 349 children, and 370 neonates. Mean question reliability was unexpectedly low (kappa = 0.447): 42.5 % of responses positive at the first interview were negative at the second, and 47.9 % of responses positive at the second had been negative at the first. Question reliability was greater for the short form of the PHMRC instrument (kappa = 0.497) and when analyzed at the level of the individual decedent (kappa = 0.610). Reliability at the level of the individual decedent was associated with COD predictive reliability and predictive accuracy.ConclusionsFamilies give coherent accounts of events leading to death but the details vary from interview to interview for the same case. Accounts are accurate but inconsistent; different subsets of symptoms are identified on each occasion. However, there are sufficient accurate and consistent subsets of symptoms to enable the Tariff Method to assign a COD.Questions which contributed most to COD prediction were also the most reliable and consistent across repeat interviews; these have been included in the short form VA questionnaire. Accuracy and reliability of diagnosis for an individual death depend on the quality of interview. This has considerable implications for the progressive roll out of VAs into civil registration and vital statistics (CRVS) systems.Electronic supplementary materialThe online version of this article (doi:10.1186/s12963-016-0104-2) contains supplementary material, which is available to authorized users.

What is the optimal recall period for verbal autopsies? Validation study based on repeat interviews in three populations.

BackgroundOne key contextual feature in Verbal Autopsy (VA) is the time between death and survey administration, or recall period. This study quantified the effect of recall period on VA performance by using a paired dataset in which two VAs were administered for a single decedent.MethodsThis study used information from the Population Health Metrics Research Consortium (PHMRC) Study, which collected VAs for “gold standard” cases where cause of death (COD) was supported by clinical criteria. This study repeated VA interviews within 3–52 months of death in PHMRC study sites in Andhra Pradesh, India, and Bohol and Manila, Philippines. The final dataset included 2113 deaths interviewed twice and with recall periods ranging from 0 to 52 months. COD was assigned by the Tariff method and its accuracy determined by comparison with the gold standard COD.ResultsThe probability of a correct diagnosis of COD decreased by 0.55 % per month in the period after death. Site of data collection and survey module also affected the probability of Tariff Method correctly assigning a COD. The probability of a correct diagnosis in VAs collected 3–11 months after death will, on average, be 95.9 % of that in VAs collected within 3 months of death.ConclusionsThese findings suggest that collecting VAs within 3 months of death may improve the quality of the information collected, taking the need for a period of mourning into account. This study substantiates the WHO recommendation that it is reasonable to collect VAs up to 1 year after death providing it is accepted that probability of a correct diagnosis is likely to decline month by month during this period.Electronic supplementary materialThe online version of this article (doi:10.1186/s12963-016-0105-1) contains supplementary material, which is available to authorized users.

Use of Smartphone for Verbal Autopsy.

Traditionally, verbal autopsies (VA) are collected on paper-based questionnaires and reviewed by physicians for cause of death assignment, it is resource intensive and time consuming. The Population Health Metrics Research Consortium VA questionnaires was made available on an Android-based application and cause of death was derived using the Tariff method. Over one year, all adult deaths occurring in 48 villages in 4 counties were identified and a VA interview was conducted using the smartphone VA application. A total of 507 adult deaths were recorded and VA interviews were conducted. Cardiovascular disease was the leading cause of death (35.3%) followed by injury (14.6%) and neoplasms (13.5%). The total cost of the pilot study was USD28 835 (USD0.42 per capita). The interviewers found use of smartphones to conduct interviews to be easier. The study showed that using a smartphone application for VA interviews was feasible for implementation in rural China.

Validating hierarchical verbal autopsy expert algorithms in a large data set with known causes of death.

BackgroundPhysician assessment historically has been the most common method of analyzing verbal autopsy (VA) data. Recently, the World Health Organization endorsed two automated methods, Tariff 2.0 and InterVA–4, which promise greater objectivity and lower cost. A disadvantage of the Tariff method is that it requires a training data set from a prior validation study, while InterVA relies on clinically specified conditional probabilities. We undertook to validate the hierarchical expert algorithm analysis of VA data, an automated, intuitive, deterministic method that does not require a training data set.MethodsUsing Population Health Metrics Research Consortium study hospital source data, we compared the primary causes of 1629 neonatal and 1456 1–59 month–old child deaths from VA expert algorithms arranged in a hierarchy to their reference standard causes. The expert algorithms were held constant, while five prior and one new “compromise” neonatal hierarchy, and three former child hierarchies were tested. For each comparison, the reference standard data were resampled 1000 times within the range of cause–specific mortality fractions (CSMF) for one of three approximated community scenarios in the 2013 WHO global causes of death, plus one random mortality cause proportions scenario. We utilized CSMF accuracy to assess overall population–level validity, and the absolute difference between VA and reference standard CSMFs to examine particular causes. Chance–corrected concordance (CCC) and Cohen’s kappa were used to evaluate individual–level cause assignment.ResultsOverall CSMF accuracy for the best–performing expert algorithm hierarchy was 0.80 (range 0.57–0.96) for neonatal deaths and 0.76 (0.50–0.97) for child deaths. Performance for particular causes of death varied, with fairly flat estimated CSMF over a range of reference values for several causes. Performance at the individual diagnosis level was also less favorable than that for overall CSMF (neonatal: best CCC = 0.23, range 0.16–0.33; best kappa = 0.29, 0.23–0.35; child: best CCC = 0.40, 0.19–0.45; best kappa = 0.29, 0.07–0.35).ConclusionsExpert algorithms in a hierarchy offer an accessible, automated method for assigning VA causes of death. Overall population–level accuracy is similar to that of more complex machine learning methods, but without need for a training data set from a prior validation study.

A shortened verbal autopsy instrument for use in routine mortality surveillance systems.

BackgroundVerbal autopsy (VA) is recognized as the only feasible alternative to comprehensive medical certification of deaths in settings with no or unreliable vital registration systems. However, a barrier to its use by national registration systems has been the amount of time and cost needed for data collection. Therefore, a short VA instrument (VAI) is needed. In this paper we describe a shortened version of the VAI developed for the Population Health Metrics Research Consortium (PHMRC) Gold Standard Verbal Autopsy Validation Study using a systematic approach.MethodsWe used data from the PHMRC validation study. Using the Tariff 2.0 method, we first established a rank order of individual questions in the PHMRC VAI according to their importance in predicting causes of death. Second, we reduced the size of the instrument by dropping questions in reverse order of their importance. We assessed the predictive performance of the instrument as questions were removed at the individual level by calculating chance-corrected concordance and at the population level with cause-specific mortality fraction (CSMF) accuracy. Finally, the optimum size of the shortened instrument was determined using a first derivative analysis of the decline in performance as the size of the VA instrument decreased for adults, children, and neonates.ResultsThe full PHMRC VAI had 183, 127, and 149 questions for adult, child, and neonatal deaths, respectively. The shortened instrument developed had 109, 69, and 67 questions, respectively, representing a decrease in the total number of questions of 40-55 %. The shortened instrument, with text, showed non-significant declines in CSMF accuracy from the full instrument with text of 0.4 %, 0.0 %, and 0.6 % for the adult, child, and neonatal modules, respectively.ConclusionsWe developed a shortened VAI using a systematic approach, and assessed its performance when administered using hand-held electronic tablets and analyzed using Tariff 2.0. The length of a VA questionnaire was shortened by almost 50 % without a significant drop in performance. The shortened VAI developed reduces the burden of time and resources required for data collection and analysis of cause of death data in civil registration systems.Electronic supplementary materialThe online version of this article (doi:10.1186/s12916-015-0528-8) contains supplementary material, which is available to authorized users.

Improving performance of the Tariff Method for assigning causes of death to verbal autopsies.

BackgroundReliable data on the distribution of causes of death (COD) in a population are fundamental to good public health practice. In the absence of comprehensive medical certification of deaths, the only feasible way to collect essential mortality data is verbal autopsy (VA). The Tariff Method was developed by the Population Health Metrics Research Consortium (PHMRC) to ascertain COD from VA information. Given its potential for improving information about COD, there is interest in refining the method. We describe the further development of the Tariff Method.MethodsThis study uses data from the PHMRC and the National Health and Medical Research Council (NHMRC) of Australia studies. Gold standard clinical diagnostic criteria for hospital deaths were specified for a target cause list. VAs were collected from families using the PHMRC verbal autopsy instrument including health care experience (HCE). The original Tariff Method (Tariff 1.0) was trained using the validated PHMRC database for which VAs had been collected for deaths with hospital records fulfilling the gold standard criteria (validated VAs). In this study, the performance of Tariff 1.0 was tested using VAs from household surveys (community VAs) collected for the PHMRC and NHMRC studies. We then corrected the model to account for the previous observed biases of the model, and Tariff 2.0 was developed. The performance of Tariff 2.0 was measured at individual and population levels using the validated PHMRC database.ResultsFor median chance-corrected concordance (CCC) and mean cause-specific mortality fraction (CSMF) accuracy, and for each of three modules with and without HCE, Tariff 2.0 performs significantly better than the Tariff 1.0, especially in children and neonates. Improvement in CSMF accuracy with HCE was 2.5 %, 7.4 %, and 14.9 % for adults, children, and neonates, respectively, and for median CCC with HCE it was 6.0 %, 13.5 %, and 21.2 %, respectively. Similar levels of improvement are seen in analyses without HCE.ConclusionsTariff 2.0 addresses the main shortcomings of the application of the Tariff Method to analyze data from VAs in community settings. It provides an estimation of COD from VAs with better performance at the individual and population level than the previous version of this method, and it is publicly available for use.Electronic supplementary materialThe online version of this article (doi:10.1186/s12916-015-0527-9) contains supplementary material, which is available to authorized users.

Validating estimates of prevalence of non-communicable diseases based on household surveys: the symptomatic diagnosis study.

BackgroundEasy-to-collect epidemiological information is critical for the more accurate estimation of the prevalence and burden of different non-communicable diseases around the world. Current measurement is restricted by limitations in existing measurement systems in the developing world and the lack of biometry tests for non-communicable diseases. Diagnosis based on self-reported signs and symptoms (“Symptomatic Diagnosis,” or SD) analyzed with computer-based algorithms may be a promising method for collecting timely and reliable information on non-communicable disease prevalence. The objective of this study was to develop and assess the performance of a symptom-based questionnaire to estimate prevalence of non-communicable diseases in low-resource areas.MethodsAs part of the Population Health Metrics Research Consortium study, we collected 1,379 questionnaires in Mexico from individuals who suffered from a non-communicable disease that had been diagnosed with gold standard diagnostic criteria or individuals who did not suffer from any of the 10 target conditions. To make the diagnosis of non-communicable diseases, we selected the Tariff method, a technique developed for verbal autopsy cause of death calculation. We assessed the performance of this instrument and analytical techniques at the individual and population levels.ResultsThe questionnaire revealed that the information on health care experience retrieved achieved 66.1% (95% uncertainty interval [UI], 65.6–66.5%) chance corrected concordance with true diagnosis of non-communicable diseases using health care experience and 0.826 (95% UI, 0.818–0.834) accuracy in its ability to calculate fractions of different causes. SD is also capable of outperforming the current estimation techniques for conditions estimated by questionnaire-based methods.ConclusionsSD is a viable method for producing estimates of the prevalence of non-communicable diseases in areas with low health information infrastructure. This technology can provide higher-resolution prevalence data, more flexible data collection, and potentially individual diagnoses for certain conditions.Electronic supplementary materialThe online version of this article (doi:10.1186/s12916-014-0245-8) contains supplementary material, which is available to authorized users.

Usefulness of the Population Health Metrics Research Consortium gold standard verbal autopsy data for general verbal autopsy methods

BackgroundVerbal Autopsy (VA) is widely viewed as the only immediate strategy for registering cause of death in much of Africa and Asia, where routine physician certification of deaths is not widely practiced. VA involves a lay interview with family or friends after a death, to record essential details of the circumstances. These data can then be processed automatically to arrive at standardized cause of death information.MethodsThe Population Health Metrics Research Consortium (PHMRC) undertook a study at six tertiary hospitals in low- and middle-income countries which documented over 12,000 deaths clinically and subsequently undertook VA interviews. This dataset, now in the public domain, was compared with the WHO 2012 VA standard and the InterVA-4 interpretative model.ResultsThe PHMRC data covered 70% of the WHO 2012 VA input indicators, and categorized cause of death according to PHMRC definitions. After eliminating some problematic or incomplete records, 11,984 VAs were compared. Some of the PHMRC cause definitions, such as ‘preterm delivery’, differed substantially from the International Classification of Diseases, version 10 equivalent. There were some appreciable inconsistencies between the hospital and VA data, including 20% of the hospital maternal deaths being described as non-pregnant in the VA data. A high proportion of VA cases (66%) reported respiratory symptoms, but only 18% of assigned hospital causes were respiratory-related. Despite these issues, the concordance correlation coefficient between hospital and InterVA-4 cause of death categories was 0.61.ConclusionsThe PHMRC dataset is a valuable reference source for VA methods, but has to be interpreted with care. Inherently inconsistent cases should not be included when using these data to build other VA models. Conversely, models built from these data should be independently evaluated. It is important to distinguish between the internal and external validity of VA models. The effects of using tertiary hospital data, rather than the more usual application of VA to all-community deaths, are hard to evaluate. However, it would still be of value for VA method development to have further studies of population-based post-mortem examinations.