Evaluation Of Health Interventions Research Articles

A principles-focused evaluation of community mental health short-term case management

Principles-focused evaluations allow evaluators to appraise each principle that guides an organization or program. This study completed a principles-focused evaluation of a new community mental health intervention called Short Term Case Management (STCM) in Toronto, Canada. STCM is a time limited intervention for clients to address unmet needs and personalized goals over 3 months. Findings showcase that a principles-focused evaluation, assessing whether program principles are guiding, useful, inspiring, developmental and/or evaluable (GUIDE), is a practical formative evaluation approach. Specifically, offering an understanding of a novel intervention, including its key components of assessment and planning, support plan implementation and evaluation and care transitions. Findings also highlight that STCM may work best for those clients ready to participate in achieving their own goals. Future research should explore how best to apply the GUIDE framework to complex interventions, including multiple principles, to increase evaluation feasibility and focus.

A Public Health Reset Through Contractualism

A Public Health Reset Through Contractualism

InterRAI Subjective Quality of Life Scale for Mental Health and Addiction Settings: A Self-Reported Measure Developed From a Multi-National Study.

Background: Measuring Quality of Life (QoL) in mental health using self-reported items is important for evaluating the quality of service and understanding the person's experience of the care received.Objective: The aim of this research was to develop and validate a self-reported QoL instrument for inpatient and community mental health settings.Methods: Data were collected from diverse research sites in Canada, Belgium, Russia, Finland, Brazil, and Hong Kong, using the 37-item interRAI Quality of Life Survey for Mental Health and Addictions. The survey was administrated to 2,218 participants from inpatient and community mental health settings, assisted living, and the general community. We randomly divided the sample into a training and a test sample (70 and 30%, respectively). We conducted principal component analysis (PCA) and exploratory factor analysis (EFA) using the training sample to identify potential factor structure. Confirmatory factor analysis (CFA) models were then fitted to finalize and externally validate the measurement model using training and test data, respectively.Results: PCA, EFA, and CFA of the training sample collectively suggested a 23-item scale measuring four latent constructs: well-being and hope (8 items), relationship (7 items), support (5 items), and activity (3 items). This model was supported by the CFA of the test sample. The goodness-of-fit statistics root mean square error, comparative fit index and Tucker-Lewis index were 0.03, 1.00, and 0.99, respectively. Estimated Cronbach's alpha based on the test data was 0.92. Raw Cronbach's alpha values for the subscales were 0.86 for well-being and hope, 0.86 for relationship, 0.69 for support, and 0.72 for activity.Conclusions: The interRAI SQoL-MHA scale is a valid instrument to measure QoL in mental health settings. The instrument will support the evaluation of the quality of care and can also be used for future research to produce SQoL-MHA values on a quality adjusted-life-year scale, facilitating the evaluation of various mental health interventions.

A Scoping Review of the Use of Indigenous Food Sovereignty Principles for Intervention and Future Directions.

ABSTRACTIndigenous food sovereignty (IFS) represents a community-led movement with potential to reduce health inequities, but no scoping review of the impact of taking an IFS approach on intervention research has been conducted. This review sought to: 1) describe intervention studies that employ IFS principles, and 2) describe the impact of studies using IFS principles on food access, eating patterns, diet quality, physical activity, and health. Through a literature review, 4 IFS principles were identified: 1) community ownership, 2) inclusion of traditional food knowledge, 3) inclusion and promotion of cultural foods, and 4) environmental/intervention sustainability. Twenty intervention studies published between January 1, 2000 and February 5, 2020 were included. Most of the studies that scored high in IFS principles saw a positive impact on diet. This review found evidence supporting the value of IFS principles in the development, implementation, and evaluation of health interventions for Indigenous communities.

PMH37 Do Education Costs Matter? A Systematic Literature Review of the Economic Impact of Psychosocial Problems on the Education Sector.

Psychosocial problems lead to costs in the health care sector but also in the education sector among other. As psychosocial problems develop during the critical period of establishing educational trajectories, education costs are particularly relevant in the context of psychosocial problems among children and adolescents. This study aimed to gain insights into the methods used for the inclusion of education costs in health economics studies and into the proportion of the education costs in relation the total costs associated with a condition or an intervention. PubMed, Embase, SSCI, CINAHL, PsycINFO, ERIC, and Econlit were systematically searched for economic evaluations of mental health, psychosocial, and educational interventions and cost-of-illness studies of mental, behavioral and neurodevelopmental disorders, conducted from a societal perspective in the population of children and adolescents. Of 6,248 screened titles and abstracts, 41 articles were included in the analysis. The most common cost items were “special education”, “school absence”, and “additional educational support”. A variety of methods were employed for the identification, measurement and/or valuation of education costs. The proportion of education costs to the total costs of condition/intervention ranged from 0% to 67% with the mean being 18.6%. Since education costs can constitute a significant proportion of the total costs of an intervention or condition, including them in health economics studies might be important in informing optimal resource allocation decisions. Although various methods are available for including education costs in health economics studies, further research is needed to develop evidence-based methods for producing comparable estimates.

MO845EUROQOL EQ-5D-3L IN DIALYSIS POPULATION IN COLOMBIA

Abstract Background and Aims The EuroQol-5D (EQ-5D-3L) is a generic questionnaire to evaluate the quality of life related to health, and simultaneously facilitates the obtaining of preference values (utilities) In the economic evaluation of health interventions. Quality-adjusted life year (QALY) is a commonly used metric. Our objective is to evaluate the quality of life (QoL) and obtaining of utilities according the dialytic treatment. Method This was an observational, multicentre cross-sectional study. Prevalent patients on dialysis for at least 90 days, older than 18, at the Baxter Renal Care Services Colombia were included between August 2, 2019, to December 11, 2020. Socio-demographic and clinical characteristics of all patients were summarized descriptively, additionally, all categorical variables were compared with Pearson’s χ2 test or Fisher’s exact test and continuous variables were analysed with Student’s t-test or Mann Whitney U test. The difference in the QoL by domains will be evaluated using Pearson's χ2 test or Fisher’s exact test. The EQ VAS provides a quantitative measure of the patient’s perception of their overall health. Results A total 254 patients were evaluated, the median age was 55 years, 55% were men. At enrolment the median dialysis vintage 3.2 years, and 63% were in peritoneal dialysis treatment (Table 1). We did not find differences in the proportions by dialysis treatment when exploring each of the 5 dimensions of the QoL (Table 2). We observed that the patient´s perception of their overall health was similar median EQ VAS score 80 in HD and EQ VAS score 80 in PD (Figure 1). We did not find statistically significant differences in health status 11111, which represents not having problems in any dimension 46.8% versus 45.6%, HD and PD respectively, p-value= 0.855. Conclusion The quality of life reported by the patients and health status in haemodialysis was similar to peritoneal dialysis.

Music for your mental health? The development and evaluation of a group mental health intervention in subacute rehabilitation

Objectives: Short-stay residents of nursing homes experience high rates of mental health (MH) distress compared to community dwelling counterparts, yet MH interventions are difficult to implement and sustain. We modified a music therapy framework to Effective Music in Psychotherapy. Using the modified model, we integrated music listening into MH group intervention and evaluated MH outcomes. This pilot study reports the development and evaluation of the Mental Health and Music Group for short-stay nursing homes residents. Method: The group was developed and refined to be non-sequential and non-cumulative, specific to the needs of short-stay nursing home residents. Pre-/post-session ratings examined affect, quality of life, and pain. Leaders monitored engagement across and between sessions. Qualitative interviews were conducted with a selection of attendees. Results: Findings indicated decreases in irritation and worry, and increases in mood, energy, and self-as-a-whole from pre- to post-session. There were no changes in pain, perception of physical health or life-as-a-whole, energetic, sad, or happy affect, or differences in engagement. Qualitative interviews demonstrated benefits of group modality and music to group cohesion and highlighted the relevance of music for mental health outcomes. Conclusion: The Mental Health and Music group was designed around a framework of Effective Music in Psychotherapy, for short-stay nursing home settings, and demonstrated promising results. Future research can solidify the interventions generalizeability to medical and rehabilitation settings addressing the specific population needs and preferences. Supplemental data for this article is available online at https://doi.org/10.1080/13607863.2021.1935463 .

Improving the Usability and Safety of Digital Health Systems: The Role of Predictive Human-Computer Interaction Modeling.

In this paper, we describe techniques for predictive modeling of human-computer interaction (HCI) and discuss how they could be used in the development and evaluation of user interfaces for digital health systems such as electronic health record systems. Predictive HCI modeling has the potential to improve the generalizability of usability evaluations of digital health interventions beyond specific contexts, especially when integrated with models of distributed cognition and higher-level sociotechnical frameworks. Evidence generated from building and testing HCI models of the user interface (UI) components for different types of digital health interventions could be valuable for informing evidence-based UI design guidelines to support the development of safer and more effective UIs for digital health interventions.

A Comparison of the SF-6Dv2 and SF-6D UK Utility Values in a Mixed Patient and Healthy Population.

BackgroundThis paper describes the first evaluation of the construct validity and performance of the newly developed preference-based measure of health, the SF-6D version 2 (SF-6Dv2).MethodUtilising data from the Multi-Instrument Comparison (MIC) project (n = 7932), we explored the descriptive differences in utility values between the SF-6Dv2 and the SF-6D and evaluated the known group validity of both measures by testing the statistical significance of differences in utility values and calculating the effect sizes across known groups. The convergent validity of the SF-6Dv2 was explored by examining whether the SF-6Dv2 is related to alternative validated measures, including the EQ-5D-5L and AQoL-8D.ResultsDifferences between the utility values of the SF-6Dv2 and SF-6D were evident; utilities were generally lower for the SF-6Dv2, with larger standard deviations resulting in larger absolute differences between groups. The SF-6Dv2 performed well in terms of known-group validity and successfully distinguished disease severity and between the disease and healthy groups, outperforming the SF-6D in some but not all groups. Convergent validity analyses indicated strong associations between the SF-6Dv2 and the SF-6D, EQ-5D-5L and AQoL-8D utilities.ConclusionsThe psychometric performance of the SF-6Dv2 is favourable with respect to known group validity and convergent validity, but does not seem to have improved, compared with the SF-6D. However, the new method of valuation has had a substantial impact on the size of absolute differences in utility values, which could impact quality-adjusted life-year results. The economic evaluation of health interventions may therefore be influenced by the choice of the SF-6Dv2 over the SF-6D.Supplementary InformationThe online version contains supplementary material available at 10.1007/s40273-021-01033-6.

Challenges in developing capability measures for children and young people for use in the economic evaluation of health and care interventions.

Methods for measuring outcomes suitable for economic evaluations of health and care interventions have primarily focused on adults. The validity of such methods for children and young people is questionable in areas including the outcome domains measured and how they are measured and valued, with most existing measures narrowly focusing on health. Novel methods for assessing benefits beyond health by focusing on a person's capability have also concentrated on adults to date. This paper aims to set out the rationale for capability measures in children and young people. It argues for the need to expand the evaluative space beyond health functioning towards broader capabilities, with children and young people playing an integral role in capability measure development. Drawing from existing literature, specific challenges related to the identification, measurement, and valuation of capabilities in children and young people are also discussed. Finally, the practical implications for conducting economic evaluation when measuring and valuing capabilities at different stages across the life-course are illustrated. We develop an alternative framework based on conceiving capabilities as evolving across the life-course. This framework may also be helpful in thinking about how to model health outcomes across the life-course.

Residents’ Perceptions of a Community-Led Intervention on Health, Well-Being, and Community Inclusion Through Photovoice

Community-centered approaches can be effective ways to engage communities and improve their health and well-being. The Grange is a community-led, multifaceted, and dynamic intervention incorporating a community hub and garden, that took place in a small area of the North-West of England, characterized by high levels of deprivation and poor health. Activities have been defined, developed, and supported by residents to meet locally defined needs. This study used photovoice methods to explore residents’ perceptions and experiences of this community-led intervention and any perceived impact on health, well-being, and community inclusion. Through photographs, semistructured interviews, a focus group discussion, and an exhibition, this study engaged intensively and creatively with a group of six residents. They identified positive and negative aspects related to The Grange and suggested recommendations that were directly communicated to policy makers during the photo-exhibition event. Participants reflected on various activities such as the community garden and the community shop. They also reflected on contextual factors and suggested that the culture of inclusivity and friendships associated with The Grange were more important to them than specific activities. This study demonstrated the value of using photovoice to (a) explore residents’ perceptions of community led interventions; and (b) meaningfully engage residents living in areas with high levels of deprivation. Public health practitioners should consider the use of photovoice (a) in the evaluation of health interventions that take place in a complex and changing context, and (b) as a powerful tool to engage with members of the community, especially traditionally disadvantaged groups, to ensure that engagement about health, well-being, and social inclusion is meaningful.

Measuring capability wellbeing in adults at different stages of life for use in economic evaluation of health and care interventions: a qualitative investigation in people requiring kidney care

PurposeCapability wellbeing measures, such as the ICECAP measures, have been proposed for use in economic evaluations to capture broader outcomes of health and care interventions. The ICECAP measures have been developed to reflect capabilities at different stages of life. Some patient groups include patients of different ages and at different stages of life, so it is not always apparent which ICECAP measure is most relevant. This study explores the impact of age and life stage on completion, where both ICECAP-A and ICECAP-O were completed by the same patient.MethodsA think-aloud study, and an associated semi-structured interview were conducted with people receiving kidney care as a renal outpatient, kidney transplant outpatient, or through receiving facility-based haemodialysis. Qualitative analysis focused on (1) differences in responses across measures by individuals, where attributes had conceptual overlap, (2) key factors in self-reported capability levels, and (3) measure preference.ResultsThirty participants were included in the study, with a mix of older and younger adults. Attributes with similar wording across measures produced similar responses compared to attributes where wording differed. Age and health were key factors for self-reported capability levels. ICECAP-A was slightly preferred overall, including by older adults.ConclusionThis study suggests use of ICECAP-A in patients with certain chronic health conditions that include a mix of adults across the life course. This study highlights the importance of considering the stage of life when using capability measures and in economic evaluations of health and care interventions more generally.

Involvement of Indigenous young people in the design and evaluation of digital mental health interventions: a scoping review protocol

BackgroundIndigenous young people worldwide are at greater risk of developing mental health concerns due to ongoing inequity and disadvantage. Digital mental health (dMH) interventions are identified as a potential approach to improving access to mental health treatment for Indigenous youth. Although involvement in the development and evaluation of dMH resources is widely recommended, there is limited evidence to guide engagement of Indigenous young people in these processes. This scoping review aims to examine the methods used to involve Indigenous young people in the development or evaluation of dMH interventions.MethodsArticles published in English, involving Indigenous young people (aged 10–24 years) in the development or evaluation of dMH interventions, originating from Australia, New Zealand, Canada and the USA will be eligible for inclusion. PubMed, Scopus and EBSCOhost databases (Academic Search Premiere, Computer and Applied Science complete, CINAHL, MEDLINE, APA PsychArticles, Psychology and Behavioural Sciences collection, APA PsychInfo) will be searched to identify eligible articles (from January 1990 onwards). Infomit and Google Scholar (limited to 200 results) will be searched for grey literature. Two reviewers will independently screen citations, abstracts and full-text articles. Study methods, methodologies, dMH intervention details, participant information and engagement, and dissemination methods will be extracted, analysed (utilising content analysis), and qualitatively assessed for alignment with best practice ethical guidelines for undertaking Indigenous health research. A narrative summary of findings will be presented. Reporting will follow the Consolidated Criteria for Strengthening Reporting of Health Research involving Indigenous peoples (CONSIDER) and the Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for scoping reviews (PRISMA-ScR) guidelines.DiscussionTo date, there are no reviews which analyse engagement of Indigenous young people in the development and evaluation of dMH interventions. This review will appraise alignment of current practice with best practice guidelines to inform future research. It will highlight appropriate methods for the engagement of young people in study processes, providing guidance for health practitioners, policy makers, and researchers working in the field of Indigenous youth and dMH.Systematic review registrationOpen Science Framework (osf.io/2nkc6).

Neutral theory: applicability and neutrality of using generic health-related quality of life tools in diseases or conditions where specific tools are available

BackgroundHealth-related quality of life (HRQoL) tools are limited by the indicators included in the construct and variation in interpretation by different researchers. Neutral Theory describes the ideal construct that includes all relevant indicators and, therefore, complete accuracy, or neutrality. Neutral Theory can thereby provide the framework to develop or test constructs. To assess the application of Neutral Theory, the neutrality of generic tools (SF-36 and EQ-5D) at measuring HRQoL was compared to disease/condition-specific tools, with the latter considered surrogates for the Neutral construct.MethodsFull descriptions of all disease/condition-specific HRQoL tools published on PubMed (to 01-Jul-19) were sourced. For each tool, the number of items with and without a direct match within the SF-36 and EQ-5D was recorded and the sensitivity/specificity calculated.ResultsThe SF-36 and EQ-5D did not achieve a sensitivity/specificity both > 50% against any of the 163 disease/condition-specific tools identified. At 20% prevalence of poor HRQoL, the false positive rate (FPR) was > 75% for all but two tools against the SF-36 and six tools against the EQ-5D. Increasing poor HRQoL to 80%, 47 tools for the SF-36 and 48 tools for the EQ-5D had a FPR < 50%. For rare disease tools (< 1/2000 population; n = 17), sensitivity/specificity ranged from 0 to 40%/5–31% for the SF-36 and 0–22%/29–100% for the EQ-5D. For non-rare (n = 75) and symptom-specific tools (n = 71) sensitivity/specificity was: 0–100%/0–100% (SF-36) and 0–50%/0–100% (EQ-5D); and 0–60%/0–19% (SF-36) and 0–25%/0–100% (EQ-5D), respectively. No concordance was recorded for 18% (2/11) of results from studies of rare disease tools versus the SF-36 (no data vs EQ-5D). For non-rare, disease-specific tools, results were discordant for 30% (25/84) and 35% (23/65) of studies against the SF-36 and EQ-5D, respectively. For symptom-specific tools, corresponding results were 36% (24/66) and 16% (5/31).ConclusionsGeneric HRQoL tools appear poorly correlated with disease/condition-specific tools, which indicates that adoption of Neutral Theory in the development and assessment of HRQoL tools could improve their relevance, accuracy, and utility in economic evaluations of health interventions.

P10 A systematic review of current literature on interventions among first-year university students and how this may influence current policy on child obesity strategies within the UK

BackgroundWith the steady rise of obesity within the UK, the Government has been increasingly developing policy to curb these figures. Great focus has been put on early implementation of a wide range of interventions in educational settings, but results show little improvement.AimTo evaluate health interventions amongst first-year university students with the intention of influencing current policy on obesity prevention amongst children.MethodsA literature search, utilising PRISMA reporting guidelines, was carried out on health intervention studies that focus on first-year university students. To formulate the search strategy, UK obesity policy was input into a PICO Framework.ResultsEight studies met the inclusion/exclusion criteria. Results showed that when measuring weight-related outcomes, no significant improvement was found; secondary outcomes, especially those regarding physical activity demonstrated greater benefit but the link to obesity prevention proved to be relatively weak.DiscussionDue to differing demographics between where the studies were based as well as cultural differences regarding views on leisure and diet led to lack of applicability to UK students.Environmental factors between compulsory and higher education must also be considered as it may be a case of creating new strategies rather than re-applying a pre-established policy.Ultimately, the lack of comparable studies reflects on the lack of understanding of the overall issue of childhood obesity and thusly its prevention.ConclusionWith evidence-based policy increasing in popularity, it is questionable as to whether the target population for child obesity strategies could widen to include first-year university students.

QALYs for COVID-19: A Comparison of US EQ-5D-5L Value Sets.

BackgroundIn economic evaluations, quality-adjusted life-years (QALYs) can serve as a unit of measurement for disease burden. Obtaining QALY values for COVID-19 presents a challenge owing to the availability of two US EQ-5D-5L value sets and the potentially asymptomatic presentation of the disease. The first value set was completed allowing for the discounting of future health outcomes while the second value set is undiscounted.ObjectiveThe objective of this study was to compare the distribution of QALY values using a national survey and the two published value sets; and to estimate the association between COVID-19 outcomes and QALY losses.MethodsBetween 9 and 11 November, 2020, 1153 US adults completed the EQ-5D-5L instrument (five items and a visual analog scale) as well as self-reported their demographics, COVID-19 symptoms, and memberships to populations that are at risk of COVID-19 infection. The two US value sets were applied to the EQ-5D-5L responses to produce QALY values. We estimated the mean QALYs by visual analog scale decile and a generalized linear model of COVID-19 outcomes.ResultsThe discounted values are higher than the undiscounted values for each visual analog scale decile owing to methodological differences. Persons at increased risk, with a fever in the past day, and with one or more other symptoms have significantly greater QALY losses (p < 0.01). Overall, non-institutionalized individuals at risk of symptomatic clinical COVID-19 equal 0.68 for the 2016 value set (95% confidence interval 0.49–0.87) and 0.10 for the 2017 value set (95% confidence interval − 0.31 to 0.51) QALYs.ConclusionsMultiple studies have shown that decision makers discount future health outcomes, which increase QALY values. This study confronts the practical implications of these methodological advances for use in COVID-19 economic evaluations. Health economists will be able to use the QALY values in this study to better evaluate health interventions against COVID-19.Supplementary InformationThe online version contains supplementary material available at 10.1007/s40271-021-00509-z.

Digital Mental Health Challenges and the Horizon Ahead for Solutions.

The demand outstripping supply of mental health resources during the COVID-19 pandemic presents opportunities for digital technology tools to fill this new gap and, in the process, demonstrate capabilities to increase their effectiveness and efficiency. However, technology-enabled services have faced challenges in being sustainably implemented despite showing promising outcomes in efficacy trials since the early 2000s. The ongoing failure of these implementations has been addressed in reconceptualized models and frameworks, along with various efforts to branch out among disparate developers and clinical researchers to provide them with a key for furthering evaluative research. However, the limitations of traditional research methods in dealing with the complexities of mental health care warrant a diversified approach. The crux of the challenges of digital mental health implementation is the efficacy and evaluation of existing studies. Web-based interventions are increasingly used during the pandemic, allowing for affordable access to psychological therapies. However, a lagging infrastructure and skill base has limited the application of digital solutions in mental health care. Methodologies need to be converged owing to the rapid development of digital technologies that have outpaced the evaluation of rigorous digital mental health interventions and strategies to prevent mental illness. The functions and implications of human-computer interaction require a better understanding to overcome engagement barriers, especially with predictive technologies. Explainable artificial intelligence is being incorporated into digital mental health implementation to obtain positive and responsible outcomes. Investment in digital platforms and associated apps for real-time screening, tracking, and treatment offer the promise of cost-effectiveness in vulnerable populations. Although machine learning has been limited by study conduct and reporting methods, the increasing use of unstructured data has strengthened its potential. Early evidence suggests that the advantages outweigh the disadvantages of incrementing such technology. The limitations of an evidence-based approach require better integration of decision support tools to guide policymakers with digital mental health implementation. There is a complex range of issues with effectiveness, equity, access, and ethics (eg, privacy, confidentiality, fairness, transparency, reproducibility, and accountability), which warrant resolution. Evidence-informed policies, development of eminent digital products and services, and skills to use and maintain these solutions are required. Studies need to focus on developing digital platforms with explainable artificial intelligence–based apps to enhance resilience and guide the treatment decisions of mental health practitioners. Investments in digital mental health should ensure their safety and workability. End users should encourage the use of innovative methods to encourage developers to effectively evaluate their products and services and to render them a worthwhile investment. Technology-enabled services in a hybrid model of care are most likely to be effective (eg, specialists using these services among vulnerable, at-risk populations but not severe cases of mental ill health).

Voice-Based Conversational Agents for the Prevention and Management of Chronic and Mental Health Conditions: Systematic Literature Review.

BackgroundChronic and mental health conditions are increasingly prevalent worldwide. As devices in our everyday lives offer more and more voice-based self-service, voice-based conversational agents (VCAs) have the potential to support the prevention and management of these conditions in a scalable manner. However, evidence on VCAs dedicated to the prevention and management of chronic and mental health conditions is unclear.ObjectiveThis study provides a better understanding of the current methods used in the evaluation of health interventions for the prevention and management of chronic and mental health conditions delivered through VCAs.MethodsWe conducted a systematic literature review using PubMed MEDLINE, Embase, PsycINFO, Scopus, and Web of Science databases. We included primary research involving the prevention or management of chronic or mental health conditions through a VCA and reporting an empirical evaluation of the system either in terms of system accuracy, technology acceptance, or both. A total of 2 independent reviewers conducted the screening and data extraction, and agreement between them was measured using Cohen kappa. A narrative approach was used to synthesize the selected records.ResultsOf 7170 prescreened papers, 12 met the inclusion criteria. All studies were nonexperimental. The VCAs provided behavioral support (n=5), health monitoring services (n=3), or both (n=4). The interventions were delivered via smartphones (n=5), tablets (n=2), or smart speakers (n=3). In 2 cases, no device was specified. A total of 3 VCAs targeted cancer, whereas 2 VCAs targeted diabetes and heart failure. The other VCAs targeted hearing impairment, asthma, Parkinson disease, dementia, autism, intellectual disability, and depression. The majority of the studies (n=7) assessed technology acceptance, but only few studies (n=3) used validated instruments. Half of the studies (n=6) reported either performance measures on speech recognition or on the ability of VCAs to respond to health-related queries. Only a minority of the studies (n=2) reported behavioral measures or a measure of attitudes toward intervention-targeted health behavior. Moreover, only a minority of studies (n=4) reported controlling for participants’ previous experience with technology. Finally, risk bias varied markedly.ConclusionsThe heterogeneity in the methods, the limited number of studies identified, and the high risk of bias show that research on VCAs for chronic and mental health conditions is still in its infancy. Although the results of system accuracy and technology acceptance are encouraging, there is still a need to establish more conclusive evidence on the efficacy of VCAs for the prevention and management of chronic and mental health conditions, both in absolute terms and in comparison with standard health care.

Outcomes reported in evaluations of programs designed to improve health in Indigenous people.

To assess the outcomes reported and measured in evaluations of complex health interventions in Indigenous communities. We searched all publications indexed in MEDLINE, PreMEDLINE, EMBASE, PsycINFO, EconLit, and CINAHL until January 2020 and reference lists from included papers were hand-searched for additional articles. Systematic review. We included all primary studies, published in peer-reviewed journals, where the main objective was to evaluate a complex health intervention developed specifically for an Indigenous community residing in a high-income country. Only studies published in English were included. Quantitative and qualitative data were extracted and summarized. Of the 3523 publications retrieved, 62 evaluation studies were included from Australia, the United States, Canada, and New Zealand. Most studies involved less than 100 participants and were mainly adults. We identified outcomes across 13 domains: clinical, behavioral, process-related, economic, quality of life, knowledge/awareness, social, empowerment, access, environmental, attitude, trust, and community. Evaluations using quantitative methods primarily measured outcomes from the clinical and behavioral domains, while the outcomes reported in the qualitative studies were mostly from the process-related and empowerment domains. The outcomes from qualitative evaluations, which better reflect the impact of the intervention on participant health, remain different from the outcomes routinely measured in quantitative evaluations. Measuring the outcomes from qualitative evaluations alongside outcomes from quantitative evaluations could result in more relevant evaluations to inform decision making in Indigenous health.

Stakeholder perspectives on the implementation and impact of Indigenous health interventions: A systematic review of qualitative studies.

BackgroundEvaluations of health interventions for Indigenous peoples rarely report outcomes that reflect participant and community perspectives of their experiences. Inclusion of such data may provide a fuller picture of the impact of health programmes and improve the usefulness of evaluation assessments.AimTo describe stakeholder perspectives and experiences of the implementation and impact of Indigenous health programmes.MethodsWe conducted a systematic review of qualitative studies evaluating complex health interventions designed for Indigenous communities in high‐income countries. We searched 6 electronic databases (through to January 2020): MEDLINE, PreMEDLINE, Embase, PsycINFO, EconLit and CINAHL and hand‐searched reference lists of relevant articles.ResultsFrom 28 studies involving 677 stakeholders (mostly clinical staff and participants), six main themes were identified: enabling engagement, regaining control of health, improving social health and belonging, preserving community and culture, cultivating hope for a better life, and threats to long‐term programme viability.ConclusionThe prominence of social, emotional and spiritual well‐being as important aspects of the health journey for participants in this review highlights the need to reframe evaluations of health programmes implemented in Indigenous communities away from assessments that focus on commonly used biomedical measures. Evaluators, in consultation with the community, should consistently assess the capacity of health professionals to meet community needs and expectations throughout the life of the programme. Evaluations that include qualitative data on participant and community‐level outcomes can improve decision‐makers' understanding of the impact that health programmes have on communities.Patient or public contributionThis paper is a review of evaluation studies and did not involve patients or the public.