Pragmatic Measures Research Articles

Persistence as a Robust Indicator of Medication Adherence-Related Quality and Performance.

Medication adherence is a priority for health systems worldwide and is widely recognised as a key component of quality of care for disease management. Adherence-related indicators were rarely explicitly included in national health policy agendas. One barrier is the lack of standardised adherence terminology and of routine measures of adherence in clinical practice. This paper discusses the possibility of developing adherence-related performance indicators highlighting the value of measuring persistence as a robust indicator of quality of care. To standardise adherence and persistence-related terminology allowing for benchmarking of adherence strategies, the European Ascertaining Barriers for Compliance (ABC) project proposed a Taxonomy of Adherence in 2012 consisting of three components: initiation, implementation, discontinuation. Persistence, which immediately precedes discontinuation, is a key element of taxonomy, which could capture adherence chronology allowing the examination of patterns of medication-taking behaviour. Advances in eHealth and Information Communication Technology (ICT) could play a major role in providing necessary structures to develop persistence indicators. We propose measuring persistence as an informative and pragmatic measure of medication-taking behaviour. Our view is to develop quality and performance indicators of persistence, which requires investing in ICT solutions enabling healthcare providers to review complete information on patients’ medication-taking patterns, as well as clinical and health outcomes.

A Brief Measure of Work Environment for Human Service Organizations

ABSTRACT Organizational characteristics are important predictors of workplace outcomes, but the length and complexity of validated instruments restrict their use in research and practice. This study tested a brief Measure of Work Environment (MWE) using data from 718 child welfare and 349 home visiting professionals. Results confirmed a three-factor structure: group cohesion, leadership support, and organizational justice. The MWE demonstrated acceptable internal consistency (α = .87) and was significantly associated with job satisfaction, burnout, and intent to leave. The MWE is a promising, pragmatic measure of organizational environment for implementation research and program improvement.

Defining Data Quality in Bibliographic and Authority Records: A Case Study of the COBISS.SI System

Various studies have shown that low data quality in bibliographic and authority records is one of the main obstacles in data management. In order to evaluate and improve data quality in the Slovenian union bibliographic database (COBIB.SI), a pragmatic dimension-based data quality measurement and evaluation case study was carried out in 2019. The article presents the set-up and application of a methodological framework that was used for the definition and measurement of bibliographic and authority records quality in COBIB.SI. The results of the case study and the framework’s applicability in data quality measurement and evaluation, as well as in data profiling that would benefit further development of COBISS + catalog, are also discussed.

Assess, Plan, Do, Evaluate, and Report: Iterative Cycle to Remove Academic Control of a Community-Based Physical Activity Program.

Purpose and ObjectivesResponsive methods and measures are needed to bridge research to practice and address public health issues, such as older adults’ need for multicomponent physical activity. The objective of this study was to detail the longitudinal, quasi-experimental work that spans 5 years to describe outcomes across RE-AIM (reach, effectiveness, adoption, implementation, and maintenance) dimensions of integrating a physical activity intervention for older adults into the Cooperative Extension System through the assess, plan, do, evaluate, report (APDER) cycle.Intervention ApproachThe participant-level intervention is Lifelong Improvements through Fitness Together (LIFT), an 8-week, group dynamics-based, strength-training program with 16 in-person sessions. The implementation intervention applies the iterative APDER cycle based on feedback for each dimension of RE-AIM. Each year, the APDER cycle was used to embed data collection procedures at the instructor and participant level to reveal the next evolution of the program.Evaluation MethodsEach evolution of LIFT was measured through a pretest and posttest quasi-experimental design. Data were collected on each RE-AIM dimension through participant surveys and functional fitness assessments, number and representativeness of trainees, and process evaluation.ResultsOverall, LIFT was expanded to 4 states with 275 instructors, reaching 816 older adults; consistently improved functional fitness outcome measures; demonstrated strong program adherence; and was seen as feasible and enjoyable by instructors and participants. LIFT is now undergoing adaptations for virtual delivery as well as updating the exercise protocol to introduce yoga postures that target flexibility and balance.Implications for Public HealthOverall, ongoing adaptations were necessary to ensure the program continued to fit the mission, values, and resources of the delivery system. Public health implications to support the need for ongoing adaptation include embedding pragmatic measures of adaptations and RE-AIM into standard evaluation pathways and using iterative APDER cycles.

Mom Expo: Bridging the Gap Between Local Women and Perinatal Resources

The purpose of this project was to describe the implementation of a perinatal health fair intended to connect local women to holistic resources. Researchers used participatory strategies to develop the health fair with local women and perinatal educators. Researchers evaluated the health fair using pragmatic measures based on the (Reach, Effectiveness, Adoption, Implementation, Maintenance) framework. Forty-two attendees were reached and 23 educators hosted booths and educational sessions. Feedback indicated strong enthusiasm for future similar events. Nearly three quarters of the time spent implementing the health fair was devoted to building relationships within the community. Overall, this project provides practical and empirical information to inform the planning, implementation, and evaluation of perinatal health fairs that establish meaningful connection between local women, perinatal educators, and health researchers.

Validation of the Implementation Climate Scale in Nursing

One critical factor in effective implementation of evidence-based practices (EBPs) in nursing is an organizational context that facilitates and supports implementation efforts. Measuring implementation climate can add useful insights on the extent to which the organizational context supports EBP implementation. This study cross-validates and examines the psychometric properties of the Implementation Climate Scale (ICS), which measures nurses' perceptions of their unit's climate for EBP implementation. This study analyzed ICS data from two cross-sectional studies, including 203 nurses from California and 301 nurses from Florida. Analyses included evaluation of internal consistency, multilevel aggregation statistics, and confirmatory factor analyses. The 18-item ICS demonstrated comparable psychometric properties to the original measure development paper in both samples. Confirmatory factor analyses provided support for the scale's factor structure in both samples. The ICS is a pragmatic measure that can be used to assess unit implementation climate in nursing contexts. Results from the ICS from nurses and nurse leaders can provide insights into implementation-specific barriers and facilitators within the organizational context.

Acceptability of employment readiness measures in youth and young adults on the autism spectrum: a mixed-methods study

Purpose Reliable, valid, and pragmatic measures are essential for monitoring and evaluating employment readiness and comparing the effectiveness of alternative implementation strategies. The Work Readiness Inventory (WRI) and Ansell–Casey Life Skills Assessment (ACLSA) are valid measures of employment readiness in neurotypical populations; however, their acceptability (i.e., user perception of measure as agreeable/satisfactory) for persons on the autism spectrum is not yet known. This investigation assesses the acceptability of the WRI and a modified ACLSA (ACLSA-M) in measuring employment readiness in youth/young adults on the spectrum. Methods A concurrent triangulation mixed-methods study design utilizing quantitative pre-post measurement of a community-based employment readiness program alongside qualitative survey assessment was employed to determine concurrent acceptability. For robustness, further explication through peer debriefing of experts evaluated the retrospective acceptability via interview and acceptability-rate assessment. Results Findings indicated that both measures are acceptable, although individual- and job-specific item modifications are advised, particularly due to disability-specific needs. Significant change in employment readiness in youth/young adults on the spectrum supports concurrent acceptability. Peer debriefing provided rich data on retrospective acceptability. Acceptability-rates of 0.84 and 0.91 confirm broad acceptability of these measures. Conclusions Implications are presented for clinicians and researchers, highlighting the relevance for autism-specific measurement development and acceptability. Implications for rehabilitation Given the lower labor force participation of persons on the autism spectrum, a combination of measures should be used in the assessment of an individual’s employment readiness. In youth and young adults on the spectrum, employment readiness can be measured using the Work Readiness Inventory (WRI) and a modified version of the Ansell–Casey Life Skills Assessment (ACLSA-M). In clinical practice and research, modifying the contents of these measures may be advised to minimize language complexity, and maximize ease in self report. When designing, developing, and testing new measures in rehabilitation practice or research, the intent should be broadened by involving diverse representation from the project outset, by engaging both those on the spectrum and neurotypical populations.

Assessment of trunk lateral flexion range of movement using a novel method in first class cricket players.

Lateral flexion range of movement (LF ROM) is used to assess and monitor recovery of side strain injury in athletes. This study established a reliable and pragmatic measure of LF ROM and investigated the stability of the measure over time in athletes. 1) Cross-sectional reliability study and 2) Cohort longitudinal study. Elite cricket teams in COUNTRY-AAA and COUNTRY-BBB Participants: Cricket players Methods: 1) The intra- and inter-rater reliability of two methods of measuring LF ROM were assessed (distance to the floor or distance to fibular head). Ten healthy first-class cricket bowlers were tested by three experienced physiotherapists. Intra-class correlations (2,1) were calculated for absolute agreement for all 3 testers. 2) Professional cricket fast bowlers were recruited from COUNTRY-AAA and COUNTRY-BBB domestic and international competitions. Lateral flexion range of movement was measured monthly during the pre- and competitive season. A one-way repeated measures analysis of variance was performed to identify difference within the pre-season, within the competitive season, and between competitive seasons. Lateral flexion range of movement towards and away from the bowling arm. Both methods had good intra- and inter-test reliability (ICC>0.84). As LF ROM to the floor was easier for clinicians it was used for the longitudinal study. Lateral flexion range of movement did not significantly alter throughout the pre- and competitive season or between seasons (p>0.05). This new method of describing LF ROM demonstrates good intra- and inter-rater reliability and stability over time and can be used as an outcome measure in side-strain injury.

Sexual and reproductive health services utilisation amongst in-school young people with disabilities in Ghana.

BackgroundSexual and reproductive health (SRH) of young people including those with disabilities is a major public health concern globally. However, available evidence on their use of sexual and reproductive health services (SRHS) is inconsistent.ObjectiveThis study investigated utilisation of SRHS amongst the in-school young people with disabilities (YPWDs) in Ghana using the healthcare utilisation model.MethodsGuided by the cross-sectional study design, a questionnaire was used to obtain data from 2114 blind and deaf pupils or students in the age group 10-24 years, sampled from 15 purposively selected special schools for the deaf and the blind in Ghana.ResultsAbout seven out of every 10 respondents had ever utilised SRHS. The proportion was higher amongst the males (67.8%) compared with the females (62.8%). Young persons with disabilities in the coastal (OR = 0.03, 95% CI = 0.01–0.22) and middle (OR = 0.06, 95% CI = 0.01–0.44) zones were less likely to have ever utilised SRHS compared with those in the northern ecological zone. The blind pupils or students were more likely to have ever utilised SRHS than the deaf (OR = 1.45, 95% CI = 1.26–3.11).ConclusionsGenerally, SRHS utilisation amongst the in-school YPWDs in Ghana is high but significantly associated with some predisposing, need and enabling or disabling factors. This underscores the need for policymakers to consider in-school YPWDs as a heterogeneous group in the design and implementation of SRHS programmes. The Ghana Education Service in collaboration with the Ghana Health Service should adopt appropriate pragmatic measures and targeted interventions in the special schools to address the SRH needs of the pupils or students.

Anesthesiology and Critical Care Response to COVID-19 in Resource-Limited Settings: Experiences from Nepal.

It is difficult to predict the future course and length of the ongoing COVID-19 pandemic, which has devastated health care systems in low- and middle-income countries. Anesthesiology and critical care services are hard hit because many hospitals have stopped performing elective surgeries, staff and scarce hospital resources have been diverted to manage COVID-19 patients, and several makeshift COVID-19 units had to be set up. Intensive care units are overwhelmed with critically ill patients. In these difficult times, low- and middle-income countries need to improvise, perform indigenous research, adapt international guidelines to suit local needs, and target attainable clinical goals.

Association between CCC-2 and Structural Language, Pragmatics, Social Cognition, and Executive Functions in Children with Developmental Language Disorder.

(1) Background: Developmental Language Disorder (DLD) is diagnosed when the child experiences problems in language with no known underlying biomedical condition and the information required for its correct evaluation must be obtained from different contexts. The Children’s Communication Checklist (CCC-2) covers aspects of a child’s communication related to structural language and pragmatic skills, which are linked to social cognition or executive functions. The aim of this article is to examine parents’ reports using the Spanish version of the CCC-2 questionnaire and its association with different formal assessments related to communication. (2) Methods: 30 children with DLD (3; 10–9 years old) and 39 age-matched (AM) children with typical development were assessed using formal measures of structural language, pragmatics, social cognition, and executive functions. Parents of children with DLD answered the Spanish version of the CCC-2. (3) Results: The performance of children with DLD was lower in all the formal assessments in comparison to AM children. The CCC-2 was significantly correlated with all the direct child assessments, although only formal measures of structural language predicted both the structural language and pragmatics scales of the CCC-2. (4) Conclusions: The CCC-2 answered by parents was consistent with formal assessments in children with DLD, and structural language seemed to be the best predictor of all the subscales.

Percent of Normal: A Pragmatic Patient-Reported Outcome Measure for the Orthopaedic Trauma Clinic.

To compare a single numerical patient-reported outcome measure (PROM) to general health and injury-specific PROMs. Retrospective cohort. Urban Level 1 trauma center. The study included 175 patients with 34 humerus, 54 pelvis, 31 acetabular, and 56 ankle fractures. Patients were administered 3 PROMs: the 12-item short-form (SF-12), an injury-specific PROM (QuickDASH-humerus; Majeed Pelvic Outcome Score (Majeed)-pelvis; modified Merle d'Aubigne score (Merle)-acetabular; Foot and Ankle Disability Index (FADI)-ankle, and the Percent of Normal (PON) PROM, a single numerical PROM, which asked, "How would you rate yourself, if 100% is back to normal?" Floor/ceiling effect, convergent validity, and responsiveness of PROMs were assessed. None of the PROMs demonstrated a floor effect. The Merle was the only PROM with a ceiling effect (19%). The PON had a strong correlation with the QuickDASH (r = 0.78) and Majeed (r = 0.78); a moderate association with the SF-12 physical component score (r = 0.63), Merle (r = 0.67), and FADI (r = 0.55); and a weak association with the SF-12 mental component score (r = 0.22). The regression coefficient for change in PROM over time, a measure of responsiveness, was greater for the PON compared with the SF-12 physical component score/mental component score, Majeed, Merle, and FADI, but not the QuickDASH. The PON is a pragmatic PROM that can be easily administered in clinic by the physician to quickly assess and manage a variety of fractures, avoiding the disadvantages of nonrelative general or region-specific PROMs.

Fragile alliances: culture, funding and sustainability in police–academic partnerships

Background Police–academic partnerships have developed significantly over the past decade or so, spurred on by the expansion of the evidence-based policing movement, the increasing value attached to impactful research in the academy, the ascendance of the professionalisation agenda in the police, and the growing necessity of cross-sectoral collaborations under conditions of post-financial crisis austerity. This trend has given rise to a burgeoning literature in the discipline of criminology which is concerned with charting the progress of these partnerships and setting out the ideal conditions for their future expansion. Aims and objectives we advance a sympathetic critique of this literature, adding a note of caution to its largely optimistic outlook. Methods we do this by combining a narrative review of the literature on police–academic partnerships with insights from elsewhere in the social sciences and observations from our experience of running the International Strand of the N8 Policing Research Partnership. Findings and discussion while we recognise that police–academic partnerships have certainly come a long way, and have the capacity to make important contributions to police work, we argue that they remain ‘fragile’ alliances, beset with fractious occupational cultures, unreliable funding streams and unsustainable inter-institutional relationships. We also reason that the structures underpinning this ‘fragility’ do not represent problems to be overcome, for they help to protect the integrity of the two professions. Conclusion we conclude by offering pragmatic measures for sustaining police–academic partnerships during those difficult periods characterised by cultural dissonance, a paucity of funding and the turnover of key personnel.

Development and preliminary evaluation of the Conventional Medicine Disclosure Index.

Non-disclosure of conventional medicine use to complementary medicine health professionals may result in patient harm. Currently, no standardised validated instrument is available to measure reasons for conventional medicine disclosure or non-disclosure. The aim of this study was to develop and evaluate a multidimensional index for identifying reasons for conventional medicine disclosure and non-disclosure by patients. Drawing upon a sub-sample of the Complementary and Alternative Medicine Use Health Literacy Disclosure Study (N=520), a formative measurement approach was used to develop a Conventional Medicine Disclosure Index (CONMED-DI). Variance-based structural equation modelling employing partial least squares evaluated multicollinearity, significance and relevance of the formative indicators to their associated primary constructs. The CONMED-DI demonstrated adequate construct validity suggesting the CONMED-DI is a pragmatic measure to determine the reasons why people choose to disclose (or not) their conventional medicine use. The CONMED-DI contains 2second-order measurement models, both with three sub-domains. The CONMED-DI serves as a preliminary instrument primarily of value to researchers interested in exploring the complementary medicine clinical encounter. The development of targeted interventions that promote disclosure of conventional medicine can be facilitated through understanding patients' reasons for disclosure and non-disclosure and optimise patients' safe use of medicines.

The TPOCS-self-reported Therapist Intervention Fidelity for Youth (TPOCS-SeRTIFY): A case study of pragmatic measure development.

Developing pragmatic assessment tools to measure clinician use of evidence-based practices is critical to advancing implementation of evidence-based practices in mental health. This case study details our community-partnered process of developing the Therapy Process Observation Coding Scale-Self-Reported Therapist Intervention Fidelity for Youth (TPOCS-SeRTIFY), a pragmatic, clinician-report instrument to measure cognitive behavioral therapy (CBT) delivery. We describe a five-step community-partnered development process. Initial goals were to create a self-report instrument that paralleled an existing direct observation measure of clinician delivery of CBT use to facilitate later assessment of measure performance. Cognitive interviews with community clinicians (n = 6) and consultation with CBT experts (n = 6) were used to enhance interpretability and usability as part of an iterative refinement process. The instrument was administered to 247 community clinicians along with an established self-reported measure of clinician delivery of CBT and other treatments to assess preliminary psychometric performance. Preliminary psychometrics were promising. Our community-partnered development process showed promising success and can guide future development of pragmatic implementation measures both to facilitate measurement of ongoing implementation efforts and future research aimed at building learning mental health systems. Developing brief, user-friendly, and accurate tools to measure how therapists deliver cognitive behavioral therapy (CBT) in routine practice is important for advancing the reach of CBT into community settings. To date, developing such "pragmatic" measures has been difficult. There is little known about how researchers can best develop these types of assessment tools so that they (1) are easy for clinicians in practice to use and (2) provide valid and useful information about implementation outcomes. As a result, there are few well-validated measures in existence that measure therapist use of CBT that are feasible for use in community practice. This paper contributes to the literature by describing our community-partnered process for developing a measure of therapist use of CBT (Therapy Process Observation Coding Scale -Self-Reported Therapist Intervention Fidelity for Youth; TPOCS-SeRTIFY). This descriptive case study outlines the community-partnered approach we took to develop this measure. This case study will contribute to future research by serving as a guide to others aiming to develop pragmatic implementation measures. In addition, the TPOCS-SeRTIFY is a pragmatic measure of clinician use of CBT that holds promise for its use by both researchers and clinicians to measure the success of CBT implementation efforts.

A pilot study comparing tools for tracking implementation strategies and treatment adaptations.

Tailoring implementation strategies and adapting treatments to better fit the local context may improve their effectiveness. However, there is a dearth of valid, reliable, pragmatic measures that allow for the prospective tracking of strategies and adaptations according to reporting recommendations. This study describes the development and pilot testing of three tools to be designed to serve this purpose. Measure development was informed by two systematic reviews of the literature (implementation strategies and treatment adaptation). The three resulting tools vary with respect to the degree of structure (brainstorming log = low, activity log = moderate, detailed tracking log = high). To prospectively track treatment adaptations and implementation strategies, three stakeholder groups (treatment developer, implementation practitioners, and mental health providers) were randomly assigned one tool per week through an anonymous web-based survey for 12 weeks and incentivized to participate. Three established implementation outcome measures, the Acceptability of Intervention Measure, Intervention Appropriateness Measure, and Feasibility of Intervention Measure, were used to assess the tools. Semi-structured interviews were conducted to gather more nuanced information from stakeholders regarding their perceptions of the tools and the tracking process. The three tracking tools demonstrated moderate to good acceptability, appropriateness, and feasibility; the activity log was deemed the most feasible of the three tools. Implementation practitioners rated the tools the highest of the three stakeholder groups. The tools took an average of 15 min or less to complete. This study sought to fill methodological gaps that prevent stakeholders and researchers from discerning which strategies are most important to deploy for promoting implementation and sustainment of evidence-based practices. These tools would allow researchers and practitioners to track whether activities were treatment adaptations or implementation strategies and what barrier(s) each targets. These tools could inform prospective tailoring of implementation strategies and treatment adaptations, which would promote scale out and spread. Strategies to support the implementation of evidence-based practices may be more successful if they are carefully customized based on local factors. Evidence-based practices themselves may be thoughtfully changed to better meet the needs of the settings and recipients. This study reports on a pilot study that aimed to create various types of tools to help individuals involved in implementation efforts track the actions they take to modify and implement interventions. These tools allow individuals to track the types of activities they are involved in, when the activities occurred, who was involved in the implementation efforts, and the reasons or rationale for the actions. The three tools in this study used a combination of open-ended and forced-response questions to test how the type of data recorded changed. Participants generally found the tools quick and easy to use and helpful in planning the delivery of an evidence-based practice. Most participants wanted more training in implementation science terminology and how to complete the tracking tools. Participating mental health providers would have liked more opportunities to review the data collected from the tools with their supervisors to use the data to improve the delivery of the evidence-based practice. These tools can help researchers, providers, and staff involved in implementation efforts to better understand what actions are needed to improve implementation success. Future research should address gaps identified in this study, such as the need to involve more participants in the tool development process.

Development of a comprehensive measure of organizational readiness (motivation \xd7 capacity) for implementation: a study protocol

BackgroundOrganizational readiness is important for the implementation of evidence-based interventions. Currently, there is a critical need for a comprehensive, valid, reliable, and pragmatic measure of organizational readiness that can be used throughout the implementation process. This study aims to develop a readiness measure that can be used to support implementation in two critical public health settings: federally qualified health centers (FQHCs) and schools. The measure is informed by the Interactive Systems Framework for Dissemination and Implementation and R = MC2 heuristic (readiness = motivation × innovation-specific capacity × general capacity). The study aims are to adapt and further develop the readiness measure in FQHCs implementing evidence-based interventions for colorectal cancer screening, to test the validity and reliability of the developed readiness measure in FQHCs, and to adapt and assess the usability and validity of the readiness measure in schools implementing a nutrition-based program.MethodsFor aim 1, we will conduct a series of qualitative interviews to adapt the readiness measure for use in FQHCs. We will then distribute the readiness measure to a developmental sample of 100 health center sites (up to 10 staff members per site). We will use a multilevel factor analysis approach to refine the readiness measure. For aim 2, we will distribute the measure to a different sample of 100 health center sites. We will use multilevel confirmatory factor analysis models to examine the structural validity. We will also conduct tests for scale reliability, test-retest reliability, and inter-rater reliability. For aim 3, we will use a qualitative approach to adapt the measure for use in schools and conduct reliability and validity tests similar to what is described in aim 2.DiscussionThis study will rigorously develop a readiness measure that will be applicable across two settings: FQHCs and schools. Information gained from the readiness measure can inform planning and implementation efforts by identifying priority areas. These priority areas can inform the selection and tailoring of support strategies that can be used throughout the implementation process to further improve implementation efforts and, in turn, program effectiveness.

ParkIndex: Validation and application of a pragmatic measure of park access and use.

Composite metrics integrating park availability, features, and quality for a given address or neighborhood are lacking. The purposes of this study were to describe the validation, application, and demonstration of ParkIndex in four diverse communities. This study occurred in Fall 2018 in 128 census block groups within Seattle(WA), Brooklyn(NY), Raleigh(NC), and Greenville County(SC). All parks within a half-mile buffer were audited to calculate a composite park quality score, and select households provided data about use of proximal parks via an online, map-based survey. For each household, the number of parks, total park acreage, and average park quality score within one half-mile were calculated using GIS. Logistic regression was used to identify a parsimonious model predicting park use. ParkIndex values (representing the probability of park use) were mapped for all study areas and after scenarios involving the addition and renovation/improvement of parks. Out of 360 participants, 23.3% reported visiting a park within the past 30 days. The number of parks (OR = 1.36, 95% CI = 1.15-1.62), total park acreage (OR = 1.13, 95% CI = 1.07-1.19), and average park quality score (OR = 1.04, 95% CI = 1.01-1.06) within one half-mile were all associated with park use. Composite ParkIndex values across the study areas ranged from 0 to 100. Hypothetical additions of or renovations to study area parks resulted in ParkIndex increases of 22.7% and 19.2%, respectively. ParkIndex has substantial value for park and urban planners, citizens, and researchers as a common metric to facilitate awareness, decision-making, and intervention planning related to park access, environmental justice, and community health.

Predicting implementation: comparing validated measures of intention and assessing the role of motivation when designing behavioral interventions

BackgroundBehavioral intention (which captures one’s level of motivation to perform a behavior) is considered a causal and proximal mechanism influencing the use of evidence-based practice (EBP). Implementation studies have measured intention differently, and it is unclear which is most predictive. Some use items referring to “evidence-based practice” in general, whereas others refer to a specific EBP. There are also unresolved debates about whether item stems should be worded “I intend to,” “I will,” or “How likely are you to” and if a single-item measure can suffice. Using each stem to refer to either a specific EBP or to “evidence-based practice,” this study compares the ability of these commonly used measures to predict future EBP implementation. The predictive validity is important for causal model testing and the development of effective implementation strategies.MethodsA longitudinal study enrolled 70 teachers to track their use of two EBPs and compare the predictive validity of six different items measuring teachers’ intention. The measures differ by whether an item refers to a specific EBP, or to “evidence-based practices” in general, and whether the stem is worded in one of the three ways: “I intend to,” “I will,” or “How likely are you to.” For each item, linear regressions estimated the variance in future behavior explained. We also compared the predictive validity of a single item versus an aggregate of items by inter-correlating the items using different stems and estimating the explained variance in EBP implementation.ResultsDepending on the EBP and how intention was measured, the explained variance in implementation ranged from 3.5 to 29.0%. Measures that referred to a specific EBP, rather than “evidence-based practices” in general, accounted for more variance in implementation (e.g., 29.0% vs. 8.6%, and 11.3% vs. 3.5%). The predictive validity varied depending on whether stems were worded “I intend to,” “I will,” or “How likely are you to.”ConclusionsThe observed strength of the association between intentions and EBP use will depend on how intention is measured. The association was much stronger if an item referred to a specific EBP, rather than EBP in general. To predict implementation, the results support using an aggregate of two or three intention items that refer to the specific EBP. An even more pragmatic measure of intention consisting of a single item can also predict implementation. As discussed, the relationship will also vary depending on the EBP, which has direct implications for causal model testing and the design of implementation strategies.

Advancing the pragmatic measurement of sustainment: a narrative review of measures

BackgroundSustainment, an outcome indicating an intervention continues to be implemented over time, has been comparatively less studied than other phases of the implementation process. This may be because of methodological difficulties, funding cycles, and minimal attention to theories and measurement of sustainment. This review synthesizes the literature on sustainment measures, evaluates the qualities of each measure, and highlights the strengths and gaps in existing sustainment measures. Results of the review will inform recommendations for the development of a pragmatic, valid, and reliable measure of sustainment.MethodsA narrative review of published sustainment outcome and sustainability measures (i.e., factors that influence sustainment) was conducted, including appraising measures in the Society of Implementation Research Collaboration (SIRC) instrument review project (IRP) and the Dissemination and Implementation Grid-Enabled Measures database initiative (GEM-D&I). The narrative review used a snowballing strategy by searching the reference sections of literature reviews and definitions of sustainability and sustainment. Measures used frequently and judged to be comprehensive and/or validated by a team of implementation scientists were extracted for analysis.ResultsEleven measures were evaluated. Three of the included measures were found in the SIRC-IRP, three in the GEM-D&I database, (one measure was in both databases) and six were identified in our additional searches. Thirteen constructs relating to sustainment were coded from selected measures. Measures covered a range of determinants for sustainment (i.e., construct of sustainability) as well as constructs of sustainment as an outcome. Strengths of the measures included, development by expert panels knowledgeable about particular interventions, fields or contexts, and utility in specific scenarios. A number of limitations were found in the measures analyzed including inadequate assessment of psychometric characteristics, being overly intervention or context specific, being lengthy and/or complex, and focusing on outer context factors.ConclusionThere is a lack of pragmatic and psychometrically sound measures of sustainment that can be completed by implementation stakeholders within inner context settings (e.g., frontline providers, supervisors).