Quality Improvement Maturity Research Articles

Using the QI Maturity Tool \u2013 Modified Ontario Version to assess the state of QI maturity in Ontario\u2019s public health units

BackgroundImplementation of quality improvement (QI) practices varies considerably among public health units (PHUs) in Ontario. With the emphasis on continuous quality improvement (CQI) in the revised Ontario Public Health Standards (OPHS), there is a need to understand the level of QI maturity in Ontario’s PHUs. The objective of this research was to establish a baseline understanding of QI maturity in Ontario’s PHUs.MethodsThe QI Maturity Tool - Modified Ontario Version was used to assess the state of QI maturity in 34 PHUs across Ontario. QI maturity was assessed through 23 questions across three dimensions: QI Organizational Culture; QI Capacity and Competency; and QI Perceived Value. QI maturity scores were classified into five stages: Beginning; Emerging; Progressing; Achieving; and Excelling. QI maturity scores were calculated for each of the 34 participating PHUs to determine their stage of QI maturity. Each PHU’s score was then used to determine the provincial average for QI maturity. Participants were also asked to answer three questions related to core CQI organizational structures.ResultsAcross the 34 PHUs, 3503 staff participated in the survey. A review of individual PHU scores indicates that Ontario’s PHUs are at varying stages of QI maturity. The average QI maturity score of 4.94 for the 34 participating PHUs places the provincial average in the “Emerging” stage of QI maturity. By QI dimensions, the participating PHUs scored in the “Emerging” stage for QI Organizational Culture (5.09), the “Beginning” stage for QI Competency and Capacity (4.58), and the “Achieving” stage for QI Perceived Value (6.00).ConclusionThere is an urgent need for Ontario’s PHUs to progress to higher stages of QI maturity. Participants place a high value on QI, but collectively are at less “mature” stages of QI in relation to QI organizational culture and the competency and capacity to engage in QI activities. PHUs should leverage the value that staff place on QI to foster the development of a culture of QI and provide staff with relevant knowledge and skills to engage in QI activities.

Rurality, Quality Improvement Maturity, and Accreditation Readiness: A Comparison Study of Colorado, Kansas, and Nebraska Local Health Departments

To compare local health department (LHD) accreditation readiness (AR) and quality improvement (QI) maturity in 3 states, between LHDs with varying levels of rurality, and across an LHD staffing-level continuum. This was a cross-sectional comparative study that included an online survey administered to LHD directors in Colorado, Kansas, and Nebraska. The survey included 10 questions assessed on a 5-point Likert scale covering 3 QI domains and 13 questions covering 5 AR domains. The median score for both QI maturity and AR was calculated by each state, by the number of full-time equivalent staff employed at the LHD, and by a measure of rurality and population density. A total of 156 LHDs from the states of Colorado, Kansas, and Nebraska. QI maturity and AR scores. A majority (59%) of the surveyed LHDs plan to apply or have already applied for Public Health Accreditation Board (PHAB) accreditation. The overall QI maturity and AR scores were highest in Nebraska, as was the intent to seek PHAB accreditation and current use of PHAB standards. Across levels of rurality and staffing, LHD QI maturity scores were similar; however, AR scores improved as LHD staffing levels increased and rurality decreased. Small LHDs and rural LHDs have QI maturity levels that are comparable to larger, less rural LHDs, but their AR is much lower. As accreditation has been found to have positive benefits, it is important that all LHDs have the capacity and resources to meet the performance standards required of accredited LHDs. Small, rural LHDs may need additional resources and support in order to improve their ability to be accredited and/or certain accreditation requirements may need modification to make accreditation more accessible to small LHDs.

Local health department factors associated with performance in the successful implementation of community-based strategies: a mixed-methods approach.

We examined which local health department (LHD)-level factors contributed to successful implementation of policy, systems, and environmental change strategies in Minnesota. We used a retrospective mixed-methods design to evaluate the relationship between the Statewide Health Improvement Plan (SHIP) grant initiative and key predictor variables (2009-2011). We obtained quantitative capacity data for 91 cities and counties in Minnesota; in addition, we conducted 15 key informant interviews to examine factors that facilitated and acted as barriers to LHD performance. Grantee performance was distributed as follows: exceeds expectations (29.7%), meets expectations (55.0%), and is approaching expectations (15.3%). Organizational quality improvement (QI) maturity was strongly positively associated with grantee performance on SHIP. Organizations with high QI maturity, effective leadership, efficient decision-making, and successful regional or cross-jurisdictional partnerships were more likely to be rated as exceeding expectations. This study successfully translated practice-based research findings into tangible outcomes, including new system-level performance measures for local public health and recommendations for shaping the statewide initiative examined in this study. The approach taken in this study to systematically monitor communications, dissemination, and translation may be a model for others.

Relationship between quality improvement implementation and accreditation seeking in local health departments.

We examined the relationship between quality improvement (QI) maturity and accreditation attributes of US local health departments (LHDs), specifically those in Nebraska. Using 2011 Nebraska LHD QI survey data, we conducted Spearman correlation analyses between QI maturity domains and accreditation attributes. Using the 2010 National Association of County and City Health Officials' National Profile of LHDs, we conducted logistic regression analyses to examine the relationships between specific QI strategies and attitude toward seeking accreditation. Leaders' commitment to and length of time engaged in QI were positively associated with LHDs' general attitude toward seeking accreditation. Use of QI strategies and integration of QI policies and practices were positively associated with LHDs' confidence in their capacity to obtain accreditation. LHDs that had used at least 1 QI framework and at least 1 QI technique in the past year were more likely to agree that they would seek accreditation within 2 years of the national accreditation program. Experience with and expertise in QI implementation play an important role in LHDs' decision to seek accreditation, and their accreditation-seeking efforts may benefit from prior implementation of systematic QI strategies.

Driving a Public Health Culture of Quality

There has been an extensive investment in building public health organizational capacity to improve performance and prepare for accreditation. An evolving perspective has focused not only on the practice of quality improvement (QI) within the health department but also upon the extent the culture of QI is embraced within the agency. No studies have examined the current national baseline of QI culture implementation, nor estimated the degree of QI sophistication local health departments (LHDs) have attained. We attempt to fill this void by aligning the findings from the QI module of the National Association of County & City Health Officials (NACCHO) 2010 Profile of LHDs against the constructs defined by the QI Maturity Tool and the NACCHO QI Roadmap (Roadmap to a Culture of Quality Improvement). Specific questions regarding QI activities from the 2010 Profile Study QI module were used to assign responding LHDs to stages within the Roadmap. We also used data from the QI Maturity Tool administered to all LHDs in the 16 participating Multi-State Learning Collaborative states in 2010 and 2011. On the basis of this matched set, we applied the summative domain scores algorithm, classified agencies into 1 of 5 groups, compared our findings with those of the NACCHO survey, and aligned our categories to those of the Roadmap. Nearly 80% of LHDs classified using the NACCHO Profile data were assigned to group 3 or 4 versus 48% using the QI Maturity Tool. Results from the cross-tabulations of the matched data set between the QI Maturity Tool classifications and the NACCHO Profile classifications revealed exact alignment 30% of the time. Forty-nine of 163 agencies were classified in the same grouping in both schemata. In addition, 84% of the agencies were classified within 1 neighboring category. The results revealed that half, if not most, LHDs fall within the middle categories of QI maturity and sophistication, regardless of which classification system was deployed.

Analysis of Local Health Department Factors that Accelerate Population-based Intervention Strategies: Preliminary Findings

Introduction: This practice-based research study capitalized on the statewide implementation of a comprehensive, locally-driven initiative to implement evidence-based policy, system and environmental changes related to obesity and tobacco use. The study examined local health department (LHD) performance and factors such as organizational quality improvement (QI) maturity, structure and governance. Methods: State health department staff reviewed grant reports and documentation pertaining to all LHD grantees, which collectively represented all 87 counties and 4 cities in Minnesota (MN), in order to designate grantees as either: “Exceeds Expectations,” “Meets Expectations” or “Approaching Expectations.” A study team of state, local and academic partners then used select measures from the QI Maturity Tool to calculate a QI Maturity Score, and subsequently place grantees into three levels of QI maturity: low, medium and high. Multivariate regression was performed to examine the potential relationship between grantee performance and QI maturity, as well as other covariates. Results: Of the final sample of 91 counties/cities, 87 had complete data for multivariate analysis. The distribution of grantee performance was: exceeds expectations (29.7%), meets expectations (55.0%) and approaching expectations (15.3%). Organizational QI maturity was strongly, positively associated with grantee performance comparing the “exceeds expectations” to those who “met or approached expectations” (OR=4.29, 95 % CI: 1.90-9.73, p=0.0005). Implications: Organizational QI maturity was strongly associated with LHD performance. More research is needed to determine whether a more mature approach to quality improvement is merely a marker for overall LHD capacity or has its own unique contribution to performance. Findings have been used to inform the next funding cycle of this initiative and provide support to implementing annual assessment of QI maturity in MN.

From the Frontier: Translating Research to Practice…QI as the Hinge Point

This article is number three in the series From the Frontier: Translating Research to Practice. The narrative describes the work of a practice-academic network in Minnesota which explored the degree to which having a culture of quality at the local health department level influenced the capacity to implement a new statewide initiative. The network conducted a mixed-methods study of grantees funded to develop and implement local policy, systems, and environmental change strategies to promote nutrition, increase activity, and reduce tobacco use and exposure. The results of their study indicated that grantees with higher performance levels in Quality Improvement (QI) were much more likely to exceed expectations in local initiatives compared to grantees with lower levels of “QI maturity”. The study results are being used at the local level to advocate for authority to bolster QI and at the state level to establish baseline capacity of new grantees in order to customize technical assistance. This provides further evidence that systems-level research is possible in such practice-academic networks, and that findings from such research are immediately translatable.

Using the QI maturity tool to classify agencies along a continuum

Major investments have been made to encourage health departments to implement quality improvement (QI) efforts. Yet, there are few empirically tested tools for public health agencies that assess these efforts and classify health departments along a QI continuum. This paper presents a new classification scheme for measuring QI Maturity in public health agencies based on a validated tool. The findings can be used to establish benchmarks, make comparisons and conduct future research linking QI and population health outcomes.

Effectiveness and Challenges for Implementing Quality Improvement Activities in Nebraska’s Local Health Departments

What is already known on this topic? Although the implementation strategies and effectiveness of quality improvement (QI) activities have been examined extensively for many industries, including the health care sector, very few studies have focused on QI activities in the public health context. What is added by this report? The study results indicated that Nebraska’s LHDs still face significant barriers for QI implementation, including low capacity, knowledge gaps, inadequate resources, and low institutional QI maturity. What are the implications for public health practice/policy/research? Policy makers and LHDs should provide QI training and external QI expertise to LHD staff and better integrate QI strategies into LHDs’ organizational culture and structure. Given the great complexity of QI methodologies, it may be helpful for LHDs to start their QI efforts by adopting and implementing relatively simple QI techniques and strategies, such as the PDSA approach, which some Nebraska LHDs (i.e., early adopters) have found to be effective, based on our qualitative research results.

Measuring Quality Improvement in Public Health

There is growing interest and investment in improving the quality of public health services and outcomes. Following the lead of other sectors, efforts are underway to introduce systematic quality improvement (QI) tools and approaches to state and local public health agencies. Little is known, however, about how to describe and reliably measure the level of QI maturity within a public health agency. The authors describe the development of a QI Maturity Tool using research from the fields of organizational design, psychology, health care, and complexity theory. The 37-item assessment tool is based on four quality domains derived from the literature: (a) organizational culture, (b) capacity and competency, (c) practice, and (d) alignment and spread. The tool was designed to identify features of an organization that may be enhancing or impeding QI; monitor the impact of efforts to create a more favorable environment for QI; and define potential cohorts of public health agencies for evaluation purposes. The article presents initial steps in testing and validating the QI Maturity Tool including: (a) developing a theoretical framework, (b) assuring face and content validity, (c) determining the tool's reliability based on estimates of internal consistency, (d) assessing the dimensionality, and (f) determining the construct validity of the instrument. The authors conclude that there is preliminary evidence that the QI Maturity Tool is a promising instrument. Further work is underway to explore whether self-reported survey results align with an agency's actions and the products of their QI efforts.