Public Health Performance Standards Program Research Articles

Implementing administrative evidence based practices: lessons from the field in six local health departments across the United States.

BackgroundAdministrative evidence based practices (A-EBPs) are agency level structures and activities positively associated with performance measures (e.g., achieving core public health functions, carrying out evidence-based interventions). The objectives of this study were to examine the contextual conditions and explore differences in local health department (LHD) characteristics that influence the implementation of A-EBPs.MethodsQualitative case studies were conducted based on data from 35 practitioners in six LHDs across the United States. The sample was chosen using an A-EBP score from our 2012 national survey and was linked to secondary data from the National Public Health Performance Standards Program. Three LHDs that scored high and three LHDs that scored low on both measures were selected as case study sites. The 37-question interview guide explored LHD use of an evidence based decision making process, including A-EBPs and evidence-based programs and policies. Each interview took 30–60 min. Standard qualitative methodology was used for data coding and analysis using NVivo software.ResultsAs might be expected, high-capacity LHDs were more likely to have strong leadership, partnerships, financial flexibility, workforce development activities, and an organizational culture supportive of evidence based decision making and implementation of A-EBPs. They were also more likely to describe having strong or important relationships with universities and other educational resources, increasing their access to resources and allowing them to more easily share knowledge and expertise.ConclusionsDifferences between high- and low-capacity LHDs in A-EBP domains highlight the importance of investments in these areas and the potential those investments have to contribute to overall efficiency and performance. Further research may identify avenues to enhance resources in these domains to create an organizational culture supportive of A-EBPs.Electronic supplementary materialThe online version of this article (doi:10.1186/s12913-015-0891-3) contains supplementary material, which is available to authorized users.

Advancing the Science of Delivery

Advancing the Science of Delivery

The Relationship Between Local Public Health Agency Characteristics and Performance of Partnership-Related Essential Public Health Services

The relationships between characteristics of local public health agencies and their self-reported scores on partnership-related indicators of the Ten Essential Public Health Services were examined. A retrospective cross-sectional study using secondary data from the National Public Health Performance Standards Program (NPHPSP) and the 2005 Profile of Local Public Health Agencies from the National Association of City and County Health Organizations (NACCHO) was completed. Participants. Local public health systems that participated in both the NPHPSP and the NACCHO surveys. Partnership-related elements from the NPHPSP data set were used as dependent variables, whereas combined focused elements from the NPHPSP and the NACCHO surveys served as independent variables. Local public health agencies' increase in partnerships over the preceding 3 years and involvement in a community health improvement process were significantly related to numerous partnership performance scores--more so than other agency characteristics. Involvement in the Mobilizing Action through Planning and Partnerships process was inversely related to some partnership performance scores. Future research must continue to identify and explore additional community- and agency-level predictors of partnership performance.

New Data, Same Story? Revisiting Studies on the Relationship of Local Public Health Systems Characteristics to Public Health Performance

This study examined the relationship between community and system characteristics of 353 local public health agencies and local public health system performance by revisiting previous research by Mays et al and Scutchfield et al. More recent and coterminous data were used. Local public health agency characteristics were obtained from the National Association of City and County Health Officials' 2005 National Profile of Local Public Health Departments and performance data were obtained from version 1 of the National Public Health Performance Standards Program local performance assessment instrument. Descriptive analyses and multivariate analyses similar to the two previous studies were employed. Population size, presence of a local board of health and whether the board makes policy, educational background of the local public health agency's top executive, and jurisdiction type were found to be important predictors of local public health performance. These findings support some of the earlier findings but do not support all the findings of the earlier studies. Variances are discussed. This study provides researchers and practitioners with an evidence base from which to make suggestions regarding characteristics, many changeable, which may influence system performance.

The National Public Health Performance Standards

Since its inception in 1998, the Centers for Disease Control and Prevention's National Public Health Performance Standards Program (NPHPSP) has helped lay the groundwork for public health quality improvement (QI) activities at the state and local levels. This article describes how the NPHPSP has promoted QI through its instruments and guidance and how it has continually strengthened the focus on QI over the years. The NPHPSP Version 2 instruments and enhanced guidance have been designed to more strongly reinforce QI and catalyze the transition from assessment to action. Despite positive reports from some state and local users that emphasize the value the NPHPSP holds for those that do successfully move forward with improvement actions, 2005 evaluation results from the Association of State and Territorial Health Officials and the National Association of County and City Health Officials indicated challenges in transitioning the assessments results into performance improvement. More recent data are promising; a 2009 postassessment survey of early Version 2 respondents indicates that the majority (75% of all respondents) report action in one or more performance improvement steps. The NPHPSP has played an important role in fostering QI in many states and local jurisdictions. Furthermore, its experiences and lessons learned in supporting QI have helped to pave the way for other initiatives, such as the emerging national accreditation system for state and local health departments.

Public Health Practice: Transcending the Known

Unabashedly, I confess to being a recovering former state and local health official. During my recent transition into academia, public health practice has continued to be my reference point for making meaningful contributions to the broader public health field. This issue of the Journal on the “business” of public health has only served to reinforce the vigor of my long-held views. Moreover, this occasion presents an opportunity to share some developing thoughts on how public health practice might better be incorporated into this venerable publication, while still preserving its appeal to a wider public health audience. Supreme Court Justice Potter Steward may have said it best: “I know it when I see it” (Jacobellis v. Ohio, 378 US 184 [1964]). Although he was referring to pornography rather than public health practice, the same test may still apply. Altogether too often, when we describe public health practice, our focus tends to be limited to that of the governmental sector. To be sure, it remains a critical unit of analysis as public health moves forward to embrace a national voluntary health department accreditation model. However, as we have recently learned through experience with the National Public Health Performance Standards Program (a product of the Centers for Disease Control and Prevention and 6 public health practice partner organizations; available at: http://www.cdc.gov/od/ocphp/nphpsp), much of what we generally recognize as public health practice is accomplished in tandem, with the public health sector interacting as partners with health care, education, transportation, and others to create the larger public health system. The shared development of strategies and interventions designed to promote and protect the health of communities (Turnock BJ. Essentials of Public Health. Sudbury, MA: Jones & Bartlett; 2007) is public health practice writ large. If the ultimate goal of public health practice is advancement of community health, then the talisman is improvement of health status indicators. Yet, this linkage between public health interventions and downstream health outcomes has often proven elusive. Given the complex environment in which public health is only one of many actors, this paucity of demonstrable impact is understandable; however, the dilemma is compounded by the relatively limited use in public health of an evidence-based approach. Prestigious medical journals, like New England Journal of Medicine and Journal of the American Medical Association, make a concerted effort to translate research findings into everyday practice relevant for their readers. Although they are not uniformly successful, I applaud the worthy attempt. They do literature reviews of contemporary science, summarize it, and outline lessons for practical application. Alternatively, at times, authors hone in on a specific (more narrow) clinical problem, and update the state of the art. Both models may have public health parallels. The equivalent approach in public health might involve soliciting the work of leading public health system researchers as well as other scientists, and encouraging translation of their findings to the public health practitioner at all levels. There are also extremely important transforming events taking place in the larger public health field. Stated more succinctly, the remodeled Government, Politics, and Law Department of the Journal will emphasize the metamorphosis that public health practice is undergoing—focusing on what it can become.

The Forgotten Instrument

This study examines the use of, and results from, the National Public Health Performance Standards Program Local Governance Instrument. It includes a compilation and analysis of 173 local governance instruments completed by local boards of health from 2003 to 2006. Only 24 of the 173 scored instruments are used because of exclusion of data from New Jersey. The study compares results from the instruments based upon demographic data reported by the local boards of health, and data on performance compiled by the National Public Health Performance Standards Program Local Public Health System Instrument. Local boards of health perform well on Essential Public Health Services #6 (78.85%), #2 (71.41%), and #7 (70.75%). Performance is far from optimal on Essential Public Health Services #10 (45.42%) and #9 (41.30%). Comparing groups based on demographic data yielded deviations too large and power too low to form any significant conclusions about local boards of health performance. It is important to note that individuals with varying levels of knowledge may have completed the governance instruments, and this may affect the results of any comparison between individual boards of health. Local boards of health need encouragement from national and state associations of local boards of health to complete the local governance instrument. This would allow local boards of health to use these data to compare performance with other boards around the nation. Identification of weak performing areas may lead to changes to improve service to the community. This instrument could also prove a useful tool in health department accreditation.

Building a Bridge to Accreditation—the Role of the National Public Health Performance Standards Program

Building a Bridge to Accreditation—the Role of the National Public Health Performance Standards Program

State Public Health Agency Expenditures

For optimal effectiveness, assessments of public health agency and system performance should include analysis to measure the amount of financial resources consumed to achieve performance levels. This pilot study was conducted to test a methodology in a state health department for comparing financial resources consumed to performance scores in each of the 10 Essential Public Health Services categories. An additional feature was to quantify the percentage of total agency expenditures utilized for administrative functions as well. The allocation of all fiscal year 2004 expenditures to the 10 Essential Public Health Services and administration categories was based on assessments of employee job functions and scope of services performed under agency contracts. Performance scores were obtained through a 2-month process of completing self-assessment surveys with system partners using the National Public Health Performance Standards Program Assessment Instrument. Investigators found no clear consistency between performance scores and agency expenditure levels. Two categories, essential service 5 (develop policies and plans) and essential service 10 (research), did have low performance and low expenditure levels. Overall though, categories with high performance scores consumed low percentages of agency expenditures and expenditure patterns were relatively high in categories with low performance scores. The study did quantify that the percentage of expenditures in the administration category was low compared to previous studies in other health departments. This knowledge was particularly useful for informing policymakers.

Use of National Public Health Performance Standards to Assess Maineʼs Diabetes System

In the fall of 2003, Maine underwent a rigorous assessment of the diabetes public health system using a modified version of the State Public Health System Performance Instrument developed by the Centers for Disease Control and Prevention's National Public Health Performance Standards Program and other national partners. The assessment was intended to serve as the impetus for the development of a statewide improvement plan. This article details the assessment process and provides a case study highlighting the successful application of systems-based model standards for a categorical issue.

The Local Public Health System Assessment of MAPP/The National Public Health Performance Standards Local Tool

The Local Public Health System Assessment (LPHSA) in Mobilizing for Action through Planning and Partnerships (MAPP), and the Local Tool of the National Public Health Performance Standards Program (LT/NPHPSP)--which are one in the same-attempt to assess the capacity to provide the essential public health services. Both tools have been utilized in various public health practice settings; however, users have been challenged with the complexity of the tools and time and human resource investment necessary to complete them. The recent Journal of Public Health Management and Practice issue focus on MAPP provided several examples of LPHSA implementation, both as a component of MAPP and as a stand-alone activity in the context of performance standards. The uniqueness in our approach involved a collaborative between a community-based organization, a public health academic program, and a governmental public health agency which conducted the LPHSA in a manner that did not require actual modification of the tool itself, was practical and feasible, and was of benefit and value to all partners.

An Instrument for Assessing Public Health System Performance: Validity in Rural Settings

This study evaluated the validity and utility of the Local Public Health System Assessment Instrument (Local Instrument) of the National Public Health Performance Standards Program in rural settings. The study compared the Local Instrument scores of 6 rural local public health systems to external assessments of those public health systems. The 6 public health systems represented 3 states in which 1 of the 2 local jurisdictions had scored well below and the other well above the state median in a pilot test of the Local Instrument. The study design featured a case study approach consisting of an iterative and integrated combination of semistructured individual and focus group interviews along with the collection of archival materials provided by the 6 public health systems. Despite differences in Local Instrument scores, the representative public health systems in each state provided roughly the same levels of public health services. Sites varied tremendously in the percentage of survey items rated highly or less relevant. The National Public Health Performance Standards Program Local Instrument can provide a useful structure and process for assessing public health system performance at the local level. Key informants provided several recommendations to improve the Local Instrument, including clarification of difficult terminology and acronyms, and development of multiple instruments structured around subsets of survey items.

National Public Health Performance Standards Assessment

North Dakota, as a rural state with a decentralized public health system, has found the National Public Health Performance Standards Program useful in assessing performance of the state's public health system. The local instrument was used for local public health systems and on Native American reservations. A description of the process as well as aggregated results of the local performance assessment is presented. An importance ranking scale was combined with the performance scores to identify priority areas. Priority needs were specifically identified for developing community health profiles, working more closely with community partnerships, and increasing emphasis on health education activities. The process was a good opportunity for bringing partners together in local public health systems and for developing interest in using the more complete strategic planning tools in Mobilizing for Action through Planning and Partnerships.

Getting What You Pay For

Governmental spending in public health varies widely across communities, raising questions about how these differences may affect the availability of essential services and infrastructure. This study used data from local public health systems that participated in the National Public Health Performance Standards Program pilot tests between 1999 and 2001 to examine the association between public health spending and the performance of essential public health services. Results indicated that performance varies significantly with both local and federal spending levels, even after controlling for other system and community characteristics. Some public health services appear more sensitive to these expenditures than others, and all services appear more sensitive to local spending than to state or federal spending. These findings can assist public health decision makers in identifying public health financing priorities during periods of change in the resources available to support local public health infrastructure.

Identifying dimensions of performance in local public health systems: results from the National Public Health Performance Standards Program.

This article uses data from the National Public Health Performance Standards Program to explore how the performance of essential public health services varies across communities and to identify underlying domains of activity that appear to drive variation in performance. Cross-sectional data were used from 315 local public health jurisdictions located within seven states that participated in the Performance Standards Program pilot tests between 1999 and 2001. Results demonstrate that local public health systems vary considerably in the extent to which they perform essential services and meet established performance standards. Factor analysis results indicate that four underlying domains of activity explain much of the variation observed in the individual performance measures, and that achieving performance standards for a single essential public health service often involves more than one underlying domain of activity. The findings suggest that composite measures constructed from the Performance Standards Program can assist public health decision makers in monitoring the performance of public health systems and identifying promising pathways for improving performance.

Local public health agency capacity and its relationship to public health system performance.

he study described in this article identifies local public health agency capacity characteristics that are related to their local public health systems' performance scores on the CDC's National Public Health Performance Standards Program assessment instrument. Public health system performance scores from a test version of the National Public Health Performance Standards instrument (5b) from county and city/county jurisdictions in three states were matched to organizational capacity data from the 1997 National Association of County and City Health Officials profile of health departments, resulting in a sample of 152 jurisdictions. Twenty-eight capacity variables from the profile and all 10 scores on the Essential Public Health Services plus the total performance score were analyzed in 11 separate multivariate regression models. Public health agency capacities in the areas of funding, organizational leadership, and certain nonprovider partnerships were found to be significantly related to public health system performance. Further study is needed to determine if these relationships between agency capacities and system performance are found, with data from other states now using the nationally released performance assessment instruments and with capacity measures that are more specific for evaluating public health system performance.

Content and criterion validity evaluation of National Public Health Performance Standards measurement instruments.

The Centers for Disease Control and Prevention's National Public Health Performance Standards Program (NPHPSP) has developed instruments to measure the performance of local and state public health departments on the 10 "Essential Services of Public Health," which have been tested in several states. This article is a report of the evaluation of the content and criterion validity of the local public health performance assessment instrument, and the content validity of the state public health performance assessment instrument. Health department performance is measured using a set of indicators developed for the 10 Essential Services of Public Health and a model standard for each indicator. Content validity of each model standard in the local instrument was addressed by community partners along the following dimensions: the importance of each standard as a measure of the associated Essential Service, its completeness as a measure, and its reasonableness for achievement. All standards for each Essential Service were then judged in terms of their completeness in measuring performance in that service. Content validity of the state instrument was evaluated in a group interview of health department staff members from three states. Criterion validity of the local instrument was assessed for a sample of eight public health departments in Florida and six in New York by examining documentary evidence for selected responses. Criterion validity was also evaluated for a sample of Florida local public health departments and one Hawaii public health department by comparing state health department staffs' judgments of performance against the instrument score. Criterion validity was upheld for a summary performance score on the local instrument, but was not upheld for performance judgments on individual Essential Services. The NPHPSP standards based on the Essential Services have validity for measuring local public health system performance, according to community partners. The model standards are valid measures of state performance, according to state public health departments in three states. Within the scope of the validity evaluations completed, the NPHPSP state and local performance assessment instruments were found to be valid measures of public health performance.

A model for public health workforce development using the National Public Health Performance Standards Program.

Workforce development programs in public health should link improvements in workers' performance with improvements in their agencies' performance. The "ten essential services" of public health provide criteria for measuring both individual worker training (as in workforce competency standards) and agency performance (as in the Centers for Disease Control and Prevention's National Public Health Performance Standards Program). This shared foundation was the basis for a model strategic training program developed for use in a 500-employee urban county health department. Full implementation of this model as a foundation for assessment, curriculum development, and evaluation requires careful attention to management issues, confidentiality of employee records, and evaluation methodologies.

Recommendations from testing of the National Public Health Performance Standards instruments.

The National Public Health performance Standards Program (NPHPSP) has developed assessment instruments based on the ten essential public health services (EPHS) for state and local health departments. The article reviews validity testing of the state and local instruments. The study employed multiple approaches to validity testing with state and local health departments in Florida, Hawaii, Minnesota, Mississippi, and New York. The New York State validity checks included the judgments of community partners. The study found that the EPHS have content and face validity as a basis for measuring public health system performance. The article includes recommendations for continued development of the NPHPS.

Assessment of validity of the national public health performance standards: the local public health performance assessment instrument.

The National Public Health Performance Standards Program (NPHPSP) has developed performance standards measurement instruments, based on the 10 "Essential Services of Public Health" that are being tested in several states. This article is a report on the face and content validity of the instrument designed for local public health systems. Judgments about the face validity of the standards were obtained in a survey of local public health systems that had used the instrument in a test state. The validity of each standard was addressed along the following dimensions: the importance of the standard as a measure of the Essential Service; its completeness as a measure; and its reasonableness for achievement. All standards for each Essential Service were then judged in terms of their completeness in measuring performance of that service. Respondents judged the standards to be highly valid measures of local public health system performance. Some respondents had reservations about whether standards related to "enforcing laws and regulations" were achievable. Holding local public health systems accountable for the activities of other agencies was a factor mentioned in conjunction with those standards. The NPHPSP standards have face and content validity for measuring local public health system performance. Further testing of their validity and reliability is continuing.