Health Program Performance Research Articles

Telepsychological services with criminal justice and substance abuse clients: A systematic review and meta-analysis.

Recent years have seen the incorporation of telepsychology into poorly accessed, rural, and underserved settings, including criminal justice and substance abuse treatment. A systematic search of the literature on telepsychological and related services with justice-involved and substance abuse clients revealed numerous descriptive reports, but few empirical studies. The results of 3 studies of criminal justice participants and 2 studies of substance-abuse participants were subjected to a series of 5 outcome-specific meta-analyses (mental health symptoms, therapeutic processes, program engagement, program performance, and service satisfaction). These 5 studies, all of which utilized a comparison group, contributed a total of 342 participants and 14 total effect sizes. Summary data on 3 additional uncontrolled studies are also presented. Results indicated that telepsychological outcomes were at least comparable with in-person outcomes. This review serves as an initial reference for clinicians and policymakers working with criminal justice and substance abuse clients, but also highlights the need for more rigorous scientific investigation into the nuances of telepsychological practice.

Assessing health program performance in low- and middle-income countries: building a feasible, credible, and comprehensive framework.

BackgroundMany health service delivery models are adapting health services to meet rising demand and evolving health burdens in low- and middle-income countries. While innovative private sector models provide potential benefits to health care delivery, the evidence base on the characteristics and impact of such approaches is limited. We have developed a performance measurement framework that provides credible (relevant aspects of performance), feasible (available data), and comparable (across different organizations) metrics that can be obtained for private health services organizations that operate in resource-constrained settings.MethodsWe synthesized existing frameworks to define credible measures. We then examined a purposive sample of 80 health organizations from the Center for Health Market Innovations (CHMI) database (healthmarketinnovations.org) to identify what the organizations reported about their programs (to determine feasibility of measurement) and what elements could be compared across the sample.ResultsThe resulting measurement framework includes fourteen subgroups within three categories of health status, health access, and operations/delivery.ConclusionsThe emphasis on credible, feasible, and comparable measures in the framework can assist funders, program managers, and researchers to support, manage, and evaluate the most promising strategies to improve access to effective health services. Although some of the criteria that the literature views as important – particularly population coverage, pro-poor targeting, and health outcomes – are less frequently reported, the overall comparison provides useful insights.

Assessment of health status and program performance in patients on long-term oxygen therapy

Despite well established clinical guidelines, performance of long-term oxygen therapy (LTOT) programs shows marked variability among territories. The current study assessed the LTOT program and the health status of patients on LTOT prior to the deployment of community-based integrated care in an urban health district of Barcelona (Spain). To assess: i) the LTOT program and health status of the patients on LTOT in the health district; ii) their frailty profile; and, iii) the requirements for effective deployment of integrated care services for these patients. Cross-sectional observational study design including all patients (n=406) on LTOT living in the health district. Health status, frailty, arterial blood gases, forced spirometry and hand-grip muscle strength were measured. Network analysis of frailty was carried out. Adequacy of LTOT prescription (n=362): 47% and 31% of the patients had PaO2≤60mmHg and ≤55mmHg, respectively. Adherence to LTOT: 31% of all patients used LTOT ≥15h/d; this figure increased to 67% in those with PaO2≤60mmHg. Assessment of frailty: Overall, LTOT patients presented moderate to severe frailty. Care complexity was observed in 42% of the patients. Adequacy and adherence to LTOT was poor and many patients were frail and complex. The outcomes of the network analysis may contribute to enhance assessment of frailty in LTOT patients. These observations suggest that an integrated care strategy has the potential to improve the health outcomes of these patients.

The quality-coverage gap in antenatal care: toward better measurement of effective coverage.

The proportion of pregnant women receiving 4 or more antenatal care visits (ANC 4+) is used prominently as a global benchmark indicator to track maternal health program performance. This has contributed to an inappropriate focus on the number of contacts rather than on the content and process of care. This paper presents analysis of specific elements of care received among women reporting 4 or more ANC visits. We conducted secondary analysis using Demographic and Health Survey data from 41 countries to determine coverage for specific elements of antenatal care. The analysis was conducted for: (1) women who delivered during the 2 years preceding the survey and who reported receiving 4 or more ANC visits, and (2) all women who delivered during the preceding 2 years. The specific ANC services that we assessed were: blood pressure measurement, tetanus toxoid vaccination, first ANC visit at less than 4 months gestation, urine testing, counseling about danger signs, HIV counseling and testing, iron-folate supplementation (≥ 90 days), and at least 2 doses of sulfadoxine/pyramethamine for malaria prevention. The difference between expected (100%) and actual coverage (the quality-coverage gap) was calculated for each service across the 41 surveys. Coverage for specific elements of care among women reporting 4 or more ANC visits was generally low for most of the specific elements assessed. Blood pressure and tetanus toxoid performed best, with median quality-coverage gaps of 5% and 18%, respectively. The greatest gaps were for iron-folate supplementation (72%) and malaria prevention (86%). Coverage for specific interventions was generally much lower among all pregnant women (reflecting population effective coverage) than among only those who had received ANC 4+ visits. Although ANC 4+ and average coverage across the elements of care correlated relatively well (Pearson r(2) = 0.56), most countries had lower average coverage for the content of care than for ANC 4+ (among all pregnant women). We argue for the adoption of a summary indicator that better reflects the content of antenatal care than does the current ANC 4+ indicator. We propose, as an alternative, the simple average of a set of ANC content indicators available through surveys and routine health information systems.

Qualitative evaluation: A critical and interpretative complementary approach to improve health programs and services

This short essay aims at commenting on the origin, development, rationale, and main characteristics of qualitative evaluation (QE), emphasizing the value of this methodological tool to evaluate health programs and services. During the past decades, different approaches have come to light proposing complementary alternatives to appraise the performance of public health programs, mainly focusing on the implementation process involved rather than on measuring the impact of such actions. QE is an alternative tool that can be used to illustrate and understand the process faced when executing health programs. It can also lead to useful suggestions to modify its implementation from the stakeholders’ perspectives, as it uses a qualitative approach that considers participants as reflective subjects, generators of meanings. This implies that beneficiaries become involved in an active manner in the evaluated phenomena with the aim of improving the health programs or services that they receive. With this work we want to encourage evaluators in the field of public health to consider the use of QE as a complementary tool for program evaluation to be able to identify areas of opportunity to improve programs’ implementation processes from the perspective of intended beneficiaries.

Spatial distribution of trachoma cases in the City of Bauru, State of São Paulo, Brazil, detected in 2006: defining key areas for improvement of health resources

The objective of this study was to analyze the spatial behavior of the occurrence of trachoma cases detected in the City of Bauru, State of São Paulo, Brazil, in 2006 in order to use the information collected to set priority areas for optimization of health resources. the trachoma cases identified in 2006 were georeferenced. The data evaluated were: schools where the trachoma cases studied, data from the 2000 Census, census tract, type of housing, water supply conditions, distribution of income and levels of education of household heads. In the Google Earth® software and TerraView® were made descriptive spatial analysis and estimates of the Kernel. Each area was studied by interpolation of the density surfaces exposing events to facilitate to recognize the clusters. Of the 66 cases detected, only one (1.5%) was not a resident of the city's outskirts. A positive association was detected of trachoma cases and the percentage of heads of household with income below three minimum wages and schooling under eight years of education. The recognition of the spatial distribution of trachoma cases coincided with the areas of greatest social inequality in Bauru city. The micro-areas identified are those that should be prioritized in the rationalization of health resources. There is the possibility of using the trachoma cases detected as an indicator of performance of micro priority health programs.

Client Satisfaction with HIV Counseling Services

Background: Client satisfaction is a good predictor of performance of health programs. Hence, clients’ perception and satisfaction studies provide insight to improve the program. Purpose: To assess clients’ perception and satisfaction with Integrated Counselling and Testing Centres (ICTCs) for HIV in an operational setting. Methods: A total of 191 client exit interviews from 12 ICTCs. The clients were stratified into general and antenatal clients. A systematic random sampling was done at high client load centers. Results: Cumulative client satisfaction was found to be 60% (±24%). Most of the clients (76%) agreed that counseling cleared doubts about HIV and found counseling beneficial (71%). Only 32% of the clients could recall issues discussed during the sessions. However, 92.5% were satisfied with ICTC facilities. Conclusions: Poor perception and low satisfaction with ICTCs needs to be addressed as this could have a direct bearing on the program.

Operational research: getting it done and making a difference [Editorial

Although The Union has been involved for years with operational research, it is in the last three years that we have collectively tried to think about what we are doing, how we can get the work done and how we can make a difference. In this regard, we have had a strong, enduring, productive and happy partnership with the Medecins Sans Frontieres (MSF) Operational Research Centre in Brussels, with whom we share a joint vision and strategy. It is worthwhile repeating our agreed definition of operational research, which is ‘research into strategies, interventions, tools or knowledge that can enhance the quality, coverage, effectiveness and performance of health systems and programmes so that better care is delivered to patients and communities’.1,2 One of our key strategies is to ensure that we build local capacity in the low- and middle-income countries in which we work. Our operational research courses and their modular structure have been described previously.2,3 Participants who are selected for our courses learn the principles of operational research while they take through a project from design to data collection to analysis, to paper writing and submission to a peer-reviewed journal. Experienced and reliable mentorship is needed and provided for this journey, and we are convinced about the effectiveness of this approach. In the three courses completed to date, we have trained 34 participants (doctors, nurses, paramedical officers, monitoring and evaluation officers) from Asia, Africa and Latin America, and seen 27 of their papers published in a variety of peer-reviewed journals. We have five other courses being implemented, with a total of 60 participants, and are confident that most will achieve their milestones and deliver the goods. We have learnt a lot along the way. We encourage our trained participants to become facilitators, and many have taken up this challenge in our subsequent courses or in courses that they have initiated and implemented locally in their own countries. We have realised the importance of a critical mass of staff, and our decisions about whom to select for our courses are strategic as well as based on merit and potential. We have learnt the meaning of the expression ‘data rich, information poor’. In many programmes or health settings from which our participants come, there is often a mass of data routinely collected over the years which is gathering dust and never used. Many of our research projects are designed around asking pertinent questions about these data sets. This is a win-win situation: the data are turned into useful knowledge, and gradual recognition is given to the concept that such data can provide information about how health care is delivered and can be changed. We have developed a sound understanding of ethics and operational research, and have redesigned our ethics application forms to relate better to retrospective data collection studies. We believe passionately in the published paper as a hard, measurable indicator that a research project has been taken through to completion and is of an acceptable standard.4 Furthermore, if we want our research to influence international or national policy, it has to be published because policy, strategy and guidance statements are increasingly evidence-based. We still have much to learn. ID Rusen summed it up well in an editorial in the International Journal of Tuberculosis and Lung Disease when he stated that ‘we need to do operational research on operational research’.5 We have changed the structure of our courses so that the first two modules on designing a research project and developing an electronic data capture instrument are done back-to-back instead of 2 months apart; we are about to implement this newly designed course, and it should save money and be more efficient. We need a robust, simple system of regularly keeping track of all the people we have trained so that we can measure whether research activities continue long after the trainings have finished. We are working on implementing this. We need to be able to measure in a standardised way whether the research has influenced policy and practice, and guidance on how to do this has just been published.6 What we need to do now is implement this in the field. The seed that we planted back in 2009 has grown into a tree. That tree and its branches continue to grow. However, like all plants, it needs nourishment without which it will wither and die. For operational research, this means international interest, support and funding. Through thick and thin, we must ensure that this happens and that operational research continues to be performed and serves its real purpose of finding health solutions for the poor.

University of Washington School of Medicine

University of Washington School of Medicine

The Process of Nurses' Experience in Public Health Program Performance

Purpose:This study was to describe and develop public health nursing. Methods:Data were collected from 19 nurses sampled from 5 public health centers. The grounded theory of Strauss and Corbin (1990) was applied. Results:Asa result, 48 concepts, 23subcategories, and 11 categories were deducedfrom opencoding. In axial coding, casual conditions facing difficulties of practical administration and attaching importance to health support for residents and context condition assessing of residents service needs impacted phenomenon searching for efficient performingmethods. Intervening conditions formingof support system and working with positive, andaction-interaction conditions improving of resident shealthand medicalaccessibility and striving for self-development lead to consequences carried out promptly for field-based problem solving, mastered of successful program planning methods and solving the issues with existing style. The periods of process were divided into 4 stages, confirming program-contents, probing program-method, developing program-strategies, andapplyingprogram-competencies.Thecorecategory,'strengtheningof practical-planningworkcompetencies incorporated the relationship between and among all categories and explained the process. Conclusion: This study described public health nurses performance in Korea. These findings have important implications for the practice and must be considered to develop competencies for planning and practice of public health.

A Correlational Study of the Relationship Between a Coordinated School Health Program and School Achievement: A Case for School Health

The study was conducted to determine whether there is a relationship between the Coordinated School Health Program (CSHP) and student academic performance. Data were collected from schools and the community for three reports for 50 states and the District of Columbia (DC). The School Health Policies and Programs Survey (SHPPS), the National Assessment of Educational Progress (NAEP), and the U.S. Census 2000 Profile were used to study the relationships among three parameters: (a) The intervention called a CSHP: (b) Student achievement; and (c) Rate of poverty in each state. A stepwise regression analysis was conducted, controlling for poverty using state-level data. Components of a CSHP had statistically significant relationships with academic achievement. Students in states with policies promoting students' health demonstrated higher academic scores and higher rates of high school completion.

Health system performance assessment in the Bulletin

Health system performance assessment in the Bulletin

A need to improve quality, rigour and dissemination of operations research

Operations research is essential for developing a strong knowledge base and identifying innovative strategies to improve performance of health programmes. We applaud John Walley et al.’s call for operations research designed from the perspective of the in-country decision-makers and agree with the approaches outlined in this editorial.1 However, in addition to the factors discussed, we think that there should also be additional focus on the quality of operations research and the dissemination process of findings from such research. This has tremendous implications for the importance of operations research technology transfer to the national level. Although the importance of quality in research might seem obvious, we have found that quality and methodological rigour are often lacking. To identify proven strategies aimed at improving routine immunization services in developing countries, we recently conducted a literature review assessing both results and methodological rigour. The lack of quality and rigour for most studies and the overall paucity of well-conducted published studies was striking, especially in light of the longstanding Expanded Programme on Immunization (EPI) and the widely recognized importance and cost-effectiveness of achieving high population coverage with vaccines.2 For our study, we conducted an exhaustive literature search of published and grey literature from 1975 to 2004 using multiple databases and 41 search terms (e.g. immunization, vaccination, developing countries) and identified more than 9000 papers and manuscripts relating to routine immunizations in less-developed countries.3 Of these, only 60 papers described results or lessons learned from a specific routine immunization strategy, thus meeting our inclusion criteria. The quality and rigour of these 60 papers were then evaluated by two reviewers using a standardized assessment instrument (except in the case of observational studies where only one reviewer was used). A rating was assigned based on the presence of elements considered critical to the scientific quality of the study and to the reviewers’ ability to appropriately understand and interpret the intervention and its reported impact. Papers received lower scores if the reviewers thought that there was an inadequate description of the methodology (39%); inappropriate selection of comparison groups (28%); inadequate sample size (41%); incomplete discussion such as lack of comparison with other study results (59%) or not addressing limitations (48%); inadequate analysis such as not accounting for confounding factors (54%); or the inability to attribute results to the intervention (28%). Through this study, it became evident that there is a need for improved research quality. We agree with Walley et al. that research should ideally focus on the operational needs of the countries and that this paradigm is sometimes challenged by realities of limited public health resources and competing interests. Operations research itself is often not a main concern nor an area of expertise for many ministries of health, especially given their many divergent priorities. On the other hand, external partners who traditionally have the resources to conduct such research may not always understand the primary operational challenges confronting ministries of health and may have research interests disparate from them. The resulting studies and their results may not be immediately applicable to the countries’ programmatic needs. It is the responsibility of both parties to ensure that relevant research of the highest quality is conducted. The Global Immunization Vision and Strategy suggests that: “Where appropriate, perform operations research and evaluation of ‘what works’ to improve the delivery of immunization and to make systems more effective, efficient and equitable in order to improve immunization coverage.”4 Additional funds for immunization and health services research have recently been made available through the GAVI Alliance’s Health System Strengthening efforts and potentially through other mechanisms to support much-needed operational research. Ministries of health and external partners must work together to ensure that these additional resources are used to identify and test innovative approaches and strategies to meet the priority programmatic needs and ensure that the operations research is of “high quality”. Finally, the paucity of studies found in our review suggests a need for additional mechanisms to disseminate findings of operations research conducted at the local level. The lack of studies may be due in part to the number of accessible journals or their requirements for submission. We support efforts to ensure that systems are in place to foster communication between ministries of health on what has been tried, what works and does not work. The failure to provide a forum to disseminate the operations research results to other countries is a missed opportunity. ■

Developments in immunization planning in Cambodia - rethinking the culture and organization of national program planning

As part of its health system reconstruction following decades of civil war, Cambodia undertook a program of health sector reform in 1996 to expand coverage of essential health services to the population of 14 million, 80% of whom are resident in over 13 000 rural villages. During this reform period, one of the major national health programs, the National Immunization Program (NIP), adapted its planning system to accommodate changes in social and health sector structure. The aims of this article are to review changes made in the approach to national immunization planning and to illustrate how these adaptations can help identify future challenges and opportunities for further improving immunization coverage in Cambodia. Sources of information for the study include immunization plans and data from international and national sources, as well as data from the national health information system. Findings of review: Management and service delivery reforms undertaken by the NIP include (1) strengthening links between immunization, health sector and international health planning; (2) development of immunization program multiyear and financial sustainability plans; (3) strengthening of national program decision making structures and processes; (4) widening of decentralized stakeholder participation in health planning; and (5) implementation of service level micro-planning. These management reforms have been associated with significant improvement in public health program performance and outcomes during this period (2003-2006). There has been an increase in vaccination coverage for children under the age of one year, over a five-year period (increase of 29% for fully immunized child at one year of age), with no significant differences in vaccination rates between urban and rural areas, and a sharp decrease in the incidence of vaccine preventable diseases. The NIP is now well positioned to take on additional challenges in coming years associated with expanding international partnerships, the continued development of civil society, further health system decentralization, and the requirement to further improve coverage in support of global and regional disease elimination goals. However, as costs continue to rise, planners in the future will need to emphasize the economic and public health benefits of immunization programs in order to sustain increasing levels of national and international investment.

Relationship Among Sociodemographic Factors, Clinical Conditions, and Health-related Quality of Life: Examining the EQ-5D in the U.S. General Population

Health-related quality of life (HRQL) measures are used increasingly in evaluations of clinical and population-based outcomes and in economic analyses. We investigate the influence of demographic, socioeconomic, and chronic disease factors on the HRQL of a representative U.S. sample. We examined data from 13,646 adults in the 2000 Medical Expenditure Panel Survey, a nationally representative sample of the U.S. general population, who completed a self-administered questionnaire containing the EQ-5D, a preference-based measure. We assessed the relationships between EQ-5D scores and sociodemographic variables, including age, sex, race/ethnicity, income and education, and six common chronic conditions. In fully adjusted models, EQ-5D scores decreased with increasing category of age and were lower for persons with a lower income and educational attainment as well as each of the six conditions. Although the EQ-5D scores were lower for females and Whites compared with Blacks such differences were not of a magnitude considered to be clinically important. In the U.S., sociodemographic factors and clinical conditions are strongly associated with scores on the EQ-5D. Population health studies and risk-adjustment models should account and adjust for these factors when assessing the performance of health programs and clinical care.

10 best resources for … measuring population health

Health information – a key ingredient in policy formation and programme planning – has often proven to be a scarce commodity in places where it is needed most. Beginning in 1978 with the declaration of the Alma Ata Conference on ‘Health for All by the Year 2000’ international momentum to monitor the performance of health programmes and hold governments accountable for progress in improving health has grown. Stimulated now by the need to measure national progress toward achieving the Millennium Development Goals and by the growing number of international initiatives such as the Global Fund to Fight AIDS TB and Malaria and the Global Alliance for Vaccines and Immunization a new lens is being focused on population health information. An important source of information on health status behaviour health care access and utilization amongst the general population are government-sponsored crosssectional and longitudinal household survey programmes. In some countries the interview is expanded to include physician assessments tests of physical functioning blood pressure lung capacity and anthropometric data and in some survey data may be linked to medical records or vital registers. The National Health Interview Survey (NHIS) is a cross-sectional sample survey conducted annually in the USA by the Centers for Disease Control and Prevention. The NHIS collects self-reported data on behaviours including tobacco use and physical activity chronic disease conditions health care utilization and satisfaction with multiple dimensions of health. The child module includes information on asthma allergies immunization and days lost from school due to illness or injury. The NHIS website provides public access to primary survey data and documentation for both the child and adult questionnaires. A wide variety of such survey programmes exists around the world and at the present time valid international comparison of data from such surveys is limited. (excerpt)

Coalition Seeks Reforms to Improve MH System Accountability

Back to table of contents Previous article Next article Government NewsFull AccessCoalition Seeks Reforms to Improve MH System AccountabilityChristine LehmannChristine LehmannSearch for more papers by this authorPublished Online:20 Aug 2004https://doi.org/10.1176/pn.39.16.0390004Congress has proposed changing federal mental health and substance abuse block grants to grants that are tied to performance-based measures and outcomes.“With so much effort and funding focused on these programs, it is crucial to understand the effectiveness of these services,” said Sen. Mike DeWine (R-Ohio) at last month's Senate hearing on performance and outcomes measures.DeWine chairs the Health, Labor, Education, and Pensions (HELP) subcommittee that oversees the funding reauthorization of the Substance Abuse and Mental Health Services Administration (SAMHSA). DeWine also serves on the Senate Appropriations Committee.The proposed Partnership Performance Grants (PPGs) are scheduled to replace the mental health block grants in Fiscal 2005 and substance abuse grants in Fiscal 2006. However, SAMHSA has not completed the reporting requirements for states to implement the PPGs.Further, state officials cannot change their data systems until SAMHSA finalizes its reporting requirements, according to a June report by the Government Accounting Office.Howard Goldman, M.D., Ph.D., testified on behalf of the Campaign for Mental Health Reform that there is widespread agreement among the various groups involved with the mental health system on the outcomes that systems and services are designed to achieve.Goldman is a professor of psychiatry and director of mental health policy studies at the University of Maryland and editor in chief of the APA journal Psychiatric Services. He has been a consultant to the President's New Freedom Commission on Mental Health and a senior scientific editor of the Surgeon General's 1999 report on mental health.The campaign's mission is to transform mental health care in America and“ serve as the mental health community's united voice in promoting federal policy changes,” Goldman testified. Among its members are APA, Children and Adults With Attention-Deficit/Hyperactivity Disorder, Depression and Bipolar Support Alliance, Bazelon Center for Mental Health Law, National Alliance for the Mentally Ill, National Association of State Mental Health Program Directors, and the National Mental Health Association.The seven consensus outcomes that health systems and services are intended to achieve are symptom reduction; enhanced social supports; community participation; improvement in employment or, in the case of youth, age-appropriate functioning; reduced homelessness and inappropriate hospitalization; improved general health status; and decreased contact with the criminal justice system.Performance and outcomes measures come with costs, however. Identifying and implementing measures for planning, budgeting, monitoring, and quality improvement is “enormously complex, expensive, and labor intensive,” Goldman said.“Resources are necessary to update or, in some cases, create information technology systems that would enable states and counties to collect, access, link, and analyze the relevant data,” he said.The federal government has an important role to play in expanding performance measurement systems by doing the following:Developing meaningful measures and definitions in consultation with the different groups involved in the mental health system.Ensuring that measures are disseminated and implemented.Funding states and counties that are creating performance and outcomes measures.“The federal commitment to this effort has been minimal so far, with states receiving grants ranging between $100,000 and $150,000 per year to move billion-dollar systems,” Goldman testified. Only a few states—Ohio, Texas, Colorado, Washington, and Oklahoma—have developed data systems capable of measuring publicly funded mental health programs' effectiveness and patient outcomes, said Goldman“SAMHSA must ensure that the data it is requiring the states to report are of value not only to the federal government, but also to the states and counties in planning, quality improvement, and contracts management.”To the extent those goals are not aligned, the federal government must be prepared to cover more of the financial burden, Goldman emphasized to the Senate subcommittee.“Transforming mental health care will require fundamental changes in all social services at the federal, state, and local levels, including Medicaid, vocational rehabilitation, and housing,” Goldman said.“If SAMHSA is to be responsible for monitoring performance and outcomes of mental health programs, then it must be given broader authority to work collaboratively with all of the other systems and agencies whose policies affect consumers and their families. Otherwise, the fragmented public mental health system will only be perpetuated.”Howard Goldman, M.D.: Identifying and implementing measures for planning, budgeting, monitoring, and quality improvement is “enormously complex, expensive, and labor intensive.”Goldman's testimony is posted online at<www.mhreform.org/news/7-19-04goldmantestimony.htm>. More information about the Campaign for Mental Health Reform is posted at<www.mhreform.org/>.▪ ISSUES NewArchived

Consumer satisfaction and incarceration after treatment.

Consumer satisfaction is increasingly used to measure community mental health program performance. Understanding the relationship between consumer satisfaction and treatment outcomes is only beginning. This article adds to this understanding by reporting on an assessment of the relationship between consumer evaluation of community mental health services and incarceration after treatment in a statewide system of care. Results indicate that satisfaction with services is related to incarceration after treatment, with satisfied consumers having lower incarceration rates. These results support the use of self-reported consumer satisfaction as a measure of mental health program performance.

Evaluation designs for adequacy, plausibility and probability of public health programme performance and impact.

The question of why to evaluate a programme is seldom discussed in the literature. The present paper argues that the answer to this question is essential for choosing an appropriate evaluation design. The discussion is centered on summative evaluations of large-scale programme effectiveness, drawing upon examples from the fields of health and nutrition but the findings may be applicable to other subject areas. The main objective of an evaluation is to influence decisions. How complex and precise the evaluation must be depends on who the decision maker is and on what types of decisions will be taken as a consequence of the findings. Different decision makers demand not only different types of information but also vary in their requirements of how informative and precise the findings must be. Both complex and simple evaluations, however, should be equally rigorous in relating the design to the decisions. Based on the types of decisions that may be taken, a framework is proposed for deciding upon appropriate evaluation designs. Its first axis concerns the indicators of interest, whether these refer to provision or utilization of services, coverage or impact measures. The second axis refers to the type of inference to be made, whether this is a statement of adequacy, plausibility or probability. In addition to the above framework, other factors affect the choice of an evaluation design, including the efficacy of the intervention, the field of knowledge, timing and costs. Regarding the latter, decision makers should be made aware that evaluation costs increase rapidly with complexity so that often a compromise must be reached. Examples are given of how to use the two classification axes, as well as these additional factors, for helping decision makers and evaluators translate the need for evaluation--the why--into the appropriate design--the how.

A risk-adjusted measure of hospitalization rates for evaluating community mental health program performance.

The authors describe the use of risk-adjusted hospitalization rates to measure community mental health treatment outcomes. The risk adjustment involves comparing rates of hospitalization subsequent to treatment with rates of hospitalization prior to treatment. The research uses a probabilistic methodology that reliably estimates caseload overlap by comparing the distribution of the dates of birth observed in data sets to the distribution of dates of birth in the general population. Findings indicate that risk-adjusted hospitalization rates are substantially different than unadjusted rates. Half of the community programs in one state consistently achieved positive outcomes in four consecutive years; other programs had mixed results or no change.