Health System Performance Measurement Research Articles

Evaluate the Implementation of Occupational Health and Safety (OHS) Management System Performance Measurement at PT. XYZ Medan to minimize Extreme Risks

OHS is an activities to ensure and protect the safety and health of the workforce through prevention of occupational injuries and occupational diseases. One way of prevention of work accident is through to implement OHS System. The OHS process should begin with a good planning process to ensure that its implementation complies with the desired policies and objectives. PT. XYZ Medan has implemented OHS System which is regulated in the procedures and manuals of the company’s management system. The purpose of this study was to assess and evaluate the Occupational Safety and Health Management System and also to evaluate Hazard Identification in the company. This research uses checklist method of all assessment criteria of OHS management system based on Indonesian Government Regulation, and for application of OHS in company use qualitative data through spreading of quisioner to managers, supervisors and workers. Based on the result of observation, it is found that the implementation of OHS in the company is not fully complied with the Indonesian Government Regulation. There are still 24 unfulfilled criteria that is concerning policy, plan of OHS, document control, product control, work security, hazard inspection, material management, skills and abilities. This study resulted in the acquisition of the results of analysis in the achievement level of OHS is 92.2% and has been included in the satisfactory category so worthy to be certified and ranked the golden flag. And analysis of the implementation and implementation of OHS to the established principles has been above 50% and can be interpreted that OHS is considered important to be applied in the company.

Appraising the value of evidence generation activities: an HIV modelling study

IntroductionThe generation of robust evidence has been emphasised as a priority for global health. Evidence generation spans a wide range of activities including clinical trials, surveillance programmes and health system...

Improving the Measurement of Health System Performance across the Rural-Urban Continuum

IntroductionWe report on key performance indicators to highlight quality and variation in health care. Given Ontario’s diverse geography, we have prioritized improving measurement across the rural-urban continuum. This will improve our ability to discern the impact of geography on health care and health status to inform planning and decision making.
 Objectives and ApproachBuilding on previous work to advance measurement of equity in health care, we struck a technical working group of experts to review methods for stratifying health system performance data by geographic location in the Ontario context. These methods were applied to a set of key performance indicators. The working group’s review of the results of this analysis will lead to recommendations for the best method to refine and standardize how geographic location is measured and stratified. This will improve our ability to discern the impact of geography on health system performance and health status for our suite of public-reporting products.
 ResultsThe technical working group identified three methodologies for consideration that used linked postal code data: Population Centre (POPCTR), Statistical Area Classification (SAC) and a hybrid POPCTR/SAC methodology. These methods were tested against a set of key performance indicators across dimensions of quality including timeliness, effectiveness, population health and health outcomes. The results show that, in the health system performance dimensions of effectiveness and timeliness, as well as for a subset of health outcomes, there is variation in performance across the urban-rural continuum, though not always in a linear way. This may reflect differences in health care access, health risk factors, sociodemographic or socioeconomic characteristics across the urban-rural continuum. More definitive conclusions and recommendations will be available when the working group meets to review the results.
 Conclusion/ImplicationsIdentifying a robust methodology for measuring performance across geographic locations will improve our ability to discern the impact of geography on health care including where geography may impact access and effectiveness of services as well as health outcomes. This information will enable better health system planning and decision-making.

Measurement of health system performance at district level: A study protocol.

BackgroundLimited efforts have been observed in low and middle income countries to undertake health system performance assessment at district level. Absence of a comprehensive data collection tool and lack of a standardised single summary measure defining overall performance are some of the main problems. Present study has been undertaken to develop a summary composite health system performance index at district level.MethodsA broad range of indicators covering all six domains as per building block framework were finalized by an expert panel. The domains were classified into twenty sub-domains, with 70 input and process indicators to measure performance. Seven sub-domains for assessing health system outputs and outcomes were identified, with a total of 28 indicators. Districts in Haryana state from north India were selected for the study. Primary and secondary data will be collected from 378 health facilities, district and state health directorate headquarters. Indicators will be normalized, aggregated to generate composite performance index at district level. Domain specific scores will present the quality of individual building block domains in the public health system. Robustness of the results will be checked using sensitivity analysis.Expected impact for public health:The study presents a methodology for comprehensive assessment of all health system domains on basis of input, process, output and outcome indicators which has never been reported from India. Generation of this index will help identify policy and implementation areas of concern and point towards potential solutions. Results may also help understand relationships between individual building blocks and their sub-components.Significance for public healthMeasuring performance of health system is important to understand progress and challenges, and create systems that are efficient, equitable and patient-focused. However, very few assessments of such nature have been observed in low and middle income countries, especially at district level, mainly because of methodological challenges. This study presents a methodology for comprehensive assessment of all domains of health system and generation of a composite Health System Performance Index on the basis of input, process, output and outcome indicators. It will help identify policy and implementation problems worthy of attention and point towards potential solutions to health system bottlenecks resulting in poor performance. The results may also help better understand the relationships between individual building blocks and their sub-components and the overall performance of the health system.

Health System Performance Measurement: A UK Perspective.

In response to the lead article by Kuluski et al. (2017), this commentary draws on UK debates around the quality of healthcare and on the author's personal experience training as a social worker. Judging whether a service has "worked" has to involve understanding what a good outcome would be for the person receiving it. While carers provide important insights here, the "user" may have a different view, and reconciling different perspectives can be challenging. Although individual services may have fulfilled their part of the process, it is important to look at the person's overall experience (although this is difficult to achieve in practice). There is also no point measuring patient experience if we are not prepared to really listen to the feedback we receive. Moving forward, the UK's personalization agenda - which focuses on giving people greater choice and control - may enhance the aims of Kuluski et al.'s work.

Experience of Care as a Critical Component of Health System Performance Measurement: Recommendations for Moving Forward.

People's experiences can provide critical guidance on how to better meet their quality of life and care needs and deploy resources more appropriately. To maximize the utility of experience data and to advance the current debate, we present four recommendations: (1) measuring experiences outside the healthcare system can provide insight into what needs to change within the healthcare system; (2) focusing on patient experience is necessary but insufficient, (family) caregiver insights and experiences require attention and can provide insight into the needs of the patient; (3) moving from "one time/single sector" measurement of experience to iterative, ongoing measurement across sectors better reflects the true lived experience of patients (especially those with complex care needs) and their caregivers; and (4) embedding measurement within engagement-capable environments that adequately resource patients, caregivers, and providers to work together is required to move from collection to meaningful change. Applying these recommendations requires a longer-term vision, shifting from provider-centred to person-centred models of care, and a deep understanding of the structural, cultural, and normative barriers to measuring care experiences.

Activities performed by pharmacists integrated in family health teams: Results from a web-based survey.

Family health teams (FHTs), an interprofessional primary care practice model, were established in Ontario in 2005. As of October 2014, 191 FHT organizations were in operation, and 111 (58%) included one or several pharmacists. The objective of this study was to document the focus of pharmacist activities in FHTs. We invited all 155 known FHT pharmacists to a web-based survey. The survey was constructed using information obtained from previously done semi-structured telephone interviews with pharmacists working in FHTs. The survey consisted of a list of activities, grouped into 5 main categories, and participants were asked to estimate time spent on each category. Free-text response questions allowed participants to identify activities not listed. Survey results were analyzed using descriptive statistics and content analysis for open-ended responses. Seventy (45%) pharmacists completed the survey. The mean respondent age was 43 years, and the average length of time working in an FHT was 4 years. All pharmacists reported being engaged in some form of direct patient care, including managing single therapeutic issues (96%), conducting general medication reviews (70%) and medication reconciliation after hospitalization (63%). Most reported providing education and drug information (83%). Pharmacists felt their work would increase patient medication adherence (94%) and physician adherence to recommended guidelines (86%), as well as reduce inappropriate prescribing (93%), polypharmacy (90%), emergency room and hospital utilization (70%-81%). Pharmacists in FHTs are strongly focused on direct patient care activities, managing specific medication issues and unstructured drug information to physicians. The majority of pharmacists reported that their activities had a substantial impact on patient medication use and health. The findings from this survey can assist pharmacists new to FHT practice, health policy planning and health research to assess associations between pharmacist activities and measures of health and health care system performance.

Patients’ Level of Satisfaction with the Health Care Services Received at Outpatient Departments in Kilimanjaro Region, Tanzania

Introduction: Patients’ satisfaction is a measure of health system performance and can be used as a foundation of encouraging health care providers to become more responsible to their patients. Satisfaction manifests itself as a distribution, access and utilization of health services. This study aimed to determine patients’ level of satisfaction with the health care services received in outpatient department in Kilimanjaro Region. Methods: A cross-sectional study design was conducted at Mawenzi Regional Referral Hospital (MRRH), Same District Hospital (DH) and Huruma Designated District Hospital (DDH) using Donabedian model to determine patients’ level of satisfaction with the health care services received at outpatient departments in Kilimanjaro Region. Systematic sampling was used to select 450 patients. Service quality questionnaire was used for data collection. Descriptive statistics was performed. Paired sample t-test, independent sample t-test and one-way ANOVA with Bonferroni posthoc test were used to compare mean gap scores. A p-value of <0.05 was considered statistically significant. Results: The overall patient’s level of satisfaction with health service provision at OPD in all three hospitals was 20%. The gap on health services provision in all three hospitals was -37.0 (± 47.0) signifying overall dissatisfaction among patients. No statistically significant difference (p<0.05) on the gap in satisfaction with quality of service provision existed between MRRH and Huruma DDH as well as between MRRH and Same DH. However, a significant gap difference was demonstrated between Huruma DDH and Same DH (p, 0.002) with a smaller gap in satisfaction at Huruma DDH compared to Same DH (-28.0 versus -46.7). The highest level of dissatisfaction in the five service dimensions tested was on empathy (-7.8). Conclusion: Patients attending OPD in all three hospitals demonstrate an overall dissatisfaction with quality of care. Hospital managements should focus on: improvement of patient-provider relationship, availability of equipment and drugs and affordability of hospital bills.

PopHR: a knowledge-based platform to support integration, analysis, and visualization of population health data.

Population health decision makers must consider complex relationships between multiple concepts measured with differential accuracy from heterogeneous data sources. Population health information systems are currently limited in their ability to integrate data and present a coherent portrait of population health. Consequentially, these systems can provide only basic support for decision makers. The Population Health Record (PopHR) is a semantic web application that automates the integration and extraction of massive amounts of heterogeneous data from multiple distributed sources (e.g., administrative data, clinical records, and survey responses) to support the measurement and monitoring of population health and health system performance for a defined population. The design of the PopHR draws on the theories of the determinants of health and evidence-based public health to harmonize and explicitly link information about a population with evidence about the epidemiology and control of chronic diseases. Organizing information in this manner and linking it explicitly to evidence is expected to improve decision making related to the planning, implementation, and evaluation of population health and health system interventions. In this paper, we describe the PopHR platform and discuss the architecture, design, key modules, and its implementation and use.

Public-private mix and performance of health care systems in CEE and CIS countries

The role of the public and private sector in health care systems remains one of the crucial problems of these systems' operation. The purpose of this research is to identify the relationships between the performance of health systems in CEE and CIS (Central and Eastern Europe and Commonwealth of Independent State) countries, and the mix of public-private sector in the health care of these countries. The study uses a zero unitarization method to construct three measures of health system performance in the following areas: (1) resources; (2) services; and (3) health status. The values of these measures are correlated with the share of public financing that represents the public-private mix in the health systems. The data used is from World Health Organization’s Health for All Database for 23 CEE and CIS countries and comprises the year 2010. The results show that the performance of health systems in the countries investigated is positively associated with a higher proportion of public financing. The strongest relationship links public financing with performance in the area of services production. For policy makers, these results imply that health systems in post-communist transition economies could be susceptible to a decreasing role of the state and that growing reliance on the market mechanism in health care can deteriorate the operation of these systems.

Using Data Envelopment Analysis to Address the Challenges of Comparing Health System Efficiency

AbstractEfficiency is one of the most potent measures of health system performance and is of particular interest to policy makers because it seeks to assess the valued outcomes of a health system in relation to the resources that are sacrificed to achieve those outcomes. However, the production process of the health care system is a complex sequence, and most indicators are only able to capture part of that process; these indicators offer limited scope for analysis. While researchers have previously constructed composite indicators which combine partial measures into a single number, the weights used for aggregating data can be contentious and may not be universally applicable across systems. Data envelopment analysis (DEA) is most often used to compare the productivity of different producing entities, including health systems. In this article, we instead propose a method that relies on DEA to construct composite health system efficiency indicators from several partial efficiency measures. Among other noted benefits, this enables the construction of composite indicators where different weights are attached to partial indicators for each country, allowing countries to be viewed according to the weights that cast each in the best light. Our application of this method suggests that there is reasonable consistency among the countries that are found to be efficient.

Exploring the state of health and safety management system performance measurement in mining organizations

Complex arguments continue to be articulated regarding the theoretical foundation of health and safety management system (HSMS) performance measurement. The culmination of these efforts has begun to enhance a collective understanding. Despite this enhanced theoretical understanding, however, there are still continuing debates and little consensus. The goal of the current research effort was to empirically explore common methods to HSMS performance measurement in mining organizations. The purpose was to determine if value and insight could be added into the ongoing approaches of the best ways to engage in health and safety performance measurement. Nine site-level health and safety management professionals were provided with 133 practices corresponding to 20 HSMS elements, each fitting into the plan, do, check, act phases common to most HSMS. Participants were asked to supply detailed information as to how they (1) assess the performance of each practice in their organization, or (2) would assess each practice if it were an identified strategic imperative. Qualitative content analysis indicated that the approximately 1200 responses provided could be described and categorized into interventions, organizational performance, and worker performance. A discussion of how these categories relate to existing indicator frameworks is provided. The analysis also revealed divergence in two important measurement issues; (1) quantitative vs qualitative measurement and reporting; and (2) the primary use of objective or subjective metrics. In lieu of these findings we ultimately recommend a balanced measurement and reporting approach within the three metric categories and conclude with suggestions for future research.

A Critique of Health System Performance Measurement.

Health system performance measurement is a ubiquitous phenomenon. Many authors have identified multiple methodological and substantive problems with performance measurement practices. Despite the validity of these criticisms and their cross-national character, the practice of health system performance measurement persists. Theodore Marmor suggests that performance measurement invokes an "incantatory response" wrapped within "linguistic muddle." In this article, I expand upon Marmor's insights using Pierre Bourdieu's theoretical framework to suggest that, far from an aberration, the "linguistic muddle" identified by Marmor is an indicator of a broad struggle about the representation and classification of public health services as a public good. I present a case study of performance measurement from Alberta, Canada, examining how this representational struggle occurs and what the stakes are.

Use of a commercial ELISA for the detection of measles-specific immunoglobulin G (IgG) in dried blood spots collected from children living in low-resource settings.

Seroepidemiological monitoring of population immunity to vaccine-preventable diseases is critical to prevent future outbreaks. Dried blood spots (DBS), drops of capillary blood dried on filter paper, are an affordable, minimally invasive alternative to venipuncture for collecting blood in field settings. However, few proven methods exist to analyze DBS for the presence of protective antibodies. This study validates a novel technique for measuring measles-specific immunoglobulin G (IgG) in capillary DBS using a commercial ELISA. The predictive performance of a new method for analyzing DBS was tested by comparing matched serum and DBS samples from 50 children. The accuracy, precision, and reliability of the procedure were evaluated, and the optimal cut points to classify positive and negative samples were determined. The method was then applied to 1,588 DBS collected during a large survey of children in Mexico and Nicaragua. Measles-specific IgG in serum samples were 62% negative, 10% equivocal, and 28% positive. In comparisons with matched serum, DBS results were 100% sensitive and 96 · 8% specific, and agreed in 46 of 50 (92%) cases. The inter-assay and intra-assay coefficients of variation from kit-provided controls were greater than desired (24.8% and 8.4%, respectively). However, in predictive simulations the average misclassification was only 3.9%. Procedures were found to be acceptable to surveyors and participants. Analyzing DBS collected in low-resources settings is a feasible and accurate means of measuring population immunity to measles and should be used to generate objective measures of health status and health system performance.

Tracking access, utilization and health system responsiveness to inform evidence-based health care policy: The case of Qatar

Background: From 2004 to 2010, the population of Qatar increased by 128%, from 744,029 to 1,699,435 inhabitants. This sharp rise in population led to declines in per-capita health care personnel and hospital bed capacity. Aims and objectives: To examine patient and provider characteristics associated with health system performance measures, the variation in these measures between Qataris and non-Qataris, and whether these measures have changed over time. Methods: We examine access, utilization, health system responsiveness and satisfaction measures, using multivariate regression analysis based on health surveys from 2006 and 2010. For purposes of representativeness, 2006 analysis applied population weights, and 2010 analysis applied weights constructed by using age, gender, nationality and occupation, based on the 2010 Census. Results: For access, we find no differences between Qataris and non-Qataris in the probability of having a usual source of care (USC). For utilization, Qatari women are 67% more likely than non-Qatari women to have an inpatient stay; both Qatari men and women are more likely to have any outpatient visits (48% and 35%) compared to non-Qataris. For health system responsiveness, for outpatient care, those who go to private clinics report higher ratings than those who go to Primary Health Care Corporation (PHCC) centers in 2006. Those who go to private clinics, private hospitals and other/employer clinics report higher ratings than those who go to PHCC centers in 2010. For satisfaction with the health care system, non-Qataris are twice as likely as Qataris to report high satisfaction in 2006. Conclusions: We find that satisfaction with outpatient services for public providers is lower than ratings for private providers, most significantly for waiting times (odd ratio of 1.20 [0.15] vs. 0.77 [0.08]). For satisfaction with the health care system, non-Qataris are twice as likely as Qataris to report high satisfaction; although for access, we find no difference between Qataris and non-Qataris in the probability of having a USC. Policy implications include improving public services, increasing provider capacity and improving access to private providers. In addition, a major policy action was the launch of a mandatory social health insurance scheme in 2013, which enabled choice of provider and access to public and private providers. As such, increased provider capacity, public private partnerships, and competition between public and private providers are likely to improve services among public providers, including waiting times.

Breaking the Deadlock: Lessons from Pan-Canadian Organizations.

The authors of the lead essay in this issue propose a set of fiscal and other levers for accelerating Canadian healthcare reform. Among the mechanisms they endorse is the concept of a learning health system, which would encourage collaboration and information sharing between jurisdictions. From health system performance measurement to exchange of best practices, a number of the foundational elements of learning health systems have parallels or antecedents in functions undertaken by pan-Canadian organizations that address healthcare issues relevant to multiple jurisdictions. Experiences and outcomes of these organizations may therefore be instructive when considering proposals for healthcare reform, such as those made by Gardner, Fierlbeck, and Levy.

Developing a dashboard to help measure and achieve the triple aim: a population-based cohort study.

BackgroundHealth system planners aim to pursue the three goals of Triple Aim: 1) reduce health care costs; 2) improve population health; and 3) improve the care experience. Moreover, they also need measures that can reliably predict future health care needs in order to manage effectively the health system performance. Yet few measures exist to assess Triple Aim and predict future needs at a health system level. The purpose of this study is to explore the novel application of a case-mix adjustment method in order to measure and help improve the Triple Aim of health system performance.MethodsWe applied a case-mix adjustment method to a population-based analysis to assess its usefulness as a measure of health system performance and Triple Aim. The study design was a retrospective, cohort study of adults from Ontario, Canada using administrative databases: individuals were assigned a predicted illness burden score using a case-mix adjustment system from diagnoses and health utilization data in 2008, and then followed forward to assess the actual health care utilization and costs in the following year (2009). We applied the Johns Hopkins Adjusted Clinical Group (ACG) Case-Mix System to categorize individuals into 60 levels of healthcare need, called ACGs. The outcomes were: 1) Number of individuals per ACG; 2) Total system costs per ACG; and 3) Mean cost per person per ACG, which together formed a health system “dashboard”.ResultsWe identified 11.4 million adults. 16.1% were aged 65 or older, 3.2 million (28%) did not use health care services that year, and 45,000 (0.4%) were in the highest acuity ACG category using 12 times more than an average adult. The sickest 1%, 5% and 15% of the population use about 10%, 30% and 50% of total health system costs respectively. The dashboard measures 2 dimensions of Triple Aim: 1) reduced costs: when total system costs per ACG or when average costs per person is reduced; and 2) improved population health: when more people move into healthier rather than sicker ACGs. It can help to achieve the third aim, improved care experience, when ACG utilization predictions are reported to providers to proactively develop care plans.ConclusionsThe dashboard, developed via case-mix methods, measures 2 of the Triple Aim goals and can help health system planners better manage their health delivery systems.

Development of evidence-based Australian medication-related indicators of potentially preventable hospitalisations: a modified RAND appropriateness method

ObjectiveIndicators of potentially preventable hospitalisations have been adopted internationally as a measure of health system performance; however, few assess appropriate processes of care around medication use, that if followed may...

Assessing Avoidable Hospitalisation using person-level administrative data

Background Avoidable hospitalization (AH) has been widely adopted as an indicator of the accessibility and overall effectiveness of primary care (PC). The objective of the study is to validate AH as a measure of health system performance in the Latium Region exploring the relationships between hospital admission for ambulatory care sensitive conditions (ACSCs) and a specific set of indicators related to GP’s activities. Methods A relational database resource has …

Public satisfaction as a measure of health system performance: A study of nine countries in the former Soviet Union

Measurement of health system performance increasingly includes the views of healthcare users, yet little research has focussed on general population satisfaction with health systems. This study is the first to examine public satisfaction with health systems in the former Soviet Union (fSU). Data were derived from two related studies conducted in 2001 and 2010 in nine fSU countries, using nationally representative cross-sectional surveys. The prevalence of health system satisfaction in each country was compared for 2001 and 2010. Patterns of satisfaction were further examined by comparing satisfaction with the health system and other parts of the public sector, and the views of health care users and non-users. Potential determinants of population satisfaction were explored using logistic regression. For all countries combined, the level of satisfaction with health systems increased from 19.4% in 2001 to 40.6% in 2010, but varied considerably by country. Changes in satisfaction with the health system were similar to changes with the public sector, and non-users of healthcare were slightly more likely to report satisfaction than users. Characteristics associated with higher satisfaction include younger age, lower education, higher economic status, rural residency, better health status, and higher levels of political trust. Our results suggest that satisfaction can provide useful insight into public opinion on health system performance, particularly when used in conjunction with other subjective measures of satisfaction with government performance.