Health System Performance Assessment Research Articles

Regional variation in access to and quality of acute stroke care: results of Germany’s Health System Performance Assessment pilot, 2014–2020

BackgroundHealth System Performance Assessments (HSPA) and analyses of unwarranted regional variation in health care both aim at identifying strengths and weaknesses of health systems to improve care. Applying HSPA’s conceptual approach of interrelated health system dimensions (e.g., access, quality) to regional levels might help to better understand variation in care to reduce inequity and improve performance.MethodsWe use four indicators identified and analysed in a pilot study for a German HSPA to assess variation in access to and quality of acute stroke care between Germany’s 16 federal states and urban vs. rural regions from 2014 to 2020. Stroke unit (SU) density, share of the population reaching a SU within 30 min by car, share of inpatient stroke cases treated in a hospital with a SU, and inpatient mortality were computed based on hospital quality reports and discharge data covering all acute care hospitals. Inpatient mortality was adjusted for age, sex, stroke type, and comorbidities.ResultsAbout 500 SU were identified, i.e., 2.0 per 1,000 inpatient stroke cases. Almost 95% of Germans could reach a SU hospital within 30 min (rural: 90%; urban: 99%; > 88% in all states but one). The share of inpatient stroke cases treated in a SU hospital increased to 93% with a decreasing span between rural (92%) and urban (95%) regions and between states (74–98%). Inpatient mortality stagnated around 8.5% and increased to 9.0% in 2020 (rural: 8.7%; urban: 9.2%; states: 7.0–9.7%, one outlier of 13.3%).ConclusionsThe results especially revealed varying performance patterns in access to and quality of acute stroke care between the federal states, indicating different areas for improvement which might be addressed by more targeted policy measures in the future.

Population assessment of health system performance in 16 countries.

To demonstrate how the new internationally comparable instrument, the People's Voice Survey, can be used to contribute the perspective of the population in assessing health system performance in countries of all levels of income. We surveyed representative samples of populations in 16 low-, middle- and high-income countries on health-care utilization, experience and confidence during 2022-2023. We summarized and visualized data corresponding to the key domains of the World Health Organization universal health coverage framework for health system performance assessment. We examined correlation with per capita health spending by calculating Pearson coefficients, and within-country income-based inequities using the slope index of inequality. In the domain of care effectiveness, we found major gaps in health screenings and endorsement of public primary care. Only one in three respondents reported very good user experience during health visits, with lower proportions in low-income countries. Access to health care was rated highest of all domains; however, only half of the populations felt secure that they could access and afford high-quality care if they became ill. Populations rated the quality of private health systems higher than that of public health systems in most countries. Only half of respondents felt involved in decision-making (less in high-income countries). Within countries, we found statistically significant pro-rich inequalities across many indicators. Populations can provide vital information about the real-world function of health systems, complementing other system performance metrics. Population-wide surveys such as the People's Voice Survey should become part of regular health system performance assessments.

Health system performance assessment and reforms, Oman.

To prioritize key areas of action and investment for the next strategic cycle of national development plans (2026-2031) in Oman, we needed a holistic view of the country's health system and its main deficiencies and inefficiencies. Informed by the World Health Organization framework, our team of seven national health ministry staff and two international experts conducted a rapid health system performance assessment. We used already available data to identify system bottlenecks and their potential root causes, verifying our findings with key informant interviews. Oman's 4.9 million population is relatively young (average age 28years) but ageing, with a mounting burden of chronic diseases. While health-care services are free for Omani nationals, more than 1.5 million expatriates rely on out-of-pocket payments for health-care services. Strengthening primary health care, improving the quality of care, providing financial protection, and ensuring that public and private health-care providers operate within the same legal and procedural framework are recognized as key national priorities. Our assessment highlighted the need to extend health service coverage to the whole population, strengthen private health-care sector governance, improve health education, increase financial investment, and expand the country's capacity for data collection and analysis. The assessment framework allowed us to identify areas where information is lacking and use already available data to analyse multiple health outcomes. As well as identifying issues that need to be addressed during the next policy development cycle, our findings have contributed towards the preparation of a more extensive assessment.

Routine data in a primary care performance dashboard, Ethiopia.

To explore the feasibility of building a primary care performance dashboard using DHIS2 data from Ethiopia's largest urban (Addis Ababa), agrarian (Oromia) and pastoral (Somali) regions. We extracted 26 data elements reported by 12 062 health facilities to DHIS2 for the period 1July 2022 to 30June 2023. Focusing on indicators of effectiveness, safety and user experience, we built 14 indicators of primary care performance covering reproductive, maternal and child health, human immunodeficiency virus, tuberculosis, noncommunicable disease care and antibiotic prescription. We assessed data completeness by calculating the proportion of facilities reporting each month, and examined the presence of extreme outliers and assessed external validity. At the regional level, average completeness across all data elements was highest in Addis Ababa (82.9%), followed by Oromia (66.2%) and Somali (52.6%). Private clinics across regions had low completeness, ranging from 38.6% in Somali to 58.7% in Addis Ababa. We found only a few outliers (334 of 816 578 observations) and noted that external validity was high for 11 of 14 indicators of primary care performance. However, the 12-month antiretroviral treatment retention rate and proportions of patients with controlled diabetes or hypertension exhibited poor external validity. The Ethiopian DHIS2 contains information for measuring primary care performance, using simple analytical methods, at national and regional levels and by facility type. Despite remaining data quality issues, the health management information system is an important data source for generating health system performance assessment measures on a national scale.

Die Bevölkerungsperspektive in der Bewertung der Leistungsfähigkeit des deutschen Gesundheitssystems: Ergebnisse einer Querschnittsbefragung GKV-Versicherter

IntroductionSince the World Health Report in 2000, Health System Performance Assessment (HSPA) has been established as a tool for the evaluation and evidence-informed governance of health systems. So far, the population perspective has not been integrated into HSPA in a systematic manner, although people's experiences and expectations are of great importance to improve health systems and especially to strengthen patient-centered care. Therefore, this study aims to conduct an HSPA of Germany's health system from the population's perspective covering all eight goals of WHO’s Health Systems Framework, and to identify strengths and needs for improvement. MethodsIn 2018, 32,000 people insured with the German sickness fund ‘AOK Nordost’ were invited by mail to participate in the survey. The questionnaire contained a total of 43 items covering the eight goals of WHO's Health Systems Framework (e.g., access, quality, safety) plus socioeconomics and other characteristics of the insured persons. The data on the health system goals were analyzed descriptively and by subgroups (age, sex, income, chronic conditions, health literacy). ResultsThe sample (n=1,481; response rate 4.6%) was 54.8% female and had a mean age of 59.1 years (±18.5). Altogether, the participants saw several needs for improvement within the German health system. For instance, 60.0% found quality differences between hospitals to be big, and between 3.9% and 8.5% reported mistakes related to their treatment or prescription medication in the previous two years. A big need for reform was especially seen regarding out-of-pocket payments (51.5%) and the coordination of ambulatory care physicians among each other (44.2%) and with hospitals (41.9%). In addition, big subgroup differences were seen, especially for income and health literacy. Of the participants in the lower income group, 37.2% reported a (very) strong financial burden due to out-of-pocket payments (vs. 20.7%). People with limited health literacy (52.1%) assessed the access to care generally as not being good, and they perceived greater quality differences and needs for reform, compared to their counterparts. For instance, 36.6% had experienced discrimination in the previous year (vs. 19.9%). DiscussionThe survey results provide a comprehensive picture of Germany's health system from the population perspective. In some areas, previous findings were confirmed (e.g., a lack of coordination between providers). Other results expand existing knowledge (e.g., the role of health literacy in health care provision) or raise new questions (e.g., the difference between the subjectively assessed burden from out-of-pocket payments and the objective measures currently used). The great differences between subgroups are a call to action on the level of both politics and practice to better consider the individual's needs in order to make health better for everyone. Further research could provide deeper insights in this regard. ConclusionStrengthening the population perspective in HSPA allows for a better understanding and evaluation of health systems and, in particular, helps to identify areas for improvement in patient-centered care.

The impact of devolution on local health systems: Evidence from Greater Manchester, England

Devolution and decentralisation policies involving health and other government sectors have been promoted with a view to improve efficiency and equity in local service provision. Evaluations of these reforms have focused on specific health or care measures, but little is known about their full impact on local health systems. We evaluated the impact of devolution in Greater Manchester (England) on multiple outcomes using a whole system approach.We estimated the impact of devolution until February 2020 on 98 measures of health system performance, using the generalised synthetic control method and adjusting for multiple hypothesis testing. We selected measures from existing monitoring frameworks to populate the WHO Health System Performance Assessment framework. The included measures captured information on health system functions, intermediatory objectives, final goals, and social determinants of health. We identified which indicators were targeted in response to devolution from an analysis of 170 health policy intervention documents.Life expectancy (0.233 years, S.E. 0.012) and healthy life expectancy (0.603 years, S.E. 0.391) increased more in GM than in the estimated synthetic control group following devolution. These increases were driven by improvements in public health, primary care, hospital, and adult social care services as well as factors associated with social determinants of health, including a reduction in alcohol-related admissions (−110.1 admission per 100,000, S.E. 9.07). In contrast, the impact on outpatient, mental health, maternity, and dental services was mixed.Devolution was associated with improved population health, driven by improvements in health services and wider social determinants of health. These changes occurred despite limited devolved powers over health service resources suggesting that other mechanisms played an important role, including the allocation of sustainability and transformation funding and the alignment of decision-making across health, social care, and wider public services in the region.

Trends in Avoidable Mortality in Kazakhstan From 2015 to 2021.

The health system performance assessment is a challenging process for decision-makers. In case of Kazakhstan's healthcare system, the calculation of avoidable mortality, which has been underutilized to date, could serve as an additional tool to prioritize areas for improvement. Therefore, the aim of the study is to analyse avoidable mortality in Kazakhstan. The data was retrieved from the Bureau of National Statistics, Kazakhstan. It covers population data by age, mortality rates from disease groups based on the Joint OECD/Eurostat classification of preventable and treatable causes of mortality. The data spans from 2015 to 2021, categorized by gender and 5-year age groups (0, 1-4, 5-9, ..., 70-74). Standardization was performed using the 2015 OECD standard population. We used joinpoint regression analysis to calculate the average annual percentage change. From 2015 to 2019, the annual percentage change in avoidable mortality per 100000 population was -3.8 (-5.7 to -1.8), and from 2019 to 2021 it increased by 17.6 (11.3 to 24.3). Males exhibited higher avoidable mortality rates compared to females. The preventable mortality rate was consistently higher than the treatable mortality. Both preventable and treatable mortality decreased from 2015 to 2019, with preventable mortality reaching 272.17 before rising to 379.23 per 100000 population in 2021. Between 2015 and 2021, treatable mortality rates increased from 179.3 (176.93- 181.67) to 205.45 (203.08-207.81) per 100000 population. In Kazakhstan, the leading causes of avoidable mortality were circulatory diseases, respiratory diseases, and cancer. To achieve the goals of Universal Health Coverage and improve the overall population health, there is an urgent need to amend the healthcare system and reduce avoidable mortality. While it is important to acknowledge the influence of COVID-19 on these trends, our study's focus on avoidable mortality provides valuable insights that complement the understanding of pandemic-related effects.

Health system performance assessment: embedding resilience through performance intelligence.

Health systems around the world are facing challenges in achieving their goals. In the wake of the coronavirus disease pandemic, the need for resilient health systems has become even more apparent. This article argues that embedding resilience into health system performance assessment (HSPA) frameworks can be a valuable approach for improving health system performance. This perspective examines key challenges threatening health systems and makes a case for the continued relevance of HSPA by embedding resilience-related performance intelligence.

Results of Germany's health system performance assessment (HSPA) pilot: the example of stroke care

Abstract Background HSPA is a tool to monitor, evaluate, and improve health systems based on their achievements within different dimensions, e.g., access to care, quality, and population health. Next to the overall system, HSPA also allows to analyze its performance for specific groups across dimensions. In a pilot for a German HSPA, a conceptual framework was developed, and indicators for every dimension were selected and analyzed. This presentation illustrates the results of the German HSPA pilot using the example of access to and quality of care for ischemic stroke. Methods Selected indicators include stroke unit (SU) density, availability of a SU within 30 minutes by car, share of inpatient cases treated in a hospital with a SU, and inpatient mortality from 2014 to 2020, respectively. Indicators were analyzed based on nation-wide hospital discharge data. Travel time was computed and provided by the Federal Institute for Research on Building, Urban Affairs and Spatial Development. Results About 475 SU were identified in hospital data (2014: 463), compared to 350 certified SU in Germany. In 75% of Germany's 11,000 municipalities, 100% of the population could reach a SU within 30 minutes by car. Number of inpatient ischemic stroke cases was about 250,000 per year. Of the 215,000 cases without transfer from another hospital, 93% were treated in a hospital with a SU in 2020 (2014: 88%). Inpatient mortality was about 6,4% for all ischemic stroke cases and 6,1% for cases without transfer, varying only slightly over time. SU treatment showed large, and inpatient mortality minor regional variation. Conclusions Access to and quality of inpatient stroke care seems comparatively good in Germany, as SU density is high and mortality low in the European context. However, regional variation should be further tackled. The example shows how HSPA can be used to assess different strands of care by connecting health system dimensions, to produce actionable evidence for policy. Key messages • Health system performance assessment (HSPA) is a tool for the evaluation and improvement of the overall system but also of the care for specific groups or diseases across health system dimensions. • The results of the German HSPA pilot show a good access and quality for the case of inpatient stroke care. Regional variations might be further reduced via improvements in hospital planning.

Identifying health inequities using a health system performance assessment: findings from Germany

Abstract Background Reducing health inequities is an important goal of high-performing health systems. Therefore, health inequities need to be assessed systematically, continuously and for all relevant aspects of a health system. The tool “health system performance assessments” (HSPA) aims at monitoring and evaluation the achievement of predefined goals and therefore the overall performance of a national health system. The aim of this study was to identify health inequities using a national HSPA. Methods A pilot study for a national HSPA in Germany was conducted between 2020 and 2023. The underlying framework defined the health system goals (e.g., access, quality, population health) and the respective indicators. Equity analyses (e.g., age, gender, socioeconomic factors, geographical region, type of health insurance) were conducted for each indicator to highlight inequities. Results Inequities were found for all health system goals. For example, unmet needs (“access”) were higher among people in the lowest income quintiles and in suburban areas. Potentially inadequate medication among the elderly (“quality”) was found more often for women and people 85+ years. The prevalence of infectious diseases (“population health”) was higher among men. Conclusions Using a national HSPA to discover health inequities can help identify these in all areas and for all goals of a health system and monitor their development over time. The systematic and continuous assessment within an HSPA is an important prerequisite for identifying and evaluating measures aiming at reducing health inequities. Key messages • A national health system performance assessment is useful for identifying and monitoring health inequities on a system level. • Health inequities (e.g., regarding age, gender, geographic region) were found for every goal of the German health care system.

Strengthening health system resilience at the European level

Abstract The resilience testing project builds on the work carried out by the ‘Expert Panel on effective ways of investing in health’ in its ‘opinion on the organisation of resilient health and social care following the COVID-19 pandemic’, adopted on 25 November 2020 as well as the December 2020 report by the EU Expert Group on Health Systems Performance Assessment (HSPA) titled ‘Assessing the resilience of health systems in Europe: an overview of the theory, current practice and strategies for improvement’. Resilience testing may contribute to the European Semester, which addresses health system effectiveness, accessibility and resilience. The encompassing resilience assessment framework (‘the testing framework’) will serve to detect critical health system weaknesses against specific shocks as well as long-term structural challenges. This will help to develop recommendations for remedial action. The heath system resilience testing methodology will support the implementation of the Regulation on serious cross-border threats to health which envisages stress tests and simulation exercises to help the development of health system functions and preparation of prevention, preparedness and response plans. It could also support the European Centre for Disease Prevention and Control (ECDC) in their assessment activities of the prevention, preparedness and response plans. By establishing a resilience testing methodology, Member States will be able to regularly and independently review their preparedness for health crisis and check their healthcare systems’ resilience against specific high-pressure scenarios and long-term structural challenges. This action will generate insight and evidence through resilience tests and will reinforce the ability of Member States and regions to become more resilient to future shocks.

The challenges of measuring: mapping and understanding resilience metrics in the context of a climate change shock

Abstract During this presentation, discussion will be held around several of the decisions made for the sample climate change scenario, suggested indicators and their implications. This provides the audience with an example process of how to put together a shock scenario and prepare a resilience test given the tools and methodology the resilience testing project provides. There is increasing evidence of climate change and its associated health risks. Whilst resilience is required against a multitude of threats, resilience to climate change induced health risks will be essential for health systems in the future. This scenario focuses on a heat wave that is complicated by a follow-on shock, the collapse of the electrical grid. Deliberate choices were made to include two shocks, the integration of heat warning systems, suggestions to increase the demand for health services and how to include considerations of vulnerable groups. These will be discussed in the context of how to customise a scenario to a country context including the current functioning, replacing the shock mechanism of heat wave with other shocks such as flood or fires, and including the lessons of previous experience of environmental shocks. Resilience testing tools, such as the resilience testing manual, other sample scenarios and peer-to-peer scenario repository website will be demonstrated in practical application. This shock initially has an impact on the service delivery and resource generation functions of the health system performance assessment (HSPA) framework but the impacts flow into the other functions. The workshop will go through the combined framework, indicators that are useful to assess health systems resilience and questions that are proposed to the guide the discussion of the resilience testing exercise. From the discussion participants will be better prepared to implement resilience testing in their countries, understanding how to customise the shock of a heat wave to their local context.

Assessing resilience though a health system performance lens

Abstract Health system resilience testing has generated keen interest since the COVID-19 pandemic but so far no methodology for this complex exercise exists. Our concept of resilience testing is inspired by the experience of stress tests in the banking sector. The novel approach is a mixed-method country-led exercise, which applies a health system lens to a particular shock scenario, and identifies vulnerabilities using the Health System Performance Assessment Framework for Universal Health and the Shock Cycle. In the first part of the presentation we describe the resilience testing methodology, based on the two conceptual frameworks. We share lessons from the required preparatory work: shock scenario selection, scope of background research, formation of a resilience testing group, resilience test day structure and process, and the facilitator's role. We also discuss the outcomes of the resilience testing process, which includes a formal assessment of the resilience of the health system. This will provide workshop participants with an overview of key concepts to conduct a country-led resilience testing exercise. The second part of the presentation focuses on the overarching results from the country-led pilots. These show that the resilience testing method can identify specific health system weaknesses at different stages of the shock cycle. Given the appropriate selection of resilience test facilitator and participants, the method can also detect system-wide challenges, shock implications and spill-overs. The participant interaction on the test day may identify key priorities for action and strengthen relevant national multiprofessional networks. Overall, the resilience testing approach is an important new tool to identify strengths and weaknesses of the current health system and in scenario-specific preparedness planning.

How do patient and care experience data inform policy and practice? Insights from the Irish National Care Experience Programme

Abstract Issue Patient experiences are central to the delivery of quality, person-centred care and an important predictor of outcomes. Prior to 2017, there was no unified, national approach to the collection of care experience data in Ireland. Description of the problem The National Care Experience Programme is a partnership between a regulator, policy makers and the Irish public health service, with a suite of surveys of inpatient, maternity, nursing home and end of life care. The surveys aim to learn from people's feedback on health and social care services to identify areas of good experience and areas requiring improvement. Results Findings have informed national policies, including a healthcare communication programme to support staff to develop communication skills with patients and families. Survey data provide a key input to the Irish health quality indicator framework and the Health Systems Performance Assessment Framework, which assess the overall performance of the Irish health system. National Maternity Experience Survey results informed the allocation of funding for post-natal supports and new lactation consultant posts, and supported the implementation of a specialist perinatal mental health services model of care, leading to increased diagnosis and vigilance of mental health issues ante- and postnatally. National Nursing Home Experience Survey findings supported legislative amendments, including provision for the delivery of in-person advocacy awareness campaigns and the strengthening of complaints processes within nursing homes. Lessons In order to maximise the use of survey findings and ensure the voices of people using health and social care services are acted on, routine surveying of care experiences is required to track the impact of policy and practice initiatives. The programme is currently expanding to include people's experiences of cancer care and mental health services, in addition to exploring options for continuous, local care experience surveys.

6.I. Workshop: Assessing health system resilience to future shocks: a new tool for policy making

Abstract The COVID-19 pandemic has highlighted the importance of health system resilience, the ability of a health system to prepare for, manage and learn from a shock. Other recent adverse events, such as economic crises, conflict, migration and extreme weather have demonstrated how a wide range of shocks can challenge health systems. Health systems vary in their ability to respond to shocks and other challenges, mitigate adverse impacts, and learn from previous experience. While emergency preparedness and planning is a well established component of health system governance in many countries, no methodology currently exists to systematically test the resilience of a health system to a broad range of shocks. In this workshop, the European Observatory on Health Systems and Policies (OBS) and the Organization for Economic Cooperation and Development (OECD) share learnings from an ongoing project with the European Commission (EC) to help EU Member States assess and understand the resilience of their health system to a wide range of future shocks. The resilience testing methodology draws on two conceptual frameworks: the Health System Performance Assessment Framework for Universal Health Coverage (HSPA Framework for UHC) and the Shock Cycle. A specific shock scenario is chosen from a broad range of possible shocks to hypothetically challenge a given health system to its limits. This highlights key structural strengths and weaknesses of the health system. The country-led process culminates in a resilience test workshop that brings together policy makers, civil servants and other relevant experts. The test day is supported by an expert facilitator and research team, who provide background knowledge and structure the test day. Results of the resilience test include an understanding of the health system strengths and vulnerabilities specific to the shock. The test day may also anticipate spill-overs effects to other health system functions, and contribute towards a participatory approach to improve preparedness going forward. The objectives of the workshop are to: - present the new methodology for resilience testing; - showcase country examples that have piloted the methodology; - engage workshop participants in discussion towards a common approach to health system resilience strengthening. The workshop consists of presentations (40 minutes) and audience interaction (20 minutes). The presentations will cover the methodology, pilot countries’ experience, and an EU view of the resilience testing process. The audience interaction will consist of moderated discussion with an element of a digital feedback and Q&A (if settings allow). Discussion will reflect on usability, potential adaptations to suit various national settings, and operationalisation for policy makers so that countries can prepare more systematically for future shocks and long-term challenges. Key messages • Resilience testing is a novel and systematic method to engage policy makers, understand health system vulnerabilities and identify weaknesses requiring action. • Country-led pilots show that shock scenarios can be employed to reveal structural vulnerabilities, which can be addressed to enhance health system resilience.

"Assessment of the Health System Performance in Ontario Major Cities (Canada)"

"This study aims to provide evidence to support policymaking by assessing health system perfor mance in the province of Ontario, Canada, and its major cities (Toronto, Mississauga, Oakville, and Brampton). It reports on the performance of health care organizations and local health systems over the years 2012-2018. The performance indicators ana lyzed are grouped into two categories: health status and quality of service (Ontario Ministry of Health). The analysis reports health care waiting times for the year 2018, focusing on the most frequently re ported disease groups and procedures such as: pediatric, cancer, cardiac, orthopedic, eye, diagnos tic imaging, and emergency room. It also looks at government spending on health, the benchmark for surgical procedures, the number of physicians, the health status of the population, and rates of hospi talization and hospital admissions. As recommen dations, among policies to improve the health sys tem, the government should pay attention to health spending, increasing the number of doctors rather than beds, and introducing the privately-owned hos pitals that could coexist with the public ones."

HPR67 Ejadah: Implementation of Value-Based Healthcare in Emirate of Dubai

Dubai Health Authority (DHA) is pursuing the implementation of Ejadah, a performance management framework designed to help transform current practice from Volume Based Care to Value Based Health Care. The aim is to develop and implement Key Performance Indicators (KPIs) according to evidence-based clinical practice guidelines to support healthcare professionals and healthcare systems adopt practices to optimize patient outcomes and reduce expenditure. To facilitate international benchmarking in compliance with Dubai Health System Performance Assessment Framework (Dubai Health SPA) that is developed collaboratively with OECD.

Comparing Hospital Efficiency: An Illustrative Study of Knee and Hip Replacement Surgeries in Spain.

WHO's Health Systems Performance Assessment framework suggests monitoring a set of dimensions. This study aims to jointly assess productivity and quality using a treatment-based approach, specifically analyzing knee and hip replacement, two prevalent surgical procedures performed with consolidated technology and run in most acute-care hospitals. Focusing on the analysis of these procedures sets out a novel approach providing clues for hospital management improvements, covering an existing gap in the literature. The Malmquist index under the metafrontier context was used to estimate the productivity in both procedures and its decomposition in terms of efficiency, technical and quality change. A multilevel logistic regression was specified to obtain the in-hospital mortality as a quality factor. All Spanish public acute-care hospitals were classified according to their average severity attended, dividing them into three groups. Our study revealed a decrease in productivity mainly due to a decrease in the technological change. Quality change remained constant during the period with highest variations observed between one period to the next according to the hospital classification. The improvement in the technological gap between different levels was due to an improvement in quality. These results provide new insights of operational efficiency after incorporating the quality dimension, specifically highlighting a decreasing operational performance, confirming that the technological heterogeneity is a critical question when measuring hospital performance.

Health System Resilience as the Basis for Explanation Versus Evaluation Comment on "The COVID-19 System Shock Framework: Capturing Health System Innovation During the COVID-19 Pandemic".

The onset and impacts of COVID-19 have prompted attention to national health system preparedness for, and capacity to adapt in response to, public health emergencies and other shocks. This preparedness and adaptive capacity are often framed as 'health system resilience' a concept previously associated more with assessments of health systems in conflict-affected and fragile states. Yet health system resilience remains a slippery concept, defined and applied in multiple ways. Reflecting on the Hodgins and colleagues' study "the COVID-19 system shock framework: capturing health system innovation during the COVID-19 pandemic," this article restates the limitations of health systems resilience as a concept capable of anchoring evaluative assessments of health system performance but stresses its value in the context of explanatory research investigating how and why health systems adapt, with due attention to the power of actors' whose choices inform the nature and direction of change.

Perspective: Lessons from COVID-19 of countries in the European region in light of findings from the health system response monitor.

Decision-makers initially had limited data to inform their policy responses to the COVID-19 pandemic. The research community developed several online databases to track cases, deaths, and hospitalizations; however, a major deficiency was the lack of detailed information on how health systems were responding to the pandemic and how they would need to be transformed going forward. In an effort to fill this information gap, in March 2020, the European Observatory on Health Systems and Policies, the WHO European Regional Office and the European Commission created the COVID-19 Health System Response Monitor (HSRM) to collect and organise up-to-date information on how health systems, mainly in the WHO European Region, were responding to the COVID-19 pandemic. The HSRM analysis and broader Observatory work on COVID-19 shone light on a range of health system challenges and weaknesses and catalogued policy options countries put in place during the pandemic to address these. Countries prioritised policies on investing in public health, supporting the workforce, maintaining financial stability, and strengthening governance in their response to COVID-19. COVID-19 is likely to continue to impact health systems for the foreseeable future; the ability to cope with this pressure, and other shocks, depends on having good information on what other countries have done so that health systems develop adequate policy options. In support of this, the country information on the COVID-19 HSRM will remain available as a repository to inform decision makers on options for actions and possible measures against COVID-19 and other public health emergencies. Building on its previous work on health systems resilience, the European Observatory on Health Systems and Policies will sustain its focus on analysing key issues related to the recovery from the pandemic and making health systems more resilient. This includes policy knowledge transfer between countries and systematic resilience testing, aiming at contributing to an improved understanding of health system response, recovery, and preparedness. The COVID-19 Health System Response Monitor (HSRM) was the first database in the WHO European Region to collect and organise up-to-date information on how health systems were responding to the COVID-19 pandemic. The HSRM provides a repository of policies which can be used to inform decision makers in health and other policy domains on options for action and possible measures against COVID-19 and other public health emergencies. This initiative proved particularly valuable, especially during the early phases of the pandemic, when there was limited information for countries to draw on as they formulated their own policy response to the pandemic. Our perspectives paper highlights some key challenges within health systems that the HSRM was able to identify during the pandemic and considers policy options countries put in place in response. Our research contributes to literature on emergency responses and recovery, health systems performance assessment, particularly health system resilience, and showcases the Observatory experience on how to design such a data collection tool, as well as how to leverage its findings to support cross-country learning.