External Quality Monitoring Research Articles

External quality monitoring facilitates improvement in already well-performing stroke units: insights from RES-Q Poland.

The Registry of Stroke Care Quality (RES-Q) is used in Poland for quality monitoring by numerous hospitals participating in the Angels Initiative. Our aim was to assess the degree of improvement in highly stroke-oriented centres that report cases to the RES-Q each year. This retrospective analysis included Polish stroke units that from January 2017 to December 2020 contributed to the RES-Q at least 25 patients annually. Seventeen out of 180 Polish stroke units reported patients each year (2017, n = 1,691; 2018, n = 2,986; 2019, n = 3,750; 2020, n = 3,975). The percentage of ischaemic stroke patients treated with alteplase remained stable (26%, 29%, 30% and 28%, respectively). The door-to-needle time progressively decreased, from a median 49 minutes to 32 minutes. The percentage of patients treated ≤ 60 minutes and ≤ 45 minutes significantly increased (from 68% to 86% and from 43% to 70%, respectively), with no change observed between 2019 and 2020. Despite a general improvement in dysphagia screening (81%, 91%, 98% and 99%), screening performed within the first 24h from admission became less frequent (78%, 76%, 69% and 65%). In-hospital mortality significantly increased (11%, 11%, 13% and 15%), while the proportion of patients discharged home remained stable. Quality-oriented projects facilitate the improvement of stroke care, even in centres demonstrating good baseline performance. Polish stroke units that consistently reported cases to the RES-Q demonstrated improvement in terms of door-to- -needle time and dysphagia screening. However, there is still a need to shorten the time to dysphagia screening, and carefully monitor stroke unit mortality following the COVID-19 pandemic.

Construction and Practice of Teaching Staff and Teaching Quality Monitoring System in University Affiliated Hospitals Under the Background of Education Informatization

Objective: To build a set of teaching staff construction and teaching quality monitoring system suitable for clinical practice teaching in affiliated hospitals of medical colleges, achieve continuous improvement of clinical education and teaching level, and ensure the quality of medical education talent cultivation. Methods: A modern clinical practice teaching quality monitoring system is constructed based on organizational structure construction, teaching staff system construction, quality control system construction, and information platform construction, combined with external audit and evaluation. Results: The hospital has established a Faculty Development and Teaching Evaluation Office specifically responsible for the cultivation of clinical teachers and the evaluation and supervision of teaching quality. A relatively complete teacher construction and teaching quality monitoring system has also been established for clinical practice teaching, thus achieving integration with the school’s quality control system in terms of management mechanism. At the same time, a set of teaching quality control mode based on the “Internet +” platform has been created by means of informatization. At present, this mode has won three national computer software copyrights and two second prizes for school-level teaching achievements. Conclusion: Through five years of practice, an “Internet +” teaching quality evaluation and monitoring system with the characteristics of teaching hospitals affiliated to local medical colleges has been established. In order to further standardize the training system of clinical teachers in affiliated hospitals, achieve self-monitoring and self-improvement in terms of teaching quality, and ensure the continuous improvement of clinical teaching quality, we will continue to promote the development of clinical teachers with quality and excellence, enrich the main team of quality monitoring, guide the transformation of the education mode from being “centered on teachers” to being “centered on students,” and realize the integration of internal and external quality monitoring systems.

A theory-informed, rapid cycle approach to identifying and adapting strategies to promote sustainability: optimizing depression treatment in primary care clinics seeking to sustain collaborative care (The Transform DepCare Study)

BackgroundFew real-world examples exist of how best to select and adapt implementation strategies that promote sustainability. We used a collaborative care (CC) use case to describe a novel, theory-informed, stakeholder engaged process for operationalizing strategies for sustainability using a behavioral lens.MethodsInformed by the Dynamic Sustainability Framework, we applied the Behaviour Change Wheel to our prior mixed methods to identify key sustainability behaviors and determinants of sustainability before specifying corresponding intervention functions, behavior change techniques, and implementation strategies that would be acceptable, equitable and promote key tenets of sustainability (i.e., continued improvement, education). Drawing on user-centered design principles, we enlisted 22 national and local stakeholders to operationalize and adapt (e.g., content, functionality, workflow) a multi-level, multi-component implementation strategy to maximally target behavioral and contextual determinants of sustainability.ResultsAfter reviewing the long-term impact of early implementation strategies (i.e., external technical support, quality monitoring, and reimbursement), we identified ongoing care manager CC delivery, provider treatment optimization, and patient enrollment as key sustainability behaviors. The most acceptable, equitable, and feasible intervention functions that would facilitate ongoing improvement included environmental restructuring, education, training, modeling, persuasion, and enablement. We determined that a waiting room delivered shared decision-making and psychoeducation patient tool (DepCare), the results of which are delivered to providers, as well as ongoing problem-solving meetings/local technical assistance with care managers would be the most acceptable and equitable multi-level strategy in diverse settings seeking to sustain CC programs. Key adaptations in response to dynamic contextual factors included expanding the DepCare tool to incorporate anxiety/suicide screening, triage support, multi-modal delivery, and patient activation (vs. shared decision making) (patient); pairing summary reports with decisional support and yearly onboarding/motivational educational videos (provider); incorporating behavioral health providers into problem-solving meetings and shifting from billing support to quality improvement and triage (system).ConclusionWe provide a roadmap for designing behavioral theory-informed, implementation strategies that promote sustainability and employing user-centered design principles to adapt strategies to changing mental health landscapes.

Interpretation of external quality assurance results on liver function test

The quality of laboratory examination results must always be guaranteed in the sense that it can provide a level of accuracy and precision that can be accounted for. Therefore, a study of External Quality Consolidation (PME) was conducted. The purpose of this study was to describe the results of External Quality Monitoring in the field of clinical chemistry parameters of Serum Glutamic Oxalocetic Transaminase (SGOT) and Serum Glutamic Pyruvic Transaminase (SGPT) at the Public Health Center Laboratory in the Surabaya area. The population in this study was a health center in the area of the Surabaya City Health Office with a sample of 15 health centers that had examinations in the field of clinical chemistry. The sample used in this study was normal level control serum which was measured on the parameters of Serum Glutamic Oxalocetic Transaminase (SGOT) and Serum Glutamic Pyruvic Oxalocetic Transaminase (SGPT). The results of data analysis showed that the percentage of PME levels of SGOT with good VIS criteria was 73.33%, sufficient criteria was 0%, less criteria was 6.67%, poor criteria was 20%. While the SGPT parameters good criteria are 53.33%, sufficient criteria are 33.33%, less criteria are 13.33%, poor criteria are 0%.©2020 JNSMR UIN Walisongo. All rights reserved.

Performance of Laboratories after 10 Years of Participating in External Quality Monitoring in Cervical Cytology

Objective: To evaluate the performance of laboratories accredited by the National Health System that perform cytopathology examination of the uterine cervix after 10 years of participation in external quality monitoring (eQM). Study Design: Seven laboratories were assessed in this study. To assess the concordance of the representation of epithelia and results, 6,536 examinations (3,433 in 2007 and 3,103 in 2017) were reviewed. Statistical analysis was performed using the κ coefficient as well as the χ² and the Fisher exact test (p < 0.05). Results: All laboratories showed adequate infrastructure and internal quality monitoring. Regarding the representation of the epithelia, the concordance remained excellent (κ between 0.84 and 0.94). In 2007, 26 false-negative results (FN), 157 false-positive results (FP), and 79 delays in clinical conduct (DCC) were identified (κ = 0.80). In 2017, 24 FN, 42 FP, and 38 DCC were identified (κ = 0.90). Five laboratories showed improvement in concordance from 2007 to 2017. We observed significant improvement in cytomorphological criteria of the results: atypical squamous cells of undetermined significance (p < 0.001), atypical squamous cells cannot exclude high-grade squamous intraepithelial lesion (p < 0.016), and low-grade squamous intraepithelial lesion (p < 0.001). We also observed a considerable improvement in the results of cellular abnormalities in glandular epithelium (p < 0.0504). Conclusions: Ten years after the implementation of eQM, improvements in the concordance of results were observed as well as reductions in FN, FP, and DCC in all laboratories monitored.

External Quality Monitoring of the Cervical Cytopathological Exams in the Rio de Janeiro City.

To discuss the implementation and contributions of the External Quality Monitoring in the city of Rio de Janeiro and to analyze the performance of the main providers of cervical cytopathology in this city from September 2013 to March 2017, here referred to as "Alpha laboratory" and "Beta laboratory." Observational, cross-sectional, retrospective study using information from the Cervical Cancer Control Information System (SISCOLO, in the Portuguese acronym), municipal coordination module, External Quality Monitoring report. The proportions of false positives, false negatives, unsatisfactory samples and rejected samples were estimated. The agreement among the observers was analyzed through the Kappa index and the reduction of disagreements in the period for each laboratory studied, comparing the results of each cycle. A total of 19,158 examinations were selected, of which 19,130 (99.85%) were monitored, 16.649 (87, 03%) were reviewed by the External Quality Monitoring Unit, 2,481 (12,97%) were rejected and 441 (2,65%) were considered unsatisfactory. The "Beta laboratory" presented excellent concordance in all cycles; the "Alpha laboratory" had good concordance in the first two cycles (K = 0.76 and 0.79), becoming excellent in the following four cycles. The average Kappa index was 0.85, with median of 0.86. The percentage of diagnostic disagreement was 6.63% of the reviewed exams, of which 5.38% required a change of conduct CONCLUSION: External Quality Monitoring is an exercise in diagnostic improvement, and its implementation was fundamental to ensure the reliability of the cytopathological exams in the city of Rio de Janeiro.

When should a clinical learning environment be ‘rested’ from the student placement circuit?

When healthcare failings occur there could be a need to consider the removal of a clinical environment from student nursing placement circuits. The decision to rest a clinical learning environment as a student education placement resulted in the need for a mechanism to support the process and so an algorithm was devised. Initially there were issues creating a flowchart to illustrate such a process and so external support was sought to enable further progress to be made. It was considered essential to have a clear and concise process that was easy to use. The algorithm was intended to contribute towards enhancing the regular internal and external quality monitoring mechanisms. The task to develop and implement the algorithm has demonstrated a willingness to promote strong cross-organisational and cross-discipline working.

Abstract 26: Retirement of a Quality Measure: The Case of Beta Blocker Use After Myocardial Infarction

Background: Quality measures are increasingly used as a basis for targeting intervention and structuring compensation. Their use has lead to substantial improvements in care for acute myocardial infarction (AMI) and congestive heart failure. Because of these improvements, many cardiovascular quality metrics are slated for retirement in 2015. The consequences of removing external quality monitoring are unclear. We evaluated the National Committee for Quality Assurance (NCQA)’s retirement in 2007 of the measure for beta-blocker prescribing after AMI because of consistently high performance. This was one of the first quality measures to be retired. Methods: We used medical and pharmacy claims from a large national insurer from 2003-2013 to compare trends in the proportion of AMI patients who filled a beta blocker prescription within 7 days of discharge according to HEDIS specifications. The quality measure reporting period was 2003 through 2006 and the period after the measure was retired was 2007 through 2013. Pre and post trends were compared using interrupted time-series design with linear regression models. Results: From 2003-2013, we identified 25,648 patients with AMI eligible for beta blocker prescription. Overall 79.2% patients filled a beta-blocker within 7 days of discharge. Beta blocker prescription rates were improving at a rate of 0.28 percentage points per month during the quality measure reporting period. Retirement of the measure lead to a flattening of this trend and an abrupt 4.8 percentage point decrease in the proportion of patients discharged with a beta-blocker post-AMI (p &lt; .001 for changes in level and slope). This decrease was highest for HMO/PPO enrollees (11.6% points, p &lt; .001). Adjusting for baseline demographic and clinical factors attenuated the decrease in beta blocker prescription rates but did not eliminate the effect. Conclusion: We found that beta blocker use among post-AMI patients appears to be much lower than publically-reported national averages. Beta blocker fill rates increased during the period of NCQA quality measure reporting but decreased significantly after this measure was retired. This finding may have relevance to upcoming plans to remove external monitoring for other cardiovascular quality measures.

External Quality Control for Dried Blood Spot-Based C-Reactive Protein Assay: Experience from the Indonesia Family Life Survey and the Longitudinal Aging Study in India

Measurement of C-reactive protein (CRP), a marker of inflammation, in dried blood spots has been increasingly incorporated into community-based social surveys internationally. Although the dried blood spot-based CRP assay protocol has been validated in the United States, it remains unclear whether laboratories in other less-developed countries can generate CRP results of similar quality. We therefore conducted external quality monitoring for dried blood spot-based CRP measurement for the Indonesia Family Life Survey and the Longitudinal Aging Study in India. Our results show that dried blood spot-based CRP results in these two countries have excellent and consistent correlations with serum-based values and dried blood spot-based results from the reference laboratory in the United States. Even though the results from duplicate samples may have fluctuations in absolute values over time, the relative order of C-reactive protein levels remains similar, and the estimates are reasonably precise for population-based studies that investigate the association between socioeconomic factors and health.

Aplicación de las especificaciones de calidad analíticas al cálculo de los límites admisibles de error en Programas de Evaluación Externa de la Calidad para concentración espermática

Introduction External quality assessment is an essential activity of the andrology laboratory. To evaluate the effectiveness of the laboratories participating in the Programme of External Quality Monitoring (PSEC) allowable limits of error are used, and the result of a laboratory is considered acceptable if it is within these limits. Currently allowable error limits depend to a large extent on the criteria of the organizers of the PSEC. Our goal is to compare the different criteria described in ISO 13 528:2005 for the calculation of the allowable limits of error discussed in the PSEC for sperm concentration, so that they can be used by all. Material and methods The data analysed in this study are the mean values and standard deviation (SD) obtained on independent control samples from two PSEC. Four different strategies for calculating the allowable limits of error according to ISO 13 528:2005 were set: 1) based on the results of participating laboratories, 2) based on the results of expert laboratories, 3) using quality specifications based on biological variability, state of the art and opinion of clinicians, and 4) using the quality specifications and adjusting for the uncertainty of the assigned value. Results Very large allowable limits of error were obtained with strategies 1 and 2, whereas strategy 3, based only on quality specifications, gave very narrow limits.. However, with strategy 4 we obtained values of intermediate acceptance intervals compared with the other strategies. There were no differences in the allowable limits of error in strategies 3 and 4 by varying the model for the calculation of analytical quality specifications. Discussion PSEC sperm concentration should use a strategy adjusted to the uncertainty of the value assigned to establish the allowable limits of error, since the acceptance intervals obtained are clinically useful, so that the values contained in them lead to clinically similar decisions. For expert laboratories to adequately define them is more important than the model chosen to estimate analytical quality specifications in a sperm concentratiońs PSEC.

Avaliação crítica das nomenclaturas diagnósticas dos exames citopatológicos cervicais utilizadas no Sistema Único de Saúde (SUS)

to identify the nomenclature for reporting cervical cytological diagnoses used by laboratories which render services to the Brazilian Unified Health System (SUS) and which participate in External Quality Monitoring (MEQ). To evaluate the information acquired from gynecologists of the SUS regarding the various diagnostic classifications that they receive in the cervical cytology diagnostic reports. we evaluated 94 cytology reports issued by laboratories which participate in the MEQ in the State of São Paulo, Brazil, and 126 questionnaires applied to gynecologists who work for the SUS. out of the 94 laboratories, 81 (86.2%) use one diagnostic classification: 79 (97.6%) use the Brazilian Nomenclature for Cytological Reports (NBLC), 1 (1.2%) uses the Papanicolaou classification and 1 (1.2%) uses the Richart diagnostic classification. Of the 13 (13.8%) laboratories that use more than one classification, 5 use 2 types and 8 use 3 to 4 types, with 9 including the Papanicolau diagnostic classification. The study showed that 52 (55.3%) laboratories presented more than one descriptive diagnosis in the same report. Out of the 126 gynecologists who filled out a questionnaire evaluating the cytopathology reports, 78 (61.9 %) stated that they received laboratory reports with only one diagnostic classification, 48 (38.1%) received reports with more than one classification and 2 received reports with all 4 classifications. Among the 93 (73.8%) gynecologists who prefer only one classification, 56 (60.2%) claimed that the NBLC contributes to clinical practice, 13 (14.0%) opted for the Richart classification, 8 (8.6%) for the Reagan classification and 16 (17.2%) for the Papanicolaou classification. Out of 33 (26.2%) gynecologists who prefer more than one classification, 5 opted for the 4 classifications. these data suggest that there is still resistance on the part of pathologists about using the official nomenclature in cytology reports for SUS. There is discrepancy between the information that gynecologists would like to see in the reports and the information provided by the pathologists. Greater efforts should be made to stimulate the use of the official nomenclature.

Quality assurance in post‐secondary education

PurposeThe common approaches to quality assurance (QA), as practiced by most post‐secondary education institutions for internal quality monitoring and most QA authorities for external quality monitoring (EQM), have been considered by many researchers as having largely failed to address the essence of educational quality. The purpose of this paper, although not meant to be exhaustive, is to review some of these approaches.Design/methodology/approachThe paper reviews the relevant issues concerning total quality management, performance indicators and EQM, the three common approaches to the QA of post‐secondary education have been reviewed.FindingsWhile from a pragmatic perspective these approaches have their respective reasons for existence, they can all be criticized as lacking rigorous theoretical foundations and being mainly driven by demands of satisfying external agendas (e.g. to enforce institutional accountability or compliance) instead of academic considerations (e.g. to facilitate student learning). As a result, a mismatch between the rhetoric and reality of educational quality has become a common experience of most practitioners, not only in western contexts from which these approaches were originated, but also in other cultural contexts that have adopted them uncritically. It is undeniable that the overall quality culture within most post‐secondary education systems worldwide, as currently manifested, tends to favor the institutional aspects rather than the student aspects of the quality issues, and tends to lean more on the accountability‐led view rather than the improvement‐led view of quality assurance.Originality/valueThe paper sheds some light on the quality debate in post‐secondary education.

Positionspapier zur Indikation und Durchführung der interventionellen Behandlung extrakranieller Karotisstenosen

Interventional treatment of extracranial carotid stenoses: current status, requirements and indications Around 30000 strokes per year in Germany are caused by extracranial obstructions of the carotid arteries. Besides the well established surgical endatherectomy, which has been proven to be superior to medical treatment alone in randomized multicenter trials, percutaneous treatment by balloon angioplasty and stent placement is increasingly performed. This consensus paper summarizes the present status of scientific studies and controlled treatment registries of carotid angioplasty and stenting and yields to recommendations regarding its performance and indication. A Medline search was done until August 2005 including all randomized comparative studies with clinical endpoints and all controlled registries with more than 500 patients included. Both completed randomized trials comparing angioplasty and stenting with surgery (CAVATAS and SAPPHIRE) did not show a significant difference with respect to stroke and death within 30 days. The incidence of this combined endpoint was between 3.3 and 6.9% in large multicenter registries. Thus, the current endovascular treatment results in a perlinterventional complication rate comparable to that which is commonly accepted for vascular surgery. A systematic preferably external quality monitoring as well as interdisciplinary cooperation most importantly with neurologists is recomendedfor all interventionalists. Based on available clinical data, this consensus paper defines recommendation classes I or II with a level of evidence B or C depending on symptoms, operative risk status of the patient and severity of the underlying stenosis.

Transforming quality evaluation

This paper outlines the preponderant approaches to external quality evaluation, including the purpose, focus, object, rationale, and methods of external evaluations. Accountability, compliance and, in some countries, control are much more frequent rationales for external monitoring than improvement. Research on the impact of quality monitoring is difficult because it is impossible to control all relevant factors to be able to map causal relationships. Arguably, such an endeavour is a positivist device that ignores the complexity and the wider socio‐political context of the quality phenomenon. Nonetheless, there is some impact research, which might be characterised as overview studies or close‐up studies. These studies reinforce the view that quality is about compliance and accountability and has, in itself, contributed little to any effective transformation of the student learning experience. Where changes to the student experience have taken place, this has arguably been the result of factors other than the external quality monitoring: at best the existence of the latter provides a legitimation for internally‐driven innovation. If quality evaluation is to be transformed to make it transforming it is time to reclaim quality evaluation from opportunistic politicians, re‐establish trust in higher education and focus attention on internal processes and motivators. Instead of politically acceptable methods, quality evaluation needs to adopt appropriate research methodologies. The paper will conclude by exploring the conditions under which quality evaluation might be transformed.

War of the Worlds: who wins in the battle for quality supremacy?

The battle‐lines are drawn up in the various struggles for control of supranational external quality monitoring (accreditation and evaluation). The different types of supranational quality processes are outlined (as they existed at the time of the presentation of the paper to the Conference). The question asked is: who benefits from the extension of quality monitoring beyond national (and sub‐national) boundaries?

Trance, Transparency and Transformation: The impact of external quality monitoring on higher education

This paper discusses the impact of external quality monitoring (EQM) on higher education, and identifies areas in higher education where changes have taken place as a result of such external initiatives. Of special interest is the question whether quality improvement actually is the result of the many EQM systems implemented. By interpreting available data an ambiguous answer is provided, highlighting some of the typical side-effects of current EQM systems at the institutional level. The paper argues that lack of effects directly related to quality improvement should not be conceived as an EQM design error alone but as a misconception of how organisational change actually takes place. In the conclusion, it is claimed that a more dynamic view on how organisations change, highlighting the responsibility of the institutional leadership as 'translators of meaning', may contribute to a more useful process.

Evaluation for What?

External quality monitoring of higher education is explored and analysed. The different types of external bodies and their modus operandi are reviewed. The reasons for undertaking the evaluation are also examined in detail. It is argued that external evaluation legitimates the status quo and fails to ask substantial questions about the nature of learning. External quality monitoring is preoccupied with method and has almost entirely ignored a quarter of a century of research into learning theory, the nature and styles of learning, and classroom innovations. It is suggested that higher education monitoring agencies need to adopt an explicit 'transformation' approach to quality and to address the implications of that for student learning. Only then will they shift from accountability- and compliance-orientated agencies to ones that raise substantial questions about improving student learning.

The Achilles' Heel of Quality: The assessment of student learning

This paper explores the dependability of assessments of student achievement when used as performance indicators for internal and external quality monitoring (IQM and EQM). Problems are identified that jeopardise attempts to monitor, control and enhance quality in higher education. Responses are suggested with preference being given to a radical approach based on accepting that reliable national data about complex student achievements are not to be had. It is argued that this means that reliance on EQM is unwise and that more attention should be paid to internal quality enhancement.

Feeding the Beast or Improving Quality?: Academics' perceptions of quality assurance and quality monitoring

Opportunities to study and explain academics' responses to the growth of external quality monitoring and the extension of institutions' own quality systems have remained largely unexploited. Based on a single-site case study of a university sector college (NewColl), the paper highlights the merits of close-up study in enabling questions to be posed that other approaches are not as well placed to address. With the aid of interview data, the 'implementation gap' between the intentions underpinning 'quality policy' and the actual outcomes is examined. A number of dimensions of the 'implementation gap' are identified, each deriving from situational factors affecting how academics view external and internal quality monitoring systems and frameworks. It is argued that if academics are to remain pivotal in efforts to improve the quality of teaching and learning, then more attention needs to be paid, by institutions and external quality bodies, to the importance of the conditions and context of academics' work. Otherwise, quality monitoring is liable to be invested with a 'beast-like' presence requiring to be 'fed' with ritualistic practices by academics seeking to meet accountability requirements.

An Evaluation of the Impact of External Quality Monitoring on a Higher Education College (1993–98)

ABSTRACT With the establishment, in August 1997, of a new UK‐wide Quality Assurance Agency for Higher Education (QAA), it is timely to reflect on the impact of the two principal forms of external quality monitoring which the QAA has replaced. First, the quality assessment by the funding councils, whose five‐year cycle of teaching quality assessment in universities and colleges in Scotland and Wales, which commenced in 1993, is now complete. Second, institutional quality audits of universities and colleges, conducted by the Higher Education Quality Council (HEQC) between 1993 and 1997. This paper reports the results of a five‐year action research study conducted at a college of higher education. The particular focus of the study was on issues arising from the development and implementation of a quality assurance system at the study institution, and attempts to reconcile requirements for accountability and efforts to encourage quality improvement. The paper reviews and evaluates the impact of each of the forms of external monitoring, teaching quality assessment and institutional audit, on the study institution. Results are presented using qualitative and quantitative data drawn from external performance measures, such as the Scottish Higher Education Funding Council (SHEFC) and HEQC quality assessment and audit reports, and internal measures of staff experience and perceptions. Lessons are drawn in relation to perceived benefits, or otherwise, of external quality assessment and quality audit, and consideration is given to the implications which the findings have beyond the study institution and the Scottish higher education sector, and in particular for the newly established Quality Assurance Agency.