External Quality Assurance Processes Research Articles

Navigating the UKAS Accreditation Process for Detecting Norovirus and Hepatitis A Virus in Fresh and Frozen Produce: a Case Study

Given the urgent requirement for more laboratories to develop proficiency in detecting foodborne viruses, this case study charts the path to accreditation, demystifying the process of validating a method for detecting norovirus and hepatitis A virus in fresh produce. Securing accreditation is crucial to ensuring dependable and precise food analysis, particularly relevant for products frequently consumed raw, which are at risk of contamination by foodborne viruses. The study provides an in-depth look at the stringent procedures integral to achieving precision and dependability in results, underscoring the pivotal role of competency checks involving artificial contamination of samples. The case study also navigates the integral role of both external and internal quality assurance processes in affirming the consistency and accuracy of laboratory testing methods. The findings of this case study are transformative, amplifying confidence in laboratory results and potentially catalysing improvements in public health by ensuring accurate virus detection and identification in food samples. Furthermore, the accreditation process, as detailed in this case study, could pioneer a path for other laboratories, fostering best practices in virus detection and identification.

Gauge against the machine: revisiting quality for multi-targeted serology platforms

Diagnosis of infections of public health significance, such as leptospirosis, often present challenges for laboratories. To counter common challenges and ensure quality driven health responses, rigorous validation and verification processes are required. Despite such rigor, however, can one be certain laboratory reports are truly reflective of infection, given the risk of rare, but potentially very significant quality oversights? Here we present a real-world scenario where diagnosis of leptospirosis cases was compromised over a 6-year period despite quality measures suggesting a well performing serological assay. A subsequent investigation revealed this was attributed to the programming of an automated microtitration plate analyser, evading detection by both quality control and external quality assurance processes. The quality oversight provides insight into potential limitations in quality processes in multi-targeted serological platforms.

EVALUATION OF PROGRAM OUTCOMES IN THE ACCREDITATION PROCESS: A RESEARCH ON THE CASE OF ATATURK UNIVERSITY LANDSCAPE ARCHITECTURE

Yükseköğretimde program akreditasyonu, bir akreditasyon kuruluşu tarafından belirli bir alanda akademik ve mesleğe ilişkin standartların bir yükseköğretim programınca hangi ölçüde karşılandığını belirleyen; ölçme, değerlendirme ve dış kalite güvence sürecidir. Yükseköğretim programlarının akreditasyon süreçlerinde değerlendirmede kullanılan bazı ölçütler bulunmaktadır. Bu ölçütlerden biri, program çıktılarıdır. Program çıktıları yükseköğretim programlarının öğrencilerine öğrenim süresi boyunca kazandırdıkları bilgi, beceri ve davranışlardır. Bu çalışmanın amacı Atatürk Üniversitesi Mimarlık ve Tasarım Fakültesi Peyzaj Mimarlığı Bölümü’nün akreditasyon çalışmalarının bir parçası olarak gerçekleştirdiği program çıktıları belirleme, güncelleme ve ölçme-değerlendirme süreçlerini incelemektir. Çalışma kapsamında bölüme ait akreditasyon öz değerlendirme raporu, bölüm kalite komisyonu raporları ve paydaş toplantısı raporları incelenerek, program çıktılarına ulaşma anketi gerçekleştirilmiştir. Çalışma ile akreditasyon süreçlerinin peyzaj mimarlığı eğitimine yönelik belirli standartların oluşturulması ve eğitim kalitesinin ölçme ve değerlendirme metotlarıyla sürekli izlenerek tedbir geliştirilmesi konusunda öncü bir rol oynayabileceği değerlendirilmiştir. Ayrıca program çıktılarına hangi düzeyde ulaşıldığını değerlendirmede sadece öğretim üyesi ve mezuniyet aşamasındaki öğrencilerin görüşlerine dayanarak yapılan bir değerlendirmenin yetersiz olacağı, programdan mezun olarak en az 1-2 yıl sektörde çalışma deneyimine ulaşmış mezunların da görüşlerine başvurulmasının daha doğru bir değerlendirme verisi sağlayacağı sonucuna ulaşılmıştır.

Agile methodology in higher education quality assurance system for SDGs 4, 8 and 9 achievement: national experience

This paper is focused on considering agile methodology as an instrument to use in education quality assurance. We propose the Scrum method applicable for education quality assurance based on adapted Manifesto for Agile Education Quality Assurance and twelve principles behind it. The Scrum procedure is described and roles are distributed for two real-life cases of external and internal educational program quality evaluation. We illustrate that proposed Scrum procedure perfectly fits existing practices and can be used to enhance both external and internal quality assurance processes in higher education. We consider achievement of SDG 4 targets through proposed methodology as the necessary step to take in achieving SDGs 8 and 9. It is concluded that stakeholders feedback about their satisfaction by economic and innovative factors should be included in each sprint review procedure in proposed Scrum methodology. We discuss SDG 4 achieving within multilayered DIKW+DM hierarchy as a framework for education quality assurance that allow to join information processing, knowledge acquisition and corresponding decision-making algorithm.

Improving of the decision-making model in the processes of external quality assurance of higher education

The peculiarities of external quality assurance processes in the higher education management system are considered. It is noted that the quality of educational programs (EP) of higher education institutions is controlled by quality assurance agencies (QAAs) using an accreditation system. The key features of the accreditation process in terms of peer review are identified. The problem of accreditation process management, namely the subjectivity and lack of consistency of expert decisions, is highlighted. The correlation method was applied to determine the interdependencies in expert assessments (competence, meaningful orientation of judgments, and perception of the linguistic rating scale). The identified types of variables make it possible to explain the existing measure of subjectivity that affects the collective conclusion of experts. A comprehensive methodology for quantitative evaluation of the EP quality under conditions of uncertainty based on the relative importance of the relevant criteria and subcriteria, as well as the levels of expert competence using the apparatus of fuzzy mathematics, is proposed. A basic model for the formation of a collegial expert opinion on the EP quality has been developed using the example of a system of quality criteria approved by the Ukrainian QAA. Variations in the expert values of the weight coefficients and parameters of fuzzy numbers in the context of the linguistic rating scale (“A – B – E – F”) made it possible to use the means of a computational experiment. The application of this model will allow managers to positively influence the existing ambiguity of the assessment method, which requires being guided by standard criteria and at the same time determining the EP innovativeness. In general, the application of the proposed evaluation tools on the quality of EP allows experts and managers to make decisions at a higher level of academic and managerial culture.

EFFECT OF EXTERNAL QUALITY ASSURANCE ON QUALITY CULTURE IN HIGHER EDUCATION INSTITUTIONS IN TANZANIA

This paper aimed at assessing the effect of external quality assurance (EQA) on quality culture (QC) in higher education institutions in Tanzania. The study used explanatory design with quantitative approach. The population included higher education institutions (HEIs) located in Dar es Salaam region and a sample of 350 respondents was selected. Data were collected using survey questionnaires. Findings revealed that external quality assurance processes such as program accreditation, university accreditation, degree recognition, resources availability, and academic standards have different effects on quality culture in higher education institutions in Tanzania. The study concluded that HEIs in Tanzania have benefited from external quality assurance practices since they have allowed them to build quality culture. The study recommended that HEIs should comply with EQA in order to continuously improve their QC.

Evaluation of the reporting quality of clinical practice guidelines on gliomas using the RIGHT checklist.

BackgroundThe reporting quality of clinical practice guidelines (CPGs) for gliomas has not yet been thoroughly assessed. The International Reporting Items for Practice Guidelines in Healthcare (RIGHT) statement developed in 2016 provides a reporting framework to improve the quality of CPGs. We aimed to estimate the reporting quality of glioma guidelines using the RIGHT checklist and investigate how the reporting quality differs by selected characteristics.MethodsWe systematically searched electronic databases, guideline databases, and medical society websites to retrieve CPGs on glioma published between 2018 and 2020. We calculated the compliance of the CPGs to individual items, domains and the RIGHT checklist overall. We performed stratified analyses by publication year, country of development, reporting of funding, and impact factor (IF) of the journal.ResultsOur search revealed 20 eligible guidelines. Mean overall adherence to the RIGHT statement was 54.6%. Eight CPGs reported more than 60% of the items, and five reported less than 50%. All guidelines adhered to the items 1a, 3, 7a, 13a, while no guidelines reported the items 17 or 18b (see http://www.right-statement.org/right-statement/checklist for a description of the items). Two of the seven domains, “Basic information” and “Background”, had mean reporting rates above 60%. The “Review and quality assurance” domain had the lowest mean reporting rate, 12.5%. The reporting quality of guidelines published in 2020, guidelines developed in the United States, and guidelines that reported funding tended to be above average.ConclusionsThe reporting quality of CPGs on gliomas is low and needs improvement. Particular attention should be paid on reporting the external review and quality assurance process. The use of the RIGHT criteria should be encouraged to guide the development, reporting and evaluation of CPGs.

English

The paper presents a possible roadmap for the definition of a European quality label for aerospace related higher education degrees. The proposal is the result of a two-years long Horizon 2020 project that has involved a great portion of the European stakeholders in aerospace: Universities, research centres, industries (both small and large) networks, associations and accreditation agencies. The core concept established is that it is possible to establish a sector-specific, content based, quality system, that can complement the existing national or European accreditation systems, providing added value to the internal and/or external quality assurance processes that are in place in most EU countries. The tools and processes proposed are sufficiently simple to be manageable by Universities in addition to their national accreditation processes or as stand-alone assessment. The main goal of the proposed process is the evaluation of the quality of the aerospace curricula in the European context, whereas the accreditation of the programme can be seen as an optional extension of the process, subject to further national regulations. The process is proposed in view of the awarding of a sector-specific, content based, quality label, to be issued by an appropriate legally recognized and qualified institution. 8 field tests with volunteering universities throughout Europe have been performed. They experienced the method as very practical and to the point.

УЧАСТЬ ЗДОБУВАЧІВ ВИЩОЇ ОСВІТИ У СИСТЕМІ ЗАБЕЗПЕЧЕННЯ ЯКОСТІ

HEI should regulate and encourage active participation in the quality assurance of higher education of all stakeholders, as well as stimulate their interaction. The introduction and support of appropriate mechanisms to ensure the maximum participation of students at different levels of the quality assurance system is one of the main tasks of the leadership of departments, faculties and institutions of higher education. Student involvement in quality can have a positive influence on the delivery and development of any aspect of the student educational experience, whether implemented by the higher education provider, a faculty, a department, or an individual member of staff. It is necessary to envisage the involvement of students in all processes and quality assurance procedures at all institutional levels. It is necessary to clearly define the rights that are granted in each case to student self- government bodies and student. Students should be provided with methodological support to improve their participation in internal and external quality assurance processes. The selection of experts for accreditation processes from the number of students should be carried out through an open competition with the assistance of national student organizations.

The Value of Peer-Evaluation for the Improvement of Quality of the Aerospace Higher Education Across Europe

The paper presents the results of a pilot project aiming at proposing a sector-specific quality system that can complement the existing national or European accreditation systems, providing added value to the internal and/or external quality assurance processes that are in place in most EU Universities. In particular, the proposed system implies a site visit from a group of peers to crosscheck and validate the data and documents prepared for the degree course under evaluation. The results of the implementation of this procedure on 8 Universities are discussed on the basis of the feedback provided, in order to assess the usefulness of the process as perceived by the provider of the aerospace degree.

A Proposal for Ensuring the Quality of Aerospace Engineering Higher Education in Europe

Abstract The paper presents a possible roadmap for the definition of a European quality label for aerospace related higher education degrees. The proposal is the result of a two-years long Horizon 2020 project that has involved a great portion of the European stakeholders in aerospace: Universities, research centres, industries (both small and large) networks, associations and accreditation agencies. The core concept established is that it is possible to establish a sector-specific, content based, quality system, that can complement the existing national or European accreditation systems, providing added value to the internal and/or external quality assurance processes that are in place in most EU countries. The tools and processes proposed are sufficiently simple to be manageable by Universities in addition to their national accreditation processes or as stand-alone assessment. The main goal of the proposed process is the evaluation of the quality of the aerospace curricula in the European context, whereas the accreditation of the programme can be seen as an optional extension of the process, subject to further national regulations. The process is proposed in view of the awarding of a sector-specific, content based, quality label, to be issued by an appropriate legally recognized and qualified institution. A set of 8 field tests with volunteering universities throughout Europe has been performed. They experienced the method as very practical and to the point.

The role of EQA in harmonization in laboratory medicine - a global effort.

There are many activities currently being undertaken in the field of laboratory medicine under the broad heading of “harmonization”. These include traceability of results to international reference standards, processes to align results from assays where traceability has not been achieved (analytical harmonization) and international or national clinical guidelines based on studies from many parts of the world. Many of these issues are global in nature, with clinical evidence derived from studies performed in all parts of the world and multinational diagnostic companies providing assays worldwide. As with all aspects of medicine, progress can only be assured where these is evidence of effectiveness of the activities. External Quality Assurance (EQA) programs are designed to meet this need. Currently EQA processes have significant limitations in meeting the global needs of the laboratory medicine community. This paper aims to identify the steps that can be taken to allow current and future EQA programs to provide information on global variation in results. It is only by being aware of result differences that steps can be taken to improve performance.

Quality assurance in diabetic retinal screening in South Africa.

Diabetic retinopathy (DR) is an important biomarker for microvascular disease and blindness. Digital fundus photography is a cost-effective way of screening for DR. Access to DR screening is difficult for many South Africans with diabetes. To perform external quality assurance (EQA) on graders registered in the Ophthalmological Society of South Africa DR screening programme. Graders registered on the South African (SA) Diabetic Register website were invited to participate in the study. The Scottish EQA software system was used to enable on-line grading of 100 retinal photographs. Expert National Health Service graders provided the consensus expert grading for the image set. Two hundred and sixty-one participants completed the EQA process, including nine ophthalmologists, 243 optometrists, and nine other graders. A wide range of outcomes were demonstrated, with a mean sensitivity of 0.905 (range 0.286 - 1.000) and mean specificity of 0.507 (0.000 - 0.935). The mean diagnostic odds ratio was calculated to be 12.3 (range 0.147 - 148.2). This is the first quality assurance study conducted with SA healthcare professionals. The outcomes are of interest to all stakeholders dealing with the diabetes epidemic. The disparity in grader performance indicates room for improvement. The results demonstrate a high referral rate to ophthalmology, suggesting that on average graders are performing safely, but with a high number of inappropriate referrals.

Assessing standardization of molecular testing for non-small-cell lung cancer: results of a worldwide external quality assessment (EQA) scheme for EGFR mutation testing.

Background:The external quality assurance (EQA) process aims at establishing laboratory performance levels. Leading European groups in the fields of EQA, Pathology, and Medical and Thoracic Oncology collaborated in a pilot EQA scheme for somatic epidermal growth factor receptor (EGFR) gene mutational analysis in non-small-cell lung cancer (NSCLC).Methods:EQA samples generated from cell lines mimicking clinical samples were provided to participating laboratories, each with a mock clinical case. Participating laboratories performed the analysis using their usual method(s). Anonymous results were assessed and made available to all participants. Two subsequent EQA rounds followed the pilot scheme.Results:One hundred and seventeen labs from 30 countries registered and 91 returned results. Sanger sequencing and a commercial kit were the main methodologies used. The standard of genotyping was suboptimal, with a significant number of genotyping errors made. Only 72 out of 91 (72%) participants passed the EQA. False-negative and -positive results were the main sources of error. The quality of reports submitted was acceptable; most were clear, concise and easy to read. However, some participants reported the genotyping result in the absence of any interpretation and many obscured the interpretation required for clinical care.Conclusions:Even in clinical laboratories, the technical performance of genotyping in EGFR mutation testing for NSCLC can be improved, evident from a high level of diagnostic errors. Robust EQA can contribute to global optimisation of EGFR testing for NSCLC patients.

Toward a Transnational Framework: Aligning Internal and External Quality Assurance Processes

This article explores the tensions between internal and external quality assurance processes, making a case for the preeminence of internal actions to ensure the capacity of institutions to respond quickly and effectively to the rapidly evolving global conditions affecting all of higher education. As the forms and means of formal and informal learning evolve more rapidly than quality assurance and accrediting bodies can adapt, institutions themselves will have to take the steps necessary to ensure that students are actually learning at levels represented by a new array of credentials and to offer credible evidence to employers and others that the credentials are indeed accurate reflections of competence. There is a place for quality assurance bodies, but these agencies will find it in their best interest to operate in the areas where they can offer the greatest societal value by articulating the standards that define integrity and quality. Both institutions and quality assurance bodies alike must join forces to recognize that expectations for what actually constitutes quality, competence, and integrity transcend national borders, cultural differences, the ages, occupations or locations of learners, and outmoded notions of prestige. It is a new era where bold ideas and strong ideals can reshape our understanding of what it means to learn throughout life at demonstrable levels of quality and competence.

213P - Results of the Second Pilot External Quality Assurance Scheme for Somatic Egfr Mutation Testing in Non-Small-Cell Lung Cancer (NSCLC)

ABSTRACT Background The External Quality Assurance (EQA) process aims at establishing performance levels and may identify systematic errors in methodology that may not be revealed by internal QA processes. The European Molecular Genetics Quality Network (EMQN), the European Society for Pathology (ESP), the European Thoracic Oncology Platform (ETOP) and the European Society of Medical Oncology (ESMO) with other leading European groups collaborated in a pilot EQA scheme for somatic EGFR gene mutational analysis in NSCLC. Material and methods Samples generated from cell lines mimicking clinical fractions of neoplastic cell content were validated by 5 laboratories and then provided to participating laboratories. Each sample was supplied with a mock clinical case. Participating laboratories registered with the EMQN, performed the analysis using their usual method(s), and submitted their results within a 6 week timeframe. Anonymous results were assessed and made available to all participants. Results Of a total of 117 labs from 30 countries registered, 91 labs returned results. Sequencing and the DxS Therascreen kit were the main methodologies used by the participants. The standard of genotyping was worrying, with a significant number of genotyping errors made. Only 42 (46.1%) participants reported the correct result for all 10 samples. The majority of errors were false-negative results (82.2%); however, 15.1% of errors were due to false-positive results and 2.7% to a combination of both errors. There were 42 (4.8%) analytical failures of which 88.1% were distributed across 6 laboratories. The performance of 18 (19.8%) labs fell below the criteria for acceptable performance (score Conclusions The technical performance of genotyping in EGFR mutation testing for NSCLC has room for improvement, evident from a high level of “true” diagnostic errors. Overall the standard of reporting was acceptable. Robust EQA can contribute to global optimization of EGFR testing. Disclosure All authors have declared no conflicts of interest.

Proficiency testing/external quality assurance for the PFA-100®

Platelet function testing is integral to haemostasis investigations and the Platelet Function Analyser-100 (PFA-100(®)) is globally the most utilised primary haemostasis-screening instrument. External Quality Assurance (EQA) (or proficiency testing) is critical to ensuring quality of test practice, but EQA for platelet function is logistically challenging and actual test-challenges generally not possible. A novel approach was therefore developed whereby a range of formulated test tubes are distributed to EQA participants to which citrated normal whole blood collected on site is added, thereby creating test material that can be locally evaluated. Several exercises have been conducted over the past four years (total of 18 challenges, most designed to mimic an aspirin effect or a mild or severe primary haemostasis defect, tested in 26-47 laboratories). Numerical results for PFA-100(®) closure times (CTs) and interpretive comments provided by participants were analysed. Reported CTs for each challenge were within limits of expectation and good reproducibility was evidenced by repeated challenges. Coefficients of variation (CVs) generated for two PFA-100(®) cartridge types (C/ADP and C/Epi) for challenges [median (range): 14.8 (3.9-29.5) and 13.9 (0.6-29.5)] was similar to those obtained using native whole blood [15.6 (14.2-18.9) and 17.3 (13.5-20.5)]. Interpretations were in general also consistent with expectations and test data provided by laboratories. In conclusion, an EQA process for the PFA-100(®) has been developed that includes a highly reproducible test-challenge process, not only proving the concept is possible for platelet function testing, but also providing a valuable mechanism for monitoring and improving laboratory performance.

Implementation of a reference standard and proficiency testing programme by the World Wide Antimalarial Resistance Network (WWARN)

BackgroundThe Worldwide Antimalarial Resistance Network (WWARN) is a global collaboration to support the objective that anyone affected by malaria receives effective and safe drug treatment. The Pharmacology module aims to inform optimal anti-malarial drug selection. There is an urgent need to define the drug exposure - effect relationship for most anti-malarial drugs. Few anti-malarials have had their therapeutic blood concentration levels defined. One of the main challenges in assessing safety and efficacy data in relation to drug concentrations is the comparability of data generated from different laboratories. To explain differences in anti-malarial pharmacokinetics in studies with different measurement laboratories it is necessary to confirm the accuracy of the assay methods. This requires the establishment of an external quality assurance process to assure results that can be compared. This paper describes this process.MethodsThe pharmacology module of WWARN has established a quality assurance/quality control (QA/QC) programme consisting of two separate components:1. A proficiency testing programme where blank human plasma spiked with certified reference material (CRM) in different concentrations is sent out to participating bioanalytical laboratories.2. A certified reference standard programme where accurately weighed amounts of certified anti-malarial reference standards, metabolites, and internal standards are sent to participating bioanalytical and in vitro laboratories.ConclusionThe proficiency testing programme is designed as a cooperative effort to help participating laboratories assess their ability to carry out drug analysis, resolve any potential problem areas and to improve their results - and, in so doing, to improve the quality of anti-malarial pharmacokinetic data published and shared with WWARN.By utilizing the same source of standards for all laboratories, it is possible to minimize bias arising from poor quality reference standards. By providing anti-malarial drug standards from a central point, it is possible to lower the cost of these standards.

Students' Roles in Maintaining Quality and in Enhancing Learning: Is There a Tension?

ABSTRACT This article is about student engagement and in particular the engagement of students in internal institutional quality assurance processes in the UK. It discusses the extent to which the introduction of more explicit internal and external quality assurance processes militate against the notion of the student as a part of a ‘cohesive learning community’ in favour of the notion of the student as ‘consumer’. Based on findings from a research study on student engagement undertaken for the Higher Education Funding Council for England, the article notes that an underlying rationale of student as consumer still prevails in many institutions' quality processes. Such a rationale is hampering students' full engagement in learning communities in ways that inform and enhance the collective student learning experience. Whilst these findings are drawn from a UK study, they may also have some relevance for other countries facing similar challenges.

On the Relationship of External Quality Assurance and Equity: Can they Converge on National Policy Agendas?

This article presents the findings of exploratory research on the relationship of external quality assurance (EQA) systems and equity policies conducted at the UNESCO Institute for Educational Planning. Authors from Australia, Brazil, India and South Africa were invited to discuss this relationship in the context of their respective higher education systems. More precisely, the study set out to analyse: (1) whether EQA systems of the four countries in their orientation explicitly refer to national equity objectives; (2) whether quality models and quality assessment procedures developed under EQA systems address national equity concerns; and (3) whether EQA processes such as follow‐up and reporting mechanisms are conducive to enhance the monitoring of national equity policies at the institutional level. The project reveals that national equity objectives can be reflected in EQA systems and that EQA can also be an instrument for the monitoring of national equity policies even where major differences occur in national EQA frameworks. The article also brings to light under which different modalities national equity concerns can be inserted in EQA systems.