External Quality Assurance Agencies Research Articles

University Actors Responding to the National Quality Assurance Regime for Higher Education in Malawi: A Case of Instrumentality, Pragmatism and Symbolic Compliance

ABSTRACTThe literature shows that the question of how to integrate quality assurance into higher education institutions is associated with several obstacles including non‐support from university actors. This study explored the university actors' response to the recently implemented external quality assurance and accreditation regime for higher education in Malawi. The findings showed that university actors respond to external quality assurance in divergent ways, mainly characterised by formal instrumentality, professional pragmatism and symbolic compliance. This meant that it would be naïve for external quality assurance agencies to assume that when university actors participate in external accreditation processes, it means that they embrace external quality assurance as a mechanism for enhancing quality. The implication was that national policy makers and quality assurance agencies should not consider university actors as ‘passive recipients’ that mutely accept quality assurance reforms, but rather seriously attend to them as both ‘makers’ and ‘shapers’ of policy in order to develop quality assurance systems that can be genuinely embraced. The study contributes to research that take a critical perspective in interpreting university personnel response to quality assurance.

‘It’s paradoxical, but it works' – Towards ambidexterity in external quality assurance: the case of Roman Catholic ecclesiastical higher education

ABSTRACT This article focuses on organizational responses to seemingly paradoxical mandates, in the context of external quality assurance in higher education. With the promulgation of the Apostolic Constitution Veritatis Gaudium in 2018, Roman Catholic ecclesiastical higher education institutions have been required to be evaluated by a designated external quality assurance agency, the Holy See’s Agency for the Evaluation and Promotion of Quality in Ecclesiastical Universities and Faculties (AVEPRO). Operating internationally, the Agency is tasked to both evaluate quality with an accountability mindset and promote quality with a continuous enhancement outlook. These two approaches to quality processes can be considered conflicting notions. Using the lens of paradox theory and organizational ambidexterity to observe the nature of and responses to tensions, this study considers how AVEPRO managed the challenge of reconciling both QA and QE imperatives. Although ambidexterity studies have mainly focused on the private sector, this analysis shows that this theoretical framework can provide a relevant lens in other contexts where paradoxes exist, such as external quality assurance agencies in higher education. Findings show that an innovative organizational structure and a creative mindset can reconcile quality assurance and quality enhancement processes through different channels.

Towards blended external quality assurance agencies: a European experience

During this pandemic period, many external quality assurance agencies organised their external assessment visits to higher education institutions virtually. This has been an experience that must be valued, determining whether in future it makes sense to continue organising visits virtually or adopting other similar formats. To this end, this article presents the results of a survey carried out among participants in 100 external assessment panels. Four different items have been analysed: satisfaction with different aspects of the virtual site visits, overall satisfaction, the visit of the facilities and the future of virtual visits. The results show that satisfaction with the process is very broad. Moreover, the physical visit to the facilities is still valued very positively. Few statistically significant differences have been found between programme and institutional accreditation processes, which suggests that whatever the future proposal is, it would follow a similar strategy in both cases.

The battle for legitimacy of ‘student involvement’ in external quality assurance of higher education from Asian QA perspectives – does policy contextualization matter?

ABSTRACT Student involvement, through formal representation in external quality assurance (EQA) has attracted the attention of governments, higher education institutions (HEIs) and quality assurance agencies worldwide. This study aims to explore current developments and the legitimacy of student involvement in EQA agencies in the Asian context. There are several major findings. First, more than a half of responding accreditors in Asia have already engaged students in the EQA systems. Second, student roles in external review processes have not been fully recognized. Third, students as HE stakeholders are considered increasingly relevant as they contribute their valuable learning experiences to HEIs.

The policy of quality assurance of university e-education in Europe and Latin America

The relevance of the study arises from the contradiction between the rapid development of e-education in different regions of the world, on the one hand, and the insufficient level of adaptation of the quality assessment mechanisms in the field of e-education in certain regions, on the other hand. This article examines and analyzes the international experience of quality assurance of higher education in the format of e-learning in Europe and Latin America. The article identifies the approaches to quality assurance of e-learning in the field of higher education in Europe and Latin America. The study is aimed at analyzing the policy of quality assurance of university e-education in the field of higher education in Europe and Latin America. The analysis of the activities of European and Latin American quality assurance organizations in the field of e-learning has shown that in the context of higher education in Europe and Latin America external quality assurance agencies play an important role in the institutional processes of ensuring the quality of university e-education. The development of a quality assurance system is an important part of the internationalization of educational programs in general and e-learning in particular since it is open and dynamic. The assessment of the quality of education is not simple; it is an important problem for the related organizations.

Impact of institutional quality audit in the Aotearoa New Zealand university system

ABSTRACT Assessing impact of external quality assurance is a critical component of the professional practice reflection of external quality assurance agencies. This paper analyses the responses of universities in Aotearoa New Zealand to recommendations made in the most recently completed cycle of academic audits using artefacts (universities’ one-year follow-up reports on responses to recommendations) from the quality assurance process. This approach allows for an assessment at system level and qualitative analysis provides further insight into thematic areas where response to recommendations is further advanced. One year after the release of audit reports, impact has been greater in the areas of leadership and management of teaching and learning and research student supervision. An index developed in the paper can support inter-institutional and inter-system comparative analyses of impact.

Institutional accreditation of online institutions in the Anglophone Caribbean: synergies and misalignments

ABSTRACT This article explores the status of quality assurance provisions, internal and external, for online education in the Anglophone Caribbean region and provides a comparative analysis of current global trends. The findings of that analysis show that there is a misalignment between the provisions of the external quality assurance agencies and needs of the online institutions, particularly in the region. However, it also highlights the existence of a regional framework policy for quality assurance of online programmes which, if implemented, could alleviate much of the disconnect between the national external quality assurance agencies and the online institutions. Finally, it suggests the need for greater harmonisation between the standards of the various national quality assurance agencies and alignment to regional and international quality assurance for online models.

Sample size and rejection limits for detecting reagent lot variability: analysis of the applicability of the Clinical and Laboratory Standards Institute (CLSI) EP26-A protocol to real-world clinical chemistry data.

To maintain the consistency of laboratory test results, between-reagent lot variation should be verified before using new reagent lots in clinical laboratory. Although the Clinical and Laboratory Standards Institute (CLSI) document EP26-A deals with this issue, evaluation of reagent lot-to-lot difference is challenging in reality. We aim to investigate a practical way for determining between-reagent lot variation using real-world data in clinical chemistry. The CLSI EP26-A protocol was applied to 83 chemistry tests in three clinical labs. Three criteria were used to define the critical difference (CD) of each test as follows: reference change value and total allowable error, which are based on biological variation, and acceptable limits by external quality assurance agencies. The sample size and rejection limits that could detect CD between-reagent lots were determined. For more than half of chemistry tests, reagent lot-to-lot differences could be evaluated using only one patient sample per decision level. In many cases, the rejection limit that could detect reagent lot-to-lot difference with≥90% probability was 0.6 times CD. However, the sample size and rejection limits vary depending on how the CD is defined. In some cases, impractical sample size or rejection limits were obtained. In some cases, information on sample size and rejection limit that met intended statistical power was not found in EP26-A. The CLSI EP26-A did not provide all necessary answers. Alternative practical approaches are suggested when CLSI EP26-A does not provide guidance.

Questioning the efficacy of quality assurance frameworks for teaching and learning: a case study from East Africa

ABSTRACTIncreasingly across many jurisdictions, external quality assurance agencies are requiring higher education institutions to develop internal quality assurance systems that grant ownership of programme improvement to those implementing teaching and learning. Whilst this development is laudable, the quality assurance frameworks used for this process appear to focus on a comprehensive list of ‘cells’ and criteria against which to assess academic standards. However, these standards vary in their degree of relevance to teaching and learning and are not always evidence-based. An internal quality assurance framework used across East Africa is analysed and comparisons are made with indicators of quality that are known to affect teaching effectiveness and student learning outcomes. These comparisons highlight the need for quality assurance agencies to develop internal quality assurance frameworks for programme review that focus on the quality of the learning experience to ensure improved programme provision.

The implementation of self-accreditation policy in Taiwan higher education and its challenges to university internal quality assurance capacity building

ABSTRACTIn 2012, the Taiwanese Ministry of Education launched a new quality assurance policy entitled ‘self-accreditation’, aimed at enhancing institutional autonomy, as well as establishing its internal quality review mechanism. In the self-accreditation policy, higher education institutions are encouraged to develop their own quality assurance frameworks based on strategic direction and institutionally specific features. The study adopts qualitative and quantitative approaches to explore the impacts and challenges of the 2012 Ministry of Education self-accreditation policy over Taiwanese universities and accrediting bodies. There are three major findings. 1. Self-accreditation policy had positive consequences and outcomes on university’s internal quality assurance capacity building. 2. Inappropriate selection processes of reviewers, arbitrary elimination of standards and indicators, inconsistency across review decisions are the challenges that self-accrediting universities encountered. 3. Under the new quality assurance policy, the roles and responsibilities of external quality assurance agencies should be redefined.

A framework for enhancing regulatory cooperation in external quality assurance in Southern Africa

ABSTRACTCross-border higher education (CBHE) has taken centre stage in the Southern Africa Development Community (SADC). There has been increased trade in higher education services among member states of the SADC. This has necessitated regional regulatory cooperation in quality assurance and accreditation. SADC has established the Southern African Quality Assurance Network as a regional network of external quality assurance agencies. This paper presents a framework that can be used to enhance regulatory cooperation within the Network on quality assurance in higher education with special focus on CBHE. The paper identifies the main areas of focus as people mobility, institutional and programme mobility, accreditation and registration and recognition of academic credentials. Proposed mechanisms for regulatory cooperation include regional credit accumulation and transfer system, regional qualifications framework, framework for accreditation and registration of foreign providers and framework for recognition of academic credentials.

External quality assurance agencies and excellence in higher education

The article presents briefly the complexity of the concept of “excellence” in higher education. Taking into account this reality, the fundamental question is how agencies can consider this complexity in building up criteria on excellence and the use of the concept “excellence” in their improvement strategies and accountability processes in the sector of higher education. Public-private nature of higher education and its growing internationalisation have significant impact on how excellence in education is described and promoted as a tool.

Adopting self-accreditation in response to the diversity of higher education: quality assurance in Taiwan and its impact on institutions

In 2012, Taiwan implemented a dual-track quality assurance system comprising accreditation and self-accreditation in higher education institutions. Self-accrediting institutions can accredit their programs without requiring approval from external quality assurance agencies. In contrast to other countries, the Ministry of Education of Taiwan authorized self-accrediting institutions to develop their own evaluation standards. This study investigated the institution-based accreditation standards and their implications on institutional internal quality assurance. Content analysis revealed that 37 % of the indicators of self-accreditation were new and not used as review indicators in the original accreditation track. Two frequently added indicators were featured indicators and levels of internationalization. The results also indicated that institutions tend to structure their internal quality assurance systems uniquely. Three types of approaches for developing institution-based standards were identified: bottom-up, hybrid, and innovative approaches. Self-accreditation has benefited institutions committed to educational quality and pursuing excellence by enabling them to employ a fitness-for-purpose approach. The diversity of higher education and educational policy changes constitute new challenges to higher education. Balancing between accountability and autonomy is critical for all stakeholders of higher education.

Impact evaluation from quality assurance agencies’ perspectives: methodological approaches, experiences and expectations

Starting from the main objective of external quality assurance (EQA) procedures to assure and improve the quality of higher education institutions and its provisions, the paper examines expected impacts of EQA procedures on institutions from the perspectives of three European quality assurance agencies. First, the paper examines the expected impacts of different standard parts of a typical peer review procedure on higher education institutions and assesses when a procedure is most likely to have an effect on the institution. The second part presents the current practices of the three EQA agencies, AQU Catalunya (Spain), Evaluationsagentur Baden-Württemberg (evalag) (Germany) and the Finnish Education Evaluation Centre (Finland) on assessing the impact of their EQA procedures.

Professionalism, profession and quality assurance practitioners in external quality assurance agencies in higher education

This article seeks to spark a dialectic discussion on the establishment of a set of professional competencies for quality assurance practitioners who serve in external quality assurance agencies in higher education. Such a need is identified due to the shortage of relevant and sufficient coverage in the quality assurance literature. To substantiate the need, the concepts of professionalism and profession, and how these concepts are linked to the quality assurance in higher education sector, are carefully examined through a desk-top research study. References are drawn from the experiences in several other occupations. The result reveals that the development of the essential competencies for external quality assurance practitioners in higher education is valuable and essential and it will be useful for the practitioners to transform themselves into a profession in the long run.

Quality assurance in higher education: Navigating the market/equity tension

The current implementation phase of quality assurance in higher education has ushered in a range of debates and ideas. In response to a recent presentation on the topic, this article provides a glimpse into some of the conditions that lend themselves to the emergence of external quality assurance agencies. A key element nationally, resonating with the international context, is the pressure on the system to widen participation to a more diverse student body. This process is occurring simultaneously with the incursion of market forces into higher education. The coalescence of these factors culminating in a set of tensions, is what is considered in this article.

The INQAAHE Guidelines of Good Practice for External Quality Assurance Agencies: Assessment and next steps

INQAAHE is working to improve the portability of degrees and credits across national boundaries. By defining ‘good practices’ for quality assurance agencies, INQAAHE will promote mutual recognition among agencies in different countries and the acceptance of their accreditation decisions outside their own country. This will allow students and others to rely on the judgments of the local quality assurance agency where the institution granting the degree or credits is located. This paper outlines the nature and importance of the Guidelines of Good Practice (GGP), describes how they were created and modified, and explains recommendations for various uses of the GGP made by INQAAHE members during their last meeting. The paper identifies ways in which the GGP could be enhanced and argues for coordination of the INQAAHE guidelines with those produced by international, regional, national, and other agencies.

Sets of Standards for External Quality Assurance Agencies: A comparison

The aim of quality assurance codes of practice and guidelines is, in theory, to give a clear indication to stakeholders, governments, financers, partners and the public at large about the various course providers and the level of education they offer. This paper compares six different quality assurance codes of practice and guidelines and uses the INQAAHE guidelines as a baseline. The paper investigates the extent to which there is a degree of transparency and comparability between the standards and criteria that are applied by the different networks. Overall, there appears to be considerable transparency and comparability between the codes of the major international networks of quality assurance agencies.

Impact of Quality Assurance: Overview of a discussion between representatives of external quality assurance agencies

There has been considerable debate about the impact of external quality assurance in higher education. This paper describes a discussion at the INQAAHE workshop in The Hague, attended by representatives of agencies, exploring the effects of external quality processes on institutions and programmes. The summary, probably for the first time, illuminates the varied impacts that quality processes have from the point of the agencies themselves. The main impacts identified include changes evident from one review to the next; improvements in performance indicators; adoption of formal internal quality processes by institutions; student feedback indicating positive changes and employer perceptions about the improvement in graduate abilities.

Evaluations of the Third Kind: External evaluations of external quality assurance agencies [1

This paper reviews several cases of external evaluations of external quality assurance (EQA) agencies. The characteristics, methodology and findings of six such evaluations from three continents are discussed and similarities and differences are analysed. The lessons drawn from these six cases show that external evaluation of agencies is a powerful means of assuring and enhancing the quality of operation of agencies, serving both their improvement and accountability. Moreover, external evaluations can facilitate the process of mutual recognition of agencies, evaluation results and national quality assurance systems.