A Conceptual Framework for Quality Assurance of LLM-based Socio-critical Systems

For autosegmentation models, the data used to train the model (e.g., public datasets and/or vendor-collected data) and the data on which the model is deployed in the clinic are typically not the same, potentially impacting the performance of these models by a process called domain shift. Tools to routinely monitor and predict segmentation performance are needed for quality assurance. Here, we develop an approach to perform such monitoring and performance prediction for cardiac substructure segmentation. To develop a quality assurance (QA) framework for routine or continuous monitoring of domain shift and the performance of cardiac substructure autosegmentation algorithms. A benchmark dataset consisting of computed tomography (CT) images along with manual cardiac substructure delineations of 241 breast cancer radiotherapy patients were collected, including one "normal" image domain of clean images and five "abnormal" domains containing images with artifact (metal, contrast), pathology, or quality variations due to scanner protocol differences (field of view, noise, reconstruction kernel, and slice thickness). The QA framework consisted of an image domain shift detector which operated on the input CT images and a shape quality detector on the output of an autosegmentation model, and a regression model for predicting autosegmentation model performance. The image domain shift detector was composed of a trained denoising autoencoder (DAE) and two hand-engineered image quality features to detect normal versus abnormal domains in the input CT images. The shape quality detector was a variational autoencoder (VAE) trained to estimate the shape quality of the auto-segmentation results. The output from the image domain shift and shape quality detectors was used to train a regression model to predict the per-patient segmentation accuracy, measured by Dice coefficient similarity (DSC) to physician contours. Different regression techniques were investigated including linear regression, Bagging, Gaussian process regression, random forest, and gradient boost regression. Of the 241 patients, 60 were used to train the autosegmentation models, 120 for training the QA framework, and the remaining 61 for testing the QA framework. A total of 19 autosegmentation models were used to evaluate QA framework performance, including 18 convolutional neural network (CNN)-based and one transformer-based model. When tested on the benchmark dataset, all abnormal domains resulted in a significant DSC decrease relative to the normal domain for CNN models ( ), but only for some domains for the transformer model. No significant relationship was found between the performance of an autosegmentation model and scanner protocol parameters ( ) except noise ( ). CNN-based autosegmentation models demonstrated a decreased DSC ranging from 0.07 to 0.41 with added noise, while the transformer-based model was not significantly affected (ANOVA, ). For the QA framework, linear regression models with bootstrap aggregation resulted in the highest mean absolute error (MAE) of , in predicted DSC (relative to true DSC between autosegmentation and physician). MAE was lowest when combining both input (image) detectors and output (shape) detectors compared to output detectors alone. A QA framework was able to predict cardiac substructure autosegmentation model performance for clinically anticipated "abnormal" domain shifts.

The higher education quality evaluation and assurance frameworks and their operating mechanisms of countries such as the United Kingdom, France, and the United States show that higher education systems, traditional culture, and social background all impact quality assurance operating mechanisms. A model analysis of these higher education quality assurance frameworks shows that quality assurance entities having clear and harmonious responsibilities and interest relations is a precondition to the good operation of higher education quality assurance mechanisms, and the model of internal self-discipline of the higher education institutions taking precedence and internal and external assurances integrating organically is a developing trend in higher education quality assurance.

:The higher education quality evaluation and assurance frameworks and their operating mechanisms of countries such as the United Kingdom, France, and the United States show that higher education systems, traditional culture, and social background all impact quality assurance operating mechanisms. A model analysis of these higher education quality assurance frameworks shows that quality assurance entities having clear and harmonious responsibilities and interest relations is a precondition to the good operation of higher education quality assurance mechanisms, and the model of internal self-discipline of the higher education institutions taking precedence and internal and external assurances integrating organically is a developing trend in higher education quality assurance.

Background and Issues: A minimum of 3 hours of rehabilitation intensity (RI) is a stroke best practice standard of care that has been established in Ontario. As collection of RI time has been mandated in Ontario, the Ontario Stroke Network has been working with stakeholders to capture RI data using workload measurement systems (WMS). Given that this is a new indicator and variability exists in use of WMS across Toronto, there is a need for standardization in RI data collection. Purpose: To develop a quality assurance (QA) framework for RI data collection in Toronto that can be used locally and provincially to ensure consistent and accurate data collection and reporting. Methods: The Toronto Stroke Networks formed a working group comprised of clinical and decision support leads from 6 rehabilitation centres. This group developed standard approaches to capturing RI data. They also examined factors that influence the quality of the data, ranked these factors by level of influence on data quality, and established mitigation strategies for the top ranked factors. Additionally, a reporting and monitoring plan was determined. Results: Through collaborative sharing of information, a QA framework was developed and adopted by the group. The top ranked factors affecting data quality included: 1) When the RI data field is left blank; 2) Inaccurate data entry by staff; 3) Variations in service interruptions and alternate level of care; and 4) Timeliness in entering data. Mitigation strategies included use of lock out periods, clear definitions, processes for staff feedback and education, and establishment of a quarterly reporting structure for organizational comparison. This QA framework has been shared with provincial stakeholders to inform system planning in other regions. Conclusions: As data accuracy in capturing RI data is important for current state analysis and benchmarking, a comprehensive QA framework was developed to support data accuracy and has been used for local monitoring and reporting of RI data. Identified factors that influence data quality related to processes for data capture. As mitigation strategies will further support these processes, this QA framework ensures accurate data collection and confidence in using RI data to support system planning.

PurposeThe purpose of this study is to examine quality assurance for archives and records management (ARM) education in an open distance e-learning (ODeL) environment in Eswatini with a view to conceptualise a framework for quality assurance in the development and implementation of an ARM programme in the context of ODeL.Design/methodology/approachThe study is anchored on the interpretive research paradigm, which surrounds a systematic literature review. The researcher searched for literature online, using scientific databases such as Ebsco, Scopus and Google Scholar. The search applied the publications from 2005 to 2019. The main search keywords are “archives”, “records management”, “open distance e-learning” and “quality assurance”. A total of 15 articles, which included documents, journal articles, reports, web pages and monographs, were retrieved, reviewed and analysed in this study. This conceptual study was preliminary, and the researcher hopes that further empirical studies based on the findings of this study could be pursued in future.FindingsARM as a form of study and delivered on ODeL platforms have been proposed in Eswatini. The major concern among stakeholders is how to ensure the quality of such programmes.Research limitations/implicationsThis conceptual study was preliminary, and the researcher hopes that further empirical studies based on the findings of this study could be pursued in future.Practical implicationsThe findings and recommendations will help in the development of ARM programmes to be offered effectively by way of ODeL, there is a need to develop a transparent quality assurance framework for such an application and its implementation.Originality/valueThis is the first study on quality assurance for ARM education in an ODeL environment in Eswatini. As part of the study, a quality assurance framework was designed for the implementation of ARM education on an ODeL platform. This quality assurance framework is intended to help higher education institutions in Eswatini such as the University of Eswatini as well as stakeholders such as the Eswatini National Archives and others to design ARM education and deliver it on an ODeL platform in a manner that guarantees adequate quality.

Molecular tests like polymerase chain reaction were widely used during the COVID-19 pandemic but as the pandemic evolved, so did SARS-CoV-2. This virus acquired mutations, prompting concerns that mutations could compromise molecular test results and be falsely negative. While some manufacturers may have in-house programs for monitoring mutations that could impact their assay performance, it is important to promptly report mutations in circulating viral strains that could adversely impact a diagnostic test result. However, commercial test target sites are proprietary, making independent monitoring difficult. In this study, SARS-CoV-2 test target sites were sequenced to monitor and assess mutations impact, and 29 novel mutations impacting SARS-CoV-2 detection were identified. This framework for molecular test target site quality assurance could be adapted to any molecular test, ensuring accurate diagnostic test results and disease diagnoses.

Challenges and opportunities for effective assessments within a quality assurance framework for MOOCs

Education and the challenges of preparing quality teachers are important priorities in many countries. Singapore is no different. The success of what Singapore hopes to achieve in education depends on the quality of its teachers. Competent and effective teachers help build a strong educational system. In response to the nation's need for quality teachers, the National Institute of Education (NIE), Singapore, reviewed and enhanced their initial teacher preparation programmes in 2005. A Values, Knowledge and Skills (VSK) model listing the attributes of beginning teachers was developed to anchor the review and enhancement. This also provided first steps in developing an overarching Quality Assurance (QA) framework. The QA framework covers key aspects of programme delivery and development, from student's entry profiles to beginning teacher's competencies. The paper has two parts. Part one details the development and conceptual underpinning of the QA framework. Part two outlines the framework's structure and components.

This research presents the NatVi Framework for Agile Software Development, Service-Oriented Architecture (SOA), and Quality Assurance (QA). The research took place in a context of profound and rapid changes in business environments that affect the software development environment. Our previous work did a Systematic Literature Review trying to find articles dealing with SOA and Agile and the challenges inherent in this combination of solutions. In order to build the Framework, this work took the shortcomings found in the solutions presented in the papers and further incorporated the necessary QA concept. In this context, the Framework attempts to provide an answer to how to develop software with quality and rapid deliveries in an ever-changing environment, where the traditional forms of software development may not handle it. Background research identified trade-offs among SOA, QA, and Agile, e.g., formality, documentation, and planning. The background research also identified strengths, weaknesses, and gaps in papers that addressed solutions to problems that arise in software development in the presented context. The results of the background research were assessed and exploited in Framework construction. In a single life-cycle, the NatVi Framework combines Agile, SOA, and QA and addresses the values and principles that guide them in 13 phases distributed in four layers. Each phase is carried out by people who perform specific roles, expressed in terms of inputs, tasks, and outputs. As a Framework, it is not exhaustive, and its main concepts may change and adapt to each environment. The Framework validation is underway, and a forthcoming paper will present the results.

Like most industrialized countries, Germany's health care system is facing two major challenges. The first is to find a sustainable financing system for increasing health care expenditures. The second is to ensure - and improve - the quality of care provided. This article describes the status quo in quality assurance in Germany and analyses the changes introduced into the SHI (Statutory Health Insurance) system with the Modernization Act of 2004. First, a theoretical framework for quality assurance that is consistent with the logic of the German social market economy is outlined. The analysis then describes new actors and their duties in the field of quality assurance, highlighting improvements in regulation and the regulatory instruments applied. Although the strategy for quality assurance is still dominated by regulation and corporatist bodies, the latest reform acts of 2004 and 2007 focus on more and better information about the quality of services provided - an important prerequisite for more competitive elements in the German health care system.

Assessment is a key aspect of curriculum design and is fundamental to the learning process. The methods used for assessment are of prime pedagogical importance as they have a direct bearing on both teaching and learning provision and the learning experience. Assessment methods must be planned so that they align well with intended learning outcomes in e-assessment procedures in particular, in order to ensure reliable and secure learner authentication and authorship in online and blended learning environments. A framework for quality assurance in trust-based e-assessment processes was developed in accordance with these objectives. The framework has been designed in line with the Standards and Guidelines for Quality Assurance in the European Higher Education Area (ESG) to assure the applicability of the e-assessment system to all online and blended learning environments while respecting the shared European framework for quality assurance (QA). This chapter provides contextual information on the quality assurance of higher education in the European Higher Education Area (EHEA) and presents in more detail the rationale behind the development of the framework, together with aspects that need to be taken into consideration when designing and implementing the quality assurance of e-assessment.

The most important problems of the higher education institutions are enhancing quality assurance. Quality assurance is at the heart of academic activities in higher education institutions in today’s world. One of the methods to determine the quality of higher education institutions is analyzing and summarizing the satisfaction of stakeholders. One of the best ways to overcome this problem is by using a decision support framework for quality assurance. That framework is analyzing higher education institutions’ historical data and supporting decision-making activities. At first, we have determined stakeholders for teaching and learning processes in NUM. In the second step, we used Weka to analyze more than 1000 students, 200 graduates’ satisfaction results. Similarly, quality improvement factors are considered. Finally, we have proposed a survey-based framework for Enhancing Higher Education Quality Assurance.

This study presents a design-integrated visual intelligence framework for micro-component quality assurance, enabling interpretable defect detection and real-time feedback in CAD-centric engineering workflows. The proposed Multi-Scale Multi-Feature Hybrid Model (MSMFHM) combines morphological and textural representations through bidirectional cross-attention with entropy-guided weighting. The architecture consists of a Resize-Focus preprocessor, Multi-Scale Mix Module, Multi-Feature Fusion Module, and dual-head decoder, aligning visual features with boundary-represented (B-Rep) CAD entities for tolerance verification and design traceability. Experiments on the TEC-Defect dataset demonstrate a Top-1 accuracy of 96.8% and a Macro-F1 score of 95.1%. Zero-shot validation on the DAGM2007 dataset confirms cross-domain generalization. The framework achieves 168 FPS at 3.8 GFLOPs on embedded hardware, ensuring deployment efficiency. Grad-CAM visualizations highlight interpretable feature attention and precise defect localization. The MSMFHM framework establishes an intelligent, CAD-integrated defect analysis pipeline, promoting proactive quality assurance and cyber-physical co-design across manufacturing processes.

The construction industry, particularly in housing projects, faces numerous challenges, including ensuring regulatory compliance, maintaining high-quality standards, and managing risks such as delays, cost overruns, and safety concerns. This paper explores the integration of Artificial Intelligence (AI) into construction to address these issues through a smart, AI-driven framework for compliance, quality assurance, and risk management. By reviewing existing literature on AI applications in construction, the paper highlights how AI technologies, such as predictive analytics, natural language processing, and machine learning, can significantly improve construction outcomes by streamlining compliance processes, enhancing quality control, and mitigating risks in real-time. The paper also discusses the key design principles for creating an AI framework tailored to construction, examining how AI can monitor regulatory adherence, ensure quality standards, and predict potential project risks. While AI presents vast opportunities, the paper identifies challenges such as technological integration, financial constraints, and regulatory hurdles that may hinder its widespread adoption. Proposed solutions, including AI-based decision-support tools and cloud-based platforms, aim to overcome these challenges. Finally, the paper emphasizes the need for future research, particularly in areas like the integration of AI with Building Information Modeling (BIM), the development of universal standards for AI in construction, and the validation of AI frameworks through case studies. This research provides valuable insights into the potential of AI to transform the construction industry, offering recommendations for further development and implementation in housing projects.

This paper is based on a study that is aimed at developing a framework for quality assurance of Library and Information Science (LIS) education in an Open Distance e-Learning (ODeL) environment. The study was anchored on the interpretive research paradigm and entailed a literature review. LIS, as a field of study that is delivered on an ODeL platform, is proposed for Eswatini. The quality of such a programme is of great concern among stakeholders. This study develops a framework for quality assurance in the development of LIS programmes in an ODeL context in Eswatini.