Improve Test Quality Research Articles

Strategies for Error Reduction: Why More Stringent Premarket Evaluations Do Little to Prevent Laboratory Errors and Traffic Accidents

Laboratory testing is a complex process with a significant error rate. Studies of laboratory errors have found that the major causes are preanalytical factors, interferences, and process errors. Efforts by regulatory agencies to improve quality via more stringent premarket evaluations of laboratory tests therefore have poor prospects of reducing laboratory errors and improving test quality. Efforts toward increasing the regulation of laboratory tests are analogous to preventing traffic accidents by increasing the premarket evaluation of automobiles. This analogy illustrates how increased premarket evaluation has limited prospects for quality improvement and, in some cases, actually contributes to errors and lower quality. Tools that are used by laboratories to detect, prevent, and address analytical errors are discussed, and the increased implementation of such tools offers approaches that can be used to improve laboratory quality.

Retesting Schemes That Improve Test Quality and Yield Using a Test Guardband

The digital integrated circuit (IC) testing model module is applied in this study to simulate the fabrication and testing of integrated circuits. The yield and quality of ICs are analyzed by assuming that the wafer devices under test conditions are normal probability distributions. The difficulties of testing and verification become increasingly great as the design function of the chip becomes remarkably complex. Conversely, the automotive industry chip supply chain has been substantially affected since the COVID-19 outbreak. The shortage of chips in the auto-market has always existed; therefore, increasing available chips under a limited production capacity has become a top priority. Therefore, this study applies the digital integrated circuit testing model (DITM) and proposes a retest plan. This method does not require considerable time to collect large wafer data, nor does it require additional hardware equipment. Furthermore, the required test quality parameters are set, and the test is repeated on the device by adjusting the test guardband (TGB). Moreover, three retesting schemes are proposed to improve the IC test quality (Yq) and test yield (Yt) to meet the requirements of consumers for product quality. A set of 2021 IEEE International Roadmap for Devices and Systems (IRDS) parameters is used to demonstrate the three proposed retesting schemes. The simulation results from the 2021 IRDS data prove that the retest method can effectively improve the test yield (Yt). A comparison of the estimated results of the three retest methods shows that using the repeat test method can maximize the test yield without sacrificing the test quality (Yq). By contrast, repeat testing can indeed improve the test yield (Yt) by 14% or more. Moreover, the increase in sellable ICs not only increases additional earnings for corporations, but also alleviates the current global shortage of automotive ICs.

A Decade of Intelligent Software Testing Research: A Bibliometric Analysis

It gets harder and harder to guarantee the quality of software systems due to their increasing complexity and fast development. Because it helps spot errors and gaps during the first phases of software development, software testing is one of the most crucial stages of software engineering. Software testing used to be done manually, which is a time-consuming, imprecise procedure that comes with errors and gaps and costs money, time, and effort. Currently, testing professionals routinely automate testing to obtain trustworthy results while saving time, cost, and labor. We’ve also moved the starting point of the software cycle to the developer, and made write tests before even writing code, or what’s known as TDD (Test Driven Development). The use of new artificial intelligence techniques will enable the generation of smart test cases to improve test quality and provide better coverage and accurate results. In this study, we used the Web of Science database to acquire bibliometric data about intelligent software testing papers which were conducted between 2012 and 2022, and we used Biblioshiny from the R bibliomerix package, alongside with VOSViewer in order to analyze the data and extract insights and answer research questions about the authors, articles, journals, organizations, and countries publishing in the field of intelligent software testing. The focus of this study is on scientific progress and collaborative trends in scholarly research, providing a blueprint for showcasing worldwide developments in the realm of intelligent software testing. By gaining a comprehensive understanding of the present state of research on the application of artificial intelligence in software testing, this study can offer valuable insights to software engineers, architects, and researchers in the field.

Improving mathematics assessment readability: Do large language models help?

AbstractBackgroundReadability metrics provide us with an objective and efficient way to assess the quality of educational texts. We can use the readability measures for finding assessment items that are difficult to read for a given grade level. Hard‐to‐read math word problems can put some students at a disadvantage if they are behind in their literacy learning. Despite their math abilities, these students can perform poorly on difficult‐to‐read word problems because of their poor reading skills. Less readable math tests can create equity issues for students who are relatively new to the language of assessment. Less readable test items can also affect the assessment's construct validity by partially measuring reading comprehension.ObjectivesThis study shows how large language models help us improve the readability of math assessment items.MethodsWe analysed 250 test items from grades 3 to 5 of EngageNY, an open‐source curriculum. We used the GPT‐3 AI system to simplify the text of these math word problems. We used text prompts and the few‐shot learning method for the simplification task.Results and ConclusionsOn average, GPT‐3 AI produced output passages that showed improvements in readability metrics, but the outputs had a large amount of noise and were often unrelated to the input. We used thresholds over text similarity metrics and changes in readability measures to filter out the noise. We found meaningful simplifications that can be given to item authors as suggestions for improvement.TakeawaysGPT‐3 AI is capable of simplifying hard‐to‐read math word problems. The model generates noisy simplifications using text prompts or few‐shot learning methods. The noise can be filtered using text similarity and readability measures. The meaningful simplifications AI produces are sound but not ready to be used as a direct replacement for the original items. To improve test quality, simplifications can be suggested to item authors at the time of digital question authoring.

An attention-based deep convolutional neural network for classification and grading of interferents in serum specimens

Accurate diagnosis depends on the provision of high-quality and efficient laboratory testing in which the serum test is one of the most important tools. However, for various reasons, interferents in serum specimens exist, which affect the accuracy of biochemical tests. Hemolysis, icterus, and lipemia (HIL) are the three most frequent interferents in serum samples. Determining their existence and degree of interference in serum samples before testing is essential to improve test quality. In this study, a deep learning model for classifying and grading HIL interferences is designed under the assumption that the serum color images contain information on the category and degree of interference. Because the major features of the classification and grading should be obtained from the liquid region of the serum, an auxiliary segmentation task was designed to assist the main classification task. The segmentation result provided the position information of the serum region for the main classification task as a spatial attention mechanism, which helped the network to focus on learning the significant features selectively with a channel attention mechanism. The method unified the two-stage task of 3-class classification and 5-degree grading of interferents into one model. The accuracy of the 15-class classification was 98.62%, and the degree grading accuracy of the three serum interferents (HIL) reached 98.74, 98.52, and 98.54%, respectively. This research provides a new approach to realize the automatic classification and grading of HIL interferents in serum specimens.

Verbesserungsmöglichkeiten von nichtinvasiven Pränataltests und der Beitrag künstlicher Intelligenz

BackgroundNoninvasive prenatal tests (NIPT) are undergoing a rapid development. The reimbursement since 1 July 2022 is contributing to an increased interest in the topic of NIPT.ObjectiveWhat is the principle of the mode of operation of NIPT? What are the starting points to improve test quality, especially by the use of artificial intelligence (AI)-based methods?Material and methodsAn evaluation of the current literature and patents was carried out as well as discussions with manufacturers of NIPTs.ResultsIn this study four major areas could be identified where improvement of the validity of NIPTs is possible. While some techniques are still in the experimental stage or have not yet progressed beyond initial studies, others have already entered routine use in the form of commercially available tests. These AI-based methods are implemented in the most promising developments.ConclusionOngoing developments lead to the fact that not all NIPTs are equally suitable for every question. Clinical trials are needed to be able to assess the progress achieved by the many AI-based innovations in clinical practice.

A Technical Survey on Delay Defects in Nanoscale Digital VLSI Circuits

As technology scales down, digital VLSI circuits are prone to many manufacturing defects. These defects may result in functional and delay-related circuit failures. The number of test escapes grows when technology is downscaled. Small delay defects (SDDs) and hidden delay defects (HDDs) are of critical importance in industries today since they are the source of most test escapes and reliability problems. Improving test quality and creating new test methods, algorithms, and test designs requires a comprehensive study of these delay defects. This article reviews the effect and impact of SDD and HDD in logic circuits. It also analyzes the relevant fault models, automatic test pattern generation (ATPG) methods, faster-than-at-speed testing (FAST), cell-aware (CA) based delay tests, test quality metrics, diagnosis of SDDs and HDDs, and commercially available Electronic Design Automation (EDA) tools. Based on the analysis, the benefits and drawbacks of several accessible approaches are addressed.

Mutation‐inspired symbolic execution for software testing

Software testing is a complex and costly stage during the software development lifecycle. Nowadays, there is a wide variety of solutions to reduce testing costs and improve test quality. Focussing on test case generation, Dynamic Symbolic Execution (DSE) is used to generate tests with good structural coverage. Regarding test suite evaluation, Mutation Testing (MT) assesses the detection capability of the test cases by introducing minor localised changes that resemble real faults. DSE is however known to produce tests that do not have good mutation detection capabilities: in this paper, the authors set out to solve this by combining DSE and MT into a new family of approaches that the authors call Mutation-Inspired Symbolic Execution (MISE). First, this known result on a set of open source programs is confirmed: DSE by itself is not good at killing mutants, detecting only 59.9% out of all mutants. The authors show that a direct combination of DSE and MT (naive MISE) can produce better results, detecting up to 16% more mutants depending on the programme, though at a high computational cost. To reduce these costs, the authors set out a roadmap for more efficient versions of MISE, gaining its advantages while avoiding a large part of its additional costs.

Disease-economy trade-offs under alternative epidemic control strategies

Public policy and academic debates regarding pandemic control strategies note disease-economy trade-offs, often prioritizing one outcome over the other. Using a calibrated, coupled epi-economic model of individual behavior embedded within the broader economy during a novel epidemic, we show that targeted isolation strategies can avert up to 91% of economic losses relative to voluntary isolation strategies. Unlike widely-used blanket lockdowns, economic savings of targeted isolation do not impose additional disease burdens, avoiding disease-economy trade-offs. Targeted isolation achieves this by addressing the fundamental coordination failure between infectious and susceptible individuals that drives the recession. Importantly, we show testing and compliance frictions can erode some of the gains from targeted isolation, but improving test quality unlocks the majority of the benefits of targeted isolation.

Using Enhanced Test Systems Based on Digital IC Test Model for the Improvement of Test Yield

In this work, we use statistical concepts to evaluate the joint probability distribution of manufacturing and test parameters and estimate the future trend of wafer test yield. Owing to the difference between the development speeds of testing technology and manufacturing technology, the testing capability of wafers is far behind the manufacturing capability of the semiconductor. Therefore, with the advancement in technology, the test yield loss caused by the tester inaccuracy has become an important problem. In this article, we propose an enhanced integrated circuit (IC) test scheme (ITS) that uses multiplex testing to improve test quality and test pass rate by retesting, and we rely on the cost evaluation mechanism to obtain the best test and the best profit. Furthermore, the International Roadmap for Devices and Systems (IRDS) 2017 data are used to estimate future test yield trends, and the results prove that the enhanced test scheme (ETS) can effectively estimate the best retest time to obtain the best test yield and the best profit.

Reducing Library Characterization Time for Cell-aware Test while Maintaining Test Quality

Cell-aware test (CAT) explicitly targets faults caused by defects inside library cells to improve test quality, compared with conventional automatic test pattern generation (ATPG) approaches, which target faults only at the boundaries of library cells. The CAT methodology consists of two stages. Stage 1, based on dedicated analog simulation, library characterization per cell identifies which cell-level test pattern detects which cell-internal defect; this detection information is encoded in a defect detection matrix (DDM). In Stage 2, with the DDMs as inputs, cell-aware ATPG generates chip-level test patterns per circuit design that is build up of interconnected instances of library cells. This paper focuses on Stage 1, library characterization, as both test quality and cost are determined by the set of cell-internal defects identified and simulated in the CAT tool flow. With the aim to achieve the best test quality, we first propose an approach to identify a comprehensive set, referred to as full set, of potential open- and short-defect locations based on cell layout. However, the full set of defects can be large even for a single cell, making the time cost of the defect simulation in Stage 1 unaffordable. Subsequently, to reduce the simulation time, we collapse the full set to a compact set of defects which serves as input of the defect simulation. The full set is stored for the diagnosis and failure analysis. With inspecting the simulation results, we propose a method to verify the test quality based on the compact set of defects and, if necessary, to compensate the test quality to the same level as that based on the full set of defects. For 351 combinational library cells in Cadence’s GPDK045 45nm library, we simulate only 5.4% defects from the full set to achieve the same test quality based on the full set of defects. In total, the simulation time, via linear extrapolation per cell, would be reduced by 96.4% compared with the time based on the full set of defects.

Measuring emotion regulation for preservice teacher selection: A theory-driven development of a situational judgment test

Competencies to manage one's own and others' emotions are highly relevant in teacher education but to assess them in the context of a selection personality questionnaires are subject to faking. Situational judgment tests (SJTs) are considered to be less biased by faking, therefore we developed a novel SJT for emotion regulation that is implemented in the admission exam for teacher education. Often SJT development is criticized to be too a theoretical; therefore, we aimed at a theory-driven SJT of interpersonal vs. intrapersonal emotion regulation in pedagogical situations. We used a mixed approach of inductive and deductive item construction to improve test quality (item homogeneity and measurement fairness). The final test comprises 22 items with four response alternatives, each expressing one of four emotion regulation strategies. In two studies, we examined psychometric quality, fairness and validity of the test and relations with cognitive ability and personality. Results support a psychometrically sound and gender-fair measurement according to the 1PL Rasch model. Correlations with tests for emotion regulation, openness, agreeableness and the dark triad were observed. Interpersonal emotion regulation predicted higher altruistic professional motives, whereas intrapersonal emotion regulation predicted higher teacher self-efficacy. Both are (negative) predictors of the intention to quit teacher education.

Use of Quality Models and Indicators for Evaluating Test Quality in an ESP Course

Qualitative methods of assessment play a decisive role in education in general and in language learning in particular. The necessity to perform a qualitative assessment comes from both increased student competition in higher education institutions (HEIs), and hence higher demands for fair assessment, and a growing public awareness on higher education issues, and therefore the need to account for a wider circle of stakeholders, including socie-ty as a whole. The aim of the present paper is to study the regulations and laws pertaining to the issue of assessment in Latvian HEIs, as well as to carry out literature sources analysis about assessment in language testing, seeking to select criteria characterizing the quality of English for Specific Purposes (ESP) tests and to apply the model of evaluating the quality of a language test on an example of a test in sport English, developed in a Latvian higher education institution. An analysis of the regulations and laws about assessment in higher education and li-terature sources about tests in language courses has enabled the development of a test qu-ality model, consisting of seven intrinsic quality criteria: clarity, adequacy, deep approach, attractiveness, originality/similarity, orientation towards student learning result/process, test scoring objectivity/subjectivity. Quality criteria comprise eleven indicators. The relia-bility of the given model is evaluated by means of the whole model, its criteria and indica-tor Cronbach’s alphas and point-biserial (item-total) correlations or discrimination indexes DI. The test was taken by 63 participants, all of them 2nd year full time students attending a Latvian higher education institution.A statistical data analysis was performed with SPSS 17.0. The results show that, altho-ugh test adequacy and clarity is sufficiently high, attractiveness and deep approach should be improved. Also the reliability of one version of the test is higher than that of the other one. One of the ways to improve test quality could be to involve other HEIs in the process of designing tests, because in a small institution it is difficult to collect authentic material for test design and create reliable language tests in a narrow field (in our case: sport English).

Product backlog rating: a case study on measuring test quality in scrum

Agile software development methodologies focus on software projects which are behind schedule or highly likely to have a problematic development phase. In the last decade, Agile methods have transformed from cult techniques to mainstream methodologies. Scrum, an Agile software development method, has been widely adopted due to its adaptive nature. This paper presents a metric that measures the quality of the testing process in a Scrum process. As product quality and process quality correlate, improved test quality can ensure high-quality products. Also, gaining experience from 8 years of successful Scrum implementation at SoftwarePeople, we describe the Scrum process emphasizing the testing process. We propose a metric Product Backlog Rating (PBR) to assess the testing process in Scrum. PBR considers the complexity of the features to be developed in an iteration of Scrum, assesses test ratings and offers a numerical score of the testing process. This metric is able to provide a comprehensive overview of the testing process over the development cycle of a product. We present a case study which shows how the metric is used at SoftwarePeople. The case study explains some features that have been developed in a Sprint in terms of feature complexity and potential test assessment difficulties and shows how PBR is calculated during the Sprint. We propose a test process assessment metric that provides insights into the Scrum testing process. However, the metric needs further evaluation considering associated resources (e.g., quality assurance engineers, the length of the Scrum cycle).

Adapting to Varying Distribution of Unknown Response Bits

Traditionally, test patterns that are generated for a given circuit are applied in an identical manner to all manufactured devices until each device under test either fails or passes each test. With increasing process variations, the statistical diversity of manufactured devices is increasing, making such one-size-fits-all approaches increasingly inefficient. Adaptive test techniques address this problem by tailoring the test decisions for the statistical characteristics of the device under test. In this article, we present several adaptive strategies to enable adaptive unknown bit masking for faster-than-at-speed testing so as to ensure no yield loss while attaining the maximum test quality based on tester memory constraints. We also develop a tester-enabled compression scheme that helps alleviate memory constraints further, shifting the tradeoff space favorably to improve test quality.

The monitoring and analysis results of 15 clinical laboratory quality indicators from 2011 to 2015

Objective To investigate how to improve test quality by monitoring and analyzing 15 clinical laboratory quality indicators from the National Health and Family Planning Commission. Methods Data were collected from clinical laboratory department of the First Affiliated Hospital of Kunming Medical University between January 2011 and August 2015. 15 quality indicators were analyzed retrospectively, including the error rate of specimen type, the coefficient variation unqualified rate of internal quality control test, the reporting rate of critical value, et al. Results The monitoring results of quality indicators basically satisfied the quality goals, except that the median of turn around time in pre-analytical phase was not established, routine internal quality control was not conducted in some laboratory tests in analytical phase and the reporting rate and reporting timely rate of critical value should be further improved in post-analytical phase. Conclusion Medical laboratory quality system can be continuously improved by means of setting up the quality goals of 15 quality indicators referring to sub-specialty and laboratory tests, as well as automated monitoring, statistics and analysis in LIS.(Chin J Lab Med, 2016, 39: 29-33) Key words: Quality indicators, health care; Quality control; Retrospective studies

Item analysis of multiple choice questions of undergraduate pharmacology examinations in an International Medical School in India

Background: Item analysis is widely used to improve test quality by observing the characteristics of a particular item and this can hence be used to ensure that questions are of an appropriate standard for inclusion in a test. Hence, this study to evaluate the multiple choice questions of an undergraduate pharmacology program. Materials and Methods: A total of 488 items were randomly selected and subjected to item analysis. Facility value (FV) and discrimination index (DI) were calculated by applying the appropriate formulae with the help of MS Excel. Results: The overall mean FV (difficulty index) and DI was 56.64% (±2.36) (mean range: 23.89-71.25%) and 0.22 (±0.84) (mean range: 0.16-0.44), respectively. 71.09% of the items analyzed were found to be “good/optimal” items based on the FV (14.13% — optimal, 56.96% — good) and 36.26% of the items analyzed were found be “very/reasonably” good items based on the DI (20.49% — very good, 15.77% — reasonably good). The number of “poor” items was 22.95% based on the FV and 18.23% based on the DI. When both the parameters were considered together, only 23% of the items were found to be “good” and 17.11% were found to be “poor.” Pearson correlation between the two indices showed a negative correlation (but statistically insignificant) between these two indices (r = −0.001379, P= 0.9774). Conclusion: Item analysis when regularly incorporated can help to develop a very useful, valid and a reliable question bank.

DIF investigations across groups of gender and academic background in a large-scale high-stakes language test

High-stakes pre-entry language testing is the predominate tool used to measure test takers’ proficiency for admission purposes in higher education in China. Given the important role of these tests, there are heated discussions about how to ensure test fairness for different groups of test takers. This study examined the fairness of the Graduate School Entrance English Examination (GSEEE) that is used to decide whether over one million test takers can enter master’s programs in China. Using SIBTEST and content analysis, the study investigated differential item functioning (DIF) and the presence of potential bias on the GSEEE with aspects to groups of gender and academic background. Results found that a large percentage of the GSEEE items did not provide reliable results to distinguish good and poor performers. A number of DIF and DBF functioned differentially and three test reviewers identified a myriad of factors such as motivation and learning styles that potentially contributed to group performance differences. However, consistent evidence was not found to suggest these flagged items/texts exhibited bias. While systematic bias may not have been detected, the results revealed poor test reliability and the study highlighted an urgent need to improve test quality and clarify the purpose of the test. DIF issues may be revisited once test quality has been improved.

Design and Implementation of Automatic Test System of Digital Device

Aiming at the digital device test, design a automatic test system, introduced the concept, architecture and technology. The PC sends SMV to digital device based on IEC61850-9-2, Then receive real data from device through the PC serial based on IEC103, constructing a loop, then completed the test. Software static structure adopts the modular programming, dynamic data exchange uses multithread. According to the configured test case, test items can be completed automatically. The system is based on PC, compared with the traditional test instrument to save hardware cost. With the help of calculation and control capabilities of the PC, realize the automatic testing, make people liberation from repetitive work, more focused on the analysis of test results, improve test quality and efficiency.

Examining Timing Path Robustness Under Wide-Bandwidth Power Supply Noise Through Multi-Functional-Cycle Delay Test

Circuits designed and fabricated with nanometer-scale technology are increasingly sensitive to power ground noise across a wide frequency range, thus necessitating a strict examination of circuit robustness against noise during manufacturing tests. Conventional at-speed testing techniques possibly result in the escape of marginal timing failures, as they are unable to account for the impact of middle- and low-frequency noise on circuit timing. To address this challenge, we propose, in this paper, a novel multi-functional-cycle test scheme that targets the noise-induced failures on critical paths of the circuit. The proposed technique explores the noise profile of at-speed functional cycles and approximates it in delay testing through the application of multiple capture operations, thus maximally detecting the timing failures that potentially take place under the worst case functional mode noise. The noise impact of individual devices on the critical paths is characterized through simulations on the power mesh model extracted from the circuit layout. This enables a computationally efficient yet SPICE-accurate estimation of the compound noise profile of the test pattern through the linear superposition of individual ones. Guided by this noise estimation technique, a test pattern transformation flow is proposed to maximize the noise in pseudo-functional test operations. Simulation results show that the proposed scheme can examine the effect of wide-bandwidth noise and thus perform a much more rigorous testing on critical paths than conventional delay testing schemes, thereby significantly improving test quality.