Software Quality Assurance Research Articles

Tuning a secondary dose verification software for a CT-guided online adaptive delivery system.

Quality assurance (QA) remains unstandardized for CT-guided online adaptive radiotherapy (CTgART) platforms (Ethos, Varian Medical Systems, Inc., Palo Alto, CA), as they become more clinically prevalent. A secondary dose calculation software (Mobius3D, Varian Medical Systems, Inc., Palo Alto, CA) is provided for this closed CTgART system. However, the clinical impact of tuning dosimetric leaf gap (DLG) correction values for specific delivery techniques for CTgART secondary dose calculations remains uninvestigated. Tuning the DLG correction value for different delivery techniques of the independent secondary dose verification software for adaptive online QA. A total of 31 volumetric arc therapy (VMAT) and 13 fixed-gantry intensity modulated radiation therapy (IMRT) plans were selected from representative anatomical sites treated in our clinic. All plans were evaluated on a patient CT dataset and a QA dataset of a solid water phantom with an embedded ion chamber placed at the center in both primary treatment planning systems (TPS) and secondary dose verification software. Primary TPS plan doses were compared with secondary calculation doses on patient CT by calculating 3D gamma passing criteria under different DLG correction values ranging from -2 to 2mm to determine the optimal DLG correction range. Ion chamber verification measurements were then compared to secondary calculation dose to determine the optimal DLG correction value by minimizing the difference for IMRT and VMAT plans, separately. The optimal DLG correction values for VMAT and IMRT techniques were -0.3 and 0.4mm respectively for the selected clinical cohort of patients. The mean gamma passing rate between primary and secondary doses for VMAT and IMRT were 99.0%±1.0% and 99.9%±0.1% with 3%/2mm and excluding 10% low dose criteria. The mean ion chamber agreement for VMAT and IMRT were 0.0%±2.1% and 0.2%±1.4%. DLG correction value should be tuned for each delivery technique (VMAT and IMRT) separately to maximize the robustness of CTgART online QA programs.

Data Analytics Tool (DAT): A Data-Driven Decision- Making Tool for Business and Data Analytics Courses

Data Analytics Tool (DAT) is an application that collects and analyzes data in order to improve business processes and aid in discovering hidden trends and patterns thereby uncovering insights to make future data- driven and intelligent decisions. This project, which is aligned with the business and data analytic course contents of the University of the East, Caloocan, is intended for particular users, such as the faculty members and students enrolled in these courses, so as to uniformly use the application in simulating business analytics cases with the given data set in their respective classes. The application development was based on the principles of lean methodology since the model eliminates waste and improves efficiency for the team, thus maximizing the time for development. Various technologies in analyzing data were incorporated in the application such as data mining techniques, statistical measures, forecasting algorithms and decision analysis methods. These technologies were implemented using Visual Studio and was coded using C# programming language. DAT was evaluated for its effectiveness and efficiency as a data analytics tool by several faculty members and students in the classes of business and analytic courses using the ISO/IEC 20510 software quality assurance standard with the following criteria: functional suitability, performance efficiency, compatibility, usability, and reliability. The general weighted mean of 3.96 and 3.49 were gained from the faculty members and students respectively. Hence, the respondents “strongly agree” with the applicability and acceptability of the proposed application as a tool that can be used in analytic courses.

How to Measure the Productivity of a Software Quality Engineer

This paper critically examines COBOL85 compiler software quality assurance project from the unintentional perspective of Total Quality. In 1988, when this project occurred, the author was not associated with TQM in any way or had any training in, or knowledge about, Total Quality. The project took place 36 years ago. The company Prime Computer, Inc. no longer exists, and the individuals involved have long since gone their separate ways. An unquestionable comfort can be derived from discussing a project that does not violate confidentiality in any way whatsoever. In 1988, this author was hired as a consultant to manage the software quality assurance effort. The charge was to significantly increase productivity in a relatively short period so that the COBOL85 compiler could be released to Prime’s customers on time. This author was directly responsible for increasing the project's productivity by 40 percent over 14 weeks, while the association with the project lasted a little over a year. Unfortunately, MAI Basic Four attempted a hostile takeover of Prime, approximately three-fourths of the way through the project. Due to financial considerations and because Prime management decided to abandon their mini-computer business entirely, the quality assurance effort finished not with a bang but with a whimper. The author was then relieved of management responsibilities so that the company could save money. The last remaining software quality assurance consultant completed his work, and the COBOL85 compiler finally went to market. Because of the lack of managerial commitment, the compiler never met the expectations of the individuals dedicated to bringing to market a quality product. At the time, the author had no formal or informal training or knowledge of TQM whatsoever but did have an understanding of how a process works. The author was convinced that the process he was hired to oversee could experience dramatic improvements in productivity in a relatively short period, which could be construed as a matter of faith or possibly overconfidence. He achieved what he set out to do.

Enhancing software quality assurance in ubiquitous learning environments through mutation testing and diverse test oracles

Ubiquitous learning enables software analysts to acquire knowledge and design test oracles from online resources including developer documentations, forums and collaborative platforms. Test oracles are primarily designed by humans and may contain quality defects. Mutation testing can facilitate the adaptability of ubiquitous learning to enhance software quality assurance. Unfortunately, mutation testing generates a large number of faulty versions known as mutants to analyze the quality of test oracles which is computationally expensive. In this paper, we proposed minimal path selection strategy to select fewer and non-trivial mutants to analyze diverse test oracles realizing ubiquitous learning environments. The proposed selection strategy explores the relationship between faulty conditions and output statements to identify feasible paths. To establish a ubiquitous learning setup, diverse test oracles are selected by incorporating user input, automated scripts and developer insights. This approach aims to enhance learner engagement and effectively design test oracles. The empirical evaluation and state-of-the-art comparison of various Java-built software programs demonstrated the potential of ubiquitous learning in software quality assurance by reducing 75% of the generated mutants. Furthermore, only 64%–85% of adequate test oracles were required to meet the desired quality assurance criteria. The study highlights the significance of personalized and adaptive approaches to assure software quality in ubiquitous learning environments.

The Most In-Demand Soft Skills for QA Professionals in Brazil

Background: Software quality assurance (QA) is carried out in tandem with software development. Essentially, QA activities are conducted in hopes of assessing the extent to which a software product aligns with predefined requirements. Although QA includes highly technical tasks, much like software development, it remains largely a human-centered endeavor. Consequently, soft skills can play a significant role in contributing to the success of a project and product quality, as well as boosting the productivity of QA professionals. Aim: Our objective is to investigate: (i) the most sought-after soft skills for QA professionals, (ii) potential correlations among these in-demand soft skills, (iii) variations in soft skill requirements based on the seniority level of positions, and (iv) potential differences in soft skills expectations according to the size of the hiring companies. Our analysis is centered around organizations in Brazil. Method: We looked for soft skills in 2,164 job advertisements from Brazilian companies. Our data extraction process followed an inductive, data-driven approach that included both manual and automated steps. Results: approximately 91% of the job advertisements list at least one soft skill. We identified 32 soft skills in our study, with five soft skills standing out as the most sought-after among them: communication-related skills, planning, innovation, collaboration, and written communication. Notably, companies of various sizes consistently prioritize communication-related and planning skills, deeming them crucial for QA professionals. Conclusions: Our findings underscore the critical importance of five soft skills: communication-related skills, planning, innovation, collaboration, and written communication. The results of our study hold potential value for QA professionals in Brazil seeking to enhance their employability and provide insights to those tasked with staffing, curriculum design, and professional development.

From Code to Care: The Impact of Software Engineering and QA on Healthcare Delivery Systems

The rapid digitalization of healthcare has led to an unprecedented reliance on software systems to improve patient care, streamline operations, and ensure regulatory compliance. This article presents a comprehensive analysis of the critical role that software engineering and quality assurance (QA) play in three key areas of modern healthcare: Electronic Health Records (EHRs), telemedicine platforms, and medical device software. Through an examination of current practices, challenges, and case studies, we demonstrate how robust software engineering methodologies and rigorous QA processes contribute to the development of reliable, secure, and efficient healthcare applications. Our findings highlight the importance of data integrity, interoperability, user experience, and regulatory compliance in healthcare software development. Furthermore, we explore emerging trends such as artificial intelligence, blockchain, and the Internet of Medical Things (IoMT), discussing their implications for future software engineering and QA practices in healthcare. This article provides valuable insights for healthcare organizations, software developers, and policymakers, offering recommendations to enhance the quality and safety of software-driven healthcare solutions.

Model-driven development for functional correctness of avionics systems: a verification framework for SysML specifications

AbstractCurrently, the most widespread software quality assurance methods in the avionics domain are semi-automated reviews and testing. However, their effort grows disproportionately to the size of the system under development. Also, these methods cannot achieve exhaustive coverage due to the complexity of today’s avionics systems and their potentially infinite set of combinations of possible inputs and system states. Furthermore, the later software issues are detected in the development process, the more expensive it is to fix them. To overcome these issues, a model-driven verification approach for modeling and analyzing avionics systems in early phases of the development is presented. To this end, semantics is given to SysML v2 models by a mapping to a theorem prover encoding. The development of a dedicated SysML v2 profile supporting event-driven data flow specifications, the encoding of corresponding structures in the theorem prover Isabelle, and a generator creating theorems from SysML v2 models are presented. The approach is evaluated by formally proving a representative liveness property of a hierarchical system model from the avionics domain. Since liveness properties can be negated only by infinite data sequences and thus cannot be covered exhaustively by testing, this case study demonstrates the added value for meeting typical safety requirements in the avionics domain. The results can be transferred from avionics to other domains, as well.

SAM: A Modern System Code for Advanced Non-LWR Safety Analysis

The System Analysis Module (SAM), developed at Argonne National Laboratory and by collaborators at other organizations, is for advanced non–light water reactor safety analysis. SAM aims to provide fast-running, modest-fidelity, whole-plant transient analysis capabilities that are essential for fast-turnaround design scoping and engineering analyses of advanced reactor concepts. To facilitate code development, SAM utilizes the MOOSE object-oriented application framework, its underlying finite element library, and linear and nonlinear solvers to leverage modern advanced software environments and numerical methods. SAM aims to solve tightly coupled physical phenomena, including fission reaction, heat transfer, fluid dynamics, and thermal-mechanical responses in advanced reactor structures, systems, and components with high accuracy and efficiency. This paper gives an overview of the SAM code development, including goals and functional requirements, physical models, current capabilities, verification and validation, software quality assurance, and examples of simulations for advanced nuclear reactor applications.

Causal reasoning in Software Quality Assurance: A systematic review

Context:Software Quality Assurance (SQA) is a fundamental part of software engineering to ensure stakeholders that software products work as expected after release in operation. Machine Learning (ML) has proven to be able to boost SQA activities and contribute to the development of quality software systems. In this context, Causal Reasoning is gaining increasing interest as a methodology to go beyond a purely data-driven approach by exploiting the use of causality for more effective SQA strategies. Objective:Provide a broad and detailed overview of the use of causal reasoning for SQA activities, in order to support researchers to access this research field, identifying room for application, main challenges and research opportunities. Methods:A systematic review of the scientific literature on causal reasoning for SQA. The study has found, classified, and analyzed 86 articles, according to established guidelines for software engineering secondary studies. Results:Results highlight the primary areas within SQA where causal reasoning has been applied, the predominant methodologies used, and the level of maturity of the proposed solutions. Fault localization is the activity where causal reasoning is more exploited, especially in the web services/microservices domain, but other tasks like testing are rapidly gaining popularity. Both causal inference and causal discovery are exploited, with the Pearl’s graphical formulation of causality being preferred, likely due to its intuitiveness. Tools to favor their application are appearing at a fast pace — most of them after 2021. Conclusions:The findings show that causal reasoning is a valuable means for SQA tasks with respect to multiple quality attributes, especially during V&V, evolution and maintenance to ensure reliability, while it is not yet fully exploited for phases like requirements engineering and design. We give a picture of the current landscape, pointing out exciting possibilities for future research.

Software Reliability Prediction by Adaptive Gated Recurrent Unit‐Based Encoder‐Decoder Model With Ensemble Empirical Mode Decomposition

ABSTRACTAccurate software reliability prediction is significant to software quality assurance. However, the rapid development and evolution of modern software pose more challenges for accurate software quality assessment. With the rapid development of machine learning and intelligent algorithms, data‐driven nonparametric models have gradually attracted increasing attention with outstanding prediction performance. However, we found that there may exist some prediction lags caused by the autocorrelation and nonstationarity of software fault data in the prediction of nonparametric models affecting their performance. To address this problem, we proposed an adaptive gated recurrent unit‐based encoder‐decoder model (ED‐GRU) with ensemble empirical mode decomposition (EEMD), effectively reducing prediction lags and performing accurate software fault number prediction. The autocorrelation and nonstationarity of fault data are first reduced by using first‐order difference and EEMD to clearly characterize the changing trend of the data. The frequency‐specific ED‐GRU networks are then combined to adaptively learn the nonlinear fluctuation trend of fault data under different frequency scales and obtain accurate prediction final results after aggregation. Experiments on eight public datasets showed that the proposed EEMD‐ED‐GRU‐PF model could effectively reduce the prediction lags and achieve the best prediction performance compared with four nonparametric and five parametric baseline methods in all datasets. Therefore, the proposed method can effectively and stably reduce the prediction lag to significantly improve the prediction accuracy. In this way, developers can accurately evaluate the current quality of the software and provide valuable guidance for software development and maintenance.

Ranking guidance actions to support engineers in fulfilling process constraints

AbstractIn safety‐critical systems engineering, regulations such as Automotive SPICE, ISO26262, or ED‐109A mandate software quality assurance measures to provide evidence that the developed system is high quality. The constraints that define quality assurance conditions during the engineering life cycle are often non‐trivial. This paper addresses the challenges, engineers face who are unfamiliar with the precise constraints of various projects (e.g., when newly joining a company or switching between departments). Understanding how to fulfill a constraint is a time‐consuming and challenging task as an engineer needs to determine the most suitable option (out of potentially many) to fulfill a constraint violation. To this end, we propose a guidance action ranking framework to provide engineers with the most relevant guidance actions. Our primary ranking algorithm analyzes in the background the actions that engineers have made in the past to resolve a constraint violation without requiring explicit feedback from them. We evaluated our framework on two real‐world data sets: an open‐source drone management and an industrial air traffic control software system. Concretely, we replay past engineering activities and measured whether, in the case of a constraint violation, our suggested guidance actions were indeed selected by the engineer. The evaluation results revealed that learning from prior guidance actions effectively identifies the most appropriate guidance actions (ranked top 1 or 2) when compared to ranking algorithms based on action simplicity and artifact property change frequency. Specifically, we achieve a median MRR of 0.95 for the first case study and 0.94 for the second case study: an improvement of 80% and 100% over the baseline. Additionally, we observed that the simplicity of a guidance action does not reliably indicate its suitability for fulfilling a constraint, whereas learning from prior change operation property out‐performed simplicity‐based ranking but did not surpass guidance frequency‐based ranking.

Improving effort-aware just-in-time defect prediction with weighted code churn and multi-objective slime mold algorithm

Effort-aware just-in-time software defect prediction (JIT-SDP) aims to effectively utilize the limited resources of software quality assurance (SQA) to detect more software defects. This improves the efficiency of SQA work and the quality of the software. However, there is disagreement regarding the representation of the key feature variable, SQA effort, in the field of effort-aware JIT-SDP. Additionally, the most recent metaheuristic optimization algorithms (MOAs) have not yet been effectively integrated with multi-objective effort-aware JIT-SDP tasks. These deficiencies, in both feature representation and model optimization (MO), result in a significant disparity between the performance of effort-aware JIT-SDP techniques and the expectations of the industry. In this study, we present a novel method called weighted code churn (CC) and improved multi-objective slime mold algorithm (SMA) (WCMS) for effort-aware JIT-SDP. It comprises two stages: feature improvement (FI) and MO. In the FI phase, we normalize the two feature variables: number of modified files (NF) and distribution of modified code across each file (Entropy). We then use an exponential function to quantify the level of difficulty of the change. The equation is as follows: DD = NFEntropy, where DD is an acronym for the degree of difficulty, NF denotes the base number, and Entropy denotes the index. We define change effort as the product of the difficulty degree in implementing the change and CC, with weighted CC representing the change effort. During the MO stage, we improve the SMA by incorporating multi-objective handling capabilities and devising mechanisms for multi-objective synchronization and conflict resolution. We develop a multi-objective optimization algorithm for hyperparameter optimization (HPO) of the JIT-SDP model in WCMS. To evaluate the performance of our method, we conducted experiments using six well-known open-source projects and employed two effort-aware performance evaluation metrics. We evaluated our method based on three scenarios: cross-validation, time-wise cross-validation, and across-project prediction. The experimental results indicate that the proposed method outperforms the benchmark method. Furthermore, the proposed method demonstrates greater scalability and generalization capabilities.

Enhancing Software Testing Quality Assurance Using Agile Development Environment and Artificial Intelligence With Research Travelogue

This paper presents the analysis of software testing quality assurance using agile development environment and artificial intelligence with research travelogue. It explores the integration of Agile development methodologies with Artificial Intelligence (AI) to elevate the quality assurance (QA) processes in software testing. By leveraging Agile practices, such as iterative development and continuous feedback, alongside AI-driven tools for automation, predictive analytics, and anomaly detection, the approach aims to significantly enhance the accuracy and efficiency of QA processes. The research travelogue documents the journey through various Agile environments, detailing how AI technologies are applied to streamline testing workflows, identify potential issues earlier in the development cycle, and adapt to changing requirements. The findings reveal that this hybrid approach not only accelerates the testing process but also improves defect detection rates and overall software reliability, offering valuable insights for future advancements in software QA practices

An Empirical Study on Code Review Activity Prediction and Its Impact in Practice

During code reviews, an essential step in software quality assurance, reviewers have the difficult task of understanding and evaluating code changes to validate their quality and prevent introducing faults to the codebase. This is a tedious process where the effort needed is highly dependent on the code submitted, as well as the author’s and the reviewer’s experience, leading to median wait times for review feedback of 15-64 hours. Through an initial user study carried with 29 experts, we found that re-ordering the files changed by a patch within the review environment has potential to improve review quality, as more comments are written (+23%), and participants’ file-level hot-spot precision and recall increases to 53% (+13%) and 28% (+8%), respectively, compared to the alphanumeric ordering. Hence, this paper aims to help code reviewers by predicting which files in a submitted patch need to be (1) commented, (2) revised, or (3) are hot-spots (commented or revised). To predict these tasks, we evaluate two different types of text embeddings (i.e., Bag-of-Words and Large Language Models encoding) and review process features (i.e., code size-based and history-based features). Our empirical study on three open-source and two industrial datasets shows that combining the code embedding and review process features leads to better results than the state-of-the-art approach. For all tasks, F1-scores (median of 40-62%) are significantly better than the state-of-the-art (from +1 to +9%).

Cognitive Inherent SLR Enabled Survey for Software Defect Prediction

Introduction:: Any software is created to help automate manual processes most of the time. It is expected from the developed software that it should perform the tasks it is supposed to do. Methods:: More formally, it should work in a deterministic manner. Further, it should be capable of knowing if any provided input is not in the required format. Correctness of the software is inherent virtue that it should possess. Any remaining bug during the development phase would hamper the application's correctness and impact the software's quality assurance. Software defect prediction is the research area that helps the developer to know bug-prone areas of the developed software. Results:: Datasets are used using data mining, machine learning, and deep learning techniques to achieve study. A systematic literature survey is presented for the selected studies of software defect prediction. Conclusion:: Using a grading mechanism, we calculated each study's grade based on its compliance with the research validation question. After every level, we have selected 54 studies to include in this study.

Bug Tracker User Guide

ABSTRACT___ The Bug Tracking Tool project addresses the pressing need for efficient bug tracking and management in contemporary software development. By leveraging modern web technologies and agile methodologies, the project aims to develop a comprehensive solution that streamlines bug identification, tracking, and resolution processes. Through a literature survey, key observations and shortcomings in existing bug tracking tools were identified, guiding the proposed work. The project’s methodology encompasses requirements gathering, design, implementation, testing, and deployment, focusing on user-centric design and functionality. Preliminary results demonstrate progress towards achieving project objectives, highlighting the tool’s potential to enhance software quality assurance practices and user satisfaction. Despite encountering challenges such as technical complexity and resource constraints, the project lays a solid foundation for future enhancements and optimizations. Overall, the Bug Tracking Tool project represents a significant step towards improving bug tracking practices and delivering higher-quality software products to end-users INDEX TEARMS – HTML, CSS , JavaScript, Python, Django Framework, SQLlite.

Data Complexity: A New Perspective for Analyzing the Difficulty of Defect Prediction Tasks

Defect prediction is crucial for software quality assurance and has been extensively researched over recent decades. However, prior studies rarely focus on data complexity in defect prediction tasks, and even less on understanding the difficulties of these tasks from the perspective of data complexity. In this article, we conduct an empirical study to estimate the hardness of over 33,000 instances, employing a set of measures to characterize the inherent difficulty of instances and the characteristics of defect datasets. Our findings indicate that: (1) instance hardness in both classes displays a right-skewed distribution, with the defective class exhibiting a more scattered distribution; (2) class overlap is the primary factor influencing instance hardness and can be characterized through feature, structural, and instance-level overlap; (3) no universal preprocessing technique is applicable to all datasets, and it may not consistently reduce data complexity, fortunately, dataset complexity measures can help identify suitable techniques for specific datasets; (4) integrating data complexity information into the learning process can enhance an algorithm’s learning capacity. In summary, this empirical study highlights the crucial role of data complexity in defect prediction tasks, and provides a novel perspective for advancing research in defect prediction techniques.

A Control Flow Graph Generation Method for Java Projects

Many software quality assurance methods depend on the control flow graph (CFG) for theanalyzing process. However, existing methods for Java projects have not described details of the CFGgeneration process, which is critical for having a fully automatic analysis method. This paper presentsa novel method, named CFG4J, for generating CFG associated with a given Java unit. The generatedCFG will be used in many other software quality assurance methods such as test data generation forunit testing, unit debugging, code coverage computation, etc. We have implemented CFG4J in a tooland performed some initial experiments with some common units with potential results. Finally, wegive some discussions about the experimental results at the end of the paper.Keywords: Control flow graph, unit testing, test data generation, Java projects.

Web Automation of Capital Management Software using Selenium and Java

Modern software development approaches have come to rely heavily on automated testing, which provides many advantages over manual testing in terms of efficiency, dependability, and scalability. This paper provides a thorough analysis of the developments in automated testing methods, emphasizing the most important approaches, instruments, and industry best practices. The paper examines several automated testing methodologies, including functional testing, regression testing, and performance testing, after providing an outline of the basics of automated testing and outlining its advantages and disadvantages. The paper also explores popular automation technologies, including Selenium, and talks about their features, capabilities, and suitability for various testing scenarios. Further- more, the paper examines emerging trends and future directions in automated testing, including the integration of artificial intel- ligence and machine learning techniques. Through this review, the paper aims to provide researchers and practitioners with valuable insights into the evolving landscape of automated testing, enabling them to make informed decisions and advancements in software quality assurance. Index Terms—Automated Testing, Software Quality Assurance, Test Automation Tools, Selenium, Functional Testing, Regression Testing, Performance Testing

Foreign experience of software testing engineers training: analysis of educational standards and programs of American universities

The article aims to explore the U.S. educational system's approach to training software testing specialists, highlighting specific features and methodologies. The study involves analyzing key standards for IT engineer training in the USA, such as the ACM/IEEE Computer Science Curricula, ABET Accreditation Criteria, ISTE Standards for Educators, and CSTA K-12 Computer Science Standards, their overall context and targeting, as well as the inclusion of both fundamental and IT-specific disciplines into their content. In the study of each standard, particular emphasis was placed on defining the place and teaching methods of theory and practice of software testing in the university curriculum guidelines. As it was revealed in the analysis, despite the extremely similar interpretation of basic concepts and practices in the context of software testing, opinions about the methodology of their teaching and the need to study as a separate discipline, differ in a set of reviewed standards. Additionally, the research examines training programs at leading American universities, including MIT, Stanford, and Carnegie Mellon University, to identify how software testing and quality assurance are reflected in their curricula, covering full-time training programs for students in computer science, software engineering and similar related fields. The analysis examined programs in both general and basic disciplines in the context of information technology, and specific to a certain technological or organizational area, such as mobile device development or IT project management. The content of the subjects, the list of modules and lectures, as well as tasks for extracurricular work, available in open sources, was also analyzed and searched for topics related to quality assurance or software testing. As revealed in the study, in most technical universities testing is not covered within a separate discipline, but is taught as a part of ones related to development practices. However, some training programs, in particular the Master's level, still have disciplines, devoted exclusively to quality control, testing management and automation