Software Quality Research Articles

How to Measure the Productivity of a Software Quality Engineer

This paper critically examines COBOL85 compiler software quality assurance project from the unintentional perspective of Total Quality. In 1988, when this project occurred, the author was not associated with TQM in any way or had any training in, or knowledge about, Total Quality. The project took place 36 years ago. The company Prime Computer, Inc. no longer exists, and the individuals involved have long since gone their separate ways. An unquestionable comfort can be derived from discussing a project that does not violate confidentiality in any way whatsoever. In 1988, this author was hired as a consultant to manage the software quality assurance effort. The charge was to significantly increase productivity in a relatively short period so that the COBOL85 compiler could be released to Prime’s customers on time. This author was directly responsible for increasing the project's productivity by 40 percent over 14 weeks, while the association with the project lasted a little over a year. Unfortunately, MAI Basic Four attempted a hostile takeover of Prime, approximately three-fourths of the way through the project. Due to financial considerations and because Prime management decided to abandon their mini-computer business entirely, the quality assurance effort finished not with a bang but with a whimper. The author was then relieved of management responsibilities so that the company could save money. The last remaining software quality assurance consultant completed his work, and the COBOL85 compiler finally went to market. Because of the lack of managerial commitment, the compiler never met the expectations of the individuals dedicated to bringing to market a quality product. At the time, the author had no formal or informal training or knowledge of TQM whatsoever but did have an understanding of how a process works. The author was convinced that the process he was hired to oversee could experience dramatic improvements in productivity in a relatively short period, which could be construed as a matter of faith or possibly overconfidence. He achieved what he set out to do.

Live or Let Die: MILSTD 1472 Type Usability Checklists in an Agile Development World

Traditional Human System Integration (HSI) processes are at risk of being eliminated given the industry driven velocity and continuous delivery methods of Agile and Development to Operations Engineering. One such process at risk is usability standard checklists like MIL-STD 1472. The DoD Design Criteria Standard—Human Engineering, or MIL-STD 1472 is a military standard that provides guidelines for the ergonomic design of military systems and equipment. It is primarily focused on ensuring that military systems are designed to optimize human performance, safety, and comfort. However, it can be time-consuming to use given that it contains over 4,400 compliance items in its entirety. In an Agile DevOps environment, the typical delivery of products is 2 weeks with some software products boasting deployments every 11 min. Incorporation of usability and rigorous human factors checklists in the design of products using a DevOps environment is a challenge given the shorter timelines and smaller increments between product development and deployment combined with the expertise and hours needed to execute the checklists. The need for regression testing, the act of re-checking items to make sure new features have not created unforeseen usability issues, exacerbates the problem. Software quality testing for each release demands automatic methods to reduce the efforts of recursive testing. Industry standard methods in agile engineering utilize heuristics for quick turn low-cost usability inspections. This paper describes the derivation of a set of usability heuristics based on MIL-STD 1472 which can be used to identify potential human factors issues more quickly, with less cost, and reduced amount of expertise required. The paper also gives an example of a case study which incorporated automatic checking to support usability experts keeping pace with the high velocity pipelines of software development in an agile DevOps environment.

Enhancing software quality assurance in ubiquitous learning environments through mutation testing and diverse test oracles

Ubiquitous learning enables software analysts to acquire knowledge and design test oracles from online resources including developer documentations, forums and collaborative platforms. Test oracles are primarily designed by humans and may contain quality defects. Mutation testing can facilitate the adaptability of ubiquitous learning to enhance software quality assurance. Unfortunately, mutation testing generates a large number of faulty versions known as mutants to analyze the quality of test oracles which is computationally expensive. In this paper, we proposed minimal path selection strategy to select fewer and non-trivial mutants to analyze diverse test oracles realizing ubiquitous learning environments. The proposed selection strategy explores the relationship between faulty conditions and output statements to identify feasible paths. To establish a ubiquitous learning setup, diverse test oracles are selected by incorporating user input, automated scripts and developer insights. This approach aims to enhance learner engagement and effectively design test oracles. The empirical evaluation and state-of-the-art comparison of various Java-built software programs demonstrated the potential of ubiquitous learning in software quality assurance by reducing 75% of the generated mutants. Furthermore, only 64%–85% of adequate test oracles were required to meet the desired quality assurance criteria. The study highlights the significance of personalized and adaptive approaches to assure software quality in ubiquitous learning environments.

Security Quality Measurement Based on ISO/IEC 25023 Quality Model Case Study: Hospital Management Information System

Hospitals need to protect the security of data assets, where data assets are an important part of the continuity of hospital operations. In connection with the protection of data and information assets in the hospital, it has become a requirement for the hospital information technology team to carry out a security audit of the hospital management information system (HMIS). In this study, we measured the quality of the HMIS software in X Hospital using ISO 25023 which focused on the security aspects of the outpatient service module and drug services in the outpatient pharmacy unit. The security aspect based on the ISO 25023 standard consists of five main characteristics, namely: confidentiality, integrity, non-repudiation, accountability and authenticity. In the early stages, calculations are carried out to find the value of each measurement standard which is denoted by (X). The X value is based on the standard calculation range of values 0 and 1. The threshold value is determined at 0.80 to categorize the point quality whether it is not good or has met the ISO 25023 quality point. The results of software quality measurements show Internal Data Corruption Prevention is worth 0.75 and is at below a predetermined threshold value. Based on these results, it is recommended to improve one of them by replication the database to minimize the possibility of Internal Data Corruption Prevention. In this study, all aspects of software quality have an average value above the threshold, so it can be concluded that HMIS in RS X meets ISO 25023 standards.

Automating TODO-missed Methods Detection and Patching

TODO comments are widely used by developers to remind themselves or others about incomplete tasks. In other words, TODO comments are usually associated with temporary or suboptimal solutions. In practice, all the equivalent suboptimal implementations should be updated (e.g., adding TODOs) simultaneously. However, due to various reasons (e.g., time constraints or carelessness), developers may forget or even are unaware of adding TODO comments to all necessary places, which results in the TODO-missed methods . These “hidden” suboptimal implementations in TODO-missed methods may hurt the software quality and maintainability in the long-term. Therefore, in this paper, we propose the novel task of TODO-missed methods detection and patching, and develop a novel model, namely TDPatcher ( T O D O-comment Patcher ), to automatically patch TODO comments to the TODO-missed methods in software projects. Our model has two main stages: offline learning and online inference. During the offline learning stage, TDPatcher employs the GraphCodeBERT and contrastive learning for encoding the TODO comment (natural language) and its suboptimal implementation (code fragment) into vector representations. For the online inference stage, we can identify the TODO-missed methods and further determine their patching position by leveraging the offline trained model. We built our dataset by collecting TODO-introduced methods from the top-10,000 Python GitHub repositories and evaluated TDPatcher on them. Extensive experimental results show the promising performance of our model over a set of benchmarks. We further conduct an in-the-wild evaluation which successfully detects 26 TODO-missed methods from 50 GitHub repositories.

Hamming distances of unsaturated orthogonal arrays

Orthogonal arrays (OAs) are applied in the statistical design of experiments, coding theory, cryptography, various types of software testing and quality control and have many connections to other combinatorial designs. The Hamming distances of OAs play a critical role in many areas such as algebraic combinatorics, quantum information theory, and wireless sensor networks. However, the calculation method of Hamming distances is difficult, especially for mixed orthogonal arrays (MOAs), since they rely on constructions or structures of OAs in many cases. In this article, by using the number of coincidences, we find some OAs with high strength and MOAs with strength 2 and 3, whose Hamming distances do not depend on their structures. We also calculate Hamming distances of a series of unsaturated asymmetrical OAs via their construction methods such as deleting columns, the expansive replacement method, and difference schemes.

NHPP Software Reliability Model with Rayleigh Fault Detection Rate and Optimal Release Time for Operating Environment Uncertainty

Software is used in diverse settings and depends on development and testing environments. Software development should improve the reliability, quality, cost, and stability of software, making the software testing period crucial. We proposed a software reliability model (SRM) that considers the uncertainty of software environments and the fault detection rate function as a Rayleigh distribution, with an explicit mean value function solution in the model. The goodness-of-fit of the proposed model relative to several existing nonhomogeneous Poisson process (NHPP) SRMs is presented based on three software application failure datasets. Further, a cost model is also presented that addresses the error-removal risk level and required time. The optimal testing release policy for minimizing the expected total cost (ETC) is also determined for NHPP SRMs. The impact of the software environment is studied by varying it, and the optimal release times and minimum ETCs are compared. The goodness-of-fit comparison confirmed that the proposed model has more accurate prediction values than other models. Further, whereas the existing models applied to the cost model do not change after a certain operation period, the proposed model yields changes in release time even for long operating periods.

Catálogo de directrices de sostenibilidad para soportar el desarrollo y operación de software sostenible desde la ingeniería de requisitos

Sustainability constitutes one of the major challenges of today's society, in particular the sustainability of ICTs and more specifically of software. Requirements engineers need guidelines to successfully address sustainability in software projects. With this goal in mind, the CRETS4DevOps method has been designed to manage requirements in DevOps environments while promoting sustainability, especially technical sustainability. CRETS4DevOps encompasses a catalog of sustainability requirements, which helps in the reuse of expert knowledge in order to address the management of software quality attributes that result in sustainability, namely: modularity, scalability, flexibility, reusability, maintainability, reliability, security, interoperability and efficiency. This catalog is intended to assist practitioners both in software development and operation as well as in a sustainability audit. A tool to support CRETS4DevOps is currently being developed as a plug in for Microsoft Azure DevOps.

Examining the Effectiveness of Continuous Integration and Continuous Deployment (CI/CD) Within SDLC Frameworks

Continuous Integration and Continuous Deployment frameworks have been a subject of significant discussion among numerous software development cycles however their impact on Software Development Life Cycles has not been investigated in detail in this research. With survey data collected from the subjects and interviews conducted on the pre-researched parameters of software quality, time-to-market of the product, and interteam collaboration, the study compares the firms that use CI/CD with those that use more conventional software development approaches. The study shows that CI/CD integration enhances software quality since defects are reduced from 1.5 to 0.5 per thousand lines of code. Furthermore, CI/CD teams have 10 releases per month against 3 of traditional and time-to-market that reduces from 6 weeks to 2 weeks. The improvement of collaboration is very visible, the CI/CD teams said that they had more meetings and their job satisfaction scores increased. As with any change, transitioning to CI/CD has potential drawbacks such as resistance to change, but these can be managed especially through training and what has been said in the article specific practices. Therefore, this research offers important insights to organisations aiming at improving their software development processes and preparedness in a relatively dynamic market environment

The Most In-Demand Soft Skills for QA Professionals in Brazil

Background: Software quality assurance (QA) is carried out in tandem with software development. Essentially, QA activities are conducted in hopes of assessing the extent to which a software product aligns with predefined requirements. Although QA includes highly technical tasks, much like software development, it remains largely a human-centered endeavor. Consequently, soft skills can play a significant role in contributing to the success of a project and product quality, as well as boosting the productivity of QA professionals. Aim: Our objective is to investigate: (i) the most sought-after soft skills for QA professionals, (ii) potential correlations among these in-demand soft skills, (iii) variations in soft skill requirements based on the seniority level of positions, and (iv) potential differences in soft skills expectations according to the size of the hiring companies. Our analysis is centered around organizations in Brazil. Method: We looked for soft skills in 2,164 job advertisements from Brazilian companies. Our data extraction process followed an inductive, data-driven approach that included both manual and automated steps. Results: approximately 91% of the job advertisements list at least one soft skill. We identified 32 soft skills in our study, with five soft skills standing out as the most sought-after among them: communication-related skills, planning, innovation, collaboration, and written communication. Notably, companies of various sizes consistently prioritize communication-related and planning skills, deeming them crucial for QA professionals. Conclusions: Our findings underscore the critical importance of five soft skills: communication-related skills, planning, innovation, collaboration, and written communication. The results of our study hold potential value for QA professionals in Brazil seeking to enhance their employability and provide insights to those tasked with staffing, curriculum design, and professional development.

Integrating AI in testing automation: Enhancing test coverage and predictive analysis for improved software quality

This research explores the integration of Artificial Intelligence (AI) in testing automation, focusing on its ability to enhance test coverage and enable predictive analysis for improved software quality. As software systems become increasingly complex, traditional testing methods often struggle to meet quality demands. This study evaluates various AI techniques, including machine learning and natural language processing, and their applications in generating test cases, optimizing testing processes, and predicting defects. Through empirical case studies from diverse organizations, we demonstrate significant improvements in test coverage and defect detection rates following AI implementation. The findings highlight the efficiency gains and quality enhancements achieved through AI-driven testing, while also addressing challenges such as data dependency, complexity of implementation, and the need for skilled personnel. This research contributes to the understanding of AI's role in software testing and encourages organizations to adopt these technologies for better quality assurance and faster development cycles.

Assessment of digital literacy and user experience in patients with cochlear implants

Abstract Introduction The cochlear implant (CI) is the treatment of choice for people affected by deafness or profound hearing loss. Patients with cochlear implants (PwCIs) can enhance their hearing outcomes using digital hearing training programs (dHTPs). Current technological trends leans towards personalized, AI-driven dHTPs, aiming to optimize user experience (UX) and adherence. This study aims to investigate the digital literacy skills and UX of PwCI before and after testing a prototype of a dHTP with an AI-driven design (TraiCI). Methods Two research-specific questionnaires developed by the Hearing Center Düsseldorf (QDL and QUX) were used to assess digital literacy skills and user experience in TraiCI. Additionally, the User Experience Questionnaire (UEQ) was employed to evaluate general user experience. The UEQ assesses software quality across dimensions such as transparency and stimulation using paired opposites. Results 22 outpatient PwCIs the Hearing Center Düsseldorf with unilateral or bilateral CI (mean age 54 years; SD = 15) were included. In QDL, PmCI demonstrated high levels of digital literacy skills. The majority of PmCI (80.6%) reported prior experience with dHTP. Usage of these programs occurred daily to multiple times per week within the first six months after CI treatment, with session durations predominantly ranging from 10 to 30 minutes. QUX data indicated a smooth initiation process (70.8%) and seamless login experience (95.5%). The UEQ addressed various aspects of user experience, with ‘transparency’ receiving the highest scores (MW = 1.9; SD = 0.6), while ‘stimulation’ received the least positive ratings (MW = 0.0; SD = 2.1). Conclusions The results show a high degree of digital literacy among PwCIs. They encountered minimal challenges in operating hardware and software components. Further refinement is necessary to align with PwCI’s expectations and maximize benefits. Early and iterative involvement of PwCIs in future projects could optimize outcomes. Key messages • The study highlights the proficient digital literacy skills of cochlear implant patients (PwCIs). • Further refinement of dHTPs is needed to better align with PwCIs’ expectations and maximize benefits, emphasizing the importance of iterative involvement for optimized outcomes.

From Code to Care: The Impact of Software Engineering and QA on Healthcare Delivery Systems

The rapid digitalization of healthcare has led to an unprecedented reliance on software systems to improve patient care, streamline operations, and ensure regulatory compliance. This article presents a comprehensive analysis of the critical role that software engineering and quality assurance (QA) play in three key areas of modern healthcare: Electronic Health Records (EHRs), telemedicine platforms, and medical device software. Through an examination of current practices, challenges, and case studies, we demonstrate how robust software engineering methodologies and rigorous QA processes contribute to the development of reliable, secure, and efficient healthcare applications. Our findings highlight the importance of data integrity, interoperability, user experience, and regulatory compliance in healthcare software development. Furthermore, we explore emerging trends such as artificial intelligence, blockchain, and the Internet of Medical Things (IoMT), discussing their implications for future software engineering and QA practices in healthcare. This article provides valuable insights for healthcare organizations, software developers, and policymakers, offering recommendations to enhance the quality and safety of software-driven healthcare solutions.

Test Case Automation: Transforming Software Testing in the Digital Era

Purpose: This research aims to explore the limitations of manual software testing, the effectiveness of test case automation in addressing these challenges, and the strategies for successful implementation of automation. It also examines the broader impact of adopting test automation on the quality, speed, and efficiency of software delivery. Methodology: The study investigates the challenges of manual testing in agile and DevOps environments and explores the role of automation as a solution. By analyzing various automation tools, frameworks, and best practices, the research provides insights into successful implementation strategies for test case automation. Additionally, it reviews the associated benefits and potential challenges organizations may face when transitioning to automated testing. Findings: Test case automation significantly mitigates the shortcomings of manual testing, particularly in fast-paced development cycles. The findings reveal that automation not only enhances the quality of software by catching defects earlier but also accelerates the testing process, improving overall efficiency. However, the study also identifies challenges, such as the initial investment in tools and skills required for automation, as well as the ongoing need for maintenance of automated test cases. Unique Contribution to Theory, Policy and Practice: This research contributes to the theoretical understanding of software testing (Tom Badgett, Corey Sandler & Glenford J. Myers, 2011) by highlighting the essential role of automation in modern software development methodologies like agile and DevOps. It bridges the gap between traditional manual testing and contemporary automated practices, providing a framework for understanding how automation fits within these rapid development paradigms. In practice, the study encourages organizations to adopt test case automation as a critical policy for improving the quality and speed of software delivery. It offers practical insights into the implementation of automation, including tool selection, training, and process integration, which can guide companies in transitioning to automated testing. Additionally, it highlights the long-term benefits of automation, making it a valuable policy consideration for organizations aiming to optimize their development and testing processes.

Construction of the Curriculum System of Software Engineering Based on the Agile Development Method

Under the background of “new engineering” construction, software engineering teaching pays more attention to cultivating students’ engineering practice and innovation ability. In view of the inconsistency between development and demand design, team division of labor, difficult measurement of individual contribution, single assessment method, and other problems in traditional practice teaching, this paper proposes that under the guidance of agile development methods, software engineering courses should adopt Scrum framework to organize course project practice, use agile collaboration platform to implement individual work, follow up experiment progress, and ensure effective project advancement. The statistical data of curriculum “diversity” assessment show that there is an obvious improvement effect on students’ software engineering ability and quality.

Improving bioinformatics software quality through teamwork.

Since high-throughput techniques became a staple in biological science laboratories, computational algorithms, and scientific software have boomed. However, the development of bioinformatics software usually lacks software development quality standards. The resulting software code is hard to test, reuse, and maintain. We believe that the root of inefficiency in implementing the best software development practices in academic settings is the individualistic approach, which has traditionally been the norm for recognizing scientific achievements and, by extension, for developing specialized software. Software development is a collective effort in most software-heavy endeavors. Indeed, the literature suggests teamwork directly impacts code quality through knowledge sharing, collective software development, and established coding standards. In our computational biology research groups, we sustainably involve all group members in learning, sharing, and discussing software development while maintaining the personal ownership of research projects and related software products. We found that group members involved in this endeavor improved their coding skills, became more efficient bioinformaticians, and obtained detailed knowledge about their peers' work, triggering new collaborative projects. We strongly advocate for improving software development culture within bioinformatics through collective effort in computational biology groups or institutes with three or more bioinformaticians.

Exploring the role of project status information in effective code smell detection

Repairing code smells detected in the code or design of the system is one of the activities contributing to increasing the software quality. In this study, we investigate the impact of non-numerical information of software, such as project status information combined with machine learning techniques, on improving code smell detection. For this purpose, we constructed a dataset consisting of 22 systems with various project statuses, 12,040 classes, and 18 features that included 1935 large classes. A set of experiments was conducted with ten different machine learning techniques by dividing the dataset into training, validation, and testing sets to detect the large class code smell. Feature selection and data balancing techniques have been applied. The classifier’s performance was evaluated using six indicators: precision, recall, F-measure, MCC, ROC area, and Kappa tests. The preliminary experimental results reveal that feature selection and data balancing have poor influence on the accuracy of machine learning classifiers. Moreover, they vary their behavior when utilized in sets with different values for the selected project status information of their classes. The average value of classifiers performance when fed with status information is better than without. The Random Forest achieved the best behavior according to all performance indicators (100%) with status information, while AdaBoostM1 and SMO achieved the worst in most of them (> 86%). According to the findings of this study, providing machine learning techniques with project status information about the classes to be analyzed can improve the results of large class detection.

Model-driven development for functional correctness of avionics systems: a verification framework for SysML specifications

AbstractCurrently, the most widespread software quality assurance methods in the avionics domain are semi-automated reviews and testing. However, their effort grows disproportionately to the size of the system under development. Also, these methods cannot achieve exhaustive coverage due to the complexity of today’s avionics systems and their potentially infinite set of combinations of possible inputs and system states. Furthermore, the later software issues are detected in the development process, the more expensive it is to fix them. To overcome these issues, a model-driven verification approach for modeling and analyzing avionics systems in early phases of the development is presented. To this end, semantics is given to SysML v2 models by a mapping to a theorem prover encoding. The development of a dedicated SysML v2 profile supporting event-driven data flow specifications, the encoding of corresponding structures in the theorem prover Isabelle, and a generator creating theorems from SysML v2 models are presented. The approach is evaluated by formally proving a representative liveness property of a hierarchical system model from the avionics domain. Since liveness properties can be negated only by infinite data sequences and thus cannot be covered exhaustively by testing, this case study demonstrates the added value for meeting typical safety requirements in the avionics domain. The results can be transferred from avionics to other domains, as well.

SAM: A Modern System Code for Advanced Non-LWR Safety Analysis

The System Analysis Module (SAM), developed at Argonne National Laboratory and by collaborators at other organizations, is for advanced non–light water reactor safety analysis. SAM aims to provide fast-running, modest-fidelity, whole-plant transient analysis capabilities that are essential for fast-turnaround design scoping and engineering analyses of advanced reactor concepts. To facilitate code development, SAM utilizes the MOOSE object-oriented application framework, its underlying finite element library, and linear and nonlinear solvers to leverage modern advanced software environments and numerical methods. SAM aims to solve tightly coupled physical phenomena, including fission reaction, heat transfer, fluid dynamics, and thermal-mechanical responses in advanced reactor structures, systems, and components with high accuracy and efficiency. This paper gives an overview of the SAM code development, including goals and functional requirements, physical models, current capabilities, verification and validation, software quality assurance, and examples of simulations for advanced nuclear reactor applications.

Continuous Integration and Continuous Deployment (CI/CD) Optimization

The advent of Continuous Integration and Continuous Deployment (CI/CD) has fundamentally altered the landscape of software development, enabling teams to deliver updates with unprecedented speed and reliability. By automating the integration of code changes from multiple developers into a central repository, CI/CD practices ensure that software is continuously tested and deployed. This ongoing cycle not only facilitates quicker release cycles but also enhances collaboration among team members and fosters a culture of shared responsibility for code quality. Despite these advancements, organizations face significant challenges in optimizing their CI/CD pipelines. As software systems grow in complexity, the demand for swift and dependable deployments intensifies. This paper explores various techniques and strategies for optimizing CI/CD processes to minimize deployment times while maintaining system reliability. Key optimization methods discussed include: Parallelization of Build Processes: This technique involves breaking down the build process into smaller, independent tasks that can be executed concurrently. By leveraging distributed computing resources, organizations can significantly reduce build times, allowing for faster iterations and deployments. Dependency Caching: Caching dependencies can drastically decrease build times by reusing previously downloaded components. This approach not only speeds up the build process but also minimizes network load and enhances the overall efficiency of the CI/CD pipeline. Incremental Builds: Unlike full builds that compile the entire codebase, incremental builds focus on compiling only the changes made since the last build. This strategy reduces the amount of work needed for each build, accelerating the overall development process. The paper also delves into advanced rollback mechanisms such as blue-green deployments and canary releases. Blue-green deployments allow teams to maintain two identical production environments, enabling smooth transitions and quick rollbacks in case of issues. Canary deployments, on the other hand, introduce new features to a small subset of users before a full rollout, allowing teams to monitor the impact and catch potential failures early. Additionally, automated rollback mechanisms play a vital role in maintaining system reliability, ensuring that any failed deployments can be reverted swiftly to avoid downtime and user disruption. The impact of automation tools on deployment speed and error reduction is another critical aspect examined in this research. Automation frameworks can streamline various stages of the CI/CD process, from code integration to testing and deployment, minimizing human error and ensuring consistent, repeatable processes. By implementing robust automation strategies, organizations can not only accelerate their deployment cycles but also improve overall software quality.