Software Testing Process Research Articles

Impact of software testing automation on the development cycle

Software development is a complex process, from its beginnings with the collection of requirements, processes are proposed for its development in order to achieve a quality product that manages to satisfy the user. Automation appears in the context of software testing development, a critical point in software development, since the quality of development depends on it. Automating software testing offers benefits in efficiency, resource optimization and product quality, also influencing the satisfaction of the development team. In conclusion, the implementation of tools that automate the software testing process will improve the software development cycle, provided they are implemented appropriately.

Component-Based Test Case Generation and Prioritization Using an Improved Genetic Algorithm

Developing test cases is the most challenging and crucial step in the software testing process. The initial test data must be optimized using a strong optimization technique due to many testing scenarios and poor testing effectiveness. Test prioritization is essential for testing the developed software products in a production line with a restricted budget in terms of time and money. A good understanding of the trade-off between costs (e.g. time and resources needed) and efficiency (e.g. component coverage) is necessary to prioritize test case scenarios for one or more software products. So, this paper proposes an efficient Multi-objective Test Case Generation and Prioritization using an Improved Genetic Algorithm (MTCGP-IGA) in Component-based Software Development (CSD). A random search-based method for creating and prioritizing multi-objective tests has been employed utilizing numerous cost and efficacy criteria. Specifically, the multi-objective optimization comprises maximizing the Prioritized Range of test cases (PR), Pairwise Coverage of Characteristics (PCC), Fault-Finding Capability (FFC), and minimizing Total Implementation Cost (TIC). For this test prioritizing problem, a unique fitness function is constructed with cost-effectiveness metrics. IGA is a robust search technique that exhibits excellent benefits and significant efficacy in resolving challenging issues, including ample space, multiple-peak, stochastic, and universal optimization. Relying on the use of IGA, this paper classifies, computes the objective function, introduces the Nondominated Sorting Genetic Algorithm-II (NSGA-II) method, evaluates each branch’s proximity on the handling route, and arranges the path set to get the best answer. The outcomes demonstrate that the proposed MTCGP-IGA with NSGA-II performed the best than other baseline algorithms in terms of prioritizing the test cases (mean value of 195.2), PCC (mean score of 0.7828), and FFC (mean score of 0.8136).

Software multi-fault localization via Chameleon clustering in parallel

Fault localization have proven to be useful in various software during the process of software testing and debugging. The presence of multiple faults simultaneously may deteriorate the effectiveness of existing fault-localization techniques. One good solution is to generate fault-focused clusters by the same bug into the same cluster in term of clustering analysis the failed execution traces. We first introduce improved Chameleon cluster analysis to locate multi-fault in parallel under a general framework. In order to make use of execution information, feature select strategy of the program execution trace is proposed. the failed program execution traces are divided into fault-focused clusters by Chameleon cluster analysis according to the likely faults that caused them, and each cluster may contain the same fault. Then, their suspiciousness is calculated after the clusters of failed program execution traces are merged with all successful program execution traces. Finally, multiple faults in the program are located simultaneously in parallel debugging mode by the suspicious ranking generated in each merged cluster. Experiments conducted on four real-world subjects shows that our approach can achieve better performance than the compared ones.

Guiding the retraining of convolutional neural networks against adversarial inputs.

When using deep learning models, one of the most critical vulnerabilities is their exposure to adversarial inputs, which can cause wrong decisions (e.g., incorrect classification of an image) with minor perturbations. To address this vulnerability, it becomes necessary to retrain the affected model against adversarial inputs as part of the software testing process. In order to make this process energy efficient, data scientists need support on which are the best guidance metrics for reducing the adversarial inputs to create and use during testing, as well as optimal dataset configurations. We examined six guidance metrics for retraining deep learning models, specifically with convolutional neural network architecture, and three retraining configurations. Our goal is to improve the convolutional neural networks against the attack of adversarial inputs with regard to the accuracy, resource utilization and execution time from the point of view of a data scientist in the context of image classification. We conducted an empirical study using five datasets for image classification. We explore: (a) the accuracy, resource utilization, and execution time of retraining convolutional neural networks with the guidance of six different guidance metrics (neuron coverage, likelihood-based surprise adequacy, distance-based surprise adequacy, DeepGini, softmax entropy and random), (b) the accuracy and resource utilization of retraining convolutional neural networks with three different configurations (one-step adversarial retraining, adversarial retraining and adversarial fine-tuning). We reveal that adversarial retraining from original model weights, and by ordering with uncertainty metrics, gives the best model w.r.t. accuracy, resource utilization, and execution time. Although more studies are necessary, we recommend data scientists use the above configuration and metrics to deal with the vulnerability to adversarial inputs of deep learning models, as they can improve their models against adversarial inputs without using many inputs and without creating numerous adversarial inputs. We also show that dataset size has an important impact on the results.

Azure DevOps The Next Era of Application Lifecycle Management

The software development, testing, and deployment process is a series of actions that are carried out in order to achieve a particular result. highly manual process, resulting in a significant number of code errors and bugs. Development and operations (Support) teams can often be out of sync, slowing software delivery and disappointing business stakeholders. DevOps creates efficiency across all tasks involved in software development, deployment, and maintenance. Connecting development with operations leads to increased visibility, more accurate requirements, improved communication, and faster time to Industry. Azure DevOps supports a joint culture and set of multiple steps and processes that bring together the development team, managers, and contributors to build software quickly. It allows the software industry to improve software at a faster pace than they can with traditional software development approaches. Compared with ADO Server, ADO Services offers the following added benefits: • Easy for server management. • Fastest access to greatest features. • Improved faster connectivity for remote users. • A transition from capital expenditures (like servers) to operational expenditures (subscriptions). DevOps is becoming a software industry standard for multiple businesses. According to a report issued by Capgemini, 61% of businesses either adopted DevOps or planned to do so during 2018. Statistics like this demonstrate that DevOps is a necessary part of your business plan if you expect to respond quickly to the market's demands, improve your business’s time-to-market, and keep your software solutions updated regularly. DevOps creates efficiency across all tasks involved in software development, deployment, and maintenance. Connecting development with operations leads to increased visibility, more accurate requirements, improved communication, and faster time to Industry.

Rancang Bangun Aplikasi Pengujian Usability Berbasis Web

Usability is one factor that influences software design success during the software testing process. Usability testing is used to evaluate whether the software meets user needs or not. Usability testing is usually carried out using a certain questionnaire method, but the distribution and measurement of the results are carried out separately. This makes the researchers spend more time testing the application. Therefore, this study provides a solution by designing and building web-based applications to measure the usability of a software design or product with the SUS, UMUX, and UMUX-lite questionnaires. The development method used in this study is Rapid Application Development (RAD) with the stages of the process starting with problem identification, literature study, needs planning, RAD design workshops, implementation, and report writing. The application was built using the Laravel framework and the database using MySQL. The main features of the online questionnaire application were the creation of a questionnaire and the analysis of the results of the questionnaire. The functionality of the application was tested by using the Black Box testing method. Then the developed application was also utilized in usability testing using the SUS, UMUX, and UMUX-lite questionnaires. The results of application usability testing using the SUS, UMUX, and UMUX-Lite methods obtained a final score of 88; 91,28, and 82,05 where all these values mean the application can be accepted and used. Keywords: Usability Testing, Rapid Application Development, SUS, UMUX, UMUX-Lite

A novel test case prioritization approach for black‐box testing based on K‐medoids clustering

AbstractRegression testing is an essential and expensive process in software testing. However, there may be insufficient resources for the execution of all test cases during regression testing. Test case prioritization (TCP) techniques improve the efficiency of regression testing by adjusting the test case execution sequence. Traditional TCP techniques usually rely on the historical execution information of the software under test for more efficient results. String distance‐based TCP (SD‐TCP) avoids these limitations; it uses only the textual difference information of the test cases themselves for prioritization. However, the time overhead on the sorting process of this method is not ideal, and the extreme test case inputs have an impact on the stability of the method. To address these problems, we propose a novel test case prioritization strategy, it first classifies the test cases more finely using the K‐medoids algorithm and then transforms the set into subsequences and improves the early diversity by greedy sorting within clusters. Finally, the test cases are selected through a polling strategy to compose the execution sequence. Extensive experimental results demonstrate that the proposed approach outperforms SD‐TCP in better time efficiency on test case prioritization; it also has a higher average percentage of fault detected (APFD) value than random prioritization (RP) and SD‐TCP.

An Automated Path-Focused Test Case Generation with Dynamic Parameterization Using Adaptive Genetic Algorithm (AGA) for Structural Program Testing

Various software engineering paradigms and real-time projects have proved that software testing is the most critical and highly important phase in the SDLC. In general, software testing takes approximately 40–60% of the total effort and time involved in project development. Generating test cases is the most important process in software testing. There are many techniques involved in the automatic generation of these test cases which aim to find a smaller group of cases that could allow for an adequacy level to be achieved which will hence reduce the effort and cost involved in software testing. In the structural testing of a product, the auto-generation of test cases that are path focused in an efficient manner is a challenging process. These are often considered optimization problems and hence search-based methods such as genetic algorithm (GA) and swarm optimizations have been proposed to handle this issue. The significance of the study is to address the optimization problem of automatic test case generation in search-based software engineering. The proposed methodology aims to close the gap of genetic algorithms acquiring local minimum due to poor diversity. Here, dynamic adjustment of cross-over and mutation rate is achieved by calculating the individual measure of similarity and fitness and searching for the more global optimum. The proposed method is applied and experimented on a benchmark of five industrial projects. The results of the experiments have confirmed the efficiency of generating test cases that have optimum path coverage.

Reviewing Software Testing Models and Optimization Techniques: An Analysis of Efficiency and Advancement Needs

Software testing is a crucial component of software engineering that aims to confirm the intended functionality of software modules and minimize the likelihood of future failures. This paper provides a comprehensive review of various software testing models and optimization techniques available in the literature, emphasizing their performance analysis and related research papers. The paper analyzes and discusses the most commonly used software testing models, including waterfall, incremental, V-model, agile, and spiral models, and identifies several areas for improvement to increase their effectiveness. These areas include using machine learning techniques to automate and optimize testing processes, reducing the number of test cases required, and introducing new metrics to gauge the success of testing. Moreover, the paper suggests developing entirely novel methods to deal with the challenges of contemporary software programs, such as the Internet of Things and artificial intelligence. This paper aims to analyze various software testing models and optimization techniques thoroughly, highlight their advantages and disadvantages, and suggest improvements to increase their efficiency and effectiveness. By continuously improving and optimizing software testing processes, software modules can function as intended, minimizing the likelihood of future failures.

Experimental Teaching Design of Software Testing Process Management

The experimental teaching of software testing course attaches great importance to technical practice. This paper mainly discusses the teaching content design of software testing process management experiment. Through this experimental teaching content, students can grasp all aspects of software testing in general, further understand the software testing activities to be completed in all aspects, and connect the scattered knowledge points of the course to form an organic whole, which is conducive to the further digestion and absorption of the course knowledge.

Exploring the challenges in software testing of the 5G system at Nokia: A survey

The ever-growing size and complexity of industrial software products pose significant quality assurance challenges to engineering researchers and practitioners, despite the constant effort to increase knowledge and improve the processes. 5G technology developed by Nokia is one example of such a grand and highly complex system with improvement potential. The following paper provides an overview of the current quality assurance processes used by Nokia to develop the 5G technology and provides insight into the most prominent challenges by an evaluation of perceived importance, urgency, and difficulty to understand the future opportunities. Nokia mode of operation, briefly introduced in this paper, has been subjected to extensive analysis by a selected group of experienced test-oriented professionals to define the most critical areas of concern. Secondly, the identified problems were evaluated by Nokia gNB system-level test professionals in a dedicated survey. The questionnaire was completed by 312 out of 2935 (10.63%) possible respondents. The challenges are seen as the most important and urgent: customer scenario testing, performance testing, and competence ramp-up. Challenges seen as the most difficult to solve are low occurrence failures, hidden feature dependencies, and hardware configuration-specific problems. Our research identified several improvement areas in the quality assurance processes used to develop the 5G technology by determining the most important and urgent problems that at the same time have a low perceived difficulty. Such initiatives are attractive from a business perspective. On the other hand, challenges seen as the most impactful yet difficult may be of interest to the academic research community. • Our survey, designed to elicit understanding of the most significant challenges in testing large and complex 5G gNB software system, received 312 out of 2935 (10.63%) system-level test practitioners in Nokia. • Challenges seen as having the highest importance are: customer scenario testing, performance testing, and competence ramp-up. • Challenges seen as having the highest urgency are: customer scenario testing, performance testing, and competence ramp-up. • Challenges seen as being the most difficult are: low occurrence failures, hidden feature dependencies, and hardware configuration-specific problems. • Obtained results imply which future improvement opportunities can have the most significant impact on the quality and cost of the software testing processes in the company. • Interestingly, respondents still seem to be not well familiar with software defect prediction models.

A hybrid novel fuzzy AHP-TOPSIS technique for selecting parameter-influencing testing in software development

Software testing is one of the most important phases in the software development life cycle. Software testing cannot ensure successful outcomes until and unless done perfectly. For enhanced testing, it is extremely important to identify and prioritize the parameters that influence the testing process. Prioritization of Parameter-Influencing Testing (PIT) to ensure the testing environment’s reliability has piqued industry practitioners’ interest. A structured framework for the prioritization of PIT will result in the timely and successful execution of the software testing process. A case study for optimal PIT selection and ranking is conducted to address this problem by devising a hybrid technique based on a Fuzzy-Analytical Hierarchy Process (AHP) and Fuzzy-Technique for Order Preference by Similarity to Ideal Solution (TOPSIS). In the proposed framework, the linguistic data obtained from the industry experts are represented as triangular fuzzy numbers used to quantify the expert’s uncertainty in the assessment of PITs with respect to selection criteria. Furthermore, the fuzzy set is integrated into the AHP to determine the selection criteria weights. Finally, a fuzzy TOPSIS is used for ranking PITs. The results reveal that ‘Automated Testing’, ‘Verification/Validation’and ‘Good Design Principles’ are the crucial PITs. The results of this study will aid industry experts in determining the applicability of PITs and demonstrating the efficacy of the proposed framework for the PITs evaluation and ranking.

Are Infinite-Failure NHPP-Based Software Reliability Models Useful?

In the literature, infinite-failure software reliability models (SRMs), such as Musa-Okumoto SRM (1984), have been demonstrated to be effective in quantitatively characterizing software testing processes and assessing software reliability. This paper primarily focuses on the infinite-failure (type-II) non-homogeneous Poisson process (NHPP)-based SRMs and evaluates the performances of these SRMs comprehensively by comparing with the existing finite-failure (type-I) NHPP-based SRMs. In more specific terms, to describe the software fault-detection time distribution, we postulate 11 representative probability distribution functions that can be categorized into the generalized exponential distribution family and the extreme-value distribution family. Then, we compare the goodness-of-fit and predictive performances with the associated 11 type-I and type-II NHPP-based SRMs. In numerical experiments, we analyze software fault-count data, collected from 16 actual development projects, which are commonly known in the software industry as fault-count time-domain data and fault-count time-interval data (group data). The maximum likelihood method is utilized to estimate the model parameters in both NHPP-based SRMs. In a comparison of the type-I with the type-II, it is shown that the type-II NHPP-based SRMs could exhibit better predictive performance than the existing type-I NHPP-based SRMs, especially in the early stage of software testing.

On the Design and Optimization of Test Cases Using an Improved Artificial Bee Colony Algorithm-Based Swarm Intelligence Approach

In this research work, a swarm intelligence-based approach, namely an improved artificial bee colony (IABC), has been proposed to design and optimize the test cases during the software testing process. The novelty of the proposed IABC algorithm is that it has three major improvement heuristics over the general ABC algorithm: (1) it replaces random population generation during the initial phase into a systematic initial solution generation by means of a novel heuristic, namely ‘Chaotic Map'; (2) to eliminate the redundant test cases, another novel heuristic, namely ‘Euclidean Distance', is applied to maintain the diversity of population; (3) to increase the convergence speed, the fitness value of the previous solution is used in the new solution generation. Further, the proposed algorithm has been evaluated with several case studies and compared with the existing works using path coverage-based test adequacy criterion. Hence, the proposed work is improved, and it outperforms the existing works and provides optimal or near optimal test case generation for efficient software testing.

API Message-Driven Regression Testing Framework

With the increase in the number of APIs and interconnected applications, API testing has become a critical part of the software testing process. Particularly considering the business-critical systems using API messages, the importance of repetitive API tests increases. Successfully performing repetitive manual API testing for a large number of test scenarios in large business enterprise applications becomes even more difficult due to the fact that human errors may prevent performing thousands of human-written tests with high precision every time. Furthermore, the existing API test automation tools used in the market cannot be integrated into all business domains due to their dependence on applications. These tools generally support web APIs over the HTTP protocol. Hence, this study is motivated by the fact that there is a lack of API message-driven regression testing frameworks in a particular area in which API messages are used in client-server communication. This study has been prepared to close the gap in a specific domain which uses business domain APIs, rather than HTTP, in client-server communication. We propose a novel approach based on the use of network packets for regression testing. We developed a proof-of-concept test automation tool implementing our approach and evaluated it in a financial domain. Unlike prior studies, our approach can provide the use of real data packets in software testing. The use of network packets increases the generalization of the framework. Overall, our study reports remarkable reuse capacity and makes a significant impact on a real-world business-critical system by reducing effort and increasing the automation level of API regression testing.

Investigation on Java Mutation Testing Tools

Software Testing is one of the most significant phases within the software development life cycle since software bugs can be costly and traumatic. However, the traditional software testing process is not enough on its own as some undiscovered faults might still exist due to the test cases’ inability to detect all underlying faults. Amidst the various proposed techniques of test suites’ efficiency detection comes mutation testing, one of the most effective approaches as declared by many researchers. Nevertheless, there is not enough research on how well the mutation testing tools adhere to the theory of mutation or how well their mutation operators are performing the tasks they were developed for. This research paper presents an investigative study on two different mutation testing tools for Java programming language, namely PIT and µJava. The study aims to point out the weaknesses and strengths of each tool involved through performing mutation testing on four different open-source Java programs to identify the best mutation tool among them. The study aims to further identify and compare the mutation operators of each tool by calculating the mutation score. That is, the operators’ performance is evaluated with the mutation score, with the presumption that the more prominent the number of killed mutants is, the higher the mutation score, thus the more effective the mutation operator and the affiliated tool.Â

The Importance of Software Testing Processes and Methods in the Banking Sector

Günümüzde ekonomik büyümeyi finanse etmenin ve bunu sürekli hale getirmenin en etkili yolu finans sektörünün güçlü ve sağlıklı bir yapıya sahip olmasıdır. Tüketim, ticaret, yapım ve üretimin finanse edilmesinde, kurumlar ve bireyler için kredinin bulunabilirliği ve bunun devamlılığının sürdürülmesi hayati öneme sahiptir. Günümüzde finansal kurumların temel işlevi müşterilerinin ve ekonominin talep ettiği finansal hizmetleri sunmaktır. Sunduğu hizmetlerle mobil ödeme sisteminin doğru çalışmasını sağlamak olan finansal kurumlar ekonomik faaliyetin önemli bir parçasıdır. Bu kritik alanda kullanılan yazılım ürünlerinin kalitesi çok önemlidir. Yazılım ürünlerindeki hatalar maddi, manevi zararların oluşmasına, müşterilerde kuruma karşı güven ve sadakat olgularının yitirilmesine neden olarak finansal kurumların kimliğini zedeler. Günümüzde internetin kullanımının yaygınlaşmasıyla birlikte Mobil Bankacılık gibi alternatif uygulama kanalları müşteriler tarafından sıkça kullanılmaya başlanmıştır. Uygulama kanallarının kullanımının yaygınlaşmasıyla birlikte uygulamalar eleştiriler almaya başlamış ve bu alanda bir rekabet ortamı oluşmuştur Bu rekabet, kurumların müşteriye sundukları yazılım ürünlerindeki hataların en aza indirgenmesi için yeni metotlar arayışına girmelerine sebep olmuştur. Yazılım ürünleri için test süreci, yazılım yaşam döngüsünün vazgeçilmez bir parçası haline gelmiştir. Makalede, yazılım test süreci ve yöntemleri anlatılmıştır.

Statistical Quality Control on Weight of Empty Lipton Tea Carton at Nampak Carton Nigeria Limited

Quality control process assures that the product being developed is of the required quality which includes inspection, deliverable peer reviews and software testing process. Quality control helps any company that manufactures a product or provides a service to improve customer satisfaction by consistently delivering quality products or services, reducing wastage of resources and increasing efficiency and profits for the company. This research work deals with the statistical quality control on the weight of empty Lipton tea at Nampak carton Nigeria Limited Oyo State, within six working days (Monday - Saturday). The data used in this research is secondary data, it was collected from the Nampak database and the method of data analysis used in the research is a control chart and test of randomness, in order to monitor the process and effectiveness of the data. After using various tools like an x-bar chart, R-chart, and Test of randomness, it was discovered that the weight of the product is within control limit both at the upper and lower control limit.

Choosing a continuous integration tool for software automated testing

Problem. Automated testing is directlyrelated to continuous integration (CI), which is the performance of frequent automated grouping of project components to identify and resolve integration issues. Choosing the adequate CI tool for automated testing is the key to effective CI process. Analysis of theory and practice shows that the choice of such an instrument to support testing remains a challenge. According to the sources, finding and combining compatible tools to create a functional pipeline is also a difficult task. The problem that arises due to the combination of tools is their configuration. The said above actualizes the search of the solution to the problem of choosing the appropriate CI tool for software testing by building a model for evaluating these tools. Goal. The aim of the work is to create a model for evaluating CI tools for the software testing process using the decision support system (DSS). Methodology. The analytical methods of research, the method of the hierarchy analysis, and the method of modeling are used. Results. In the course of the work, the possibilities of the main CI tools for automated testing were analyzed and the criteria for their evaluation were determined. Originality. Based on the defined criteria, a model for evaluating CI tools using DSS was developed and used to select the appropriate instrument. Practical value. The tool for continuous integration recommended by DSS, CircleCI, has been implemented at the enterprise to test a real software product. The results of using the evaluation model are obtained and analyzed.

An Empirical Investigation to Understand the Issues of Distributed Software Testing amid COVID-19 Pandemic

Generally, software developers make errors during the distributed software development process; therefore, software testing delay is a significant concern. Some of the software mistakes are minor, but others may be costly or harmful. Since things can still go wrong—individuals encounter mistakes from time to time—there is a need to double-check any software we develop in a distributed environment. The current global pandemic, COVID-19, has exacerbated and generated new challenges for IT organizations. Many issues exist for distributed software testing that prevent the achievement of successful and timely risk reduction when several of the mechanisms on which testing is based are disrupted. The environment surrounding COVID-19 is quickly evolving on a daily basis. Moreover, the pandemic has exposed or helped to develop flaws in production systems, which obstruct software test completion. Although some of these issues were urgent and needed to be evaluated early during the distributed software development process, this paper attempts to capture the details that represent the current pandemic reality in the software testing process. We used a Fuzzy TOPSIS-based multiple-criteria decision-making approach to evaluate the distributed software testing challenges. The statistical findings show that data insecurity is the biggest challenge for successful distributed software testing.