Software Testing Process Research Articles

Dynamic Software Testing and Evaluation With State Space Method

Abstract Dynamic software testing and evaluation are important software engineering subjects and can be used to improve reliability. In this paper, we propose a dynamic software testing and evaluation model based on the state space method. In this model, the run time features of software are quantified into one state vector; the program scale, difficulties, structures, and development expenses are quantified into one input vector; the total software complexity is quantified into one output vector; and the software testing and evaluation process is regarded as a feedback control process. Dynamic software testing and evaluation with the state space method can adaptively regulate the critical factors that drastically affect software reliability. Experimental results show that the proposed method is useful for reducing costs, shortening developing time, and improving reliability.

Metaheuristic approach for constructing functional test‐suites

Today, software systems are complex and have many possible configurations. A deficient software testing process often leads to unfortunate consequences, including data losses, large economic losses, security breaches, and even bodily harm. Thus, the problem of performing effective and economical testing is a key issue. Combinatorial testing is a method that can reduce cost and increase the effectiveness of software testing for many applications. It is based on constructing economical sized test-suites that provide coverage of the most prevalent configurations. Mixed covering arrays (MCAs) are combinatorial structures that can be used to represent these test-suites. MCAs are combinatorial objects represented as matrices having a test case per row. MCAs are small, in comparison to an exhaustive approach, and guarantee a level of interaction coverage among the parameters involved. This study presents a metaheuristic approach based on a simulated annealing (SA) algorithm for constructing MCAs. This algorithm incorporates several distinguishing features, including an efficient heuristic to generate good quality initial solutions, and a compound neighbourhood function that combines two carefully designed neighbourhood functions. The experimental design involved a benchmark reported in the literature and two real cases of software components. The experimental evidence showed that the SA algorithm equals or improves the obtained results by other approaches reported in the literature, and also finds the optimal solution in some of the solved cases.

Particularities of Verification Processes for Distributed Informatics Applications

This paper presents distributed informatics applications and characteristics of their development cycle. It defines the concept of verification and there are identified the differences from software testing. Particularities of the software testing and software verification processes are described. The verification steps and necessary conditions are presented and there are established influence factors of quality verification. Software optimality verification is analyzed and some metrics are defined for the verification process.Keywords: Distributed Informatics Applications, Software Testing, Software Verification, Verification Process, Software Optimality(ProQuest: ... denotes formulae omitted.)1 Characteristics of Distributed Informatics ApplicationsDistributed informatics applications are software constructions that are based on architectures whose components, through interaction, realize allocations of resources in real time. Distributed informatics applications include:- a heterogeneous group of users, having many elements that, through interaction, solve their well-defined problems, as data entry volume, sequence operations that activates and with concrete results that marks the success of performing the interaction or the need to retake some components from the operations chain specifying the cause and the manner of disposal: after a few replays each user successfully completes the interaction getting the message meaning that his problem has been solved correctly and completely;- dynamic definition of computer network through which is realized the messages transfer of user exempted in the ground indefinitely, the only restrictions being those related to hardware resources that ensure compatibility with data acquisition system and connection performance;- conducting an achievements cycle that includes steps, such as: defining the target group and setting its size; defining the set of distinct problems, which are subject to processing; in this respect for each problem are used appropriate definition tools to reduce the risk of sub-definitions or supra-definitions, situations attracting reversals of prior steps when minuses of information in the case of sub-definitions, or the excess of information, in the case of supra-definition of the problem, generate effects that lead to discontinuation of the development cycle with the impossibility to pass to the next stage; the stage of clear specifications, consistent, the development stage of informatics solutions variants accompanied by performance estimation models requires choosing the suitable variant against the criterion with which the multiplication effects is managed at the moment of implementation; the code elaboration stage as optimal resource allocation process, knowing that the instructions, data structures defining mechanism and building of sequences procedures must be understood as infinite resource use, but which differ from each other in terms of performance criteria of informatics applications, taken as a whole; the testing stage [1] play a very special role for the distributed informatics applications because these operates independently of the developer; the user has limited possibilities to manage uncontrolled situations resulting from existing errors in procedures that allocate resources wrongly or generates random processing behavior, which ultimately creates users discomfort (elaboration, documentation, implementation).Modern distributed informatics applications are investments, so they include:- the investor who pays the development of the distributed informatics application;- staffproviding application development;- staffensuring exploitation management;- users that solves the problem with the distributed informatics application, which becomes service provider;- from the amounts transferred by the users to other destinations, a part returns to the investor, a part to the management of the application and the investment is recovered, and those who ensure the management have their profit. …

Analyzing an automotive testing process with evidence-based software engineering

ContextEvidence-based software engineering (EBSE) provides a process for solving practical problems based on a rigorous research approach. The primary focus so far was on mapping and aggregating evidence through systematic reviews. ObjectivesWe extend existing work on evidence-based software engineering by using the EBSE process in an industrial case to help an organization to improve its automotive testing process. With this we contribute in (1) providing experiences on using evidence based processes to analyze a real world automotive test process and (2) provide evidence of challenges and related solutions for automotive software testing processes. MethodsIn this study we perform an in-depth investigation of an automotive test process using an extended EBSE process including case study research (gain an understanding of practical questions to define a research scope), systematic literature review (identify solutions through systematic literature), and value stream mapping (map out an improved automotive test process based on the current situation and improvement suggestions identified). These are followed by reflections on the EBSE process used. ResultsIn the first step of the EBSE process we identified 10 challenge areas with a total of 26 individual challenges. For 15 out of those 26 challenges our domain specific systematic literature review identified solutions. Based on the input from the challenges and the solutions, we created a value stream map of the current and future process. ConclusionsOverall, we found that the evidence-based process as presented in this study helps in technology transfer of research results to industry, but at the same time some challenges lie ahead (e.g. scoping systematic reviews to focus more on concrete industry problems, and understanding strategies of conducting EBSE with respect to effort and quality of the evidence).

A Discussion of Software Reliability Growth Models with Time-Varying Learning Effects

Over the last few decades, software reliability growth models (SRGM) has been developed to predict software reliability in the testing/debugging phase. Most of the models are based on the Non-Homogeneous Poisson Process (NHPP), and an S or exponential-shaped type of testing behavior is usually assumed. Chiu et al. (2008) provided an SRGM that considers learning effects, which is able to reasonably describe the S and exponential-shaped behaviors simultaneously. This paper considers both linear and exponential-learning effects in an SRGM to enhance the model in Chiu et al. (2008), assumes the learning effects depend on the testing-time, and discusses when and what learning effects would occur in the software development process. This research also verifies the effectiveness of the proposed models with R square (Rsq), and compares the results with these of other models by using four real datasets. The proposed models consider constant, linear, and exponential-learning effects simultaneously. The results reveal the proposed models fit the data better than other models, and that the learning effects occur in the software testing process. The results are helpful for the software testing/debugging managers to master the schedule of the projects, the performance of the programmers, and the reliability of the software system.

An Integrated Approach of Software Development and Test Processes to Distributed Teams

An Integrated Approach of Software Development and Test Processes to Distributed Teams

Improving software testing process: feature prioritization to make winners of success‐critical stakeholders

SUMMARYFor a successful software project, acceptable quality must be achieved within an acceptable cost, demonstrating business value to customers and satisfactorily meeting delivery timeliness. Testing serves as the most widely used approaches to determine that the intended functionalities are performed correctly and achieve the desired level of services; however, it is also a labor‐intensive and expensive process during the whole software life cycle. Most current testing processes are often technique‐centered, rather than organized to maximize business value. In this article, we extend and elaborate the ‘4+1’ theoretical lenses of Value‐based Software Engineering (VBSE) framework in the software testing process; propose a multi‐objective feature prioritization strategy for testing planning and controlling, which aligns the internal testing process with value objectives coming from customers and markets. Our case study in a real‐life business project shows that this method allows reasoning about the software testing process in different dimensions: it helps to manage the testing process effectively and efficiently, provides information for continuous internal software process improvement, and increases customer satisfaction, which makes winners of all success‐critical stakeholders (SCSs) in the software testing process. Copyright © 2010 John Wiley & Sons, Ltd.

Optimal test sequence generation using firefly algorithm

Software testing is an important but complex part of software development life cycle. The optimization of the software testing process is a major challenge, and the generation of the independent test paths remains unsatisfactory. In this paper, we present an approach based on metaheuristic firefly algorithm to generate optimal test paths. In order to optimize the test case paths, we use a modified firefly algorithm by defining appropriate objective function and introducing guidance matrix in traversing the graph. Our simulations and comparison show that the test paths generated are critical and optimal paths.

Optimal Software Release Using Time and Cost Benefits via Fuzzy Multi-Criteria and Fault Tolerance

As we know every software development process is pretty large and consists of different modules. This raises the idea of prioritizing different software modules so that important modules can be tested by preference. In the software testing process, it is not possible to test each and every module regressively, which is due to time and cost constraints. To deal with these constraints, this paper proposes an approach that is based on the fuzzy multi-criteria approach for prioritizing several software modules and calculates optimal time and cost for software testing by using fuzzy logic and the fault tolerance approach.

Parallel Study of Integrated Test in Software Testing Process

With the view of the phenomenon that different test modules have significant differences in the assembly test system, and there is some dependence between these modules, through the dependence theory of the parallel technology, there is the possibility of parallelization used in the integrated test system. In this paper, we analyses various control flows of each program or relation between the modules, and then gets all kinds of dependency between these modules, at the same time transforms the dependency into control flow graph, lastly gets the dependency of actual test cases. On this basis, according to the former implementation plan in parallel research, we can arrange parallel orders of modules the integrated test system, it can effectively improve the efficiency of software testing, and greatly saves the test time.

An optimum test stopping criterion based on software reliability modelling and Taguchi methods

A software company will get maximum benefit from testing only when the software testing process is optimised. The decision on when to stop testing and release the software to client is very important for software service companies. A lot of research has been conducted in the past to determine the optimal testing time and many stopping criteria have been proposed for software testing. The important among them are criteria based on estimated reliability, statistical similarity and cost benefit analysis. This paper discusses an optimum test stopping criterion combining the cost benefit analysis and reliability modelling. The approach focuses on fitting various software reliability models to the data and identifying the best-fit model using Taguchi’s loss function. Later on, the testing process is optimised by identifying the optimum testing effort required to detect the maximum bugs under the given cost constraint. Two case studies demonstrating the application of the proposed criterion are also presented in the paper.

Test sequence optimisation: an intelligent approach via cuckoo search

Exhaustive testing is never the appropriate approach in software testing as this violates the principle of cost effectiveness in software development as well as puts time and other resources at stake. This explains the need to optimise the process of software testing. The present paper gives an overview of cuckoo search algorithm and its role in software coverage optimisation. This paper proposes an algorithm to generate optimised test sequence(s), which obtains 100% software coverage based on cuckoo search.

Design of a competence model for testing teams

More and more software development organisations are paying attention to the improvement of the software testing process, because it is considered a key factor to ensure the quality of software products. However, the staff usually has problems developing testing activities because they do not have the appropriate competences to carry out these activities effectively; this results in low performance in organisations and increased costs of software products. A way to reduce this gap is developing a competence model that defines the roles who participate in the software testing activities as well as the general and technical competences required for them. Therefore this model could be applied to train staff in software testing activities and to recruit the appropriate profiles, which contribute to improving their performance. Considering that there is no published competence model specifically addressed to software testing, this study presents one that has been developed, analysing the literature and testing jobs and validated by experts in the software testing field using a survey as a validation method. So, as a result of this work, a competence model for software testing close to the software industry has been obtained.

Using Test Employee Capability Maturity Model for Supporting Gaps Bridging in Software Testing

This study addresses the challenge of assessing gaps among the differences of test people in eight groups by matching them based on four scenarios. The proposed model called Test Employee Capability Maturity Model (TEC-MM) helps find the gaps and measure the capability and maturity levels of each sub-maturity models in four dimensions as technicality, management, business, and personality. We applied TEC-MM to a software company which has instituted test people improvement strategy and plans to be a front runner in software testing industry. The findings reveal that gaps existed and consequently affect software testing process quality.

Simulation-Based Evaluation for the Impact of Personnel Capability on Software Testing Performance

This study presents a decision making process in three steps of knowledge management for test organization using process simulation and financial analysis. First, project cost assessment of test knowledge management process subjects to different project duration and number of staffs. Two knowledge management simulation models representing experienced personnel with knowledge sharing and inexperienced personnel with internal training respectively are employed to contrast test personnel capability. Second, performance evaluation of software testing process by different personnel capability is conducted to simulate system test using three project metrics, namely, duration, effort cost, and quality. Third, a comparative financial analysis is prepared to determine the best solution by return on investment, payback period, and benefit cost ratio. The results from three stages of finding are discussed to arrive at the final scenario. We provide a case study evaluating how software testing industry needs to build effective test organization with high quality personnel for sustainable development and improvement.

Software Reliability Prediction Based on Least Squares Estimation

Of the main interest in practical software testing would be the prediction of the number of detected faults in future rather than the estimation of the quantitative reliability function, which is defined as the probability that the software does not fail during a pre-specified time interval after release. In other words, practitioners wish to know the number of remaining faults and the saturated level of the cumulative number of faults, in order to control the software testing process. In not only such a situation but also the case where the likelihood information of statistical models is not fully available, the least squares estimation (LSE) may be simple but very useful in estimating model parameters. Hence, the LSE methods should be comprehensively studied in the context of software reliability engineering. In this paper we examine four LSE methods with application to the software reliability prediction. In real data analyses based on several software fault data, we show that LSE methods are still attractive in terms of goodness-of-fit performance and predictive performance in many cases. Also, we analyze the software reliability models with trend change point based on the LSE methods.

Software Testing in Small IT Companies: A (not only) South African Problem

Small software companies make for the majority of software companies around the world, but their software development processes are often not as clearly defined and structured as in their larger counterparts. Especially the test process is often the most neglected part of the software process. This contribution analyses the software testing process in a small South African IT company, here called X, to determine the problems that currently cause it to deliver software fraught with too many defects. The findings of a survey conducted with all software developers in company X are discussed, and several typical problems are identified. We also discuss two prevalent test process improvement models that can be used to reason about the possibilities of process improvement. Solutions to those (or similar) problems often already exist, but a major part of the problem addressed in this contribution is the unawareness, or unfamiliarity, of many small industrial software developers and IT managers as far as the scientific literature on software science and engineering, and especially in our case: software testing, is concerned.

Software Testing Model for Quality

Software testing plays an important role during the software developing period. Any saving on software test will significantly reduce the total cost of software. The key point of this paper is to build a software testing process with the cost control management and to make tradeoffs between the software quality and the software cost. Typical software testing model are analysed and compared. A Quality controlled 3-D testing model is introduced. The model combined with software test process, software quality cost and the testing type. The computational formula is defined to calculate software quality cost. A software quality balance law is introduced to balance control and failure cost. By adjusting software testing method and strategy, organization can reach the objective of the best balance between software quality and the software cost. The result shows an optimistic point that will prove the accurate of the model and the law introduced.

Transfer learning for cross-company software defect prediction

Context Software defect prediction studies usually built models using within-company data, but very few focused on the prediction models trained with cross-company data. It is difficult to employ these models which are built on the within-company data in practice, because of the lack of these local data repositories. Recently, transfer learning has attracted more and more attention for building classifier in target domain using the data from related source domain. It is very useful in cases when distributions of training and test instances differ, but is it appropriate for cross-company software defect prediction? Objective In this paper, we consider the cross-company defect prediction scenario where source and target data are drawn from different companies. In order to harness cross company data, we try to exploit the transfer learning method to build faster and highly effective prediction model. Method Unlike the prior works selecting training data which are similar from the test data, we proposed a novel algorithm called Transfer Naive Bayes (TNB), by using the information of all the proper features in training data. Our solution estimates the distribution of the test data, and transfers cross-company data information into the weights of the training data. On these weighted data, the defect prediction model is built. Results This article presents a theoretical analysis for the comparative methods, and shows the experiment results on the data sets from different organizations. It indicates that TNB is more accurate in terms of AUC (The area under the receiver operating characteristic curve), within less runtime than the state of the art methods. Conclusion It is concluded that when there are too few local training data to train good classifiers, the useful knowledge from different-distribution training data on feature level may help. We are optimistic that our transfer learning method can guide optimal resource allocation strategies, which may reduce software testing cost and increase effectiveness of software testing process.

Software testing process

Early identification of defects and prevention of defects migration are key goals of the testing process. It is highly desirable to optimize the test activities for a fast time-to-market while delivering a product that meets quality expectation. Software engineering research has led to a raft of systematic testing activities applicable during the development life cycle. Software test process elaborates various testing activities and describes which activity is to be carried out when. Given the need and significance of phased approach of testing, this paper proposes a prescriptive framework elaborating testing activities to be carried out while integrating it within the development life cycle.