Software Testing Methods Research Articles

Quality assurance strategies for machine learning applications in big data analytics: an overview

Machine learning (ML) models have gained significant attention in a variety of applications, from computer vision to natural language processing, and are almost always based on big data. There are a growing number of applications and products with built-in machine learning models, and this is the area where software engineering, artificial intelligence and data science meet. The requirement for a system to operate in a real-world environment poses many challenges, such as how to design for wrong predictions the model may make; How to assure safety and security despite possible mistakes; which qualities matter beyond a model’s prediction accuracy; How can we identify and measure important quality requirements, including learning and inference latency, scalability, explainability, fairness, privacy, robustness, and safety. It has become crucial to test thoroughly these models to assess their capabilities and potential errors. Existing software testing methods have been adapted and refined to discover faults in machine learning and deep learning models. This paper covers a taxonomy, a methodologically uniform presentation of all presented solutions to the aforementioned issues, as well as conclusions about possible future development trends. The main contributions of this paper are a classification that closely follows the structure of the ML-pipeline, a precisely defined role of each team member within that pipeline, an overview of trends and challenges in the combination of ML and big data analytics, with uses in the domains of industry and education.

LITERATURE REVIEW: ADVANTAGES AND DISADVANTAGES OF BLACK BOX AND WHITE BOX TESTING METHODS

Software testing methods play a crucial role in ensuring the quality, security, and performance of applications. The two main approaches often used are black box and white box testing. This article presents a comprehensive literature review of the characteristics, advantages and disadvantages of both testing methods with the aim of providing in-depth insights for practitioners and researchers in the field of software engineering. The research method used is Systematic Literature Review (SLR), which is a technique of collecting information through the process of searching for literature based on research questions, filtering search results, evaluating, and analyzing relevant research from various academic database sources to gain a comprehensive understanding. Through the analysis of various literature studies, the results of this research are the characteristics of each testing method that is more effectively used in best practice to achieve efficient testing with the coverage of literature review sources in the range of 2020 to 2024. The review of these characteristics is based on seven criteria, namely: test focus, test base, tester's point of view, test case design, program error identification method, program code knowledge, and test effectiveness. In addition, this article presents a comparison of the two methods so that the differences in testing methods can be clearly recognised in terms of how they are implemented. Based on the results of review literature, both methods are better used together in order to ensure that the functional and internal code structure of the software is functioning properly.

WEB-BASED TOURISM INFORMATION SYSTEM IN BUNGUS TELUK KABUNG SUB-DISTRICT OFFICE

This system design aims to develop a web-based tourism information system for Bungus Teluk Kabung sub-district office and create applications that can be implemented in this sub-district office. This system research method is built on a website platform and runs online. The research method uses the system development life cycle (SDLC) with the waterfall model approach, which consists of 5 phases, namely: a) Analysis b) Design c) Implementation d) Maintenance e) Planning. The programming language used is (PHP), and the database application uses a data processor (MySql) as the database (RDBMS). The system design tools use Unified Modeling Language (UML) method with 4 (four) diagrams according to the design requirements, namely use case diagrams, activity diagrams, class diagrams and sequence diagrams. The tourism information application in the sub-district office of Bungus Teluk Kabung facilitates visitors to find information and tourist places in the study area. This system was tested and validated using the software testing method, using black box testing. This system can overcome the problems that exist in the area of Bungus Teluk Kabung sub-district.

QuanTest: Entanglement-Guided Testing of Quantum Neural Network Systems

Quantum Neural Network (QNN) combines the Deep Learning (DL) principle with the fundamental theory of quantum mechanics to achieve machine learning tasks with quantum acceleration. Recently, QNN systems have been found to manifest robustness issues similar to classical DL systems. There is an urgent need for ways to test their correctness and security. However, QNN systems differ significantly from traditional quantum software and classical DL systems, posing critical challenges for QNN testing. These challenges include the inapplicability of traditional quantum software testing methods to QNN systems due to differences in programming paradigms and decision logic representations, the dependence of quantum test sample generation on perturbation operators, and the absence of effective information in quantum neurons. In this paper, we propose QuanTest, a quantum entanglement-guided adversarial testing framework to uncover potential erroneous behaviors in QNN systems. We design a quantum entanglement adequacy criterion to quantify the entanglement acquired by the input quantum states from the QNN system, along with two similarity metrics to measure the proximity of generated quantum adversarial examples to the original inputs. Subsequently, QuanTest formulates the problem of generating test inputs that maximize the quantum entanglement adequacy and capture incorrect behaviors of the QNN system as a joint optimization problem and solves it in a gradient-based manner to generate quantum adversarial examples. Experimental results demonstrate that QuanTest possesses the capability to capture erroneous behaviors in QNN systems (generating 67.48%-96.05% more high-quality test samples than the random noise under the same perturbation size constraints). The entanglement-guided approach proves effective in adversarial testing, generating more adversarial examples (maximum increase reached 21.32%).

Code-Aware Prompting: A Study of Coverage-Guided Test Generation in Regression Setting using LLM

Testing plays a pivotal role in ensuring software quality, yet conventional Search Based Software Testing (SBST) methods often struggle with complex software units, achieving suboptimal test coverage. Recent work using large language models (LLMs) for test generation have focused on improving generation quality through optimizing the test generation context and correcting errors in model outputs, but use fixed prompting strategies that prompt the model to generate tests without additional guidance. As a result LLM-generated testsuites still suffer from low coverage. In this paper, we present SymPrompt, a code-aware prompting strategy for LLMs in test generation. SymPrompt’s approach is based on recent work that demonstrates LLMs can solve more complex logical problems when prompted to reason about the problem in a multi-step fashion. We apply this methodology to test generation by deconstructing the testsuite generation process into a multi-stage sequence, each of which is driven by a specific prompt aligned with the execution paths of the method under test, and exposing relevant type and dependency focal context to the model. Our approach enables pretrained LLMs to generate more complete test cases without any additional training. We implement SymPrompt using the TreeSitter parsing framework and evaluate on a benchmark challenging methods from open source Python projects. SymPrompt enhances correct test generations by a factor of 5 and bolsters relative coverage by 26% for CodeGen2. Notably, when applied to GPT-4, SymPrompt improves coverage by over 2x compared to baseline prompting strategies.

Research on the Application of Embedded Real-time Software in Computer Software Design

With the continuous development of computer technology, embedded real-time software is also developing and improving. In the design of computer software, embedded real-time software is the core part, and its reasonable application can effectively improve the function and performance of computer software and ensure its stable and reliable operation. At the same time, embedded real-time software can effectively realize the real-time operation of computers and bring convenience to people's daily life and work. This paper deeply studies the application of embedded real-time software in computer software design. The key technologies and methods such as real-time operating system, real-time task scheduling algorithm, embedded software design mode and embedded software testing and verification method are discussed in detail. Combined with specific cases, the application practice of embedded real-time software in computer software design is analyzed, including real-time requirements analysis, performance optimization, resource management and application in specific fields. This paper analyzes the specific application of embedded real-time software in computer software design, and provides some reference for related workers.

Fine-grained Coverage-based Fuzzing

Fuzzing is a popular software testing method that discovers bugs by massively feeding target applications with automatically generated inputs. Many state-of-the-art fuzzers use branch coverage as a feedback metric to guide the fuzzing process. The fuzzer retains inputs for further mutation only if branch coverage is increased. However, branch coverage only provides a shallow sampling of program behaviors and hence may discard interesting inputs to mutate. This work aims to take advantage of the large body of research in defining finer-grained code coverage metrics (such as control-flow, data-flow, or mutation coverage) and to evaluate how fuzzing performance is impacted when using these metrics to select interesting inputs for mutation. We propose to make branch coverage-based fuzzers support most fine-grained coverage metrics out of the box (i.e., without changing fuzzer internals). We achieve this by making the test objectives defined by these metrics (such as conditions to activate or mutants to kill) explicit as new branches in the target program. Fuzzing such a modified target is then equivalent to fuzzing the original target, but the fuzzer will also retain inputs covering the additional metric objectives for mutation. In addition, all the fuzzer mechanisms to penetrate hard-to-cover branches will help in covering the additional metric objectives. We use this approach to evaluate the impact of supporting two fine-grained coverage metrics (multiple condition coverage and weak mutation) over the performance of two state-of-the-art fuzzers (AFL++ and QSYM) with the standard LAVA-M and MAGMA benchmarks. This evaluation suggests that our mechanism for runtime fuzzer guidance, where the fuzzed code is instrumented with additional branches, is effective and could be leveraged to encode guidance from human users or static analyzers. Our results also show that the impact of fine-grained metrics over fuzzing performance is hard to predict before fuzzing and most of the time either neutral or negative. As a consequence, we do not recommend using them to guide fuzzers, except maybe in some possibly favorable circumstances yet to be investigated, like for limited parts of the code or to complement classical fuzzing campaigns.

Optimization of White Box Testing by Utilizing Branching and Repeating Structures in Java Programs Using Base Path

Software testing is a process created to detect anomalies in the operation of a program or system in order to achieve the expected results. White box testing is a software testing method that tests the internal structure, design and program code. This research aims to produce an optimization method for white box testing in Java programs by utilizing branching and repetition structures using the basis path method, as well as analyzing the effectiveness of the proposed method in generating test cases. The Java program tested in this research includes input function calls, loops, branching, and exception handling. Test case design is carried out by applying the basis path method to achieve comprehensive coverage. The test results show that the base path method is able to produce effective test cases for testing control structures without redundancy. Test case design is assisted by flowgraph and matrix graph modeling. It is hoped that this research can contribute to the optimization of white box testing techniques.

Black Box Testing on Website-Based Guestbook Registration Applications

In the rapidly developing digital era, website-based applications have become an important means of meeting the needs of various uses. One application that has received attention is the guest book registration application, which aims to record and manage guest information efficiently. In this context, application testing becomes crucial to ensure service quality, security and availability. This research focuses on black box testing, a software testing method that emphasizes the functionality and reliability of applications from the user's perspective. Testing covers key aspects such as data input, and key functionality. Through this approach, it is hoped that the research results can contribute to the development of a website-based guestbook registration application that is more reliable and suits user needs.

LandSAR software testing method for Lu Tan-1 SAR standard product generation in natural resource monitoring

LandSAR software testing method for Lu Tan-1 SAR standard product generation in natural resource monitoring

Test data generation for covering mutation-based path using MGA for MPI program

Message Passing Interface (MPI) is a communication protocol used for parallel programming in various languages, valued for its reliability and broad applicability. Mutation testing is a software testing method for systematically simulating software faults. However, the significant number of inserted mutation branches in the program escalates the testing cost. To address this issue, we propose the comprehensive PMTDGM framework. The framework first generates mutation-based paths based on the relevancy of mutant branches and the difficulty of mutant branch coverage, followed by the establishment of a multitask model of path coverage. Finally, we employ a multi-population genetic algorithm (MGA) to generate test data. Our experiments, performed on six MPI programs of varying sizes and structures, demonstrate that the mutation-based paths have the small test sets and are easy to cover. Additionally, the multitask model and MGA can significantly improve the efficiency of generating test data and reduce the cost of mutation testing for MPI programs compared to traditional methods.

Application and Practice of Intelligent Algorithms in Computer Software Design and Testing

With the rapid development of information technology, software has become a part of social life. However, for complex software systems, traditional design and testing methods face a series of problems such as low testing coverage and low efficiency. The main purpose of this article is to study the optimization of software design and testing processes using intelligent algorithms, in order to improve the quality and efficiency of software. This article established a real-time optimization and feedback adjustment mechanism during the software design and testing phase, enabling intelligent algorithms to respond promptly to the environment and requirements. Monitoring and control strategies were designed to ensure the algorithm runs stably for a long time. Test method 1 (traditional method) had 100 test cases, a total execution time of 1200 seconds, and an average execution time of 12 seconds per test case; test method 2 (Intelligent Algorithm Optimization) had 100 test cases, a total execution time of 800 seconds, and an average execution time of 8 seconds per test case. The research results of this article play a good promoting role in improving software testing coverage and software structure optimization, and also provide new ideas and methods for the promotion of intelligent algorithms in software engineering.

Retracted: Component-Based Software Testing Method Based on Deep Adversarial Network

Retracted: Component-Based Software Testing Method Based on Deep Adversarial Network

Detecting system defects of digital devices in the course of malfunction imitation using fuzzing

Aim. The paper proposes approaches to the organisation of testing of digital systems through malfunction imitation for the purpose of ensuring compliance of international and Russian failure and fault resistance standards for the purpose of efficient (in terms of time) detection of software defects as part of mass production of products. The paper proposes a structure and operating algorithm of a hardware and software test bed for malfunction simulation intended for testing a system’s devices. The test bed collects and processes data for fuzzing, hardware error identification, as well as defines the scope of testing.Methods. The paper used basic systems approach, classical methods of the probability theory and mathematical statistics, decision theory, methods of hardware and software testing and development, mathematical theory of fuzzy sets and fuzzy logic.Results. Malfunction simulation algorithms were developed for the purpose of testing hardware and software systems using fuzzing that ensure probabilistic estimation of the termination time of testing with a specified accuracy.Conclusions. The above set of algorithms allows detecting system defects in the process of software and hardware integration into a single system that cause new malfunctions (emergence) that cannot be taken into consideration at the design stage.

Exhaustive Test Case Generation for Nuclear Safety Software Based on the Software Logic Model

An issue regarding the incorporation of software reliability within the nuclear power plant (NPP) probabilistic risk assessment model has emerged in the licensing processes of digitalized NPPs. Since software failure induces common-cause failure of the processor modules, the reliability of the software used in the NPP safety-critical instrumentation and control systems must be quantified and verified with proper test cases and environments. In this study, a software testing method based on the minimal cut set (MCS)–based exhaustive test case generation scheme is proposed where the software logic model is developed from available information on the software development and the MCSs that represent the necessary and sufficient conditions for the software variables’ states to produce safety software outputs are generated. The MCSs are then converted into the test cases, which can be used as inputs to the test bed to verify that the test cases produce correct outputs after software execution. The effectiveness of the proposed method is demonstrated with the safety-critical trip logic software of the APR-1400 reactor protection system. The method provides a systematic way to conduct exhaustive software testing and prove the functionality of the nuclear safety software based on the test result without uncertainties.

ENHANCING THE EFFECTIVENESS OF USABILITY TESTING FOR USER INTERFACES

The paper analyses the problems of maintaining software quality, namely, usability testing as a direction of ensuring this quality, analyses publications that consider the methods of electro-oculography, electro-retinography, mouse-tracking, eye-tracking, etc. and proves that the issue of improving the quality of software products largely depends on the effectiveness of usability testing. The study is based on the analysis of traditional software testing methods and tools and the proposal of mouse-tracking and eye-tracking technologies as an alternative solution to the problem. Criteria and metrics for assessing the usability of web applications were identified, an analytical approach was used as a methodological basis, which involves a comprehensive consideration of the research object, methods of comparative analysis and classification were used to obtain data, the results were processed using standard statistical methods, and the necessity and importance of usability testing of websites was assessed. The article considers generally accepted rules and recommendations in the field of usability testing, analyses quantitative and qualitative methods of usability testing evaluation, for a more detailed study of the subject of research the ergonomic interaction of a user with an information system, namely with a web resource, studies the main categories of users depending on a number of indicators, and pays special attention to establishing quality assessment criteria based on existing standards of recommendations. the article reviews the process of usability testing of information system interfaces, its analysis and evaluation, analyses the methods of electro-oculography and electro-retinography, mouse-tracking and eye-tracking technologies, and concludes that the use of eye-tracking technology will allow collecting and systematising quantitative and qualitative data on user interaction with the system and will make it possible to optimise the process of usability testing by reducing the time of its implementation. The prerequisites for conducting an experimental study of theoretical results using eye-tracking technologies have been created.

Software Testing Integration-Based Model (I-BM) Framework for Recognizing Measure Fault Output Accuracy Using Machine Learning Approach

In software development, the software testing phase is an important process in determining the quality level of the software. Software testing is a process of executing a program aimed at finding errors in module access, units, and involves the execution of the system being tested on a number of test inputs, and determining whether the output produced is correct. In this study, a model-based testing (MBT) called integration-based model (I-BM) framework will be developed. This I-BM framework integrates testing variables from several software testing methods, namely black-box testing, white-box testing, unit testing, system testing, and acceptance testing. The integrated variables are function, interface, structure, performance, requirement, documentation, positives, and negatives. Then, this framework will document software errors to form a dataset, which will be measured for the level of accuracy of expected manual fault output using neural network algorithm and support vector machine. From the experiment results, it shows that the accuracy level of predicting fault output values from the I-BM framework using the neural network algorithm is on average 80%, and it produces a superior SVM architecture model in predicting I-BM framework output errors with an accuracy value of 0.99, precision of 0.99, recall of 0.99, and [Formula: see text]-score of 0.99. Compared to other MBT, the IBM framework has the advantage of being a more comprehensive software testing model because it starts from the identification of problems, analysis, design, documentation of software testing, and recommendations for each fault output found. Thus, software errors can be classified systematically in the form of a dataset, and not only focus on software testing for product lines and module mappings.

Web-Based Motorcycle Loan Management System with credit risk and approval analysis using Data Mining Techniques

Using information systems can dramatically improve a business's productivity and efficiency. Moreover, the ability of information systems to provide users with the information they need to complete jobs effectively can be the most significant advantage. This project development plan proposes a Web-Based Motorcycle Loan Management System with credit risk and approval analysis using Data Mining Techniques for a private motorcycle dealer who encounters challenges, particularly in the decision-making of motorcycle loans. It will facilitate the automation of business processes, including online motorcycle loan applications and payments, access to real-time data and reporting, and credit risk and approval analysis to assist the credit department in decision- making by applying data mining techniques. The project will employ the Scrum framework for software development and testing method. A total of 150.625 man-days with a total cost of P1,350,231.78 were computed to complete the proposed information system. A one-year financial viability analysis was defined to determine the proposed project's total cost and benefits. Based on the results, time spent on activities is reduced to 89.19%, indicating that the business can handle more applications every day compared to the average twenty daily applications since processing time is significantly decreased. It is recommended for future improvement of this plan to implement Augmented Reality for a better user online experience in previewing motorcycle units.

PENERAPAN FRAMEWORK CODEIGNITER UNTUK WEBSITE TOKO ONLINE PADA DASTER HITS PONTIANAK

Daster Hits pontianak has a need to expand its market share and manage valid data systems. The sales system currently used is traditional sellers and social media such as Instagram and WhatsApp, therefore the interaction between the store and customers is still limited, recording of transactions is still manual using a transaction book, and if customers want to find products they have to scroll Instagram or come directly to the store. The purpose of this research is to apply the CodeIgniter framework to build an online shop website which is expected to help the shop in promoting the product, manage product data, and expedite the product sales process. The research method used in this research is Design Science Research and the design method uses Extreme Programming. The system modeling used is the Unified Modeling Language (UML) and the software testing method uses white-box testing. The advantages of Daster Hits Pontianak Online Shop website are the features such as managing user profiles, product sales, detailed product information, online product orders and payment confirmation, searches using sequential search algorithms, managing product stocks using the Apriori algorithm, and notifications using push notifications. The process of making this program uses the PHP programming language and the MariaDB database.

Debugging Error with The Help of Software Testing Methods

Debugging Error with The Help of Software Testing Methods