Software Fault Research Articles

Adaptive scheduling-based fine-grained greybox fuzzing for cloud-native applications

Coverage-guided fuzzing is one of the most popular approaches to detect bugs in programs. Existing work has shown that coverage metrics are a crucial factor in guiding fuzzing exploration of targets. A fine-grained coverage metric can help fuzzing to detect more bugs and trigger more execution states. Cloud-native applications that written by Golang play an important role in the modern computing paradigm. However, existing fuzzers for Golang still employ coarse-grained block coverage metrics, and there is no fuzzer specifically for cloud-native applications, which hinders the bug detection in cloud-native applications. Using fine-grained coverage metrics introduces more seeds and even leads to seed explosion, especially in large targets such as cloud-native applications. Therefore, we employ an accurate edge coverage metric in fuzzer for Golang, which achieves finer test granularity and more accurate coverage information than block coverage metrics. To mitigate the seed explosion problem caused by fine-grained coverage metrics and large target sizes, we propose smart seed selection and adaptive task scheduling algorithms based on a variant of the classical adversarial multi-armed bandit (AMAB) algorithm. Extensive evaluation of our prototype on 16 targets in real-world cloud-native infrastructures shows that our approach detects 233% more bugs than go-fuzz, achieving an average coverage improvement of 100.7%. Our approach effectively mitigates seed explosion by reducing the number of seeds generated by 41% and introduces only 14% performance overhead.

StructuredFuzzer: Fuzzing Structured Text-Based Control Logic Applications

Rigorous testing methods are essential for ensuring the security and reliability of industrial controller software. Fuzzing, a technique that automatically discovers software bugs, has also proven effective in finding software vulnerabilities. Unsurprisingly, fuzzing has been applied to a wide range of platforms, including programmable logic controllers (PLCs). However, current approaches, such as coverage-guided evolutionary fuzzing implemented in the popular fuzzer American Fuzzy Lop Plus Plus (AFL++), are often inadequate for finding logical errors and bugs in PLC control logic applications. They primarily target generic programming languages like C/C++, Java, and Python, and do not consider the unique characteristics and behaviors of PLCs, which are often programmed using specialized programming languages like Structured Text (ST). Furthermore, these fuzzers are ill suited to deal with complex input structures encapsulated in ST, as they are not specifically designed to generate appropriate input sequences. This renders the application of traditional fuzzing techniques less efficient on these platforms. To address this issue, this paper presents a fuzzing framework designed explicitly for PLC software to discover logic bugs in applications written in ST specified by the IEC 61131-3 standard. The proposed framework incorporates a custom-tailored PLC runtime and a fuzzer designed for the purpose. We demonstrate its effectiveness by fuzzing a collection of ST programs that were crafted for evaluation purposes. We compare the performance against a popular fuzzer, namely, AFL++. The proposed fuzzing framework demonstrated its capabilities in our experiments, successfully detecting logic bugs in the tested PLC control logic applications written in ST. On average, it was at least 83 times faster than AFL++, and in certain cases, for example, it was more than 23,000 times faster.

An empirical study of data sampling techniques for just-in-time software defect prediction

An empirical study of data sampling techniques for just-in-time software defect prediction

Integration of Finite Element Analysis and Laboratory Analysis on 3D Models for Methodology Calibration.

To better address mechanical behavior, it is necessary to make use of modern tools through which it is possible to run predictions, simulate scenarios, and optimize decisions. sources integration. This will increase the capability of detecting material modifications that forerun damage and/or to forecast the stage in the future when very likely fatigue is initiating and propagating cracks. Early warning outcomes obtained by the synergetic implementation of NDE-based protocols for studying mechanical and fatigue and fracture behavior will enhance the preparedness toward economically sustainable future damage control scenarios. Specifically, these early warning outcomes will be developed in the form of retopologized models to be used coupled with FEA. This paper presents the first stage of calibration and the combination of a system of different sensors (photogrammetry, laser scanning and strain gages) for the creation of volumetric models suitable for the prediction of failure of FEA software. The test objects were two components of car suspension to which strain gauges were attached to measure its deformation under cyclic loading. The calibration of the methodology was carried out using models obtained from photogrammetry and experimental strain gauge measurements.

SeDPGK: Semi-supervised software defect prediction with graph representation learning and knowledge distillation

Context:Constructing an effective defect prediction model relies on a substantial number of labeled program modules. Unfortunately, program module labeling is often time-consuming and error-prone. Semi-supervised software defect prediction (SSDP) can alleviate this issue by incorporating some labeled modules and the remaining unlabeled modules from the same project. Objective:However, previous SSDP methods ignore the significant influence of dependencies between software modules. The potential of knowledge distillation in leveraging labeled instances to guide the learning process and effectively utilizing information from unlabeled instances to improve SSDP performance has not been fully investigated. Method:We propose a novel approach SeDPGK. Specifically, to exploit the graph-structured knowledge, we first construct the program dependence graph to extract control and data dependencies among modules. Then we use graph neural networks (GNNs) to learn the graph representation of the module relationships and encode with the statement semantics of abstract syntax tree and traditional static features for diversity. Second, we integrate multiple GNNs jointly trained as teacher models to ensemble various styles of graph-based networks and generate trustworthy labels for unlabeled modules. Further, to preserve the teacher model’s sufficient structure and semantic knowledge, we adopt a trainable label propagation and multi-layer perception as the student model and mitigate the differences between the teacher and student models using two widespread knowledge distillation functions. Results:We conducted our experiments on 17 real-world projects. The experimental results show that SeDPGK outperforms semi-supervised baselines with an average improvement of 16.9% for PD, 42.5% for FAR, and 8.9% for AUC, respectively. Moreover, the performance improvement is consistently significant across multiple statistical tests. Conclusion:The effectiveness of SeDPGK comes from the aggregation of the different GNNs with heterogeneity. Moreover, the graph structure and semantic features hidden behind the source code play a crucial role in the distillation framework.

SDP-MTF: A Composite Transfer Learning and Feature Fusion for Cross-Project Software Defect Prediction

Software defect prediction is critical for improving software quality and reducing maintenance costs. In recent years, Cross-Project software defect prediction has garnered significant attention from researchers. This approach leverages transfer learning to apply the knowledge from existing projects to new ones, thereby enhancing the universality of predictive models. It provides an effective solution for projects with limited historical defect data. Nevertheless, current methodologies face two main challenges: first, the inadequacy of feature information mining, where code statistical information or semantic information is used in isolation, ignoring the benefits of their integration; second, the substantial feature disparity between different projects, which can lead to insufficient effect during transfer learning, necessitating additional efforts to narrow this gap to improve precision. Addressing these challenges, this paper proposes a novel methodology, SDP-MTF (Software Defect Prediction using Multi-stage Transfer learning and Feature fusion), that combines code statistical features, deep semantic features, and multiple feature transfer learning methods to enhance the predictive effect. The SDP-MTF method was empirically tested on single-source cross-project software defect prediction across six projects from the PROMISE dataset, benchmarked against five baseline algorithms that employ distinct features and transfer methodologies. Our findings indicate that SDP-MTF significantly outperforms five classical baseline algorithms, improving the F1-Score by 8% to 15.2%, thereby substantively advancing the precision of cross-project software defect prediction.

Parameter-Efficient Multi-classification Software Defect Detection Method Based on Pre-trained LLMs

Software Defect Detection (SDD) has always been critical to the development life cycle. A stable defect detection system can not only alleviate the workload of software testers but also enhance the overall efficiency of software development. Researchers have recently proposed various artificial intelligence-based SDD methods and achieved significant advancements. However, these methods still exhibit limitations in terms of reliability and usability. Therefore, we introduce MSDD-(IA)3, a novel framework leveraging the pre-trained CodeT5+ and (IA)3 for parameter-efficient multi-classification SDD. This framework constructs a detection model based on pre-trained CodeT5+ to generate code representations while capturing defect-prone features. Considering the high overhead of pre-trained LLMs, we injects (IA)3 vectors into specific layers, where only these injected parameters are updated to reduce the training cost. Furthermore, leveraging the properties of the pre-trained CodeT5+, we design a novel feature sequence that enriches the input data through the combination of source code with Natural Language (NL)-based expert metrics. Our experimental results on 64K real-world Python snippets show that MSDD-(IA)3 demonstrates superior performance compared to state-of-the-art SDD methods, including PM2-CNN, in terms of F1-weighted, Recall-weighted, Precision-weighted, and Matthews Correlation Coefficient. Notably, the training parameters of MSDD-(IA)3 are only 0.04% of those of the original CodeT5+. Our experimental data and code can be available at (https://gitee.com/wxyzjp123/msdd-ia3/).

Evaluation of the Character Home Visit Program Using the Context, Input, Process, Product (CIPP) Model at SMP Islam Al Madinah Jombang

Evaluation of the character program at SMP Islam Al Madinah Jombang is very important to determine the effectiveness of the program in developing moral and ethical values ??among students. This evaluation can also help identify deficiencies and errors in the character program, so that better improvements and development can be made. This research aims to find out how to evaluate the character home visit program. Research using the CIPP evaluation model includes Context, Input, Process, Product. Data collection techniques use interview, observation and documentation methods.The data analysis used is data collection, data reduction, data presentation, and verification or drawing conclusions. The results of the program evaluation include (1) in terms of context, the components are home visit background, character, social conditions of students, and school policies. (2) in terms of input, the components are available resources, teacher preparation, and parental involvement. (3) viewed from the process, its components are planning, implementation, evaluation and challenges. (4) in terms of product, the components are the impact of student behavior, parental satisfaction with program results, and recommendations for program improvement and development.

Two sides of the same coin: A study on developers' perception of defects

SummarySoftware defect prediction is a subject of study involving the interplay of software engineering and machine learning. The current literature proposed numerous machine learning models to predict software defects from software data, such as commits and code metrics. Further, the most recent literature employs explainability techniques to understand why machine learning models made such predictions (i.e., predicting the likelihood of a defect). As a result, developers are expected to reason on the software features that may relate to defects in the source code. However, little is known about the developers' perception of these machine learning models and their explanations. To explore this issue, we focus on a survey with experienced developers to understand how they evaluate each quality attribute for the defect prediction. We chose the developers based on their contributions at GitHub, where they contributed to at least 10 repositories in the past 2 years. The results show that developers tend to evaluate code complexity as the most important quality attribute to avoid defects compared with the other target attributes such as source code size, coupling, and documentation. At the end, a thematic analysis reveals that developers evaluate testing the code as a relevant aspect not covered by the static software features. We conclude that, qualitatively, there exists a misalignment between developers' perceptions and the outputs of machine learning models. For instance, while machine learning models assign high importance to documentation, developers often overlook documentation and prioritize assessing the complexity of the code instead.

Handling class overlap and imbalance using overlap driven under-sampling with balanced random forest in software defect prediction

Handling class overlap and imbalance using overlap driven under-sampling with balanced random forest in software defect prediction

The Role of Machine Learning in Software Development

In today's rapidly evolving technological era, the role of machine learning in software development has become increasingly vital and influential. Machine learning has revolutionized various aspects of software development, from code analysis and optimization to prediction and decision-making. Moreover, machine learning algorithms have the potential to significantly enhance the software development process by automating repetitive tasks, improving code quality, and reducing the time and effort required for software testing and debugging. By gaining lots of data and powerful computing resources, machine learning algorithms can be able to analyse patterns and make accurate predictions about software performance, identify potential bugs or security issues, and assist in improving software design and development processes. Furthermore, machine learning can facilitate software maintenance and debugging by detecting anomalies and identifying potential causes of software failures. Albeit, using machine learning techniques into the software development process can greatly improve efficiency, productivity, and overall software quality. In this paper it is presenting the tools, techniques and the application of Machine Learning (ML) in different phases of Software Development Life Cycle (SDLC) for enhancing and improving the software development process.

Development of a Novel Classifier Model Implementing Bio-inspired Algorithms to Optimize Existing Models for Software Defect Detection

Development of a Novel Classifier Model Implementing Bio-inspired Algorithms to Optimize Existing Models for Software Defect Detection

WSO-KELM: War Strategy Optimization-Based Kernel Extreme Learning Machine for Automatic Software Fault Prediction Model

WSO-KELM: War Strategy Optimization-Based Kernel Extreme Learning Machine for Automatic Software Fault Prediction Model

Towards a framework for reliable performance evaluation in defect prediction

Enhancing software reliability, dependability, and security requires effective identification and mitigation of defects during early development stages. Software defect prediction (SDP) models have emerged as valuable tools for this purpose. However, there is currently a lack of consensus in evaluating the predictive performance of newly proposed models, which hinders accurate measurement of progress and can lead to misleading conclusions. To tackle this challenge, we present MATTER (a fraMework towArd a consisTenT pErformance compaRison), which aims to provide reliable and consistent performance comparisons for SDP models. MATTER incorporates three key considerations. First, it establishes a global reference point, ONE (glObal baseliNe modEl), which possesses the 3S properties (Simplicity in implementation, Strong predictive ability, and Stable prediction performance), to serve as the baseline for evaluating other models. Second, it proposes using the SQA-effort-aligned threshold setting to ensure fair performance comparisons. Third, it advocates for consistent performance evaluation by adopting a set of core performance indicators that reflect the practical value of prediction models in achieving tangible progress. Through the application of MATTER to the same benchmark data sets, researchers and practitioners can obtain more accurate and meaningful insights into the performance of defect prediction models, thereby facilitating informed decision-making and improving software quality. When evaluating representative SDP models from recent years using MATTER, we surprisingly observed that: none of these models demonstrated a notable enhancement in prediction performance compared to the simple baseline model ONE. In future studies, we strongly recommend the adoption of MATTER to assess the actual usefulness of newly proposed models, promoting reliable scientific progress in defect prediction.

Integrity Testing Performance Analysis in Embedded Systems using IDI

Multiple modules that interact with one another to exchange data make up embedded systems. Incorrect resource data exchanged between modules might cause anomalies or mistakes in operation. Resources that interact create dependencies between two modules, such that modifications made to one module's resources will impact modifications made to another module's functionality. Interaction errors between modules are one of the main causes of severe software failures, according to several embedded system investigations, such as those conducted on aerospace or automotive systems. Interaction testing is therefore a crucial step in minimizing risk and reducing interaction errors. When modules interact both directly and indirectly, interaction failures arise. Indirect interactions occur beneath the interface, where a data reliance relationship with resources may lead to an unexpected result

A novel software defect prediction model using two-phase grey wolf optimisation for feature selection

A novel software defect prediction model using two-phase grey wolf optimisation for feature selection

Long-term software fault prediction with wavelet shrinkage estimation

Wavelet shrinkage estimation received considerable attentions to estimate stochastic processes such as a non-homogeneous Poisson process in a non-parametric way, and was applied to software reliability estimation/prediction. However, it lacks the prediction ability for unknown future patterns in long term and penalizes assessing the software reliability in practice. In this paper, we focus on the long-term prediction of the number of software faults detected in the testing phase and propose many novel long-term prediction methods based on the wavelet shrinkage estimation. The fundamental idea is to adopt both the denoised fault-count data and prediction values, and to minimize several kinds of loss functions to make effective predictions. We also develop an automated wavelet-based software reliability assessment tool, W-SRAT2, which is a drastic extension of the existing tool, W-SRAT, by adding those prediction algorithms. In numerical experiments with 6 actual software development project data, we investigate the predictive performance of our long-term prediction approaches, which consist of 2,640 combinations, and compare them with the common software reliability growth models with the maximum likelihood estimation. It is shown that our wavelet shrinkage estimation/prediction methods outperform the existing software reliability growth models.

TR-Fuzz: A syntax valid tool for fuzzing C compilers

Compilers play a critical role in current software construction. However, the vulnerabilities or bugs within the compiler can pose significant challenges to ensuring the security of the resultant software. In recent years, many compilers have made use of testing techniques to address and mitigate such concerns. Fuzzing is widely used among these techniques to detect software bugs. However, when fuzzing compilers, there are still shortcomings in terms of the diversity and validity of test cases. This paper introduces TR-Fuzz, a fuzzing tool specifically designed for C compilers based on Transformer. Leveraging position embedding and multi-head attention mechanisms, TR-Fuzz establishes relationships among data, facilitating the generation of well-formed C programs for compiler testing. In addition, we use different generation strategies in the process of program generation to improve the performance of TR-Fuzz. We validate the effectiveness of TR-Fuzz through the comparison with existing fuzzing tools for C compilers. The experimental results show that TR-Fuzz increases the pass rate of the generated C programs by an average of about 12% and improves the coverage of programs under test compared with the existing tools. Benefiting from the improved pass rate and coverage, we found five bugs in GCC-9.

Development of optimised software fault prediction model using machine learning

Software fault prediction is a crucial task, especially with the rapid improvements in software technology and increasing complexity of software. As identifying and addressing bugs early in the development process can significantly minimize the costs and enhance the software quality. Software fault prediction using machine learning algorithms has gained significant attention due to its potential to improve software quality and save time in the testing phase. This research paper investigates the impact of classification models on bug prediction performance and explores the use of bio-inspired optimization techniques to enhance model results. Through experiments, it is demonstrated that applying bio-inspired algorithms improves the accuracy of fault prediction models. The evaluation is based on multiple performance metrics and the results show that KNN with BACO (Binary Ant Colony Optimization) generally outperform the other models in terms of accuracy. The BACO-KNN fault prediction model attains the accuracy of 96.39% surpassing the previous work.

Software framework for error detection in software using the spot fuzzing method

The paper describes the architecture and principles of operation of the software complex that provides detection of errors in software using spot fuzzing, presented in earlier works of the author. The functions and interrelationship of the modules that make up the software complex are described, and the results of experimental studies in terms of operability and efficiency of the complex are presented.