Test Suite Research Articles

Smartphone tests quantify lower extremities dysfunction in multiple sclerosis

IntroductionIncreasing shortage of neurologists compounded by the global aging of the population have translated into suboptimal care of patients with chronic neurological diseases. While some patients might benefit from expanding telemedicine, monitoring neurological disability via telemedicine is challenging. Smartphone technologies represent an attractive tool for remote, self-administered neurological assessment. To address this need, we have developed a suite of smartphone tests, called neurological functional test suite (NeuFun-TS), designed to replicate traditional neurological examination. The aim of this study was to assess the ability of two NeuFun-TS tests—short walk and foot tapping—to quantify motor functions of lower extremities as assessed by a neurologist.MethodsA cohort of 108 multiple sclerosis (MS) patients received a full neurological examination, imaging of the brain, and completed the NeuFun-TS smartphone tests. The neurological exam was digitalized using the NeurEx™ platform, providing calculation of traditional disability scales, as well as quantification of lower extremities-specific disability. We assessed unilateral correlations of 28 digital biomarkers generated from the NeuFun-TS tests with disability and MRI outcomes and developed machine learning models that predict physical disability. Model performance was tested in an independent validation cohort.ResultsNeuFun-TS-derived digital biomarkers correlated strongly with traditional outcomes related to gait and lower extremities functions (e.g., Spearman ρ > 0.8). As expected, the correlation with global disability outcomes was weaker, but still highly significant (e.g., ρ 0.46–0.65; p < 0.001 for EDSS). Digital biomarkers also correlated with semi-quantitative imaging outcomes capturing locations that can affect lower extremity functions (e.g., ρ ~ 0.4 for atrophy of medulla). Reliable digital outcomes with high test-retest values showed stronger correlation with disability outcomes. Combining strong, reliable digital features using machine learning resulted in models that outperformed predictive power of best individual digital biomarkers in an independent validation cohort.DiscussionNeuFun-TS tests provide reliable digital biomarkers of lower extremity motor functions.

Multi-strategy cooperative enhancement dung beetle optimizer and its application in obstacle avoidance navigation.

Path planning is a crucial component of unmanned driving systems designed to address the challenge of autonomous navigation. This article introduces a novel approach called the multi-strategy cooperative enhanced dung beetle optimization algorithm (RCDBO) and applies it to dispose of path planning issues. Its primary aim is to mitigate the shortcomings of the fundamental dung beetle optimization algorithm (DBO), namely, its tendency to prematurely converge to local optima and its limited global planning capability. Initially, population initialization leverages Bernoulli-based chaotic mapping to enhance diversity and ensure uniform randomness, thereby improving both the quality of initial solutions and optimization efficiency. Subsequently, a random walk strategy is employed to perturb the rolling behavior of the dung beetle population during the initial stage, thus mitigating potential algorithmic local stagnation. Finally, adopting a vertical and horizontal crossover strategy introduces perturbations to the current best value of the dung beetle population, thereby enhancing the DBO method's global optimization capability during the later stages of evolution. The RCDBO method was evaluated against several well-established swarm intelligence algorithms using twelve benchmark test functions, the CEC2021 test suite, the Wilcoxon rank-sum test, and the Friedman test, to rigorously assess its feasibility. Additionally, the effectiveness of each strategy within the RCDBO framework was validated, and the algorithm's capability to achieve an optimal balance between global exploration and local exploitation was systematically analyzed through the CEC2021 test suite. Furthermore, in path planning simulation experiments targeting maps of varying sizes and complexities, the RCDBO algorithm, compared to the basic DBO algorithm, results in shorter planned path lengths. In simpler map environments, it exhibits fewer turning points and smoother paths, indicating that the RCDBO algorithm also possesses certain advantages in addressing path-planning challenges.

An Empirical Study on the Suitability of Test-based Patch Acceptance Criteria

In this article, we empirically study the suitability of tests as acceptance criteria for automated program fixes, by checking patches produced by automated repair tools using a bug-finding tool, as opposed to previous works that used tests or manual inspections. We develop a number of experiments in which faulty programs from IntroClass , a known benchmark for program repair techniques, are fed to the program repair tools GenProg, Angelix, AutoFix and Nopol, using test suites of varying quality, including those accompanying the benchmark. We then check the produced patches against formal specifications using a bug-finding tool. Our results show that, in the studied scenarios, automated program repair tools are significantly more likely to accept a spurious program fix than producing an actual one. Using bounded-exhaustive suites larger than the originally given ones (with about 100 and 1,000 tests) we verify that overfitting is reduced but a) few new correct repairs are generated and b) some tools see their performance reduced by the larger suites and fewer correct repairs are produced. Finally, by comparing with previous work, we show that overfitting is underestimated in semantics-based tools and that patches not discarded using held-out tests may be discarded using a bug-finding tool.

Patch Correctness Assessment: A Survey

Most automated program repair methods rely on test cases to determine the correctness of the generated patches. However, due to the incompleteness of available test suites, some patches that pass all the test cases may still be incorrect. This issue is known as the patch overfitting problem. Overfitting problem is a longstanding problem in automated program repair. Due to overfitting patches, the patches obtained by automated program repair tools require further validation to determine their correctness. Researchers have proposed many methods to automatically assess the correctness of patches, but no systematic review provides a detailed introduction to this problem, the existing solutions, and the challenges. To address this deficiency, we systematically review the existing approaches to patch correctness assessment. We first offer a few examples of overfitting patches to acquire a more detailed understanding of this problem. We then propose a comprehensive categorization of publicly available techniques and datasets, examine the commonly used evaluation metrics, and perform an in-depth analysis of the effectiveness of the existing models in addressing the challenge of overfitting. Based on our analysis, we provided the difficulties encountered by current methodologies, alongside the possible avenues for future research exploration.

Transferring knowledge by budget online learning for multiobjective multitasking optimization

Multiobjective multitasking optimization (MO-MTO) has attracted increasing attention in the evolutionary computation field. Evolutionary multitasking (EMT) algorithms can improve the overall performance of multiple multiobjective optimization tasks through transferring knowledge among tasks. Negative transfer resulting from the indeterminacy of the transferred knowledge may bring about the degradation of the algorithm performance. Identifying the valuable knowledge to transfer by learning the historical samples is a feasible way to reduce negative transfer. Taking this into account, this paper proposes a budget online learning based EMT algorithm for MO-MTO problems. Specifically, by regarding the historical transferred solutions as samples, a classifier would be trained to identified the valuable knowledge. The solutions which are considered containing valuable knowledge will have more opportunity to be transfer. For the samples arrive in the form of streaming data, the classifier would be updated in a budget online learning way during the evolution process to address the concept drift problem. Furthermore, the exceptional case that the classifier fails to identify the valuable knowledge is considered. Experimental results on two MO-MTO test suits show that the proposed algorithm achieves highly competitive performance compared with several traditional and state-of-the-art EMT methods.

Testing ADAS/ADS – from critical scenarios to automated testing oracles

AbstractAutomated and autonomous driving systems are increasingly integrated into modern vehicles to support the vision of safe, efficient, and comfortable transportation. Given the complexity of these systems, thorough testing and close monitoring of their behavior is inevitable to prevent hazardous situations and unexpected behaviors. This paper presents different approaches for testing ADAS/ADS, focusing on generating critical scenarios for virtual validation and monitoring approaches applied during operation. Specifically, we provide a comprehensive overview of our previous work on combinatorial and search-based testing methodologies, highlighting their application in generating robust test suites. Additionally, we summarize our work on intelligent monitoring approaches to detect operational issues. Our findings emphasize the necessity of advanced testing solutions and continuous monitoring to identify and mitigate potential failures, demonstrating their applicability in enhancing the safety and trustworthiness of ADAS/AD.

Mechanical and Civil Engineering Optimization with a Very Simple Hybrid Grey Wolf—JAYA Metaheuristic Optimizer

Metaheuristic algorithms (MAs) now are the standard in engineering optimization. Progress in computing power has favored the development of new MAs and improved versions of existing methods and hybrid MAs. However, most MAs (especially hybrid algorithms) have very complicated formulations. The present study demonstrated that it is possible to build a very simple hybrid metaheuristic algorithm combining basic versions of classical MAs, and including very simple modifications in the optimization formulation to maximize computational efficiency. The very simple hybrid metaheuristic algorithm (SHGWJA) developed here combines two classical optimization methods, namely the grey wolf optimizer (GWO) and JAYA, that are widely used in engineering problems and continue to attract the attention of the scientific community. SHGWJA overcame the limitations of GWO and JAYA in the exploitation phase using simple elitist strategies. The proposed SHGWJA was tested very successfully in seven “real-world” engineering optimization problems taken from various fields, such as civil engineering, aeronautical engineering, mechanical engineering (included in the CEC 2020 test suite on real-world constrained optimization problems) and robotics; these problems include up to 14 optimization variables and 721 nonlinear constraints. Two representative mathematical optimization problems (i.e., Rosenbrock and Rastrigin functions) including up to 1000 variables were also solved. Remarkably, SHGWJA always outperformed or was very competitive with other state-of-the-art MAs, including CEC competition winners and high-performance methods in all test cases. In fact, SHGWJA always found the global optimum or a best cost at most 0.0121% larger than the target optimum. Furthermore, SHGWJA was very robust: (i) in most cases, SHGWJA obtained a 0 or near-0 standard deviation and all optimization runs practically converged to the target optimum solution; (ii) standard deviation on optimized cost was at most 0.0876% of the best design; (iii) the standard deviation on function evaluations was at most 35% of the average computational cost. Last, SHGWJA always ranked 1st or 2nd for average computational speed and its fastest optimization runs outperformed or were highly competitive with their counterpart recorded for the best MAs.

Expanded Functionality and Portability for the Colvars Library.

Colvars is an open-source C++ library that provides a modular toolkit for collective-variable-based molecular simulations. It allows practitioners to easily create and implement descriptors that best fit a process of interest and to apply a wide range of biasing algorithms in collective variable space. This paper reviews several features and improvements to Colvars that were added since its original introduction. Special attention is given to contributions that significantly expanded the capabilities of this software or its distribution with major MD simulation packages. Collective variables can now be optimized either manually or by machine-learning methods, and the space of descriptors can be explored interactively using the graphical interface included in VMD. Beyond the spatial coordinates of individual molecules, Colvars can now apply biasing forces to mesoscale structures and alchemical degrees of freedom and perform simulations guided by experimental data within ensemble averages or probability distributions. It also features advanced computational schemes to boost the accuracy, robustness, and general applicability of simulation methods, including extended-system and multiple-walker adaptive biasing force, boundary conditions for metadynamics, replica exchange with biasing potentials, and adiabatic bias molecular dynamics. The library is made available directly within the main distributions of the academic software GROMACS, LAMMPS, NAMD, Tinker-HP, and VMD. The robustness of the software and the reliability of the results are ensured through the use of continuous integration with a test suite within the source repository.

A Multimodal Multi-Objective Coati Optimization Algorithm Based on Spectral Clustering

Multimodal multi-objective problems refer to situations where more than one Pareto set in the decision space corresponds to the same Pareto front in the objective space. In this paper, we propose a multimodal multi-objective coati optimization algorithm based on spectral clustering (MMOCOA-SC) for use in multimodal multi-objective problems. The algorithm starts by creating stable subpopulations via a spectral clustering method, grouping similar individuals within each subpopulation. Then, an improved coati optimization algorithm is proposed in order to increase the diversity of the population and continuously search for near-Pareto optimal solutions. Finally, we use an improved crowding distance method combined with a non-dominated sorting method to maintain and retain multiple near-Pareto optimal solutions. The MMOCOA-SC is evaluated alongside five state-of-the-art algorithms using the 2020 CEC test suite, IDMP test problems, and twelve classic engineering application problems. In comparison to the other five state-of-the-art algorithms, the experimental results show that MMOCOA-SC exhibits a superior performance when applied to the problems. This study explores multimodal multi-objective optimization algorithms with a focus on the concept of symmetry, which is crucial for multimodal multi-objective optimization problems in terms of achieving a balanced decision space and a diversity of solutions in the objective space.

Combinatorial transition testing in dynamically adaptive systems: Implementation and test oracle

Due to the large number of possible interactions and transitions among features in dynamically adaptive systems, testing such systems poses significant challenges. To verify that such systems behave correctly, combinatorial interaction testing (CIT) can create concise test suites covering all valid pairs of features of such systems. While CIT claims to find all errors caused by two features, it does not cover certain errors occurring only for specific transitions between features. To address this issue we study the technique of Combinatorial Transition Testing (CTT), which includes both generation and detection of what we call behavioural transition errors. From an initial generation algorithm that combines both interaction and transition coverage but lacks scalability, we propose an optimised version that enables CTT even for hundreds of features. From a valid test suite covering all transitions, we complete our testing approach with a test oracle that detects all behavioural transition errors without any prior knowledge of the system’s behaviour. After a comprehensive analysis over a large number of feature models, we conclude that size of CTT-generated test suites and test effort needed to use our test oracle are linearly correlated to CIT-generated ones and that CTT grows logarithmically in the number of features.

Dual-path differential perturbation sand cat swarm optimization algorithm integrated with escape mechanism

In this paper, based on the sand cat swarm optimization (SCSO) algorithm, a dual-path differential perturbation sand cat swarm optimization algorithm integrated with escape mechanism (EDSCSO) is proposed. EDSCSO aims to solve the problems of the original SCSO, such as the limited diversity of the population, low efficiency of solving complex functions, and ease of falling into a local optimal solution. First, an escape mechanism was proposed to balance the exploration and exploitation of the algorithm. Second, a random elite cooperative guidance strategy was used to utilize the elite population to guide the general population to improve the convergence speed of the algorithm. Finally, the dual-path differential perturbation strategy is used to continuously perturb the population using two differential variational operators to enrich population diversity. EDSCSO obtained the best average fitness for 27 of 39 test functions in the IEEE CEC2017 and IEEE CEC2019 test suites, indicating that the algorithm is an efficient and feasible solution for complex optimization problems. In addition, EDSCSO is applied to optimize the three-dimensional wireless sensor network coverage as well as the unmanned aerial vehicle path planning problem, and it provides optimal solutions for both problems. The applicability of EDSCSO in real-world optimization scenarios was verified.

Differential evolution with multi-strategies for UAV trajectory planning and point cloud registration

The present study introduces a novel adaptive algorithm, MELSHADE-cnEpSin, which aims to enhance the performance of LSHADE-cnEpSin, which is not only stands out as one of the most competitive versions of differential evolution but also holds the distinction of being one of the CEC winner algorithms. Compared to the original methodology, four main distinctions are presented. To begin with, we adopt an adaptive selection mechanism (ASM) of crossover rate Cr value based on the external archive to rechoose a suitable value. In the next place, a nonlinear population reduction strategy using Sigmoid function is employed to improve population distribution. Additionally, a restart strategy is implemented to mitigate the risk of algorithmic convergence towards suboptimal solutions. Furthermore, the performance of MELSHADE-cnEpSin was evaluated using standard CEC2017 and CEC2022 test suites in conjunction with nine CEC-winning algorithms (L-SHADE, EBOwithCMAR, AGSK, LSHADE-SPACMA, LSHADE-cnEpSin, ELSHADE-SPACMA, EA4eig, MadDE and APGSK-IMODE) as well as two novel algorithms (ACD-DE and MIDE). Furthermore, MELSHADE-cnEpSin was effectively employed to address the challenge of UAV trajectory planning in intricate mountainous terrain and underwent simulation with point cloud registration cases utilizing a rapid global registration dataset, thereby showcasing the potential of MELSHADE-cnEpSin in tackling real-world optimization problems.

Artificial Optimizer Algorithm for Power System Stabilizer design problem and multidisciplinary engineering applications

The novelty of this research lies in presenting a fresh stochastic algorithm enthused via ‘Velociraptor’ social intelligence in wildlife (or nature), known as the Velociraptor Group Optimization algorithm (VROA). In this strategy, ‘Velociraptor’ co-operative natural life cycle is mathematically framed, and novel mechanisms are presented to perform search (or exploration) and hunting (or exploitation). This suggests that the proposed method reveals a noteworthy ability in both exploitation and exploration. Furthermore, it successfully stabilities exploration and exploitation, supporting the search process. In the direction of evaluating the VROA, we utilized it on 51 CEC'17, CEC'20, and CEC'22 standard benchmark suites and six multidisciplinary engineering optimization functions. Furthermore, well-known statistical methods like Wilcoxon rank-sum test and Friedman's test have been used to verify the strength of the proposed algorithm against various optimizers. The tabulated numerical and statistical solutions show that VROA performs better than recent optimizers on most standard test suites and has efficiently attained the most competent resolution while concurrently upholding adherence to the designated constraints. The results validate that VROA can offer efficient and accurate optimal solutions in evaluation with recent metaheuristics.

LTM: Scalable and Black-Box Similarity-Based Test Suite Minimization Based on Language Models

LTM: Scalable and Black-Box Similarity-Based Test Suite Minimization Based on Language Models

Influence of graphene nanoplatelets on the passivation and corrosion resistance of carbon steel bars in ordinary portland cement mortar

This study investigates the influence of adding graphene nanoplatelets (GNPs) to mortar mixtures on the corrosion behavior of embedded carbon steel rebars. Mortar mixtures were prepared with a water-to-cement ratio of 0.42 and GNP replacement levels of 0, 0.05, and 1 percent by weight of cement. After 28 days of curing in an environmental chamber, the passivity and kinetics of chloride induced corrosion initiation for the rebars were studied using a suite of electrochemical tests. The results suggest that incorporating GNPs at low concentrations may enhance the passivity and resistance to chloride-induced corrosion of carbon steel rebars; however, due to the large variability in the data, the findings are inconclusive. Potentiodynamic measurements revealed that the estimated average polarization resistance of steel rebars in 0 G (no GNP), 0.05 G, and 1 G specimens in the passive state was on average −182.3, −276.5, and −304.2 kΩ, respectively. Upon exposure to chloride solution, 0 G specimens, with an average corrosion potential of −600 mV vs. SCE, exhibited the lowest resistance, whereas 0.05 G specimens, with an average of −491 mV vs. SCE, showed the highest corrosion resistance. Results from electrical impedance spectroscopy were consistent with the potentiodynamic findings. Surface characterization also revealed that the inclusion of GNPs does not alter the surface roughness of the interface between steel rebar and paste. However, the interface between rebar ribs and mortar paste had higher surface roughness and porosity in all specimens regardless of GNPs replacement level. Visual inspections confirmed that the area around the rebar ribs at the interface is more conducive to the formation of crevice and pitting corrosion.

Dual-tiered insights: cross-examining entities in free text electronic health records

Abstract Background Poor determination and processing of actions at the primary care level are found to be associated with unstructured and cluttered electronic health records (EHRs). Physicians recommend that EHRs must highlight entities, such as action items, findings, follow-ups, and reasons for medication changes. There’s consensus around the success of large language models (LLMs) in detecting clinical and non-clinical entities in EHRs. However, end-to-end systems make limited efforts to go beyond entity and relationship detection to support clinical decision-making. Methods We propose a two-tier system that generates case-based comparative reports for entities highlighted by physicians in EHRs. First, we fine-tuned a custom Bidirectional Encoder Representations from Transformers (BERT) model that detects the required entities within the EHRs. Second, a clustering-based service generates a report comparing the detected entities of a specific EHR with other EHRs sharing the same conditions determined by the healthcare professional. Results The system leverages the MIMIC IV deidentified free-text clinical notes dataset. The first-tier successfully classified entities with a macro-score average range of [0.87 - 0.94]. For the second tier, preliminary simulations on a set of test samples were made to cluster the entities. The simulations leverage a custom statistical tests engine offering a suite of entity-type specific tests. The results of these tests are used to generate the case-based detected entities comparative report. Conclusions Highlighting clinical and non-clinical entities bridges the communication gap between primary and secondary care. Clustering these entities extends the capability of LLMs in clinical support decision systems by providing healthcare professionals with the ability to run comparative analyses between EHRs. The comparative analysis reports provide a factual comparison between a selected number of detected entities and their thresholds. Key messages • Clustering clinical and non-clinical entities extends the capacity of named entity recognition in clinical decision-making systems. • Ability to generate a comparative analysis report based on figures extracted from recognised entities in free-text electronic health records.

DeLoSo: Detecting Logic Synthesis Optimization Faults Based on Configuration Diversity

Logic synthesis tools are the core components of digital circuit design, which convert programs written in hardware description languages into gate-level netlists, and optimize the netlists. However, the netlist optimization is complex, with numerous optimization parameters to be configured. Any minor optimization faults in logic synthesis tools may cause circuit diagrams to significantly deviate from the original design, posing risks in target systems. We propose DeLoSo, the De tector of Lo gic S ynthesis o ptimization faults, the first method specifically designed to identify potential faults in the optimization processes of logic synthesis tools. DeLoSo relies on netlist differences and parameter variations to guide the generation of diverse Logic Synthesis Optimization Configuration (LSOC) combinations to thoroughly test the optimization process. DeLoSo consists of three components: LSOC generator, which generates diverse LSOC combinations through configuration recombination and mutation; LSOC diversity evaluator, which assesses the diversity of optimization configurations; and LSOC validator, which validates the generated LSOC combinations to discover optimization faults. DeLoSo identified 19 faults in two established logic synthesis tools (i.e., Vivado and Yosys); 15 of them have been fixed by vendors. Particularly, the community maintainers of Yosys have considered incorporating DeLoSo into Yosys’ existing test suite.

Algorithm XXXX: UltimateKalman, Flexible Kalman Filtering and Smoothing Using Orthogonal Transformations

UltimateKalman is a flexible linear Kalman filter and smoother implemented in three popular programming languages: MATLAB, C, and Java. UltimateKalman is a slight simplification and slight generalization of an elegant Kalman filter and smoother that was proposed in 1977 by Paige and Saunders. Their algorithm appears to be numerically superior and more flexible than other Kalman filters and smoothers, but curiously has never been implemented or used before. UltimateKalman is flexible: it can easily handle time-dependent problems, problems with state vectors whose dimensions vary from step to step, problems with varying number of observations in different steps (or no observations at all in some steps), and problems in which the expectation of the initial state is unknown. The programming interface of UltimateKalman is broken into simple building blocks that can be used to construct filters, single or multi-step predictors, multi-step or whole-track smoothers, and combinations. The paper describes the algorithm and its implementation as well as a test suite of examples and tests.

Suppressing the gauge problem in local hybrid functionals without a calibration function: The choice of local mixing function.

Modern functionals based on the exact-exchange (EXX) energy density like local hybrid functionals (LHs) or range-separated LHs have recently received additional attention due to their advantages over established functionals when it comes to the local balance between self-interaction errors and static-correlation errors. A possible theoretical drawback of such functionals over the years has been the so-called gauge problem due to the inherent ambiguity of exchange-energy densities. Modern LHs like LH20t or more sophisticated functionals based thereon have been constructed using suitably optimized calibration functions (CFs) to minimize the mismatch of the semi-local and EXX energy densities. Here, we show that the unphysical contributions arising from the gauge problem may also be reduced significantly without a CF by tailoring the position-dependence of the EXX admixture (local mixing function, LMF) in a way to suppress spurious positive energy-density contributions locally in space. This is achieved by building the so-called x-LMFs upon the ratio between EXX and semi-local exchange-energy densities. The resulting LH24x functional provides similar accuracy, e.g., for the GMTKN55 test suite, as LH20t, but without introduction of a CF! We provide detailed comparative analyses of integrated energies and spatially resolved energy densities. The good performances of LHs for chemically relevant energy differences are to some extent due to the core nature of unphysical artifacts that cancel out efficiently.

Understanding Test Convention Consistency as a Dimension of Test Quality

Unit tests must be readable to help developers understand and evolve production code. Most existing test quality metrics assess test code's ability to detect bugs. Few metrics focus on test code's readability. One standard approach to improve readability is the consistent application of conventions. We investigated test convention consistency as a dimension of test quality. We formalized test suite consistency as the extent to which alternatives are used within a code base and introduce two complementary metrics to capture this extent. We elaborated a catalog of over 30 test conventions for the Java language organized in 10 convention classes that group mutual alternatives. We developed tool support to detect occurrences of conventions, compute consistency metrics over a test suite, and view occurrences of conventions in the corresponding code. We applied our tools to study the consistency of the test suites of 20 large open-source Java projects. The study validates the design of the test convention classes, provides descriptive statistics on the range of consistency values for ten different convention classes, and enables us to link observed changes in consistency values to specific events in the change history of our target systems, thus providing evidence of the construct validity of the metrics. We conclude that analyzing test suite consistency via static analysis shows promise as a practical approach to help improve test suite quality.