System Test Cases Research Articles

Second-order optimization strategies for neural network quantum states.

The Variational Monte Carlo (VMC) method has recently seen important advances through the use of neural network quantum states. While more and more sophisticated ansatze have been designed to tackle a wide variety of quantum many-body problems, modest progress has been made on the associated optimization algorithms. In this work, we revisit the Kronecker-Factored Approximate Curvature (KFAC), an optimizer that has been used extensively in a variety of simulations. We suggest improvements in the scaling and the direction of this optimizer and find that they substantially increase its performance at a negligible additional cost. We also reformulate the VMC approach in a game theory framework, to propose a novel optimizer based on decision geometry. We find that on a practical test case for continuous systems, this new optimizer consistently outperforms any of the KFAC improvements in terms of stability, accuracy and speed of convergence. Beyond VMC, the versatility of this approach suggests that decision geometry could provide a solid foundation for accelerating a broad class of machine learning algorithms. This article is part of the theme issue 'The liminal position of Nuclear Physics: from hadrons to neutron stars'.

Optimization methods to solve the set covering problems

Phasor Measurement Unit Placement aims to decrease the cost this phasor equipment while maintaining the observability of the electrical distribution systems. This placement can be modeled as a set covering problem. Synchronized phasor data provides electrical systems’ information with higher quality and frequency than Remote Terminal Units. These data enable the implementation of faster than real-time state estimators. Thus, supervisory control and data acquisition systems can be replaced by automatically monitored systems that add reliability, efficiency, and resilience to smart grid systems. Due to the flexibility and large-scale in the distribution systems, heuristic methods have frequently been applied to this placement problem. Since the use of Chemical Reaction Optimization (CRO) algorithm is not recurrent, we have decided to implement it together with a topological system observability analysis. Initial population creation, parameterization, elementary reactions of CRO, and local search strategies are presented in this study. Computational results were performed in IEEE power system test cases. The optimization method proposed has obtained optimal results at a lower computation time than the literature. To test in large-scale, we have applied another optimization method to Set Covering Problems from Beasley library, with different parameters. However, the results were not satisfactory indicating that the CRO might not be suitable to some large-scale instances of the Set Covering Problems this library. In this case, further studies are required.

Requirement based Test Case Prioritization for System Testing

System Testing encompasses a large number of test cases, which may not be able to get executed due to constrained time, budget and limitation of the resources. Therefore, the test cases must be prioritized in some order such that the critical and most required functionality can be tested early. In this paper, a hierarchical approach for system test case prioritization based on requirements has been proposed that maps requirements on the system test cases. This approach analyzes and assigns value to each requirement based on a comprehensive set of twelve factors thereby prioritizing the requirements. Further, the prioritized requirement is mapped on the highly relevant module and then prioritized set of test cases. To analyze the effectiveness of this approach, a case study of income tax calculator software [1] has been taken. The existing as well as the proposed approach were applied and analyzed on this software. The results show the efficacy of the proposed approach in terms of fault detection and severity early.

Generic and industrial scale many-criteria regression test selection

While several test case selection algorithms (heuristic and optimal) and formulations (linear and non-linear) have been proposed, no multi-criteria framework enables Pareto search — the state-of-the-art approach of doing multi-criteria optimization. Therefore, we introduce the highly parallelizable, openly available Many-Criteria Test-Optimization Algorithm (MC-TOA) framework that combines heuristic Pareto search and optimality gap knowledge per criterion. MC-TOA is largely agnostic to the criteria formulations and can incorporate many criteria where existing approaches offer limited scope (single or few objectives/constraints), lack flexibility in the expression and assurance of constraints, or run into problem complexity issues. For two large-scale systems with up to six criteria and thousands of system test cases, MC-TOA not only produces, over the board, superior Pareto fronts in terms of HVI score compared to the state-of-the-art many-objective heuristic baseline, it also does that within minutes of runtime for worst-case executions, i.e., assuming that a regression affects the entire test-suite. MC-TOA depends on convex solvers. We find that the evaluated open-source solvers are slower but suffice for smaller systems, while being less robust for larger systems. Linear formulations execute faster and obtain near-optimal results, which led to faster and better overall convergence of MC-TOA compared to integer formulations.Editor’s note: Open Science material was validated by the Journal of Systems and Software Open Science Board.

Security Enhancement of Power System State Estimation With an Effective and Low-Cost Moving Target Defense

Moving target defense (MTD) is a new defensive mechanism developed in power systems to thwart false data injection attacks (FDIAs). However, since the MTD works by perturbing the branch parameters with the distributed flexible ac transmission system (D-FACTS), it might cause additional infrastructure and operation costs and affect the system dynamics. This is a complicated problem because it is closely related to which branches should be perturbed and how much they are changed. In this article, we analyze the essentials of MTD and construct an effective and low-cost MTD. To begin with, we provide a sufficient and necessary condition for MTD to protect a bus from being affected by the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">intended FDIA</i> . Based on this result, we propose a new metric to quantify the protection level of MTD and an efficient algorithm to minimize the number of required D-FACTS devices for protecting a specific set of buses. To reduce the operation cost, we develop two strategies to make the increasing operation cost zero for activating the MTD. Furthermore, we analyze the impact of MTD on the system dynamics with a special emphasize on small signal stability. Finally, we conduct extensive simulations to validate our findings with the test cases of power systems in MATPOWER.

Instance Space Analysis of Search-Based Software Testing

Search-based software testing (SBST) is now a mature area, with numerous techniques developed to tackle the challenging task of software testing. SBST techniques have shown promising results and have been successfully applied in the industry to automatically generate test cases for large and complex software systems. Their effectiveness, however, has been shown to be problem dependent. In this paper, we revisit the problem of objective performance evaluation of SBST techniques in light of recent methodological advances - in the form of Instance Space Analysis (ISA) - enabling the strengths and weaknesses of SBST techniques to be visualised and assessed across the broadest possible space of problem instances (software classes) from common benchmark datasets. We identify features of SBST problems that explain why a particular instance is hard for an SBST technique, reveal areas of hard and easy problems in the instance space of existing benchmark datasets, and identify the strengths and weaknesses of state-of-the-art SBST techniques. In addition, we examine the diversity and quality of common benchmark datasets used in experimental evaluations.

A non-intrusive probabilistic multi-energy flow calculation method and its application in operation risk analysis of integrated energy systems

With the deep coupling of electricity, heat, and gas systems, the uncertainties in renewable energy sources and loads significantly impact the energy flow distribution of integrated energy systems. Probabilistic multi-energy flow calculations considering these uncertain factors have become essential for risk analysis, optimal management, and operational control. However, it is still difficult to efficiently and accurately deal with the diverse and large numbers of correlated random variables. This paper proposes a non-intrusive probabilistic multi-energy flow calculation method and explores its application in the operation risk analysis of integrated energy systems. The probabilistic multi-energy flow model is established considering the uncertainties and correlations of renewable energy sources and loads. The proposed model is solved within the sparse polynomial chaos expansion framework based on Bayesian compressive sensing. Thus, the probabilistic density functions of the risk indices of each subsystem can be obtained. On this basis, the conditional value-at-risk method is employed for the operation risk analysis. The feasibility and advantages of the proposed method are verified using a typical integrated energy system test case.

Optimal Investment Planning of Bankable Multi-Carrier Microgrid Networks

Multi-carrier microgrids are increasingly recognized to play a critical role in capturing the synergies between various vectors of energy, including electricity, gas, and heating/cooling systems, thereby providing an effective platform for handling the complexities associated with the dynamic interactions of the system components. While a recent, growing body of the literature has conclusively shown the effectiveness of multi-carrier microgrids in improving the business cases of localized energy solutions with high penetrations of renewables, the potential benefits associated with interconnecting neighboring multi-carrier microgrids to form networked systems are less well-explored. This is particularly important as networked multi-carrier microgrids capable of trading energy with each other during grid outages can significantly improve the resilience of the aggregated system, whilst at the same time reducing the total discounted costs. In response, this paper introduces an integrated planning framework, formulated as a mixed-integer linear programming problem, for the optimal planning of neighboring multi-carrier microgrids in the presence of demand-side flexibility resources with the aim of minimizing the total discounted cost of the networked microgrids. Based on the numerical evidence obtained from the application of the proposed method to a test-case system, the integrated planning optimization framework's effectiveness in improving the economics and resilience of a cluster of electrically coupled multi-carrier microgrids is validated. More specifically, the numerical simulations confirm the proposed model's effectiveness in a significant reduction of the under-utilized storage and dispatchable generation capacities, as well as the otherwise-curtailed non-dispatchable generations. Accordingly, the findings from this study have important implications for accelerating the transition to locally more reliable, affordable, autonomous microgrids.

Spatial Network Decomposition for Fast and Scalable AC-OPF Learning

This paper proposes a novel machine-learning approach for predicting AC-OPF solutions that features a fast and scalable training. It is motivated by the significant training time needed by existing machine-learning approaches for predicting AC-OPF. The proposed approach is a 2-stage methodology that exploits a spatial decomposition of the power network that is viewed as a set of regions. The first stage learns to predict the flows and voltages on the buses and lines coupling the regions, and the second stage trains, in parallel, the machine-learning models for each region. The predictions can then seed a power flow to eliminate the physical constraint violations, resulting in minor violations only for the operational bound constraints. Experimental results on the French transmission system (up to 6,700 buses) and large test cases from the <monospace xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">pglib</monospace> library (up to 9,000 buses) demonstrate the potential of the approach. Within a short training time, the approach predicts AC-OPF solutions with very high fidelity, producing significant improvements over the state-of-the-art. The proposed approach thus opens the possibility of training machine-learning models quickly to respond to changes in operating conditions.

Intravenous Fluids—A Test Case for Learning Health Systems

Intravenous Fluids—A Test Case for Learning Health Systems

Research on design method of human resource system test case in construction enterprise

Abstract In the process of human resource system test design in construction enterprises, various test design methods are often used, such as equivalence class division, boundary value analysis, causal graph, error inference and so on. How to choose the correct test design method is related to the rationality of test case design and high efficiency of test execution. Therefore, the selection of appropriate test design method plays a vital role in use case design. In this paper, the basic use of equivalence class partition and boundary value analysis and their relationship are briefly described. It is hoped that it can be used for reference in the study of human resource system test cases in construction enterprises.

Resilience-Oriented Planning of Multi-Carrier Microgrids under Cyber-Attacks

Microgrids are inherently subject to a variety of cyber-physical threats due to potential vulnerabilities in their cyber systems. In this context, this paper introduces a cyber-attack-resilient design of a multi-carrier microgrid to avoid the loss of critical loads. The objective of the proposed model is to minimize the total planning cost of multi-carrier microgrids, which incorporates the investment and replacement costs of distributed energy resources, operation and maintenance costs, peak demand charges, emission costs, unserved energy costs, and potential reinforcement costs to handle cyber-physical attacks. Not only is the proposed multi-carrier microgrid planning approach able to determine the optimal size of multi-carrier microgrids, but it also identifies and reinforces the system to handle cyber-physical attacks by serving critical loads. The proposed multi-carrier microgrid planning model is formulated as a mixed-integer programming problem and solved using the GAMS 24.1 software. To evaluate the effectiveness of the proposed integrated resource planning model, it is applied to a real-world industrial park test-case system. Numerical simulations demonstrate the effectiveness of the resilience-oriented multi-carrier microgrid planning model. Importantly, the simulation results indicate the economic viability of multi-carrier microgrids optimized by the proposed model. Also, the model sensitivity of various decision variables has been analyzed.

Strategic Protection Against FDI Attacks With Moving Target Defense in Power Grids

Moving target defense (MTD) is a new defensive strategy protecting the power system state estimation from cyberattacks. Using the distributed flexible ac transmission system (D-FACTS), MTD works by actively perturbing the branch parameters that are needed to construct the false data injection (FDI) attacks. Although there are many pioneer works on MTD, the relationship between the construction of MTD and detection of FDI attacks has not been revealed. In this article, we reveal the correlation between MTD design and FDI detection and optimize MTD’s performance in terms of detecting FDI attacks. We provide a sufficient condition for a specially designed MTD to detect and identify the FDI attack and a necessary condition for general MTDs to protect the state estimates from being independently modified. With the aim to reduce the number of measurements that can be manipulated by the attacker after MTD, we develop a heuristic algorithm to compute a near-optimal solution for the deployment of D-FACTS devices. Moreover, we prove that the coordinated design of consecutive perturbation schemes within an MTD cycle can improve the performance of MTD in terms of detecting FDI attacks. Finally, we conduct extensive simulations with the IEEE power system test cases to validate our findings.

Automatic Generation and Optimization of Combinatorial Test Cases from UML Activity Diagram Using Particle Swarm Optimization

Generation of test cases is one of the essential activities of the software testing process. The process of executing a programme to identify defects to improve the system's quality is known as software testing. Manually writing test cases takes time, effort, and money. On the other hand, generating test cases automatically is the solution to this problem. For this automation process, a model-based test case generation technique would be acceptable. A model is usually required to generate test cases in the model-based testing technique. Nowadays, researchers have relied on the activity diagram to generate test cases. Test cases for combinatorial logic systems are required. Combinatorial testing is essential for producing a small number of test cases and identifying errors occurred by interactions between system input parameters. Information about constraints, parameters and its values are required for generation of test cases. It is difficult to extract information regarding constraints, parameters, its values, and interactions between parameters from an Activity Diagram. A novel approach is proposed to extract this information from an Activity Diagram. The authors created a tool that automatically generates combinatorial test cases using UML Activity Diagrams. The proposed tool has two main parts. First, the combinatorial test design model is developed for extraction of input parameters. Second part is generation of optimized number of combinatorial test cases using Particle Swarm Optimization algorithm. Finally, the authors experimented on a real-world case study namely viz. Railway Reservation using the proposed tool, and it is shown that the proposed tool generated optimum number of combinatorial test cases.

Two-Stage MILP Model for Optimal Skeleton-Network Reconfiguration of Power System for Grid-Resilience Enhancement

Skeleton-network reconfiguration is an important task and plays a most important role during power system restoration (PSR) after a blackout. By determining the skeleton network, the power system can be restored as soon as possible to minimize the burden of network reconfiguration. In this paper, a resilience-based skeleton-network reconfiguration (NR) strategy, i.e., a two-stage mixed-integer linear programming (MILP)–based NR strategy for grid-resilience enhancement, is proposed to minimize the impact of emergency power outages on power systems. The start-up sequence for non-black-start generators (NBSGs) and line energization sequences are determined in the first-stage optimization model. A serial restoration constraint and a transient frequency constraint are considered to achieve the serial restoration scheme of NBSGs. Except for the generator power, all the variables attained during the first stage are assumed fixed in the second stage. The second-stage optimization model determines the optimal skeleton network by determining the target transmission lines and critical load pickup during the NR phase. Furthermore, the sets of metrics (i.e., system flexibility, power loss ratio, and recovery time resilience) are presented to determine the grid operational resilience in the skeleton-network reconfiguration. Then, two stages of skeleton-NR are formulated as MILP, and the branch-and-bound method is presented to solve both models. Finally, simulation studies are performed on the two modified power system test cases and the results validate that the proposed strategy can efficiently determine the robust skeleton network for practical and detailed PSR planning.

A Risk-Averse Approach for Distribution Grid Expansion Planning

Recent episodes of natural disasters have challenged the resilience of power grids. Adequate distribution grid planning that properly captures the risk aversion of the utility system planner is a key factor to increase the flexibility of distribution networks to circumvent these events. In this paper, we propose a methodology to determine the optimal portfolio of investments in lines and storage devices in order to minimize a convex combination between expected value and CVaR of operational costs, including energy not served, while taking into account the multistage nature of the energy storage management within this context. While the expected value of energy not served has been traditionally employed to tackle routine failures, we also minimize the CVaR of energy not served to address high-impact, low-probability (HILP) events. We illustrate the performance of the proposed methodology with a 54-Bus system test case.

Model checking, testing and debugging

This issue contains four papers. The first paper focuses on model checking, the second and third papers focus on testing and the last paper focuses on debugging. The first paper, ‘Model checking C++ programs’ by Felipe R. Monteiro, Mikhail R. Gadelha and Lucas C. Cordeiro, is motivated by memory safety issues and how these have proved to be a source of security vulnerabilities. The authors devised a novel bounded model checking approach. The first step was to encode a number of C++ features in a decidable fragment of first-order logic. SMT solvers were then used to carry out verification. In experiments, the proposed approach was found to outperform state-of-the-art verifiers. The prototype tool also found arithmetic-overflow errors in a commercial application. (Recommended by Professor Pretscher) The second paper, ‘GPU acceleration of finite state machine input execution: Improving scale and performance’, by Vanya Yaneva, Ajitha Rajan and Christophe Dubach looks at the problem of executing a large number of tests on a finite state machine (FSM). The motivation for this work is model validation. The approach devised uses GPUs to allow multiple tests to be run in parallel. The authors built on their previous work, which showed how FSM execution can be performed on a GPU, by addressing a number of limitations. In particular, the authors addressed the data transfer overhead and they also performed experiments with FSMs that were too large to fit into GPU memory. In the experiments, the novel optimisations led to further improvements, with the GPU being over four times faster, on average, than a 16-core CPU. (Recommended by Professor Pretscher) The third paper, ‘Survey on test case generation, selection and prioritization for cyber-physical systems’, by Zahra Sadri-Moshkenani, Justin Bradley and Gregg Rothermel, presents a survey of approaches that generate, select or prioritise test cases for cyber-physical systems. The authors identified 34 related papers (26 papers on test generation, 6 papers on test selection and 7 papers on test prioritisation) and classified them according to 8 properties distilled by the authors from past experience. From the survey results, the authors identified a number of open challenges. To address some of these challenges, existing approaches may be adapted or new approaches may be developed. (Recommended by Professor Phil McMinn) The fourth paper, ‘Effective fault localization and context-aware debugging for concurrent programs’, by Justin Chu, Tingting Yu, Jane Huffman Hayes, Xue Han and Yu Zhao, presents Coadec, an approach for automatically generating interthread control flow paths to diagnose concurrency bugs. Coadec consists of two phases: concurrency fault localization and context-aware debugging. The authors evaluated Coadec on 10 real-world multithreaded Java applications and showed that Coadec outperforms state-of-the-art approaches for localising concurrency faults and that Coadec's context debugging can help developers understand concurrency fault by inspecting a small percentage of code. (Recommended by Marc Roper)

A Rapid Monte Carlo Reliability Evaluation Method for Integrated Energy Systems Based on Transformer

In this paper, a Transformer-based fast Monte Carlo reliability evaluation method for integrated energy systems is proposed. First, the faults of the system components are sampled and the corresponding minimum cut load amounts are calculated to obtain the sample data for training the machine learning model. Then, Transformer is used as a machine learning algorithm for mining the nonlinear mapping relationship between system component faults and the minimum cut load, and the estimation model of the minimum cut load under different faults is trained. Finally, the model is combined with Monte Carlo simulation method to randomly sample component states, and for each state, the minimum cut load amount is directly given by the trained estimation model, thus realizing fast evaluation of integrated energy system fast reliability. The proposed method is applied to the integrated energy system test case, which verifies its effectiveness.

Research on automatic test case generation method of flight control system based on UML state diagram

In order to realize the automatic test of flight control system, the automatic generation method of flight control system test case is studied by using UML state diagram. Using UML extension mechanism, the real-time extension scheme of UML state diagram of flight control system is given. Aiming at the real-time performance of flight control system, the migration equivalence class is constructed by using the time domain equivalence partition method. After the test path is generated according to the corresponding test standard, the test case is generated automatically. Taking a simplified flight control system as an example, the feasibility and practicability of the method are verified by comparing the time of generating test cases with that of generating test cases manually.

An order-adaptive compact approximation Taylor method for systems of conservation laws

We present a new family of high-order shock-capturing finite difference numerical methods for systems of conservation laws. These methods, called Adaptive Compact Approximation Taylor (ACAT) schemes, use centered (2p+1)-point stencils, where p may take values in {1,2,…,P} according to a new family of smoothness indicators in the stencils. The methods are based on a combination of a robust first order scheme and the Compact Approximate Taylor (CAT) methods of order 2p-order, p=1,2,…,P so that they are first order accurate near discontinuities and have order 2p in smooth regions, where (2p+1) is the size of the biggest stencil in which large gradients are not detected. CAT methods, introduced in [3], are an extension to nonlinear problems of the Lax-Wendroff methods in which the Cauchy-Kovalesky (CK) procedure is circumvented following the strategy introduced in [22] that allows one to compute time derivatives in a recursive way using high-order centered differentiation formulas combined with Taylor expansions in time. The expression of ACAT methods for 1D and 2D systems of conservative laws is given and the performance is checked in a number of test cases for several linear and nonlinear systems of conservation laws, including Euler equations for gas dynamics.