Simulation-based Verification Research Articles

Hybrid AI-based 4D trajectory management system for dense low altitude operations and Urban Air Mobility

Urban Air Mobility (UAM) has emerged as a promising solution to address some of the challenges of transportation congestion and associated pollution, especially in large cities. However, the development of drone transportation and UAM services is limited by the capacity of the low altitude airspace where these new vehicles will operate. Without suitable regulatory advancements and associated traffic management systems, air traffic in the densest low-altitude sectors may incur congestion, which, in addition to affecting operational efficiency, can increase systemic risk and fuel emergency occurrences, thereby affecting the safety of people and property in the air and on the ground. To address these challenges, this study aims to develop an intelligent Uncrewed Aircraft Traffic Management (UTM) system that leverages the complementary strengths of metaheuristic and machine learning algorithms for an effective management of dense low altitude airspace. The UTM system determines time-based three-dimensional airspace Demand-Capacity Balancing (DCB) solutions by processing real-time data updates and dynamically replanning flight paths and DCB decisions in any given context, while also providing UAM operators with relevant inputs for autonomous decision-making. Simulation-based verification activities in representative conditions show that the proposed UTM system has the ability to effectively resolve overload instances and minimize potential conflicts in low-altitude airspace, with an operationally acceptable running time. We conclude that the proposed hybrid algorithm can support a successful implementation of UAM services in and around cities, and it has high potential to address critical airspace resource constraints also in traditional Air Traffic Flow Management (ATFM).

Survey of Machine Learning for Software-assisted Hardware Design Verification: Past, Present, and Prospect

With the ever-increasing hardware design complexity comes the realization that efforts required for hardware verification increase at an even faster rate. Driven by the push from the desired verification productivity boost and the pull from leap-ahead capabilities of machine learning (ML), recent years have witnessed the emergence of exploiting ML-based techniques to improve the efficiency of hardware verification. In this paper, we present a panoramic view of how ML-based techniques are embraced in hardware design verification, from formal verification to simulation-based verification, from academia to industry, and from current progress to future prospects. We envision that the adoption of ML-based techniques will pave the road for more scalable, more intelligent, and more productive hardware verification.

Investigation of navigation information correction techniques for master-slave AUV formations in unstable communication environments

Cooperative navigation within master–slave Autonomous Underwater Vehicles (AUVs) necessitates the periodic transmission of high-precision positional data from the master AUV, endowed with advanced navigational instrumentation, to the slave AUV characterized by limited positional accuracy. This data exchange is facilitated through an acoustic sensor mechanism, enabling the quantification of relative spatial separation between the two vehicles. Employing an Extended Kalman Filter (EKF), the positioning error of the slave AUV can be estimated and corrected, enhancing its positional accuracy. However, this collaborative navigation methodology encounters dual challenges. Firstly, the intricate underwater acoustic environment introduces the potential for data loss or outlier measurements in submarine acoustic communication, thereby jeopardizing the efficacy of collaborative navigation. Secondly, the current approach solely rectifies the slave AUV's positional error upon the receipt of data from the master AUV, neglecting the fundamental divergence in positional error that accrues between successive data receptions, stemming from the inherent imprecision in the dead reckoning process. To contend with these complexities, this study presents a novel approach to rectifying navigation data within master–slave AUV formations, particularly within the context of unstable communication conditions. The proposed method harnesses an outlier adaptive discriminant EKF framework, proficiently addressing the slave AUV's positional error while simultaneously estimating and compensating for azimuthal angular errors during underwater acoustic communication between the master and slave AUVs. This method establishes a pivotal groundwork for heightening the accuracy of independent autonomous navigation within the slave AUV during subsequent operational phases. The accuracy and applicability of the proposed method are rigorously validated through simulation-based verification and lake-water field experiments. The empirical results unequivocally highlight the efficacy of the underwater acoustic communication approach put forth in this study, with an outlier recognition rate exceeding 80%. Furthermore, the method adeptly rectifies azimuthal angular errors and positional inaccuracies within the AUV, thus effecting a comprehensive enhancement of overall positioning precision. This research bears significant implications for advancing autonomous underwater navigation, providing a robust framework to bolster the precision and reliability of master–slave AUV formations, particularly under challenging communication circumstances.

Virtual Environment Model Generation for CPS Goal Verification using Imitation Learning

Cyber-Physical Systems (CPS) continuously interact with their physical environments through embedded software controllers that observe the environments and determine actions. Field Operational Tests (FOT) are essential to verify to what extent the CPS under analysis can achieve certain CPS goals, such as satisfying the safety and performance requirements, while interacting with the real operational environment. However, performing many FOTs to obtain statistically significant verification results is challenging due to its high cost and risk in practice. Simulation-based verification can be an alternative to address the challenge, but it still requires an accurate virtual environment model that can replace the real environment interacting with the CPS in a closed loop. In this article, we propose ENVI (ENVironment Imitation), a novel approach to automatically generate an accurate virtual environment model, enabling efficient and accurate simulation-based CPS goal verification in practice.To do this, we first formally define the problem of the virtual environment model generation and solve it by leveraging Imitation Learning (IL), which has been actively studied in machine learning to learn complex behaviors from expert demonstrations. The key idea behind the model generation is to leverage IL for training a model that imitates the interactions between the CPS controller and its real environment as recorded in (possibly very small) FOT logs. We then statistically verify the goal achievement of the CPS by simulating it with the generated model. We empirically evaluate ENVI by applying it to the verification of two popular autonomous driving assistant systems. The results show that ENVI can reduce the cost of CPS goal verification while maintaining its accuracy by generating accurate environment models from only a few FOT logs. The use of IL in virtual environment model generation opens new research directions, further discussed at the end of the article.

Automatic Generation of Scenarios for System-level Simulation-based Verification of Autonomous Driving Systems

Automatic Generation of Scenarios for System-level Simulation-based Verification of Autonomous Driving Systems

Optimizing Highly-Parallel Simulation-Based Verification of Cyber-Physical Systems

Optimizing Highly-Parallel Simulation-Based Verification of Cyber-Physical Systems

Development and Verification of an IoT-enabled Air Quality Monitoring System Based on Petri Nets

An internet of things (IoT) system for monitoring environmental parameters on campus has been successfully developed in this study. Various sensors are adopted in front of the IoT system for detecting air pollutants. The architecture of LongRange in Class C uses low-power and long-distance transmission technologies, which has been set up on the large campus so that the terminal equipment can reach a balance between downlink latency and battery life, making it the best communication protocol. In addition, this monitoring system uses the Petri net software tool to build a correct IoT platform based on the fundamental design process and to demonstrate the feasibility of the IoT system through simulation-based verification. Finally, the experimental results indicate that the IoT system for monitoring the environmental parameters on campus achieves the goal of an acceptable data transmission success rate of more than 95%. Thus, it can facilitate the air quality trends for policy making as well as the hazardous prediction and prevention.

Simulation-based Verification of Autonomous Navigation Systems by Stepwise Scenario Extraction based on Risk Assessment

Simulation-based Verification of Autonomous Navigation Systems by Stepwise Scenario Extraction based on Risk Assessment

SIMULATION-BASED ACCURACY INVESTIGATION OF A PHOTOGRAMMETRIC SETUP TO MEASURE A DYNAMIC PROCESS

Abstract. The verification of a measurement system is an essential part of system development. For this purpose, various guidelines can be used to evaluate and validate photogrammetric systems. However, these guidelines are only designed to validate systems that observe a static scene. Hence, these guidelines cannot validate measurement systems that observe dynamic scenes. In addition, reference data is not available for most systems, making verification significantly more difficult or not a practical solution. In this work, a simulation-based verification approach is presented. The presented approach allows the analysis of complex systems and the investigation of specific processing steps. The approach is based on a Monte Carlo simulation, which only requires the probability density distributions of the input data and synthetic reference data. For this purpose, the probability density distributions of the input data are determined by kernel density estimation to generate realistic input data. The application is a wind tunnel test, where aerodynamic and structural dynamic phenomena are observed at a wind turbine model. The measurement system consists of four high-speed cameras, which acquire the rotor blades' deformations. The objective of the simulation is to evaluate the complete process regarding the accuracy and precision of the measurement system. Experimental data can be used to estimate the quality of the simulation. It was shown that the simulation produces realistic results and that it is suitable for validating dynamic measurement systems. The simulation showed that the precision and accuracy of the system are highly dependent from the estimation of the self-motion. The achieved accuracy is still high and allows the detection of small-scale blade deformations.

Implementing and verifying release-acquire transactional memory in C11

Transactional memory (TM) is an intensively studied synchronisation paradigm with many proposed implementations in software and hardware, and combinations thereof. However, TM under relaxed memory, e.g., C11 (the 2011 C/C++ standard) is still poorly understood, lacking rigorous foundations that support verifiable implementations. This paper addresses this gap by developing TMS2-ra, a relaxed operational TM specification. We integrate TMS2-ra with RC11 (the repaired C11 memory model that disallows load-buffering) to provide a formal semantics for TM libraries and their clients. We develop a logic, TARO, for verifying client programs that use TMS2-ra for synchronisation. We also show how TMS2-ra can be implemented by a C11 library, TML-ra, that uses relaxed and release-acquire atomics, yet guarantees the synchronisation properties required by TMS2-ra. We benchmark TML-ra and show that it outperforms its sequentially consistent counterpart in the STAMP benchmarks. Finally, we use a simulation-based verification technique to prove correctness of TML-ra. Our entire development is supported by the Isabelle/HOL proof assistant.

Verification and validation of digital twins and virtual testbeds

<p>Our society is surrounded by technical systems that increasingly influence and<br />take over aspects of everyone’s everyday life. As we build on the correct functioning, the need for appropriate methodologies that support the development of reliable products persists. However, the development and in particular the qualification become challenging due to their increasing inherent complexity. Virtual testbeds and digital twins are modern key concepts in this context and offer a wide range of opportunities for simulation-based verification and validation processes with virtual prototypes, especially in cases where real tests are hard to perform. However, the validity of test results derived from virtual experiments depends on the quality of the underlying simulation models.<br />Of course, the models themselves must be verified before. But in the context<br />of virtual testbeds and digital twins this becomes challenging since the corresponding models are usually complex themselves. Therefore, we presented an approach to establish structured verification and validation activities for digital twins and virtual testbeds, based on a modular configuration framework that supports the development of complex simulation models.</p>

Simulation-based verification of the shape measurement mechanism of micro structures beyond the diffraction limit using speckle interferometry

A method that can measure three-dimensional shapes beyond the diffraction limit of a lens has been proposed that does not require focusing on an image of the measured object, as in traditional processing. In this method, the shape of the microstructure can be measured by high-resolution detection of the amount of phase change of the speckle pattern with a lateral shift of the object. In this study, the measurement mechanism of the method was investigated using computer simulations. It was found that the measurement is realized by a combination of the perfect optical system and speckle interferometry using the scattered light as the illumination light. In addition, the validity of the simulation results was verified using a real optical system.

A Low Temperature Coolant Conditioning Cart for Testing DEF Thawing Systems

<div class="section abstract"><div class="htmlview paragraph">Diesel Exhaust Fluid (DEF) systems are required to function in cold ambient temperatures below the freezing point of DEF. Manufacturers may demonstrate compliance by following an EPA guidance procedure described below [<span class="xref">2</span>], using whole vehicles at winter test sites at −18 deg C or lower. However, commercial trucks may have multiple variants with different DEF system layouts, so it is impractical to test every possible configuration. A climatic chassis dynamometer (CCD) can also be used for this test, but this is still expensive and time consuming, and does not address the problem of complexity. Instead, much time and expense can be saved by using simulation methods to identify worst case configurations, and to demonstrate with confidence that a limited number of tests will cover the whole possible range. This methodology can further be used to show that a range of vehicles can be represented with selected rig tests in a cold chamber. Such a methodology depends on the ability to generate repeatable test data in a cold room environment. Previously published work [<span class="xref">1</span>] has shown that it is difficult to achieve a repeatable test. In particular, it is challenging to provide coolant with repeatable and accurate temperature and flow profiles, between −40 deg C and +95 deg C. A coolant cart has been developed to provide a stream of conditioned coolant, starting at −40 deg C or higher, utilizing user-specified profiles of temperature and flow rate versus time. The desired profiles were designed to match test data logged from truck engines during warmup from a typical cold overnight temperature of −22 deg C. Surprisingly, no suitable test equipment was found to be commercially available, and the task of designing and building such equipment was found to be challenging. The need to function at −40 deg C restricted the choice of components. The requirement to follow user-defined profiles for both flow and temperature, within tight error bands and following sudden changes, also required unusual care in design. A dedicated coolant cart was therefore commissioned and built. This paper documents the design and operation of the cart. Major component descriptions are provided, with circuit and input/output detail, recharge time and operator interface and processing. The end result is a cart that provides for repeatable cold room testing of tank thaw performance to support a simulation-based verification method, thereby reducing the need for whole-vehicle testing by 75%.</div></div>

전압 안정도 향상을 위한 칼만필터 알고리즘 기반의 비선형 P-Q 드룹 커브 추정

An increase in a distributed generation (DG) causes an excessive voltage problem unless there is proper complement action. The direct voltage control, however, might cause control issues of hunting between adjacent DGs. This paper proposes an indirect voltage control method based on a non-linear P-Q droop curve that is estimated using the Kalman-filter algorithm. For effective estimation, PQV curved surface is estimated, from which the P-Q droop curve is extracted with the condition of zero voltage variation. The effectiveness is verified based on a practical microgrid model composed of real island power system data. For the simulation-based verification, power system computer aided design/electromagnetic transient including DC (PSCAD/EMTDCTM) is used.

Обзор методов функционального онлайн-тестирования микропроцессоров

Online testing is a process of functional verification of microprocessors produced in silicon or their FPGA-prototypes, i.e. post-silicon verification. This type of testing differs both from the manufacturing testing, aimed at checking the workability of manufactured chips (e.g., absence of physical defects, admissibility of physical characteristics) and from simulation-based pre-silicon functional verification of microprocessors models (where internal microprocessor signals are available for observing, and the execution process can be controlled). Post-silicon verification enables to rapidly run huge numbers of tests and detect bugs missed during pre-silicon functional verification. Tests for microprocessors are usually represented by executable programs. Accordingly, the main tasks of online testing are high-performance generation of test programs in the given ISA and creation of a test environment responsible for launching programs, assessing the correctness of their execution by a microprocessor, diagnosing errors, and interacting with the outside world. This paper examines the problems arising in the development of online testing systems (online test program generators), reviews existing solutions in this area, and, on the base on them, proposes a promising approach to organizing online testing.

Real-time automated register abstraction active power-aware electronic system level verification framework

All modern low power system on a chip (SoC) architectures are equipped with an in-built power management system. Every new system is expected to have more features and lower power consumption, resulting in a continuous demand to improve energy efficiency. To cope up with the ever increasing demand, an active power-aware management verification architecture is necessary to minimize the power consumption. Power reduction techniques include clock-gating, power-gating, multi-voltage, and voltage-frequency scaling. The proposed verification architecture utilizes the Unified Power Format (UPF) 2.1 libraries to achieve early design verification at the Electronic System-Level (ESL) of abstraction. The proposed testbench can verify several designs of different power management schemes. The presented work offers a reduction in power states, CPU time and simulation time as compared to existing techniques. The interactive formal and simulation-based verification methods are used in this paper to remove the simulation artifacts during functional and power co-simulation. Additionally, this paper incorporates functional correctness and power-aware checks for different modules of Design Under Verification (DUV) at Transaction-Level Modeling (TLM).

Review on set-theoretic methods for safety verification and control of power system

Increasing penetration of renewable energy introduces significant uncertainty into power systems. Traditional simulation-based verification methods may not be applicable due to the unknown-but-bounded feature of the uncertainty sets. Emerging set-theoretic methods have been intensively investigated to tackle this challenge. The paper comprehensively reviews these methods categorized by underlying mathematical principles, that is, set operation-based methods and passivity-based methods. Set operation-based methods are more computationally efficient, while passivity-based methods provide semi-analytical expression of reachable sets, which can be readily employed for control. Other features between different methods are also discussed and illustrated by numerical examples. A benchmark example is presented and solved by different methods to verify consistency.

DEv-PROMELA: modeling, verification, and validation of a video game by combining model-checking and simulation

Modeling, verifying, and validating are essential steps in order to build systems and software that do what designers expect. If formal verification, and especially model-checking, is a popular method for proving the correctness of properties, its efficiency depends on the accuracy of the used models and the quality of abstractions. As a consequence, applying verification techniques on large-scale complex software like video games is hard without strong assumptions and simplifications. Simulation models are generally more accurate than verification models, but it is often harder to verify them. Combined formalisms that take the benefits of both model-checking and discrete-event simulation represent a good deal between both of these families, although strong engineering expertise remains necessary to define the relevant tests and scenarios. This paper proposes an approach to build this kind of formalism through the example of DEv-PROMELA, which is built by combining Discrete-event System Specification formalism and PROMELA language. Then, it shows how combined formalisms can be used for modeling, verifying, and validating complex software like video games by using both formal-based and simulation-based verification and validation.

Enhanced Design Debugging With Assistance From Guidance-Based Model Checking

Design debugging is one of the most important steps in the modern integrated circuits (ICs) development cycle. Simulation-based verification is never sufficient for ensuring design correctness because of its incomplete nature. Formal techniques such as model checking promise to solve this issue through a complete state-space traversal approach. However, because of increasing design complexity, such methods suffer from scalability issues. Guidance-based state-space traversal techniques have been proposed in the past to assist the model checkers in overcoming the complexity bottleneck. Automatically identifying these guidance hints is relatively difficult and requires heuristic-based reasoning procedures. Additionally, to come up with quick fixes during the debug stage, an effective bug localization strategy is needed. In this article, we revisit the paradigm of guidance-based model checking and propose a methodology to improve these guidance generation mechanisms for achieving fine-grained bug localization. In particular, this work proposes a systematic methodology to localize the buggy RTL lines from the erroneous RTL simulation trace. The proposed technique involves the mining of invariant-like assertions from simulation traces. The mined assertions act as probable guidance candidates for the model checking exercise. To identify useful guidance hints from possible ones, we use the Bayesian networks that explore conditional dependence between the various hints at different levels and the target property. These guidance hints are utilized for obtaining possible buggy subregions, which are analyzed via an iterative model checking methodology for fine-grained bug localization. By using the proposed framework, bugs can be localized to within a few lines of RTL description.

Towards simulation-based verification of autonomous navigation systems

Autonomous ships are expected to change water-based transport of both cargo and people, and large investments are being made internationally. There are many reasons for such transformation and interest, including shifting transport of goods from road to sea, reducing ship manning costs, reduced dangerous exposure for crew, and reduced environmental impact.Situational awareness (SA) systems and Autonomous navigation systems (ANS) are key elements of autonomous ships. Safe deployment of ANS will not be feasible based on real-life testing only, but will require large-scale, systematic simulation-based testing in addition to assurance of the development process.DNV GL proposes to use a digital twin, meaning a digital representation of key elements of the autonomous ship as a key tool for simulation-based testing. The digital twin contains comprehensive mathematical models of the ship and its equipment, including all sensors and actuators. The complete simulation-based test system complementing the digital twin should consist of a virtual world to simulate environmental conditions, geographical information and interaction with other maritime traffic. Finally, the test system must include a test management system that controls simulation of the digital twin and the virtual world, generates test scenarios as well as evaluates the test scenario results. An automatic scenario generation tool should search for low ANS performance, and ultimately establish sufficient coverage of the possible scenario space. The test scenario evaluation should automatically consider safety, conformance to collision regulations at sea (COLREG), and possibly also the efficiency of the ship navigation.This paper presents a comprehensive prototype of a test system for ANS. Key topics are simulation-based testing, interfacing the simulator and ANS, cooperation with ANS manufacturers, dynamic test scenario generation, automatic assessment towards COLREG and experiences from the cooperation with The Norwegian Defense Establishment (FFI).