Test Scenarios Research Articles

Using cellular automata and multi-criteria evaluation to simulate the wildfire expansion in Prince George, British Co-lumbia

Wildfires pose a critical and ongoing challenge in British Columbia, Canada, threatening human life, property, and natural ecosystems. To understand and predict the behavior of such fires, our study employs Cellular Automata (CA), a mathematical model adept at simulating complex systems through grid-based cell interactions. This model, validated by prior research, incorporates a wind propagation rule that significantly enhances the prediction of wildfire spread in the direction of prevailing winds. Research centers on a wildfire event in Prince George, utilizing CA to simulate fire dynamics influenced by var-ious factors. The models strength lies in its ability to represent detailed local interactions and its flexibility in scenario testing, which is instrumental in understanding model uncer-tainties. By simulating different fire scenarios, the study aims to grasp the complexities and potential variables affecting wildfire behavior. The research provides a foundation for decision-makers to analyze and study wildfire events, leveraging a Multi-Criteria Evaluation (MCE) Model to assess the susceptibility of cells to fire. This comprehensive approach combines CA with MCE, offering a robust framework for simulating and manag-ing wildfire expansion in British Columbia.

Bayesian Safety Validation for Failure Probability Estimation of Black-Box Systems

Estimating the probability of failure is an important step in the certification of safety-critical systems. Efficient estimation methods are often needed due to the challenges posed by high-dimensional input spaces, risky test scenarios, and computationally expensive simulators. This work frames the problem of black-box safety validation as a Bayesian optimization problem and introduces a method that iteratively fits a probabilistic surrogate model to efficiently predict failures. The algorithm is designed to search for failures, compute the most-likely failure, and estimate the failure probability over an operating domain using importance sampling. We introduce three acquisition functions that aim to reduce uncertainty by covering the design space, optimize the analytically derived failure boundaries, and sample the predicted failure regions. Results show this Bayesian safety validation approach provides a more accurate estimate of failure probability with orders of magnitude fewer samples and performs well across various safety validation metrics. We demonstrate this approach on three test problems, a stochastic decision-making system, and a neural-network-based runway detection system. This work is open-sourced (https://github.com/sisl/BayesianSafetyValidation.jl) and is currently being used to supplement the FAA certification process of the machine learning components for an autonomous cargo aircraft.

Development of a dynamic traffic microsimulator for on-demand transit operations within an integrated modelling system

ABSTRACT This study develops an intelligent dynamic agent-based microsimulation (iDAMS) module for on-demand transit (ODT) operations within an integrated transport, land-use and energy (iTLE) model. A real-time optimization component within the iDAMS is formulated by the utilization of travelling salesman problem and simulated annealing metaheuristics that perform dynamic passenger-vehicle assignment. It simulates ODT operations for meeting the 24-hour auto-trip demand of Halifax, Canada, to compare the performance of the proposed system with personal cars (PCs). The optimization objectives are to determine the optimal fleet size and seat capacity that satisfies maximum trip requests while minimizing waiting time, travel time and vehicle kilometres travelled (VKT). Simulation results indicate that the ODT system can deliver service similar to PCs in Halifax while decreasing cost and emissions (13% reduction in VKT). The tools developed in this research will provide transit planners ability to conduct ODT scenario simulation and test system performance in real time.

MULTI-BLOCK: A novel ML-based intrusion detection framework for SDN-enabled IoT networks using new pyramidal structure

The ever-expanding Internet of Things (IoT) landscape faces significant security challenges due to limitations in traffic monitoring, device heterogeneity, and weak security practices, leaving networks vulnerable to multi-target and coordinated attacks like large-scale botnets and Distributed Denial-of-Service (DDoS). Existing intrusion detection systems can be computationally expensive and resource-intensive. This can be problematic for resource-constrained IoT devices with limited processing power, memory, and battery life. This paper proposes MULTI-BLOCK, a novel, multi-module framework that leverages machine learning, stateful P4 processing, and a Software-Defined Networking (SDN)-based multi-controller architecture. MULTI-BLOCK tackles critical management tasks within IoT networks, including synchronized communication, traffic monitoring, intrusion detection, and attack mitigation. This comprehensive framework comprises four modules. The first, the proposed pyramidal conceptually decentralized multi-controller structure (PCDMCS), introduces Decentralized Control Interfaces (DCIs) for real-time threat identification through a Decentralized Warning Conduit (DWC). The second module provides comprehensive network monitoring using P4-enabled 24-state tables. The third module leverages 30 new P4-extracted/calculated features and the Enhanced Weighted Ensemble Algorithm (EWEA) for enhanced anomaly detection. The final module presents a novel mitigation approach with 22 steps distributed across multiple controllers for scalability. Extensive evaluation using established IoT datasets (X-IIoTID, TON_IoT, and Edge-IIoTset) and diverse test scenarios (including single-victim and multi-victim attacks) demonstrates MULTI-BLOCK's exceptional performance, achieving high accuracy (99.75 %), precision (99.32 %), F1-score (99.53 %), recall (99.67 %), specificity (99.60 %), low false positive rates (FPR) (0.40 %), and low Average Detection Time (ADT) (2.11 ms). By offering a robust and adaptable solution against evolving threats, MULTI-BLOCK represents a significant advancement in IoT network security.

Secure authentication of identity information for quantum resistant IoT devices based on R-LWE and QIBE

ABSTRACT In order to improve the security of Internet of Things device authentication, a security framework is built in this paper by combining the ring-learning band error problem with quantum indistinguishable bilinear encryption. The research method meticulously designs different test scenarios and conducts multiple rounds of testing to evaluate the accuracy of the system authentication. The results indicated that the authentication accuracy was improved to 93.3% on average after using the proposed algorithm. The authentication speed is significantly improved with a shorter operation cycle. After parameter optimization, repeated tests indicated that the accuracy rate remained stable at over 90%, and the accuracy rate reached 100% in specific phases. Combining the ring learning with error problem technique with quantum indistinguishable bilinear encryption greatly improves the authentication security of Internet of Things devices against quantum computing threats. This discovery provides new strategies for Internet of Things security against quantum attacks and has far-reaching theoretical and practical implications for protecting critical infrastructure.

Improvement of Computer Adaptive Multistage Testing Algorithm Based on Adaptive Genetic Algorithm

Multistage testing (MST) is a portion of computational adaptive testing that adapts assessment structure at the sublevel rather than the component level. The goal of the MST algorithm is to identify bugs in computer programming, and there is a significant cost to utilising MST due to its decreased versatility during software development and maintenance. The efficiency of most algorithms drastically reduces for adaptive MST with complex feasible regions, while some modern algorithms function well while tackling computerised MST with a basic practicable range. The study offers an automated Adaptive Multistage Testing algorithm based on Adaptive Genetic Algorithm (AMST-AGA) for optimisation and scalability problems, in which constraints are successively introduced and dealt with at various evolutionary phases. In this paper, many test cases will aid in finding bugs and meeting completeness goals. Each time test cases are created, these testing scenarios must continue to pass.

Repository of MRI-derived models of the breast with single and multiple benign and malignant tumors for microwave imaging research.

The diagnosis of breast cancer through MicroWave Imaging (MWI) technology has been extensively researched over the past few decades. However, continuous improvements to systems are needed to achieve clinical viability. To this end, the numerical models employed in simulation studies need to be diversified, anatomically accurate, and also representative of the cases in clinical settings. Hence, we have created the first open-access repository of 3D anatomically accurate numerical models of the breast, derived from 3.0T Magnetic Resonance Images (MRI) of benign breast disease and breast cancer patients. The models include normal breast tissues (fat, fibroglandular, skin, and muscle tissues), and benign and cancerous breast tumors. The repository contains easily reconfigurable models which can be tumor-free or contain single or multiple tumors, allowing complex and realistic test scenarios needed for feasibility and performance assessment of MWI devices prior to experimental and clinical testing. It also includes an executable file which enables researchers to generate models incorporating the dielectric properties of breast tissues at a chosen frequency ranging from 3 to 10 GHz, thereby ensuring compatibility with a wide spectrum of research requirements and stages of development for any breast MWI prototype system. Currently, our dataset comprises MRI scans of 55 patients, but new exams will be continuously added.

Generation of nighttime pedestrian fatal precrash scenarios at junctions in Tamil Nadu, India, using cluster correspondence analysis

Objective Modern transportation amenities and lifestyles have changed people’s behavioral patterns while using the road, specifically at nighttime. Pedestrian and driver maneuver behaviors change based on their exposure to the environment. Pedestrians are more vulnerable to fatal injuries at junctions due to increased conflict points with vehicles. Generation of precrash scenarios allows drivers and pedestrians to understand errors on the road during driver maneuvering and pedestrian walking/crossing. This study aims to generate precrash scenarios using comprehensive nighttime fatal pedestrian crashes at junctions in Tamil Nadu, India. Methods Though numerous studies were available on identifying pedestrian crash patterns, only some focused on identifying crash patterns at junctions at night. We used cluster correspondence analysis (CCA) to address this research gap to identify the patterns in nighttime pedestrian fatal crashes at junctions. Further, high-risk precrash scenarios were generated based on the positive residual means available in each cluster. This study used crash data from the Road Accident Database Management System of Tamil Nadu State in India from 2009 to 2018. Characteristics of pedestrians, drivers, vehicles, crashes, light, and roads were input to the CCA to find optimal clusters using the average silhouette width, Calinski-Harabasz measure, and objective values. Results CCA found 4 clusters with 2 dimensions as optimal clusters, with an objective value of 3.3618 and a valence criteria ratio of 80.03%. Results from the analysis distinctly clustered the pedestrian precrash behaviors: Clusters 1 and 2 on pedestrian walking behaviors and clusters 3 and 4 on crossing behaviors. Moreover, a hidden pattern was observed in cluster 4, such as transgender drivers involved in fatal pedestrian crashes at junctions at night. Conclusion The generated precrash scenarios may be used to train drivers (novice and inexperienced for nighttime driving), test scenario creation for developing advanced driver/rider assistance systems, hypothesis creation for researchers, and planning of effective strategic interventions for engineers and policymakers to change pedestrian and driver behaviors toward sustainable safety on Indian roads.

CPR-SLAM: RGB-D SLAM in dynamic environment using sub-point cloud correlations

Abstract The early applications of Visual Simultaneous Localization and Mapping (VSLAM) technology were primarily focused on static environments, relying on the static nature of the environment for map construction and localization. However, in practical applications, we often encounter various dynamic environments, such as city streets, where moving objects are present. These dynamic objects can make it challenging for robots to accurately understand their own position. This paper proposes a real-time localization and mapping method tailored for dynamic environments to effectively deal with the interference of moving objects in such settings. Firstly, depth images are clustered, and they are subdivided into sub-point clouds to obtain clearer local information. Secondly, when processing regular frames, we fully exploit the structural invariance of static sub-point clouds and their relative relationships. Among these, the concept of the sub-point cloud is introduced as novel idea in this paper. By utilizing the results computed based on sub-poses, we can effectively quantify the disparities between regular frames and reference frames. This enables us to accurately detect dynamic areas within the regular frames. Furthermore, by refining the dynamic areas of keyframes using historical observation data, the robustness of the system is further enhanced. We conducted comprehensive experimental evaluations on challenging dynamic sequences from the TUM dataset and compared our approach with state-of-the-art dynamic VSLAM systems. The experimental results demonstrate that our method significantly enhances the accuracy and robustness of pose estimation. Additionally, we validated the effectiveness of the system in dynamic environments through real-world scenario tests.

Managing deepsea oil spills through a systematic modeling approach

Offshore oil exploration and production in deepwater are associated with environmental risks to marine ecosystems. This research introduces DWOSM (Deep Water Oil Spill Model), a three-dimensional Lagrangian model, which is developed to simulate the transport and fate of oil spills resulting from subsea blowouts. DWOSM comprises three interconnected modules: DWOSM-DSD, which predicts the oil droplet size distribution from a blowout release; DWOSM-NearField, simulating plume dynamics and tracking oil droplets within the plume region; and DWOSM-FarField, modeling the evolution of dispersed oil beyond the near-field. Compared to other oil spill models, this integrated approach improves the transition between near and far fields using a near-field particle tracking algorithm. It also employs the thermodynamic models to enable the prediction of oil properties under varying deep water pressure and temperature. To gauge the reliability and efficacy of DWOSM, a hypothetical case situated within a North American context is employed for model testing. The DWOSM and its each module are juxtaposed with other established oil spill models. The outcomes indicate that DWOSM yields comparable results to these models by providing reasonable predictions of a deepwater blowout. The model's verification through case scenario testing and comparison underscores its potential as a decision tool for assessing and managing the potential environmental impacts of offshore petroleum activities.

Designing and Planning of Studies of Driver Behavior at Pedestrian Crossings Using Whole-Vehicle Simulators

The paper presents a proposed methodology for designing and planning research on driver behavior at pedestrian crossings using whole-vehicle simulators. It was assumed that dedicated research should be conducted in specific problem contexts. The problems identified were the identification of hazards and the risk of accidents involving vulnerable road users. The purpose of this identification is to determine the determinants of safety at pedestrian crossings, which should constitute guidance when designing new solutions for safety support systems at pedestrian crossings. A number of hazard factors were identified; divided into categories, including types of crossings, location, and surroundings; behavior of vulnerable road users; and attention (focus) distractors, both inside and outside the vehicle. A method for defining and selecting research scenarios and selecting a group of research participants was proposed. Additionally, it was proposed to conduct repeatable test scenarios for different driving speeds and different weather conditions. With respect to the publications on this topic, this work focuses on the process of designing and planning dedicated simulation studies, which may provide a source of guidance and good practices for other researchers. This is an example of how interdisciplinary research involving human factors, traffic organization, and ITS systems can be planned and implemented.

A Dynamic Programming Approach to the Collision Avoidance of Autonomous Ships

The advancement of autonomous capabilities in maritime navigation has gained significant attention, with a trajectory moving from decision support systems to full autonomy. This push towards autonomy has led to extensive research focusing on collision avoidance, a critical aspect of safe navigation. Among the various possible approaches, dynamic programming is a promising tool for optimizing collision avoidance maneuvers. This paper presents a DP formulation for the collision avoidance of autonomous vessels. We set up the problem framework, formulate it as a multi-stage decision process, define cost functions and constraints focusing on the actual requirements a marine maneuver must comply with, and propose a solution algorithm leveraging parallel computing. Additionally, we present a greedy approximation to reduce algorithm complexity. We put the proposed algorithms to the test in realistic navigation scenarios and also develop an extensive test on a large set of randomly generated scenarios, comparing them with the RRT* algorithm using performance metrics proposed in the literature. The results show the potential benefits of an autonomous navigation or decision support framework.

Robotic Process Automation Efficiency for Mobile App Testing: An Empirical Investigation

In today’s rapidly evolving software environment, the graphical user interface (GUI) plays a crucial role in providing intuitive, user-friendly interaction. However, traditional intrusive GUI testing methods often face challenges such as interrupting user workflows, requiring significant manual effort and insufficient test scenario coverage. Non-intrusive testing methods, such as Robotic Process Automation (RPA), offer a solution to validate GUI functionality without modifying the application’s code or affecting the user experience. RPA systems automate repetitive tasks by simulating user interactions, becoming valuable tools in GUI testing. However, challenges like limited computational resources, time constraints, or restricted exploration capabilities may limit the efficiency of individual RPA agents, thus restricting coverage and effectiveness. To address this issue, this study explores the performance of a single RPA agent versus an RPA cluster under different testing conditions, using three popular testing methods: Monkey, Stoat and Q-testing. Experimental results indicate that an RPA cluster outperforms a single RPA in GUI coverage and error detection, making a significant contribution to the field of non-intrusive GUI exploration testing. The findings of this study provide directions for future research to ensure the delivery of high-quality mobile applications.

Weed species classification with UAV imagery and standard CNN models: Assessing the frontiers of training and inference phases

Accurate weed species identification is crucial for effective site-specific weed management (SSWM), enabling targeted and timely control measures for each weed in crop field. This study advanced the current approach to species-level weed identification during the early growth stage by integrating unmanned aerial vehicles (UAVs) imagery with standard convolutional neural networks (CNNs) models such as VGG16, Resnet152 and Inception-Resnet-v2. For this, a robust dataset was created with 33,467 labels of weeds (Atriplex patula, Chenopodium album, Convolvulus arvensis, Cyperus rotundus, Lolium rigidum, Portulaca oleracea, Salsola kali, Solanum nigrum) and crops (maize, tomato), which was subjected to different training, validation and test scenarios. Model inputs were adjusted in order to align them with the information represented by the UAV images. Initially, models were developed in balanced scenarios, gradually increasing label numbers to assess their performance. Inception-ResNet-v2 achieved over 90% accuracy with 400 labels, while ResNet152 and VGG16 required 600 and 800 labels, respectively, for similar accuracy. In a more complex and realistic scenarios with unbalanced datasets, Inception-ResNet-v2 outperformed, likely due to its deeper architecture and enhanced capability to capture intricate features and patterns within UAV images. The study emphasized the importance of the minority-to-majority species ratio in unbalanced datasets, which affects minority species classification. To prevent misclassification, it is crucial to determine the right number of labels for CNN model training and validation. Weed maps were generated after species classification using the Faster R–CNN algorithm as an object detector. This advancement in methodology facilitates the precise and efficient implementation of SSWM techniques.

Contrasting YOLOv7, SSD, and DETR on Insulator Identification under Small-sample Learning

Background:: Daily inspections of insulators are necessary because they are indispensable components for power transmission lines. Using deep learning to monitor insulators is a newly developed method. However, most deep learning-based detection methods rely on a large training sample set, which consumes computing resources and increases the workload of sample labeling. The selection of learning models to monitor insulators becomes problematic. Objective:: Through comparative analysis, a model suitable for small-sample insulator learning is found to provide a reference for the research and application of insulator detection. Methods:: This paper compares some of the latest deep learning models, YOLOv7, SSD, and DETR, for insulator detection based on small-sample learning. The small sample here means that the number of samples and their proportion to the total sample are relatively small. Two public insulator image sets, InsulatorDataSet with 600 insulator images and Transmission-line-pictures (TLP) with 1230 insulator images in the natural background are selected to test the performance of these models. Results:: Tests on two public insulator image sets, InsulatorDataSet and TLP, show that the recognition rates of YOLOv7, DETR, and SSD are arranged from high to low. The DETR and the YOLOv7 have stable performance, while the SSD lacks stable performance in terms of the learning time and recognition rate. Conclusion:: The in-domain and cross-domain scenario tests show that YOLOv7 is more suitable for insulator detection under small-sample conditions among the three models.

Research on AEB-P Control Strategy Considering Braking Process Comfort

A control approach for the pedestrian-oriented automobile automatic emergency braking system (AEB-P) that takes driving comfort into account is suggested in an effort to further optimize the AEB-P system's control algorithm. Firstly, for the AEB-P assessment standard of C-NCAP, a braking safety distance model that takes the driver's comfort into account during braking is created. The next step was to construct a hierarchical controller using PID and MPC principles. Lastly, CarSim and Matlab/Simulink are used to build and validate the test scenarios and control methods. The simulation results demonstrate that both ride comfort and driving safety can be guaranteed by the suggested control technique.

Impacts of information quantity and display formats on driving behaviors in a connected vehicle environment

The emerging connected vehicle (CV) technologies facilitate the development of integrated advanced driver assistance systems (ADASs), with which various functions are coordinated in a comprehensive framework. However, challenges arise in enabling drivers to perceive important information with minimal distractions when multiple messages are simultaneously provided by integrated ADASs. To this end, this study introduces three types of human–machine interfaces (HMIs) for an integrated ADAS: 1) three messages using a visual display only, 2) four messages using a visual display only, and 3) three messages using visual plus auditory displays. Meanwhile, the differences in driving performance across three HMI types are examined to investigate the impacts of information quantity and display formats on driving behaviors. Additionally, variations in drivers’ responses to the three HMI types are examined. Driving behaviors of 51 drivers with respect to three HMI types are investigated in eight field testing scenarios. These scenarios include warnings for rear-end collision, lateral collision, forward collision, lane-change, and curve speed, as well as notifications for emergency events downstream, the specified speed limit, and car-following behaviors. Results indicate that, compared to a visual display only, presenting three messages through visual and auditory displays enhances driving performance in four typical scenarios. Compared to the presentation of three messages, a visual display offering four messages improves driving performance in rear-end collision warning scenarios but diminishes the performance in lane-change scenarios. Additionally, the relationship between information quantity and display formats shown on HMIs and driving performance can be moderated by drivers’ gender, occupation, driving experience, annual driving distance, and safety attitudes. Findings are indicative to designers in automotive industries in developing HMIs for future CVs.

Flexural performance and damage of reinforced fly ash and slag-based geopolymer concrete after coupling effect of freeze-thaw cycles and sustained loading

To better represent field conditions, the bending performance of reinforced geopolymer concrete (GPC) beams after the coupling effect of freeze-thaw cycling and sustained flexural load was investigated. Two types of GPC (G10 and G50) with different content of fly ash and slag were included and compared with ordinary Portland cement concrete (OPC). The mechanical strength of concrete and bond strength of rebars in concrete before and after freeze-thaw cycles were measured. It was found that the frost resistance of G10 was the weakest followed by OPC and G50. Three scenarios of bending tests on reinforced concrete beams were included: (1) control (without freeze-thaw damage); (2) individual freeze-thaw cycling; (3) coupling effect before bending. Rebound tests were conducted to evaluate frost damages at different areas of the beams as the base to analyze bending behaviors of the beams. The failure mode, load-displacement(strain) relationship, ultimate flexural strength, and bending stiffness of beams from bending tests are analyzed. All beams in scenario (1) were dominant in flexural failure. For scenario (2), G10 and G50 failed in debonding failure due to significant reduction in bond strength, while OPC failed in diagonal tension failure as OPC could sustain bond stiffness, thus preventing debonding failure. For scenario (3), damages in GPC were exacerbated, while damages in OPC were limited.

BFTDiagnosis: An automated security testing framework with malicious behavior injection for BFT protocols

In peer-to-peer computing networks, Byzantine Fault Tolerance (BFT) protocols are a popular solution to ensure the consistency security in the presence of some malicious nodes, thus are widely employed in blockchain systems. However, BFT protocols still face various security threats in practice. Currently, testing the security of BFT protocols often requires manual operations. I.e., it lacks comprehensive automation testing techniques. This makes it difficult to cover various scenarios of malicious node behaviors. Therefore, it is an urgent requirement to design an automated testing framework that can comprehensively and efficiently evaluate the security of BFT protocols.This paper proposes BFTDiagnosis, an automated security testing framework with malicious behavior injection for BFT protocols. The framework can automatically configure and initiate protocol testing tasks, and construct consensus nodes to execute different BFT protocols. During testing, consensus nodes can inject malicious behaviors based on pattern matching strategies and malicious behavior triggering mechanisms to simulate various malicious scenarios. Through an Analyzer, BFTDiagnosis can collect protocol runtime data from consensus nodes and calculate four security quantification indicators to evaluate protocol performance.We conducted a load consumption evaluation of BFTDiagnosis. The results indicate its acceptable performance. Additionally, we tested PBFT, Basic-HotStuff, and Chained-HotStuff protocols using the BFTDiagnosis framework, validating the effectiveness of the framework and the rationality of security quantification indicators. Finally, we compared it with nine related technologies and found that BFTDiagnosis is good at testing scenario comprehensiveness and fault localization capability, and can intuitively evaluate the security of BFT protocols through four quantification indicators. The above results show that BFTDiagnosis is an effective and comprehensive security testing framework for BFT protocols.

Pre-crash scenarios for safety testing of autonomous vehicles: A clustering method for in-depth crash data

Autonomous vehicles (AVs) provide an opportunity to enhance traffic safety. However, AVs market penetration is still restricted due to their safety concerns and dependability. For widespread adoption, it is crucial to thoroughly assess the safety response of AVs in various high-risk scenarios. To achieve this objective, a clustering method was used to construct typical testing scenarios based on the China In-depth Mobility Safety Study-Traffic Accident (CIMSS-TA) database. Initially, 222 car-to-powered two-wheelers (PTWs) crashes and 180 car-to-car crashes were reconstructed from CIMSS-TA database. Second, six variables were extracted and analyzed, including the motion of the two vehicles involved, relative movement, lighting condition, road condition, and visual obstruction. Third, these variables were clustered using the k-medoids algorithm, identifying five typical pre-crash scenarios for car-to-PTWs and seven for car-to-car. Additionally, we extracted the velocities and surrounding environmental information of the crash-involved parties to enrich the scenario description. The approach used in this study used in-depth case review and thus provided more insightful information for identifying and quantifying representative high-risk scenarios than prior studies that analyzed overall descriptive variables from Chinese crash databases. Furthermore, it is crucial to separately test car-to-car scenarios and car-to-PTWs scenarios due to their distinct motion characteristics, which significantly affect the resulting typical scenarios.