Simulation-based Performance Evaluation Research Articles

ITRPL: An intelligent and trusted RPL protocol based on Multi-Agent Reinforcement Learning

Routing Protocol for Low Power and Lossy Networks (RPL) is the de-facto routing standard in IoT networks. It enables nodes to collaborate and autonomously build ad-hoc networks modeled by tree-like destination-oriented direct acyclic graphs (DODAG). Despite its widespread usage in industry and healthcare domains, RPL is susceptible to insider attacks. Although the state-of-the-art RPL ensures that only authenticated nodes participate in DODAG, such hard security measures are still inadequate to prevent insider threats. This entails a need to integrate soft security mechanisms to support decision-making. This paper proposes iTRPL, an intelligent and behavior-based framework that incorporates trust to segregate honest and malicious nodes within a DODAG. It also leverages multi-agent reinforcement learning (MARL) to make autonomous decisions concerning the DODAG. The framework enables a parent node to compute the trust for its child and decide if the latter can join the DODAG. It tracks the behavior of the child node, updates the trust, computes the rewards (or penalties), and shares them with the root. The root aggregates the rewards/penalties of all nodes, computes the overall return, and decides via its ϵ-Greedy MARL module if the DODAG will be retained or modified for the future. A simulation-based performance evaluation demonstrates that iTRPL learns to make optimal decisions with time.

A simulation-based performance evaluation of new generation dynamic shading devices with multi-objective optimization

Shading devices are the integrated components of building envelopes, which are designed to protect interiors from the excessive amount of direct and indirect solar radiation. Blocking the sunlight, these devices reduce the operational cost of cooling systems, which implies an inverse proportionality between the cooling energy consumption and desired thermal comfort. Therefore, in this study, a unique design for dynamic shading devices of an office building, located in the hot and humid climatic region, is proposed and presented from the early stages of the architectural design process. Above all, an innovative parametric model is created using the Grasshopper algorithmic modeling environment with the Honeybee and Ladybug plug-ins to overcome all difficulties of the manual design process. By employing the optimization plug-in of the Grasshopper software named Octopus, a performance evaluation based multi-objective optimization (MOO) method is introduced to find different cell dimensions of the devices for various orientations, south, east and west, of the office building. This research also investigates the potential of the proposed shading devices based on two major performance aspects, reducing the cooling load while increasing the thermal comfort of the office building, located at Bayrakli, Izmir, Turkey, latitude: 38° 27′ 44.00″ N and longitude: 27° 10′ 0.00″ E. The lowest cooling energy consumption is estimated as 8.35 kWh for 22nd of July 12:00 a.m. for the west orientation with the dimension of 100 × 100 cm, among other cell dimensions, which are 50 × 200 cm and 200 × 50 cm. The novelty lies behind the optimization of the conflicting performance features and the design of the new generation dynamic shading devices which would shed light on new shading device era.

HLPerf: Demystifying the Performance of HLS-based Graph Neural Networks with Dataflow Architectures

The development of FPGA-based applications using HLS is fraught with performance pitfalls and large design space exploration times. These issues are exacerbated when the application is complicated and its performance is dependent on the input data set, as is often the case with graph neural network approaches to machine learning. Here, we introduce HLPerf, an open-source, simulation-based performance evaluation framework for dataflow architectures that both supports early exploration of the design space and shortens the performance evaluation cycle. We apply the methodology to GNNHLS, an HLS-based graph neural network benchmark containing 6 commonly used graph neural network models and 4 datasets with distinct topologies and scales. The results show that HLPerf achieves over 10 000 × average simulation acceleration relative to RTL simulation and over 400 × acceleration relative to state-of-the-art cycle-accurate tools at the cost of 7% mean error rate relative to actual FPGA implementation performance. This acceleration positions HLPerf as a viable component in the design cycle.

A simulation-based performance evaluation model for decision support on drone location and delivery scheduling

PurposeThis paper aims to propose a simulation-based performance evaluation model for the drone-based delivery of aid items to disaster-affected areas. The objective of the model is to perform analytical studies, evaluate the performance of drone delivery systems for humanitarian logistics and can support the decision-making on the operational design of the system – on where to locate drone take-off points and on assignment and scheduling of delivery tasks to drones.Design/methodology/approachThis simulation model captures the dynamics and variabilities of the drone-based delivery system, including demand rates, location of demand points, time-dependent parameters and possible failures of drones’ operations. An optimization model integrated with the simulation system can update the optimality of drones’ schedules and delivery assignments.FindingsAn extensive set of experiments was performed to evaluate alternative strategies to demonstrate the effectiveness for the proposed optimization/simulation system. In the first set of experiments, the authors use the simulation-based evaluation tool for a case study for Central Florida. The goal of this set of experiments is to show how the proposed system can be used for decision-making and decision-support. The second set of experiments presents a series of numerical studies for a set of randomly generated instances.Originality/valueThe goal is to develop a simulation system that can allow one to evaluate performance of drone-based delivery systems, accounting for the uncertainties through simulations of real-life drone delivery flights. The proposed simulation model captures the variations in different system parameters, including interval of updating the system after receiving new information, demand parameters: the demand rate and their spatial distribution (i.e. their locations), service time parameters: travel times, setup and loading times, payload drop-off times and repair times and drone energy level: battery’s energy is impacted and requires battery change/recharging while flying.

Control parameter optimization of underwater gliders with uncertainty using hierarchical optimization method and interval analysis theory

This article attempts to solve the engineering optimization problem for control parameter configuration of underwater gliders with parameter uncertainty. The optimization objectives are maximizing the glider energy utilization rate and average voyage velocity; design variables include the net buoyancy adjustment amount and movable mass block translation amount; and uncertain parameters include the glider control parameter errors, manufacturing errors and sensor measurement errors. First, a dynamic simulation-based performance evaluation model of the glider is established, and interval number order theory is used to quantify the uncertainty. Then, surrogate models of the performance evaluation model are established, and the Sobol’ method-based sensitivity analysis is executed to identify the key uncertain parameters. Thus, a hierarchical optimization framework is introduced, and its system level and sub-level are used for determining optimal control parameter values and performance evaluation parameter intervals, respectively. Finally, optimization calculation is executed under different parameter configurations, and some rules are summarized.

A systematic decision-support approach for healthcare facility layout design integrating resource flow and space adjacency optimization with simulation-based performance evaluation

Healthcare facilities are recognized as significant public architectural design, given their design complexity. Particularly in the early stages of design process, the spatial configuration cannot be overlooked and it has a significant impact on the functional performance of a building. Existing research on healthcare facility layout design focused on the methods of simulation, procedural and algorithmic approaches with limited consideration of building operational performance, or the well-being of future occupants. In light of this, this article proposes a systematic decision-support approach that combines multi-objective optimization and simulation-based occupancy performance evaluation to offer a well-performance healthcare facility layout solution. The mathematical model's purpose is to optimize building operational objectives, namely, minimizing resource transportation costs and maximizing adjacent requirements between facilities. Then the performance of the mathematical model's Pareto solutions is evaluated using a simulation module that takes user-building interactions into account. To further validate the effectiveness of our approach, we conducted a comparative experiment involving three healthcare architects in the design of a surgical ward layout. Results indicate that the proposed approach can assist architects in achieving a balance between efficiency and safety in healthcare facility layout design, resulting in a more occupant-centric building design. Moreover, this systematic approach provides the theoretical basis on describing how to integrate the widely researched models for architectural layout planning and occupancy performance evaluation.

Correction to: Simulation-based Performance Evaluation of Skewed Uncontrolled Intersections

Correction to: Simulation-based Performance Evaluation of Skewed Uncontrolled Intersections

CAMELAMA: Cooperative awareness and spaceborne monitoring enabled by location-assisted medium access

In beaconing systems such as AIS or ADS-B, used by ships and aircraft, each node periodically broadcasts its navigational state to nearby nodes to increase traffic safety. Nowadays, these beacons are also used as a source for satellite-based global traffic monitoring. This dual use imposes competing needs on the medium access control protocol as the size of the collision domains varies by a large factor between the use cases. Even the subproblem of solely avoiding terrestrial nodes’ packets to collide from the perspective of a receiving satellite is not trivial to solve if the satellite’s collision domain spans multiple hops in the terrestrial network. Based on ideas of the LAMA protocol, we propose CAMELAMA, a novel contention-free medium access control protocol for position awareness beaconing. Our low-overhead approach uses neither forwarding of node state nor handshakes but only the navigational data that is shared between the terrestrial nodes anyway. CAMELAMA provides local cooperative awareness while at the same time desynchronizing transmissions within a satellite’s collision domain even in terrestrially disconnected topologies. In a simulation-based performance evaluation we find that CAMELAMA outperforms SO-TDMA (the MAC protocol of AIS) and also scales better with respect to high node densities.

Simulation-based Performance Evaluation of Skewed Uncontrolled Intersections

Simulation-based Performance Evaluation of Skewed Uncontrolled Intersections

Optimal Design of Flow Control Fins for a Small Container Ship Based on Machine Learning

In this study, the optimal design of flow control fins (FCFs) for a container ship was carried out via a machine learning approach. The conventional design practice for the FCF relied on simulation-based performance evaluation, which demands a large amount of analysis time. Instead of computational fluid dynamics (CFD)-based prediction, artificial neural network (ANN)-based prediction was attempted. Prior to the machine learning process, the wake distribution data were collected systematically via CFD. Based on the collected data, the wake distributions and resistance performance dependent on varying the fin positions were learned using the ANN, and the optimal fin position was selected with relevant optimization techniques. When multi-objective optimization was employed, it was found that both wake distributions and resistance performance were improved in a practically applicable timeframe. The current process is superior to conventional simulation-based optimization in terms of speed. From the viewpoint of prediction accuracy, in this study, ANN-based prediction was found to be equally accurate as CFD-based prediction. Thus, the results can provide a novel and reliable design methodology for the optimal design of ship appendages.

The Impact of a Multifaceted Simulation Education and Feedback Program for Community Emergency Departments on Pediatric Diabetic Ketoacidosis Management.

We sought to determine if general emergency departments (GEDs) were managing pediatric diabetic ketoacidosis (DKA) correctly and if management could be improved using a multilayered educational initiative. We hypothesized that a multifaceted program of in situ simulation education and formal feedback on actual patient management would improve community GED management of pediatric DKA. This study combined a prospective simulation-based performance evaluation and a retrospective chart review. A community outreach simulation education initiative was developed followed by a formal patient feedback process. Fifteen hospitals participated in simulation sessions and the feedback process. All hospitals were scored for readiness to provide care for critically ill pediatric patients using the Emergency Medical Services for Children (EMSC) Pediatric Readiness Assessment. Six of the 15 have had a second hospital visit that included a DKA scenario with an average performance score of 60.3%. A total of 158 pediatric patients with DKA were included in the chart review. The GEDs with higher patient volumes provided best practice DKA management more often (63%) than those with lower patient volumes (40%). Participating in a DKA simulated scenario showed a trend toward improved care, with 47.2% before participation and 68.2% after participation ( P = 0.091). Participating in the formal feedback process improved best practice management provided to 68.6%. Best practice management was further improved to 70.3% if the GED participated in both a DKA simulation and the feedback process ( P = 0.04). A multifaceted program of in situ simulation education and formal feedback on patient management can improve community GED management of pediatric patients with DKA.

Traffic Characterization Based Stochastic Modelling of Network-on-Chip

The trend towards multi-core and many-core processors has changed the landscape of computers and servers. Now the performance of a microprocessor heavily depends not only on the data path but also on the memory technology and communication technology. On-chip communication networks or Network-on-Chip is a component which facilitates communication between the cores of a microprocessor. In this manuscript we propose an analytical model for the analysis of Network-on-Chip performance based on Jackson queuing networks for applications that exhibit Poisson injection process. The injection process statistics and the traffic profile of PARSEC benchmarks have been characterized. We have shown that injection process of PARSEC benchmarks does not match with the well known probability distributions using Quantile-Quantile plot and Kolmogorov-Smirnov test. For realistic benchmarks that does not follow Poisson injection process we propose <inline-formula><tex-math notation="LaTeX">$G/D/1$</tex-math></inline-formula> queuing model for the NoC router. We have carried out performance analysis using the traffic characterization in conjunction with our analytical model for prediction of performance metrics. The analytical model offers a speed up of around 13 times in comparison with the simulation based performance evaluation. The percentage of error between the analytical model and simulation is less than 6% for most of the benchmarks except Canneal and Ferret. We have also compared our analytical model results with the results of SNIPER simulator.

Simulation-based performance evaluation of large-scale thermal energy storage coupled with heat pump in district heating systems

Large-scale water-based thermal energy stores (TES) coupled with heat pumps (HPs) are a key element in District Heating (DH) systems to achieve an increase of the share of renewables.The implementation of a 1D equation- based TES model, coupled with a 2D or 3D finite element-based model of the surrounding ground provides a robust and reliable tool that can be used in the study of large-scale TES systems. In this work, a simplified equation-based model for the HP using the TES as evaporator heat source is implemented to investigate its impact on the TES performance and on the surrounding ground temperatures. Two TES types (buried tank and shallow pit) are analysed under the same conditions (size, envelope characteristics, ambient temperature, ground properties and system boundary conditions). The parameters used to evaluate the coupled TES-HP performance consider both the environmental aspect (ground temperature and CO2 emissions) and the technical performance (energy and exergy efficiency).The integration of the HP proves to be effective in increasing the TES efficiency by 6 % (from 87 %) compared to the reference case without an HP in case of an insulated tank TES, and by 16 % (from 64 %) in case of an insulated shallow pit. The TES exergy efficiency analysis emphasizes the better performance of the tank over the shallow pit, with the tank geometry outperforming the shallow pit in all the investigated configurations (+ 25 % in the reference case without HP, + 20 % with 6 MW HP). However, when extending the exergy analysis to the system, the difference in exergy efficiency between the two geometries is strongly reduced (+ 5 % on average for the tank with respect to the shallow pit); the impact of thermal insulation is also less noticeable. In particular, in the system analysis the configuration of shallow pit with HP shows a performance comparable with the tank without HP. In terms of CO2 emissions, the integration of the HP coupled with a TES allows for a reduction of 33 % for the tank and 29 % for the shallow pit compared to the respective reference cases, due to the better utilization of the TES capacity. The resulting ground temperature near the TES is also influenced by the use of the HP, with an average ground temperature 5 K lower than in the reference case without HP at a ground depth between of 7 to 15 m. The contribution of thermal insulation appears however necessary for both geometries, as it contributes most to the reduction of ground temperature.

Energy-Efficient Artificial Intelligence of Things With Intelligent Edge

Artificial Intelligence of Things (AIoT) is an emerging area of future Internet of Things (IoT) to support intelligent IoT applications. In AIoT, intelligent edge computing technologies accelerate intelligent services’ processing speed with much lower cost than simple cloud-aided IoT architecture. However, there is still a lack of resource strategy to optimize the energy efficiency of AIoT with intelligent edge computing. Therefore, in this article, we focus on the energy consumption of edge devices and cloud services in processing AIoT tasks and formulate the optimization problem in scheduling tasks in the edge and the cloud. Meanwhile, a novel online method is proposed to solve the optimization problem. We investigate the energy consumption of several typical intelligent edge devices and the cloud service in an intelligent edge computing testbed. Extensive simulation-based performance evaluation shows that the proposed method outperforms other strategies with lower energy consumption.

Evaluating the power and limitations of genome-wide association studies in Caenorhabditis elegans.

Quantitative genetics in Caenorhabditis elegans seeks to identify naturally segregating genetic variants that underlie complex traits. Genome-wide association studies scan the genome for individual genetic variants that are significantly correlated with phenotypic variation in a population, or quantitative trait loci. Genome-wide association studies are a popular choice for quantitative genetic analyses because the quantitative trait loci that are discovered segregate in natural populations. Despite numerous successful mapping experiments, the empirical performance of genome-wide association study has not, to date, been formally evaluated in C. elegans. We developed an open-source genome-wide association study pipeline called NemaScan and used a simulation-based approach to provide benchmarks of mapping performance in collections of wild C. elegans strains. Simulated trait heritability and complexity determined the spectrum of quantitative trait loci detected by genome-wide association studies. Power to detect smaller-effect quantitative trait loci increased with the number of strains sampled from the C. elegans Natural Diversity Resource. Population structure was a major driver of variation in mapping performance, with populations shaped by recent selection exhibiting significantly lower false discovery rates than populations composed of more divergent strains. We also recapitulated previous genome-wide association studies of experimentally validated quantitative trait variants. Our simulation-based evaluation of performance provides the community with critical context to pursue quantitative genetic studies using the C. elegans Natural Diversity Resource to elucidate the genetic basis of complex traits in C. elegans natural populations.

A Novel Joint Dataset and Computation Management Scheme for Energy-Efficient Federated Learning in Mobile Edge Computing

In this letter, a novel joint dataset and computation management (DCM) scheme for energy-efficient federated learning (FL) in mobile edge computing (MEC) is proposed. For this purpose, with respect to the amount of dataset and computation resources, we rigorously formulated analytical models for i) learning efficiency, which considers the estimated global accuracy tendency according to the amount of dataset and service latency, and ii) the overall energy consumption of FL participants, including local training and model parameter transmission. To consider the trade-off between these two factors in the FL procedure with MEC, a theoretical framework for the DCM problem that jointly optimizes the amount of dataset and the computation resources used for local training over multiple FL clients was designed. Additionally, the extensive simulation-based performance evaluations validate the superior performance of the proposed DCM; compared to the various benchmarks in terms of the proposed cost function and test accuracy on the MNIST dataset with independent identically distributed (IID) / non-IID settings.

Interference and QoS-Aware Resource Allocation Considering DAS Behavior for C-RAN Power Minimization

This work aims to minimize the power consumption of cloud virtualization components by addressing the important challenge of designing allocation schemes that cater for both radio and computational resources in a virtualized cloud-based radio environment. Unlike previous efforts, we consider the realistic behavior of radio resource heads (RRHs) associated with one baseband unit (BBU), acting as a DAS. We first formulate the admission control (AC) and the RRH–BBU mapping problem, subject to constraints on user throughput requirements, computational capacity in the BBU pool, and according to the assumption of DAS behavior. As the optimal solution is practically intractable for large-scale dynamic networks, we propose a two-level resource allocation framework, based on two developed algorithms: a first one for AC at the radio level and a second one for RRH–BBU mapping at the computational level. The two algorithms are designed to allow both the levels to consider each other’s constraints and particularities. Results obtained from an extensive simulation-based performance evaluation show the high performance of our proposal in terms of radio-related metrics, number of accepted users, power saving, and energy and spectrum efficiency, compared with three baseline reference schemes.

Learning-based restoration sequence ordering for multi-site traffic signal failure

Traffic signal failures could result in significant local and system-level performance degradation. Sequencing the restoration of failed signals with limited resources is a challenging problem—capturing dynamic changing transportation system performance following a feasible solution requires tedious computation, and the short time frame for restoring failed signals makes these decisions time-sensitive and should be determined in a timely manner. Feasible Signal Restoration Sequence Ordering (SRSO) problem, as a critical building block to optimize the restoration sequence, has not been well-studied in the existing literature, nor have solutions to address the computational burden issue. In this work, a machine learning model based on Structural Recurrent Neural Network (SRNN) is proposed to predict system performance, i.e., aggregated accumulated total delay, following a given restoration sequence, to address the computational burden of the simulation-based performance evaluation. Spatio-temporal (ST) graph representation is leveraged in this methodology to take the topological information, i.e., how adjacent movements interact with each other, into consideration. Although microscopic simulation is used to obtain the ground truth performance, which is still time-consuming for a group of signal failure scenarios, a trained machine learning model can surrogate the tedious computation in the decision-making process in a timely manner. The challenges to build this machine learning model effectively and efficiently are two-folds. First, a transportation system is a typical dynamic system whose behavior is constrained by the topology of the network. Therefore, both spatial and temporal interactions between road sections should be captured to predict system performance effectively. Second, in the context of signal failure, system performance is exposed to a disrupted control strategy, which makes it even more challenging to predict system performance effectively. The original SRNN model could address the first challenge, and the movement feature representation and its integration to the SRNN model proposed in this work could address the second. A case study was conducted to demonstrate the operability and effectiveness of the proposed methodology. It is demonstrated that the signal restoration sequence could impact system performance during and after the restoration process significantly. Then, both Aggregated Accumulated Total Delay (AATD) prediction accuracy and the performance of restoration sequence ordering is evaluated for the case study network. Outcomes of the case study show that the learning-based model can help identify sequences ranked in top 8% of optional sequences referring to ground truth information. Furthermore, in terms of the fine-tune of the learning rate, the Cosine-Annealing-LR strategy leads to both lower loss value, better ordering performance, and shorter delay experienced in the restoration process, compared to the Step-LR strategy.

Congestion-aware heterogeneous connected automated vehicles cooperative scheduling problems at intersections

More and more vehicles are connected today via emerging connected and automated vehicle (CAV) technologies. An intriguing application of CAVs is to cross intersections without stops through cooperative scheduling by traffic control infrastructure. Nonetheless, with the increase of CAVs’ requests for green, two problems will surface: (I) accommodating too many CAVs’ green requests will generate severe interruptions to general traffic; (II) simple scheduling policies like first-come-first serve is inappropriate due to heterogeneous importance of CAVs. To overcome these challenges, we present a mixed-integer linear programming (MILP) formulation for congestion-aware heterogeneous CAV scheduling problems at intersections in this paper. The objective is to ensure that intensive and heterogeneous green requests by CAVs can be scheduled at intersections while the mobility of background traffic is still maintained. The MILP formulation is developed in the context of discrete space-time and phase-time networks whose variables are space-time arc choice variables with respect to individual vehicles and phase-time arc choice variables. We also build an efficient search algorithm based on the “A-D curves” for real-time applications. Three experiments are conducted to validate the proposed MILP formulation and search algorithm. The simulation-based performance evaluation for the congestion-aware CAV scheduling reveal promising results for real-world applications in the future.

Wireless Geophone Networks for Land Seismic Data Acquisition: A Survey, Tutorial and Performance Evaluation.

Seismic data acquisition in oil and gas exploration employs a large-scale network of geophone sensors deployed in thousands across a survey field. A central control unit acquires and processes measured data from geophones to come up with an image of the earth’s subterranean structure to locate oil and gas traps. Conventional seismic acquisition systems rely on cables to connect each sensor. Although cable-based systems are reliable, the sheer amount of cable required is tremendous, causing complications in survey logistics as well as survey downtime. The need for a cable-free seismic data acquisition system has attracted much attention from contractors, exploration companies, and researchers to lay out the enabling wireless technology and architecture in seismic explorations. This paper gives a general overview of land seismic data acquisition and also presents a current and retrospective review of the state-of-the-art wireless seismic data acquisition systems. Furthermore, a simulation-based performance evaluation of real-time, small-scale wireless geophone subnetwork is carried out using the IEEE 802.11 g technology based on the concept of seismic data acquisition during the geophone listen or recording period. In addition, we investigate an optimal number of seismic samples that could be sent by each geophone during this period.