System Simulation Environment Research Articles

Mobile Robot Navigation Based on Noisy N-Step Dueling Double Deep Q-Network and Prioritized Experience Replay

Effective real-time autonomous navigation for mobile robots in static and dynamic environments has become a challenging and active research topic. Although the simultaneous localization and mapping (SLAM) algorithm offers a solution, it often heavily relies on complex global and local maps, resulting in significant computational demands, slower convergence rates, and prolonged training times. In response to these challenges, this paper presents a novel algorithm called PER-n2D3QN, which integrates prioritized experience replay, a noisy network with factorized Gaussian noise, n-step learning, and a dueling structure into a double deep Q-network. This combination enhances the efficiency of experience replay, facilitates exploration, and provides more accurate Q-value estimates, thereby significantly improving the performance of autonomous navigation for mobile robots. To further bolster the stability and robustness, meaningful improvements, such as target “soft” updates and the gradient clipping mechanism, are employed. Additionally, a novel and powerful target-oriented reshaping reward function is designed to expedite learning. The proposed model is validated through extensive experiments using the robot operating system (ROS) and Gazebo simulation environment. Furthermore, to more specifically reflect the complexity of the simulation environment, this paper presents a quantitative analysis of the simulation environment. The experimental results demonstrate that PER-n2D3QN exhibits heightened accuracy, accelerated convergence rates, and enhanced robustness in both static and dynamic scenarios.

Numerical and experimental investigation of stratified water storage tanks: An enhanced adaptive-grid model

Stratified water storage tanks are key in thermal energy systems, effectively balancing energy supply with heat demand, thus facilitating operational flexibility. Accurately modeling both energy balance and thermocline evolution with minimal computational effort is essential for models implementation in real-time control applications. One-dimensional, static-grid models are favored for their easy implementation; however, they require dense grids in order to minimize numerical diffusion and to represent mixing effect correctly, resulting in rather expensive computational effort. This study introduces an adaptive-grid model that comprehensively represents the water storage tank’s geometrical features, including inlet and outlet duct heights and dead volume sizes. It also details the spatially discretized heat diffusion and characterizes the inflow region’s mixing using an empirical equation. Implemented in both TRaNsient SYstem Simulation (TRNSYS) and Modelica environments, the model is validated through laboratory experiments with a hot water storage tank operating under various mass flow and temperature conditions, representative of heat pump-driven heating systems. The results indicate improvements in computational efficiency compared to static-grid models, specifically a 55% reduction in simulation duration with the same level of uncertainty.

Dynamic path planning via Dueling Double Deep Q-Network (D3QN) with prioritized experience replay

Path planning is a key requirement for mobile robots employed for different tasks such as rescue or transport missions. Conventional methods such as A* or Dijkstra to tackle path planning problem need a premise map of the robot's environment. Nowadays, dynamic path planning is a popular research topic, which drives mobile robots without prior static requirements. Deep reinforcement learning (DRL), which is another popular research area, is being harnessed to solve dynamic path planning problem by the researchers. In this study, Deep Q-Networks, which is a subdomain of DRL are opted to solve dynamic path planning problem. We first employ well known techniques Double Deep Q-Networks (D2QN) and Dueling Double Deep Q-Networks (D3QN) to train a model which can drive a mobile robot in environments with static and dynamic obstacles within 3 different configurations. Then we propose D3QN with Prioritized Experience Replay (PER) extension in order to further optimize the DRL model. We created a test bed to measure the performance of the DRL models against 99 randomly generated goal locations. According to our experiments, D3QN-PER method performs better than D2QN and D3QN in terms of path length and travel time to the goal without any collisions. Robot Operating System and Gazebo simulation environment is utilized to realize the training and testing environments, thus, the trained DRL models can be deployed to any ROS compatible robot seamlessly.

The future of road freight transport and alternative technologies: A case study for Italy

The truck sector produces one-fourth of the greenhouse gas emissions in Italy. Therefore, it calls for an urgent emission reduction to achieve the national mid-century net-zero pledges. In this work, a methodology is developed, based on a model for estimating the long-term energy transitions, the ModUlar energy system Simulation Environment (MUSE), to project the long-term freight heavy-duty sector evolution in Italy. The methodology applies a bottom-up description of the sector, where technologies are characterised in terms of costs, efficiencies, and emissions. Results show that efficiency improvements and technological cost reduction due to learning can only reduce the sector emissions by a limited amount. In fact, although, lowering costs of alternative powertrains and fuels will have a significant effect on reducing tailpipe emissions, this should occur alongside a ban of internal combustion engines. Additional renewable generation should need to grow between 30 and 64 TW/y to substantially reduce emissions and leading to a zero-tailpipe truck sector by 2050. The replacement of conventional engines by alternative powertrains remains dependent on distribution and fuelling infrastructure expansion, which should go hand in hand with the promotion of electric and hydrogen-fuelled vehicles.

MUSE-RASA captures human dimension in climate-energy-economic models via global geoAI-ML agent datasets

This article provides a combined geospatial artificial intelligence-machine learning, geoAI-ML, agent-based, data-driven, technology-rich, bottom-up approach and datasets for capturing the human dimension in climate-energy-economy models. Seven stages were required to conduct this study and build thirteen datasets to characterise and parametrise geospatial agents in 28 regions, globally. Fundamentally, the methodology starts collecting and handling data, ending with the application of the ModUlar energy system Simulation Environment (MUSE), ResidentiAl Spatially-resolved and temporal-explicit Agents (RASA) model. MUSE-RASA uses AI-ML-based geospatial big data analytics to define eight scenarios to explore long-term transition pathways towards net-zero emission targets by mid-century. The framework and datasets are key for climate-energy-economy models considering consumer behaviour and bounded rationality in more realistic decision-making processes beyond traditional approaches. This approach defines energy economic agents as heterogeneous and diverse entities that evolve in space and time, making decisions under exogenous constraints. This framework is based on the Theory of Bounded Rationality, the Theory of Real Competition, the theoretical foundations of agent-based modelling and the progress on the combination of GIS-ABM.

Optimization Control Strategy for a Central Air Conditioning System Based on AFUCB-DQN

The central air conditioning system accounts for 50% of the building energy consumption, and the cold source system accounts for more than 60% of the total energy consumption of the central air conditioning system. Therefore, it is crucial to solve the optimal control strategy of the cold source system according to the cooling load demand, and adjust the operating parameters in time to achieve low energy consumption and high efficiency. Due to the complex and changeable characteristics of the central air conditioning system, it is often difficult to achieve ideal results using traditional control methods. In order to solve this problem, this study first coupled the building cooling load simulation environment and the cold source system simulation environment to build a central air conditioning system simulation environment. Secondly, noise interference was introduced to reduce the gap between the simulated environment and the actual environment, and improve the robustness of the environment. Finally, combined with deep reinforcement learning, an optimal control strategy for the central air conditioning system is proposed. Aiming at the simulation environment of the central air conditioning system, a new model-free algorithm is proposed, called the dominant function upper confidence bound deep Q-network (AFUCB-DQN). The algorithm combines the advantages of an advantage function and an upper confidence bound algorithm to balance the relationship between exploration and exploitation, so as to achieve a better control strategy search. Compared with the traditional deep Q-network (DQN) algorithm, double deep Q-network (DDQN) algorithm, and the distributed double deep Q-network (D3QN) algorithm, the AFUCB-DQN algorithm has more stable convergence, faster convergence speed, and higher reward. In this study, significant energy savings of 21.5%, 21.4%, and 22.3% were obtained by conducting experiments at indoor thermal comfort levels of 24 °C, 25 °C, and 26 °C in the summer.

Multi-agent-based collaborative regulation optimization for microgrid economic dispatch under a time-based price mechanism

• This paper provides a multi-agent based coordinated dispatch strategy for the economic dispatch of the microgrid under a time-based price mechanism. • We develop a microgrid optimization model for the microgrid operation process, which includes battery regulation and user satisfaction. • The established optimization model is solved using a MACPSO algorithm, and the agent communication mechanism in the microgrid is examined. • The performance comparison using different algorithms (PSO, IPSO, MAPSO and MACPSO) has been presented. • The proposed multi-agent coordinated dispatch strategy is validated by designing a multi-agent system simulation environment and using the Java agent development (JADE) framework. The economic optimal dispatch of a microgrid is a challenging task with significant economic and social implications. Under a time-based price mechanism, this paper proposes a multi-agent-based coordinated dispatch strategy for the microgrid's economic dispatch. The information between the agents in the microgrid can be fed back in time to reduce microgrid operation failures by coordinated control between the demand-side load, the battery, and the power generation side micro-sources. The output of other micro-sources is then coordinated using an appropriate battery charging and discharging plan based on the characteristics of the time-based electricity price mechanism. Following that, an economical microgrid operation model is established and solved using a multi-agent chaotic particle swarm optimization (MACPSO) algorithm, which considers user satisfaction. Finally, a multi-agent system (MAS) simulation environment is built using the Java agent development (JADE) framework. The proposed microgrid with multi-agent coordinated control can effectively improve its operation efficiency, according to a comparative analysis of examples.

LBSS: A Lightweight Blockchain-Based Security Scheme for IoT-Enabled Healthcare Environment.

Recently, global healthcare has made great progress with the use of Internet of Things technology. However, for there to be excellent patient care, there must be a high degree of safety for the IoT health system. There has been a massive increase in hacking systems and the theft of sensitive and highly confidential information from large health centers and hospitals. That is why establishing a highly secure and reliable healthcare system has become a top priority. In this paper, a security scheme for the IoT-enabled healthcare environment, LBSS, is proposed. This security scheme comprises three security mechanisms. The first mechanism is based on the blockchain technology and is used for transaction integrity. The second mechanism is used to store the healthcare system data in a secure manner through the distribution of its data records among multiple servers. The third mechanism is used to access the healthcare data after applying a proposed authorization test. To minimize the security overhead, the healthcare data is prioritized in regard to its importance. Therefore, each security mechanism has specific steps for each level of data importance. Finally, the NS3 package is used to construct a simulation environment for IoT-enabled healthcare systems to measure the proposed security scheme performance. The simulation results proved that the proposed healthcare security scheme outperformed the traditional models in regard to the performance metrics.

Modular Fuel Cell System Simulation Environment with Focus on Membrane Water Management

Modular Fuel Cell System Simulation Environment with Focus on Membrane Water Management

Energy-Economic Assessment of Islanded Microgrid with Wind Turbine, Photovoltaic Field, Wood Gasifier, Battery, and Hydrogen Energy Storage

Island energy systems are becoming an important part of energy transformation due to the growing needs for the penetration of renewable energy. Among the possible systems, a combination of different energy generation technologies is a viable option for local users, as long as energy storage is implemented. The presented paper describes an energy-economic assessment of an island system with a photovoltaic field, small wind turbine, wood chip gasifier, battery, and hydrogen circuit with electrolyzer and fuel cell. The system is designed to satisfy the electrical energy demand of a tourist facility in two European localizations. The operation of the system is developed and dynamically simulated in the Transient System Simulation (TRNSYS) environment, taking into account realistic user demand. The results show that in Gdansk, Poland, it is possible to satisfy 99% of user demand with renewable energy sources with excess energy equal to 31%, while in Agkistro, Greece, a similar result is possible with 43% of excess energy. Despite the high initial costs, it is possible to obtain Simple Pay Back periods of 12.5 and 22.5 years for Gdansk and Agkistro, respectively. This result points out that under a high share of renewables in the energy demand of the user, the profitability of the system is highly affected by the local cost of energy vectors. The achieved results show that the system is robust in providing energy to the users and that future development may lead to an operation based fully on renewables.

MUSE: An open-source agent-based integrated assessment modelling framework

Integrated assessment models (IAMs) are a cornerstone of an effective approach to climate change mitigation. Despite the variety of methodologies for characterising the energy system, land use change, economics, and climate response, the modelling community has an open and urgent request for tools capable of more realistic interpretation of the energy transition, capturing human behaviour, and embodying the principles of transparency, reproducibility, and flexibility of use.This paper presents an open-source modelling framework designed to fill that gap. Named MUSE (ModUlar energy systems Simulation Environment), this new agent-based model supports flexible characterisation of agent decision-making, including individual goals, bounded-rationality, imperfect foresight, and limited knowledge during the decision process. MUSE integrates this agent-based approach in a partial-equilibrium framework and enables a technology-rich description of the energy systems with an unprecedented degree of flexibility for including technological, temporal, and geographical granularity. The structure of MUSE creates the ability to produce climate change mitigation assessments that are more grounded, and more tangible model outputs for conceiving effective approaches to mitigation. MUSE is available open source under a GNU General Public License v3.0 on GitHub at this link https://github.com/SGIModel/MUSE_OS.

A Socio-Inspired Methodology and Model for Advanced and Opportunistic Interactions between Industrial IoT Objects

The concept of the Internet of Things (IoT) is widely discussed. IoT is one of the emerging technologies that have caught the attention of many researchers. The increase in the number of exchanges of services between heterogeneous or homogeneous connected objects with the integration of social networking concepts gives rise to the concept of the Social Internet of Things (SIoT). The SIoT concept contributes to the evolution of interactions between industrial objects by improving deterministic mechanisms towards intelligent interactions. The integration of the SIoT concept into the Industrial Internet of Things (IIoT) gives rise to the Social Internet of Industrial Things (SIoIT) and plays an important role in improving system performance in Industry 4.0. In this article, we propose an innovative methodology and a model of socio-inspired interaction between industrial communicating objects inspired by sociological approaches. Thanks to this model, socialized industrial communicating objects form a community of objects, autonomously and dynamically, by exchanging messages to know each other perfectly, and service requests between objects are executed adaptively according to the principles of social interaction governed by socio-inspired strategies and conditions. The model is implemented, tested and validated in a Netlogo multi-agent system simulation environment.

Towards monocular vision-based autonomous flight through deep reinforcement learning

This paper proposes an obstacle avoidance strategy for small multi-rotor drones with a monocular camera using deep reinforcement learning. The proposed method is composed of two steps: depth estimation and navigation decision making. For the depth estimation step, a pre-trained depth estimation algorithm based on the convolutional neural network is used. On the navigation decision making step, a dueling double deep Q-network is employed with a well-designed reward function. The network is trained using the robot operating system and Gazebo simulation environment. To validate the performance and robustness of the proposed approach, simulations and real experiments have been carried out using a Parrot Bebop2 drone in various complex indoor environments. We demonstrate that the proposed algorithm successfully travels along the narrow corridors with the texture free walls, people, and boxes.

Longitudinal Inter-Vehicle Distance Control of Autonomous Vehicle Platoons Subjected to Internal and External Disturbances

The idea of autonomous vehicle platoons presents a variety of social, economic, and safety benefits to the transportation industry. However, implementing and deploying autonomous vehicle platoons is still a challenge. In this paper, we present a PID-based computationally cost-efficient controller to aid in the longitudinal control of the inter-vehicle distance between successive platoon members. The proposed approach is facilitated by inter-vehicle communication. The algorithm was implemented using the Robotics Operating System and Gazebo simulation environment. In order to evaluate the performance and applicability of the proposed approach, meticulous simulations under numerous scenarios were performed using 3D vehicle models so as to mimic the real world. The algorithm successfully maintains the longitudinal inter-vehicle distance within the desired range, ensures that no collisions occur among platoon members, and preserves the platoon formation.

State Estimation Model Reduction Through the Manifold Boundary Approximation Method

This paper presents a procedure for estimating the systems state when considerable Information and Communication Technology (ICT) component outages occur, leaving entire system areas unobservable. For this task, a novel method for analyzing system observability is proposed based on the Manifold Boundary Approximation Method (MBAM). By utilizing information geometry, MBAM analyzes boundaries of models in data space, thus detecting unidentifiable system parameters and states based on available data. This approach extends local, matrix-based methods to a global perspective, making it capable of detecting both structurally unidentifiable parameters as well as practically unidentifiable parameters (i.e., identifiable with low accuracy). Beyond partitioning identifiable/unidentifiable states, MBAM also reduces the model to remove reference to the unidentifiable state variables. To test this procedure, cyber-physical system (CPS) simulation environments are constructed by co-simulating the physical and cyber system layers.

Socialization of Smart Communicative Objects in Industrial Internet of Things

In the recent past the Internet of Things (IoT) is moving from interactive objects to smart objects. The number of smart devices or sensors built-in becomes larger. The increase in the number of services exchanged between connected objects has led to the creation of a new concept called Social IoT (SIoT), in which heterogeneous or homogeneous objects can interact and request or exchange services, which makes it possible to include social or human actions in the IoT platform. The use of technologies based on the concept of the Internet of Things in applications and industrial sectors has built a new concept of research called the Industrial Internet of Things (IIoT). The integration of the SIoT concept into the Industrial Internet of Things (IIoT) plays an important role in improving system performance in Industry 4.0 and gives rise to the Social Internet of Industrial Things (SIoIT). In this context, we have revealed a new socio-inspired interaction model between industrial communicating objects inspired by sociological approaches that transform objects into socialized industrial communicating objects. These objects form a community, autonomously and dynamically, by exchanging messages to know each other perfectly, and service requests between objects are realized in an adaptive way according to the principles of social interaction governed by socio-inspired strategies and conditions. The model is evaluated using Netlogo multi-agent system simulation environment.

Research on Digital Twin Modeling and Simulation Technology for Distributed Resource Aggregation Regulation

Virtual power plants need to build awareness, transmission and application of information flow model, require multiple levels between virtual power plant, the virtual power plant each participation main body, regulation and control of virtual power plant and quantification of communication information transmission between the main constraints, this paper studies the distributed resources polymerization technology, the modeling and simulation of mechanical and electrical characteristics of virtual power plant communication network transmission characteristics of digital simulation technology, This paper proposes a simulation method of diversified resource aggregation regulation based on virtual group structure. By collecting the electrical characteristic change curve of each distributed unit, and stacking the transmission characteristics of communication network, digital simulation aggregation is carried out to obtain the digital model of cooperative operation and control characteristics of virtual power plant "information flow + energy flow". Through this model will be connected to Matlab/Simulink power system simulation environment for power regulation and operation simulation verification.

Socio-technical modeling of smart energy systems: a co-simulation design for domestic energy demand

To tackle the climate crisis, the European energy strategy relies on consumers taking ownership of the energy transition, accelerating decarbonisation through investments in low-carbon technologies and ensuring system stability and reliability by actively participating in the market. Therefore, tools are needed to better understand an increasingly complex and actor-dense energy system, tracking socio-technical dynamics that occur at its margins and then predicting the effects on larger scales. Yet, existing domestic energy demand models are not flexible enough to incorporate a wide range of socio-technical factors, and to be incorporated into larger energy system simulation environments. Here, a co-simulation design for domestic energy demand modeling is presented and motivated on the basis of four design principles: granularity, scalability, modularity and transparency. Microsimulation of domestic energy demand, through the Python open source library demod, shows that it is possible to achieve high detail and high temporal resolution without compromising scalability. Furthermore, mosaik, an open source co-simulation framework, makes it possible to generate, integrate and orchestrate a multitude of demod-based instances with other independent models, which for the illustrative purposes of this study are represented by a heat pump model. The authors hope that the detailed documentation of the proposed solution will encourage interdisciplinary and collaborative efforts to develop a simulation ecosystem capable of investigating alternative energy transition pathways and evaluating policy interventions through the socio-technical lens.

Operation and Performance Assessment of a Hybrid Solar Heating and Cooling System for Different Configurations and Climatic Conditions

Energy needs of air conditioning systems are constantly growing worldwide, due to climate change and growing standards of buildings. Among the possible systems, solar heating and cooling based on reversible heat pumps and thermally driven chillers are a viable option for ensuring space heating and cooling for different users. The high installation costs are a limit to their diffusion, however, under specific circumstances (climate, type of the building, type of the user, etc.), the investment in this technology can be profitable in a long term. The presented paper describes an energy-economic assessment of a solar heating and cooling system integrating a solar dish concentrator with thermal collectors coupled with a reversible heat pump and an absorption or adsorption chiller. The system integrated with a household building is developed and dynamically simulated in the Transient System Simulation (TRNSYS) environment under different circumstances –adoption of absorption or adsorption chiller, use of auxiliary thermal energy to drive the sorption chillers, and locality. The results show that space cooling demand in Cracow is matched by solar energy, in a range between 49.0 and 97.6%, while for Naples the space cooling demand is provided by solar heat from 46.1 to 99.1% depending on the adopted sorption chiller and or the use of auxiliary heat for a natural gas boiler. The proposed system is not profitable in case Cracow, since a Simple Pay Back period of about 20 years is achieved. Conversely, case of Naples, the same index achieves a value between 8 and 12 years showing that the proposed system may be a viable solution for heating and cooling installation.

A unified seakeeping and maneuvering analysis of multiple linked towing system with triangular Bodies

Three-legged triangular tension leg platforms (TLPs), which are a kind of compliant-type floating structure, are drawing attention from industrial society. To reduce the transportation cost, a multiple linked ship towing system that the platform and anchor are linked and wet towed is studied. A unified seakeeping and maneuvering method which can take account of the hydrodynamic interaction between the bodies and utilize the experimental results for tuning the damping coefficients is proposed to model the towing system. In the method, a low-pass filter is applied to separate the slowly and fast motion components for the maneuvering and seakeeping calculations. The maneuvering and seakeeping problems can thus be solved simultaneously with a same time scale. SimMechanics™, which provides multi-body simulation environment for mechanical systems, is applied to implement the numerical model for studying the fully coupled six degrees-of-freedom (DOFs) motion of the towing system. Besides, experiments are carried out to demonstrate the feasibility of the system and the effectiveness of the numerical model. A circular motion in irregular waves is studied at the end and the issues of the system are discussed accordingly.