Agent-based Control Research Articles

Long-term experimental evaluation and comparison of advanced controls for HVAC systems

The tremendous energy usage from buildings leads to research studies on their improvement, among which advanced building control plays an important role. In advanced building controls, data-driven predictive control (DDPC), differentiable predictive control (DPC), and reinforcement learning (RL) have shown advantages, but their comparison often lacks in existing studies. The simulation-based prior comparison studies have inconsistent results due to different assumptions and simplifications. Therefore, to comprehensively compare the three advanced strategies for real-time building HVAC controls, we implemented DDPC, specifically, hierarchical DDPC (HDDPC), DPC, and RL in a real building testbed for more than 5 months. The results show that all three advanced controls maintained the indoor environmental quality (IEQ) cost-effectively. Overall, HDDPC outperformed the baseline control with more than 50% energy savings, followed by RL with 48%, and DPC with 30.6%. Most control failures were related to API communication issues. Besides, the information gaps between room and system level controllers and non-optimal control decisions will degrade HDDPC's performance. Such degradation did not happen in DPC and RL, which led to better performance of agent-based control over HDDPC. Moreover, HDDPC needs minutes to make control decisions whereas DPC and RL need milliseconds, indicating higher online computing resources required by HDDPC. For agent training, DPC is faster than RL, as DPC training needs minutes and RL needs hours, but its performance is not as good as RL. This study provides a comprehensive understanding and assessment of the pros and cons of advanced building controls and sheds light on future research on building controls.

Agent-Based System for Continuous Control and its Application to Activated Sludge Process

This paper presents a concept of architecture and ontology layouts for development of multiagent model-based predictive control systems. The presented architecture provides guidelines to simplify development of agent-based systems and improve its maintainability. The proposed multiagent system (MAS) layout is split into multiple subsystems that include agents dedicated to performing assigned tasks. MAS implementation was prepared which can use provided algorithms, actuators and can react to changes in its environment to reach the best available control quality. An example of MAS based on the proposed architecture is shown in application of dissolved oxygen (DO) concentration control in a laboratory activated sludge setup with biological reactor. For that application, MAS incorporates agent-based controllers from Boundary-Based Predictive Controllers (BBPC) family. Presented experiments prove flexibility, resilience and online reconfiguration ability of the proposed multiagent system.

Water level control of nuclear steam generators using intelligent hierarchical autonomous controller

The challenge of water level control in steam generators, particularly at low power levels, has always been a critical aspect of nuclear power plant operation. To address this issue, this paper introduces an IHA controller. This controller employs a CPI controller as the primary controller for direct water level control, coupled with an agent-based controller optimized through a DRL algorithm. The agent dynamically optimizes the parameters of the CPI controller in real-time based on the system’s state, resulting in improved control performance. Firstly, a new observer information is obtained to get the accurate state of the system, and a new reward function is constructed to evaluate the status of the system and guide the agent’s learning process. Secondly, a deep ResNet with good generalization performance is used as the approximator of action value function and policy function. Then, the DDPG algorithm is used to train the agent-based controller, and an advanced controller with good performance is obtained after training. Finally, the popular UTSG model is used to verify the effectiveness of the algorithm. The results demonstrate that the proposed method achieves rise times of 73.9 s, 13.6 s, and 16.4 s at low, medium, and high power levels, respectively. Particularly, at low power levels, the IHA controller can restore the water level to its normal state within 200 s. These performances surpass those of the comparative methods, indicating that the proposed method excels not only in water level tracking but also in anti-interference capabilities. In essence, the IHA controller can autonomously learn the control strategy and reduce its reliance on the expert system, achieving true autonomous control and delivering excellent control performance.

Analysis on Intelligent Agent based Approach for Software Engineering

A broad area of research, known as "Agent-Based Computing", focuses on developing "agent-based" intelligent software using agent-based techniques. However, there is a scarcity of research focusing on providing enough evidence of the superiority of agent-based techniques in creating complex software systems. This article has attempted to provide evidence for why agent-based techniques are superior to traditional methods for creating complex software systems, such as control systems. A case of a distinct agent-based control system (the power transportation management system used by Iber-26 drola) is used to illustrate these broader principles. This line of reasoning allows advocates of complex software engineering paradigms to accurately assert that their method can replicate the essential ideas behind agent-based computing. When broken down to their most basic components, agent-based systems are just computer programmes, and every programme has the same set of computable functionalities. The value of a paradigm lies in the way of thinking and the tools it teaches to programmers. As such, agent-based ideas and approaches are not just an extension of those now accessible within existing paradigms, but also well suited to the development of large, networked systems.

Demonstration of automated curative power flow control with a multi-agent system using distributed series reactors in a PHIL simulation

This paper demonstrates the real-time applicability of an automated curative power flow control system in a High Voltage (HV) grid. The system is realized in a laboratory environment as an agent-based control algorithm implemented on commercially available hardware. Measurements and control signals are exchanged between a Real-Time Simulation (RTS) of an HV grid model and the agent system. To control power flows, the agents use simulated flexible loads and feeders as well as physical Distributed Series Reactors (DSRs), which are looped into the RTS via a Power Hardware-in-the-Loop (PHIL) interface. The control algorithm is tested in various scenarios and the agents’ performance is analyzed regarding overload reduction and reaction time. The results show that the heuristic agent algorithm can solve congestions ad-hoc and improve the overall system state. The agents’ reaction times depend on measurement availability: While a fast measurement sending rate can enable the agents’ base reaction time of approximately 11s, longer intervals between measurements - which may be favorable from a control center operator’s point of view - can cause longer delays until congestions are solved. These results put previous works in perspective, which did not consider real-time simulation and proposed possible agent reactions within milliseconds.

Learning to cooperate in agent-based control of queueing networks

Learning to cooperate in agent-based control of queueing networks

Distributed control of battery energy storage systems in distribution networks for voltage regulation at transmission–distribution network interconnection points

This paper describes a control framework that enables distributed battery energy storage systems (BESS) connected to distribution networks (DNs) to track voltage setpoints requested by the transmission system operator (TSO) at specific interconnection points in an optimal and coordinated manner. The control design is based on an optimisation problem whose objective is to minimise the real-time voltage-tracking mismatch while satisfying local physical network constraints. A novel agent-based control scheme adopting an online convex optimisation (OCO) framework is developed and solved in a distributed fashion to guarantee the solution’s scalability and the service provision within the required time. The BESS agents under the proposed control framework automatically adapt to the time-varying network conditions so as to track the required voltage setpoints whilst fulfilling the technical operating requirements of the local network. The designed OCO-based framework addresses the existing conflict between the accuracy and optimality of the solution and the communication and computational efficiency. The convergence to the optimal solution is demonstrated. Several case studies are performed to corroborate the analytical results and assess the performance of the proposed approach.

Agent-based target evaluation and fire doctrine: an aspect-oriented programming view

Threat evaluation is a vital process for any defense system, and it consists of a series of calculation and evaluation based on the inferred capabilities and intents of the targets that aim to give damage to defended assets. Target evaluation is proceeded in a wargame and the aim is to compare weapon target pairs according to a set of criteria. The target evaluation cycle is repeated anytime a new detection is received and when any change happens in the target currently detected. The whole process consists of a set of tasks that are shared between Command and Control units and the tasks require different responsibilities. Each task is succeeded by a specific behavior that is represented as a reasonably ordered set of actions. The task sharing is organized by taking the C2 architectures into account. In this paper, an agent-based command and control entity, which is in charge of target evaluation and giving engagement decision, is designed and it is situated in an air defense simulation environment. The study aims to propose an agent design in military decision-making domain, bringing analytic methods with the first-order logic together, and combine aspect orientation with agent design. The study also improves dynamic aspect management in agent programming using the relation concept.

Agent-based access control framework for enterprise content management

Telework is an important alternative to work that seeks to enhance employees’ safety and well-being while reducing the company costs. Employees can work anytime, any where and under high mobility conditions using new devices. Therefore, the access control of remote exchanges of Enterprise Content Management systems (ECM) have to take into consideration the diversity of users’ devices and context conditions in a telework open network. Different access control models were proposed in the literature to deal with the dynamic nature of users’ context and devices. However, most access control models rely on a centralized management of permissions by an authorization entity which can reduce its performance with the increase of number of users and requests in an open network. Moreover, they often depend on the administrator’s intervention to add new devices’ authorization and to set permissions on resources. In this paper, we suggest a distributed management of access control for telework open networks that focuses on an agent-based access control framework. The framework uses a multi-level rule engine to dynamically generate policies. We conducted a usability test and an experiment to evaluate the security performance of the proposed framework. The result of the experiment shows that the ability to resist deny of service attacks over time increased in the proposed distributed access control management compared with the centralized approach.

Agent-Based Coordinated Control of Power Electronic Converters in a Microgrid

This paper presents the implementation of an agent-based architecture suitable for the coordination of power electronic converters in stand-alone microgrids. To this end, a publish-subscribe agent architecture was utilized as a distributed microgrid control platform. Over a distributed hash table (DHT) searching overlay, the publish-subscribe architecture was identified based on a numerical analysis as a scalable agent-based technology for the distributed real-time coordination of power converters in microgrids. The developed framework was set up to deploy power-sharing distributed optimization algorithms while keeping a deterministic time period of a few tens of milliseconds for a system with tens of converters and when multiple events might happen concurrently. Several agents participate in supervisory control to regulate optimum power-sharing for the converters. To test the design, a notional shipboard system, including several converters, was used as a case study. Results of implementing the agent-based publish-subscribe control system using the Java Agent Development Framework (JADE) are presented.

Relaxing Platform Dependencies in Agent-Based Control Systems

Agent-based systems have been widely used to develop industrial control systems when they are required to address issues such as flexibility, scalability and portability. The most common approach to develop such control systems is with agents embedded in a platform that provides software libraries and runtime services that ease the development process. These platforms also bring challenges to the agent-based control system engineering. For example, they might introduce default design features, such as a global directory of agents. Furthermore, agents are generally locked in a platform and depend on the platform’s available support for deployment across computing infrastructures. This article addresses these challenges through an approach for building agent-based control systems, that relaxes the dependencies in multiagent system (MAS) platforms, through the use of container-based virtualisation. The proposed approach is elaborated via a reference architecture that enables the implementation of agents as self-contained applications that can be deployed, on-demand, in independent environments but still are able to communicate and coordinate with other agents of the control system. We built a prototype using this approach and evaluated it in the context of a case study for the supervisory control of digital network infrastructures. This case study enabled us to demonstrate feasibility of the approach and to show the flexibility, of the resulting control system, to adopt several topologies as well as to operate at different scales, over emulated networks. We also concluded that designing agents as individual deployment units is also cost-effective especially in control scenarios with low number of stable agents.

Planning the Future of Smart Cities With Swarms of Fully Autonomous Unmanned Aerial Vehicles Using a Novel Framework

The autonomy of unmanned aerial vehicles (UAVs) - self-governing in the aerospace discipline has been a remarkable research area with the development of the advanced bespoke microcontrollers embedded with advanced AI techniques for the last several decades. The road forward about the operational environment is certain about the swarms of fully automated UAVs (FAUAVs), that is, urban areas. FAUAVs with self-learning and self-decision-making abilities by executing non-trivial sequences of events with decimetre-level accuracy based on a set of rules, control loops and constraints using dynamic flight plans and trajectories are taking their indispensable parts within smart cities (SCs). Therefore, their integration with the SC components using real-time data analytics is urgent. This is mainly required to establish a better swarm intelligence along with a safer and optimised harmonious smart ecosystem that enables cooperative FAUAV-SC automation systems with collaborative automated intelligence engaging in the concepts of Internet of Everything (IoE) and Automation of Everything (AoE). Planning the future of cities with swarms of FAUAVs is explored in this paper to optimise the use of FAUAVs with a diverse range of applications and a contemporary methodology is proposed using a holistic framework - FAUAVinSCF equipped with various effective and efficient techniques along with a novel FAUAV routing technique customisable to the constraints of FAUAVs and urban areas. With the methodology, the components of SC and FAUAVs involving recent and impending technological advancements are moulded together to make this inevitable transformation a harmonious part of the inhabitants contributing to the cities' liveability and sustainability. The framework consists of a decentralized agent-based control architecture that monitors and controls the swarms of resource-constraint FAUAVs for their real-time requirements in optimising their urban uses. The outcomes of the methodology suggest that the constraints of FAUAVs can be mitigated significantly in urban areas and consequently, their efficacy can be increased in realising their diverse range of missions.

Scalable assessment method for agent‐based control in cyber‐physical distribution grids

This study proposes a scalable method to assess agent-based control concepts and deployment systems for cyber-physical electrical distribution grids. The deployment system of an agent-based control concept is the system that executes the agents and their interactions in a real implementation. Due to the increasing number of controllable loads, generators, and storage systems installed in distribution grids, scalability of control concepts and deployment systems plays a key role. Hence, the assessment method for both has to be scalable itself. For this reason, the proposed method bases on DistAIX, an open-source scalable simulation tool for cyber-physical distribution grids designed for execution on distributed computing clusters. DistAIX is extended with an interface for coupling with the deployment system of the agent-based control approach. The performance of the proposed method is evaluated for an agent-based control concept called SwarmGrid-X and the deployment system cloneMAP. The results demonstrate the feasibility of the proposed method and exemplify the functionality and scalability of SwarmGrid-X and cloneMAP.

Design, Application, and Evaluation of a Multiagent System in the Logistics Domain

The increasing demand for flexibility of automated production systems also affects the automated material flow systems (aMFS) they contain and demands reconfigurable systems. However, the centralized control concept usually applied in aMFS hinders an easy adaptation, as the entire control software has to be re-tested, when manually changing sub-parts of the control. As adaption and subsequent testing are a time-consuming task, concepts for splitting the control from one centralized to multiple, decentralized control nodes are required. Therefore, this paper presents a holistic agent-based control concept for aMFS, whereby the system is divided into so-called automated material flow modules (aMFM), each being controlled by a dedicated module agent. The concept allows the reconfiguration of aMFS, consisting of heterogeneous, stationary aMFM, during runtime. Furthermore, it includes aspects such as uniform agent knowledge bases through metamodel-based development, a communication ontology considering different information types and properties, strategic route optimization in decentralized control architecture and a visualization concept to make decisions of the module agents comprehensible to operators and maintenance staff. The evaluation of the concept is performed by means of material flow simulations as well as a prototypical implementation on a lab-sized demonstrator.

A Toolchain for the Development of Agent-Based Smart Grid Control Solutions

The energy transition necessitates a variety of control solutions to integrate renewables-based energy generation and unlock demand-side flexibility. To contribute to solving this challenge, the authors built a toolchain for developing smart grid control solutions. This toolchain consists of methodological, conceptual and technological components. In this contribution, we detail the way in which the toolchain facilitates the continuous utilization of control software artifacts across simulations, testbeds and deployment. This capability has the potential to significantly reduce engineering effort and build trust in agent-based control solutions for smart grids.

Comparing Agent-based Control Architectures For Next Generation Telecommunication Network Infrastructures

Multi-agent systems have been an effective choice for designing control systems that are flexible and agile. However, few attention has been given to the evaluation of the architectures of such systems. This becomes critical with the emerging requirements in complex domains such as digital network infrastructures. In this paper, we propose an approach for the evaluation of agent-based control architectures and introduce three multi-agent based architectures for the supervisory control of network service operations of the next generation of digital infrastructures. With the proposed approach, we evaluated the architectures and the implemented control systems prototypes under a realistic network infrastructure environment. Our approach has been effective to evaluate the candidate architectures. The results of communication overhead and reaction time, have shown that agent-based hierarchical and heterarchical-ring architectures have outperformed the heterarchical-complete network architecture.

Agent‐based system to control the air‐conditioner and EV charging for residents in smart cities

Air-conditioner (AC) accounts for a significant share of residential energy consumption. Considering the widespread rise in electric vehicle (EV) usage, its charging would also contribute a considerable percentage of consumer's total energy consumption. Consequently, the concurrent operation of AC and EV charging would result in peaky load curves. Hence, this study proposes a system of agents for AC and EV charging applications, which incorporates load-management strategies to flatten the load curve. Thereby, the presented system includes two agents, namely: a smart load node for a thermostatically controlled load (SLN-TCL) and a smart battery charge controller. Subsequently, a subagent, namely micro-node, has been introduced to support SLN-TCL and to implement the concept of distributed temperature sensing (DTS). The implementation of DTS subdues the conventional temperature sensing mechanism of AC and ensures a more flexible operation. This study includes the design, development, and features of agents and subagents for AC and EV applications. Furthermore, this study also demonstrates the agent-based control actions for peak-shaving under real conditions to showcase the performance of this system.

A cloud-based operation optimization of building energy systems using a hierarchical multi-agent control

This work presents an agent-based control concept that we integrate into a cloud framework for controlling building energy systems. The agents are arranged in a hierarchical structure, where a coordinator agent sends optimized set point values to sub-agents. Each sub-agent controls a subsystem in order to reach the given set points and provides the coordinator agent with a cost function for the overall set point optimization. The multi-agent system and the building energy system exchange data via the cloud framework FIWARE. The components of the building energy system (e.g. boiler, air-handling unit) are connected to the cloud framework and send measurements (e.g. temperature values or volume flow) via a scalable IoT-Interface to the corresponding data object in the platform. The sub-agents of the agent control receive the measurements and send the calculated control action back to the corresponding components of the energy system. In a simulation study, we use the framework to control an air-handling unit. The aim of the agent control is to reach a given supply air temperature while reducing the total consumed energy. The implemented cloud-based control is capable of efficiently reaching the given temperature set points while communicating via the IoT interface.

Towards modular composition of agent-based voltage control concepts

In the last years, diverse agent-based concepts for voltage control in distribution grids were presented in literature. All these approaches are developed manually. Up to now, no tailoring approach has been presented to adapt these concepts to different – likewise specific – grid situations. As the effectiveness of voltage control schemes is highly dependent on the specific grid situation, this can result in multiple problems with regard to performance, applicability and expandability of the proposed agent-based control. The ideal case would be a customizable agent system that is automatically tailored to the grid it shall be applied to, considering its specific characteristics. To address that complex task, this paper proposes an approach for the modular composition of voltage control agents by recombining existing concepts from literature to new fully functional agent systems. This approach makes the first move to an optimized and scenario-specific creation of agent-based control systems that are specifically designed and automatically tailored to a given power system.

Agent-based Indoor Air Quality Control System

Agent-based Indoor Air Quality Control System