Multi-agent Simulation Environment Research Articles

A Multi-Agent Centralized Strategy Gradient Reinforcement Learning Algorithm Based on State Transition

The prevalent utilization of deterministic strategy algorithms in Multi-Agent Deep Reinforcement Learning (MADRL) for collaborative tasks has posed a significant challenge in achieving stable and high-performance cooperative behavior. Addressing the need for the balanced exploration and exploitation of multi-agent ant robots within a partially observable continuous action space, this study introduces a multi-agent centralized strategy gradient algorithm grounded in a local state transition mechanism. In order to solve this challenge, the algorithm learns local state and local state-action representation from local observations and action values, thereby establishing a “local state transition” mechanism autonomously. As the input of the actor network, the automatically extracted local observation representation reduces the input state dimension, enhances the local state features closely related to the local state transition, and promotes the agent to use the local state features that affect the next observation state. To mitigate non-stationarity and reliability assignment issues in multi-agent environments, a centralized critic network evaluates the current joint strategy. The proposed algorithm, NST-FACMAC, is evaluated alongside other multi-agent deterministic strategy algorithms in a continuous control simulation environment using a multi-agent ant robot. The experimental results indicate accelerated convergence and higher average reward values in cooperative multi-agent ant simulation environments. Notably, in four simulated environments named Ant-v2 (2 × 4), Ant-v2 (2 × 4d), Ant-v2 (4 × 2), and Manyant (2 × 3), the algorithm demonstrates performance improvements of approximately 1.9%, 4.8%, 11.9%, and 36.1%, respectively, compared to the best baseline algorithm. These findings underscore the algorithm’s effectiveness in enhancing the stability of multi-agent ant robot control within dynamic environments.

Deep contextual reinforcement learning algorithm for scalable power scheduling

This article introduces a deep contextual reinforcement learning (DCRL) optimization algorithm to tackle the NP-hard power scheduling problem. The algorithm is based on a multi-agent simulation environment that decomposes the power scheduling problem into sequential Markov decision processes (MDPs). The simulation environment inherently corrects incorrect decisions and adjusts supply capacities so that the MDPs adhere to the optimization constraints. A deep reinforcement learning (DRL) model is trained on these MDPs to provide optimal solutions. Demonstrating its applicability and effectiveness, the proposed method is evaluated on various test systems with different unit numbers, constraints, and production cost functions. The experimental results show that the proposed method has better performance relative to alternative methods, such as binary alternative moth-flame optimization (BAMFO), binary particle swarm optimization (BPSO), teaching learning-based optimization (TLBO), new binary particle swarm optimization (NBPSO), binary learning particle swarm optimization (BLPSO), quasi-oppositional teaching learning-based algorithm (QOTLBO), binary-real-coded genetic algorithm (BRCGA), hybrid genetic-imperialist competitive algorithm (HGICA), three-stage priority list (TSPL), and hybridized evolutionary algorithm and sequential quadratic programming (HEPSQP). The novelties of the proposed method lie in its adaptability to longer planning horizons and linear dimensionality complexity, rendering it suitable for large-scale power systems.

Reinforcement learning-based optimization for power scheduling in a renewable energy connected grid

Power scheduling is an NP-hard optimization problem that demands a delicate equilibrium between economic costs and environmental emissions. In response to the growing concern for climate change, global environmental policies prioritize decarbonizing the electricity sector by integrating renewable energies (REs) into power grids. While this integration brings economic and environmental benefits, the intermittency of REs amplifies the uncertainty and complexity of power scheduling. Existing optimization approaches often grapple with a limited number of units, overlook critical parameters, and disregard the intermittency of REs. To address these limitations, this article introduces a robust and scalable optimization algorithm for renewable integrated power scheduling based on reinforcement learning (RL). In this proposed methodology, the power scheduling problem is decomposed into Markov decision processes (MDPs) within a multi-agent simulation environment. The simulated MDPs are used to train a deep reinforcement learning (DRL) model for solving the optimization. The validity and effectiveness of the proposed method are validated across various test systems, encompassing single-to tri-objective problems with 10–100 generating units. The findings consistently demonstrate the superior performance of the proposed DRL algorithm compared to existing methods, such as multi-agent immune system-based evolutionary priority list (MAI-EPL), binary real-coded genetic algorithm (BRCGA), teaching learning-based optimization (TLBO), quasi-oppositional teaching learning-based algorithm (QOTLBO), hybrid genetic-imperialist competitive algorithm (HGICA), three-stage priority list (TSPL), real-coded grey wolf optimization (RCGWO), multi-objective evolutionary algorithm based on decomposition (MOEAD), and non-dominated sorting algorithms (NSGA-II and NSGA-III). Regarding the experimental results, it is important to highlight the importance of integrating RESs into larger power systems. In a 10-unit system with 2.81 % RE penetration, reductions of 3.42 %, 4.03 %, and 3.10 % were observed in costs, CO2 emissions, and SO2 emissions, respectively. Similarly, in a 100-unit system with a RE penetration rate of only 0.28 %, reductions of 3.75 % in cost, 4.42 % in CO2, and 3.34 % in SO2 were observed. These findings emphasize the effectiveness of RES integration, even at lower penetration rates, in larger-scale power systems.

A multi-agent reinforcement learning approach for ART adaptive control in automated container terminals

Intelligent transportation equipment represented by Artificial Intelligence Robot of Transportation (ART) has been gradually applied to automated container terminals (ACT). ARTs have higher intelligence and distributed autonomous decision-making capability compared to AGVs. The production and operation of ACT require strict loading plans based on container types and destination ports, which puts strict requirements on the ART loading task sequence. ART travel time from the yard to the quay is highly uncertain due to special scenarios such as traffic flow convergence and turning in narrow areas, which will lead to long waits for ART due to out-of-sequence arrivals at the quay. Therefore, it is necessary to regulate the speed of ART to adjust the timing of ART arriving at the quay. However, existing scheduling systems tend to ignore the impact of ART speed and loading sequence constraints on terminal operations. To optimize the multi-ART control problem of ACT, the waiting time of ART at the quay is taken as the optimization goal, and the speed regulation of ART during each task is defined as a Markov decision process. Then, by incorporating reinforcement learning and a multi-agent approach, an adaptive decision-making method is developed to generate the optimal speed regulation strategy. This method interacts with the multi-agent simulation environment to obtain real-time status information and learns strategies online to improve the intelligence of individual agents and the collaborative capabilities of the agent group. Finally, a multi-agent simulation model embedded with a reinforcement learning agent is established on AnyLogic software. Simulation results show that this method can effectively reduce the waiting time of ART, improve the coherence and efficiency of loading operations, and exhibit good performance in online scenarios that consider emergency tasks. The results indicate that the speed of ART in ACTs with parallel layout of the yard can impact the operational efficiency of the system. Furthermore, by appropriately defining the reward function and training method, collaborative behaviour among equipment can be enhanced, ultimately optimizing the overall performance of the system.

Reinforcement Learning-Based Multi-Objective Optimization for Generation Scheduling in Power Systems

Multi-objective power scheduling (MOPS) aims to address the simultaneous minimization of economic costs and different types of environmental emissions during electricity generation. Recognizing it as an NP-hard problem, this article proposes a novel multi-agent deep reinforcement learning (MADRL)-based optimization algorithm. Within a custom multi-agent simulation environment, representing power-generating units as collaborative types of reinforcement learning (RL) agents, the MOPS problem is decomposed into sequential Markov decision processes (MDPs). The MDPs are then utilized for training an MADRL model, which subsequently offers the optimal solution to the optimization problem. The practical viability of the proposed method is evaluated across several experimental test systems consisting of up to 100 units featuring bi-objective and tri-objective problems. The results demonstrate that the proposed MADRL algorithm has better performance compared to established methods, such as teaching learning-based optimization (TLBO), real coded grey wolf optimization (RCGWO), evolutionary algorithm based on decomposition (EAD), non-dominated sorting algorithm II (NSGA-II), and non-dominated sorting algorithm III (NSGA-III).

Neurodynamic approaches for multi-agent distributed optimization

This paper considers a class of multi-agent distributed convex optimization with a common set of constraints and provides several continuous-time neurodynamic approaches. In problem transformation, l1 and l2 penalty methods are used respectively to cast the linear consensus constraint into the objective function, which avoids introducing auxiliary variables and only involves information exchange among primal variables in the process of solving the problem. For nonsmooth cost functions, two differential inclusions with projection operator are proposed. Without convexity of the differential inclusions, the asymptotic behavior and convergence properties are explored. For smooth cost functions, by harnessing the smoothness of l2 penalty function, finite- and fixed-time convergent algorithms are provided via a specifically designed average consensus estimator. Finally, several numerical examples in the multi-agent simulation environment are conducted to illustrate the effectiveness of the proposed neurodynamic approaches.

Experimental Evaluation of AGV Dispatching Methods in an Agent-Based Simulation Environment and a Digital Twin

A critical part of Automated Material Handling Systems (AMHS) is the task allocation and dispatching strategy employed. In order to better understand and investigate this component, we here present an extensive experimental evaluation of three different approaches with randomly generated, as well as custom designed, environment configurations. While previous studies typically focused on use cases based on highly constrained navigation capabilities (e.g., overhead hoist transport systems), our evaluation is built around highly mobile, free-ranging vehicles, i.e., Autonomous Mobile Robots (AMR) that are gaining popularity in a broad range of applications. Consequently, our experiments are conducted using a microscopic agent-based simulation, instead of the more common discrete-event simulation model. Dispatching methods often are built around the assumption of the asynchronous evaluation of an event-based model, i.e., vehicles trigger a cascade of individual dispatching decisions, e.g., when reaching intersections. We find that this does not translate very well to a fleet of highly mobile systems that can change direction at any time. With this in mind, we present formulations of well known dispatching approaches that are better suited for a synchronous evaluation of the dispatching decisions. The formulations are based on the Stable Marriage Problem (SMP) and the Linear Sum Assignment Problem (LSAP). We use matching and assignment algorithms to compute the actual dispatching decisions. The selected algorithms are evaluated in a multi-agent simulation environment. To integrate a centralised fleet management, a digital twin concept is proposed and implemented. By this approach, the fleet management is independent of the implementation of the specific agents, allowing to quickly adapt to other simulation-based or real application scenarios. For the experimental evaluation, two new performance measures related to the efficiency of a material handling system are proposed, Travel Efficiency and Throughput Effort. The experimental evaluation indicates that reassignment mechanisms in the dispatching method can help to increase the overall efficiency of the fleet. We did not find significant differences in absolute performance in terms of throughput rate. Additionally, the difference in performance between SMP- and LSAP-based dispatching with reassignment seems negligible. We conclude with a discussion, where we consider potential confounding factors and relate the findings to previously reported results found in the literature.

Multi-agent-based collaborative regulation optimization for microgrid economic dispatch under a time-based price mechanism

• This paper provides a multi-agent based coordinated dispatch strategy for the economic dispatch of the microgrid under a time-based price mechanism. • We develop a microgrid optimization model for the microgrid operation process, which includes battery regulation and user satisfaction. • The established optimization model is solved using a MACPSO algorithm, and the agent communication mechanism in the microgrid is examined. • The performance comparison using different algorithms (PSO, IPSO, MAPSO and MACPSO) has been presented. • The proposed multi-agent coordinated dispatch strategy is validated by designing a multi-agent system simulation environment and using the Java agent development (JADE) framework. The economic optimal dispatch of a microgrid is a challenging task with significant economic and social implications. Under a time-based price mechanism, this paper proposes a multi-agent-based coordinated dispatch strategy for the microgrid's economic dispatch. The information between the agents in the microgrid can be fed back in time to reduce microgrid operation failures by coordinated control between the demand-side load, the battery, and the power generation side micro-sources. The output of other micro-sources is then coordinated using an appropriate battery charging and discharging plan based on the characteristics of the time-based electricity price mechanism. Following that, an economical microgrid operation model is established and solved using a multi-agent chaotic particle swarm optimization (MACPSO) algorithm, which considers user satisfaction. Finally, a multi-agent system (MAS) simulation environment is built using the Java agent development (JADE) framework. The proposed microgrid with multi-agent coordinated control can effectively improve its operation efficiency, according to a comparative analysis of examples.

Boundary consensus control strategies for fractional-order multi-agent systems with reaction-diffusion terms

Multi-agent systems (MASs), as an essential research topic of distributed artificial intelligence, mainly refer to the coordination behavior among the agents to achieve a global goal. The issue dealing with the security threats of the oil and gas industry has always attracted the attention of numerous researchers. Meanwhile, MASs play a significant role in the security and defense of oil pipelines. This paper mainly studies the spatial boundary control for consensus of fractional-order MASs with reaction–diffusion term. First of all, two kinds of consensus boundary control methods are studied in fractional-order MASs, both based on Neumann boundary conditions. Furthermore, by employing fractional-order inequalities, consensus criteria of the MASs are presented in terms of LMIs. Finally, two numerical examples in the multi-agent simulation environment indicate the effectiveness of the proposed theoretical results.

A Socio-Inspired Methodology and Model for Advanced and Opportunistic Interactions between Industrial IoT Objects

The concept of the Internet of Things (IoT) is widely discussed. IoT is one of the emerging technologies that have caught the attention of many researchers. The increase in the number of exchanges of services between heterogeneous or homogeneous connected objects with the integration of social networking concepts gives rise to the concept of the Social Internet of Things (SIoT). The SIoT concept contributes to the evolution of interactions between industrial objects by improving deterministic mechanisms towards intelligent interactions. The integration of the SIoT concept into the Industrial Internet of Things (IIoT) gives rise to the Social Internet of Industrial Things (SIoIT) and plays an important role in improving system performance in Industry 4.0. In this article, we propose an innovative methodology and a model of socio-inspired interaction between industrial communicating objects inspired by sociological approaches. Thanks to this model, socialized industrial communicating objects form a community of objects, autonomously and dynamically, by exchanging messages to know each other perfectly, and service requests between objects are executed adaptively according to the principles of social interaction governed by socio-inspired strategies and conditions. The model is implemented, tested and validated in a Netlogo multi-agent system simulation environment.

Knowledge transfer in multi-agent reinforcement learning with incremental number of agents

In this paper, the reinforcement learning method for cooperative multi-agent systems (MAS) with incremental number of agents is studied. The existing multi-agent reinforcement learning approaches deal with the MAS with a specific number of agents, and can learn well-performed policies. However, if there is an increasing number of agents, the previously learned in may not perform well in the current scenario. The new agents need to learn from scratch to find optimal policies with others, which may slow down the learning speed of the whole team. To solve that problem, in this paper, we propose a new algorithm to take full advantage of the historical knowledge which was learned before, and transfer it from the previous agents to the new agents. Since the previous agents have been trained well in the source environment, they are treated as teacher agents in the target environment. Correspondingly, the new agents are called student agents. To enable the student agents to learn from the teacher agents, we first modify the input nodes of the networks for teacher agents to adapt to the current environment. Then, the teacher agents take the observations of the student agents as input, and output the advised actions and values as supervising information. Finally, the student agents combine the reward from the environment and the supervising information from the teacher agents, and learn the optimal policies with modified loss functions. By taking full advantage of the knowledge of teacher agents, the search space for the student agents will be reduced significantly, which can accelerate the learning speed of the holistic system. The proposed algorithm is verified in some multi-agent simulation environments, and its efficiency has been demonstrated by the experiment results.

Connected and Autonomous Vehicle Cohort Speed Control Optimization via Neuroevolution

Predictive Energy Management (PrEM) research is at the forefront of modern transportation’s energy consumption reduction efforts. The development of PrEM optimization algorithms has been tailored to selfish vehicle operation and implemented in the form of vehicle dynamics and/or adaptive powertrain control functions. With the progress in vehicle automation, this paper focuses on extending PrEM into the realm of a System of Systems (SoS). The proposed approach uses the shared information among Connected and Automated Vehicles (CAV) and the infrastructure to synthesize a reduced energy speed trajectory at the cohort level within urban environments. Neuroevolution is employed to incorporate a generalized optimum controller, robust to the emergent behaviors typical of multi-agents SoS. The authors demonstrated the use of heuristics and systems engineering processes in abstracting and integrating the resulting neural network within the control architecture, which enables novel added-value features such as green wave pass/fail classification and e-Horizon velocity prediction. The resulting controller is faster than real-time and was validated with a multi-agent simulation environment and on a real-world closed-loop track at the American Center for Mobility (ACM). The GM Bolt and Volt CAV mixed cohort testing at ACM demonstrated energy reductions from 7% to 22% depending on scenarios.

Socialization of Smart Communicative Objects in Industrial Internet of Things

In the recent past the Internet of Things (IoT) is moving from interactive objects to smart objects. The number of smart devices or sensors built-in becomes larger. The increase in the number of services exchanged between connected objects has led to the creation of a new concept called Social IoT (SIoT), in which heterogeneous or homogeneous objects can interact and request or exchange services, which makes it possible to include social or human actions in the IoT platform. The use of technologies based on the concept of the Internet of Things in applications and industrial sectors has built a new concept of research called the Industrial Internet of Things (IIoT). The integration of the SIoT concept into the Industrial Internet of Things (IIoT) plays an important role in improving system performance in Industry 4.0 and gives rise to the Social Internet of Industrial Things (SIoIT). In this context, we have revealed a new socio-inspired interaction model between industrial communicating objects inspired by sociological approaches that transform objects into socialized industrial communicating objects. These objects form a community, autonomously and dynamically, by exchanging messages to know each other perfectly, and service requests between objects are realized in an adaptive way according to the principles of social interaction governed by socio-inspired strategies and conditions. The model is evaluated using Netlogo multi-agent system simulation environment.

A novel search and survey technique for unmanned aerial systems in detecting and estimating the area for wildfires

In recent years Unmanned Aerial Vehicles (UAVs) have progressively been utilized for wildfire management, and are especially in prevalent in forest fire monitoring missions. To ensure the fast detection and accurate area estimation of forest fires, a two-step search and survey algorithm for multi-UAV system is proposed to address these fire scenarios. Initially, a grid-based partition method is applied to divide the area-of-interest into several search areas. Then, an archetype search pattern is used to provide timely UAV exploration within those sub-areas. Once the fire zones are detected, a novel survey strategy is employed for UAVs to discover the boundary points of the fire zones, so that the area of the fire zones can be estimated using the sampled boundary points. In addition, the effect of wind is accounted for improving fire zone boundary estimates. The proposed search-and-survey procedure is validated on multiple simulated scenarios using the U.S. Air Force’s mission-realistic Aerospace Multi-Agent Simulation Environment (AMASE) software. Simulation results showcase that the proposed search pattern can effectively discover the seeded fire zones within 40 min of the mission. This is relatively faster than the other two well-known search patterns. Moreover, the proposed survey technique provides a coverage estimate with at least 85% accuracy for the area of interest within 90 min of the mission.

Multi-Unmanned-Aerial-Vehicle Wildfire Boundary Estimation Using a Semantic Segmentation Neural Network

This paper presents a system to command and control a team of fixed-wing unmanned aerial vehicles (UAVs) to sense dynamic wildfire boundaries. UAV team task and trajectory planning strategies enable the team to rapidly find, rally around, and map the wildfire boundaries. A novel boundary estimation algorithm generates two-dimensional concave polygonal estimates of multiple dynamic boundaries given sparse observation data. The algorithm was tested with simulated wildfire scenario binary fire or free-point observations collected by the UAV team. First, all gathered observations are used to classify groups of points into clusters belonging to individual wildfires; then, spatiotemporal information from wildfire observations is encoded as an image with observation age represented as pixel brightness. A neural network performs semantic segmentation on each image and outputs a predicted binary image of the wildfire. This image is decoded back into a point set that feeds into a boundary estimation algorithm (Polylidar) to extract a concave boundary. Benchmarks for planner and boundary estimation times and accuracy comparisons are provided. Our boundary estimation algorithm and supporting multiagent planning strategies were used to win the 2019 U.S. Air Force Research Laboratory’s Swarm and Search Wildfire challenge using the aerospace multiagent simulation environment.

Multi-Agent Simulation Environment for Logistics Warehouse Design Based on Self-Contained Agents

We propose a multi-agent simulation environment for logistics warehouses. Simulation is a crucial part of designing industrial systems, such as logistics warehouses. A warehouse is a multi-agent system (MAS) that consists of various autonomous subsystems with robots, material-handling equipment, and human workers. It is generally difficult to analyze the performance of a MAS thus, it is important to model a warehouse and conduct simulations to design and evaluate the possible system configurations. However, the cost of modeling warehouses and modifying the models is high because there are various components and interactions compared to conventional multi-agent simulations. We proposed a self-contained agent architecture and message architecture of a multi-agent simulation environment for logistics warehouses to reduce the simulation-model development and modification costs. We quantitatively evaluated our environment in terms of development costs by comparing such costs of our environment and a widely used multi-agent simulation environment.

Inferring User Intent using Bayesian Theory of Mind in Shared Avatar-Agent Virtual Environments.

We present a real-time algorithm to infer the intention of a user's avatar in a virtual environment shared with multiple human-like agents. Our algorithm applies the Bayesian Theory of Mind approach to make inferences about the avatar's hidden intentions based on the observed proxemics and gaze-based cues. Our approach accounts for the potential irrationality in human behavior, as well as the dynamic nature of an individual's intentions. The inferred intent is used to guide the response of the virtual agent and generate locomotion and gaze-based behaviors. Our overall approach allows the user to actively interact with tens of virtual agents from a first-person perspective in an immersive setting. We systematically evaluate our inference algorithm in controlled multi-agent simulation environments and highlight its ability to reliably and efficiently infer the hidden intent of a user's avatar even under noisy conditions. We quantitatively demonstrate the performance benefits of our approach in terms of reducing false inferences, as compared to a prior method. The results of our user evaluation show that 68.18% of participants reported feeling more comfortable in sharing the virtual environment with agents simulated with our algorithm as compared to a prior inference method, likely as a direct result of significantly fewer false inferences and more plausible responses from the virtual agents.

Application of adaptive strategy for supply chain agent

With the tremendous increase in the globalization of trade the corresponding supply chains supporting the manufacture, distribution and supply of goods has become extremely complex. Intelligent agents can help with the problem of effective management of these complex supply chains. In this paper we introduce the design, implementation and testing of an intelligent agent for handling procurement, customer sales, and scheduling of production in a stylized supply chain environment. The supply chain environment used in this paper is modeled after the trading agent competition that is held annually to choose the best agent for managing a supply chain. Our supply chain agent, which we call SCMaster, uses dynamic inventory control and various reinforcement learning techniques like Q-learning, Softmax, e-greedy, and sliding window protocol to make our agent adapt dynamically to the changing environment created by competing agents. A multi-agent simulation environment is developed in Java to test the efficacy of our agent design. Two competing agents are created modeled after the winners of past trading agent competitions and are tested against our agent in various experimental designs. Results of simulations show that our agent has better performance compared to the other agents.

Distributed MASON: A scalable distributed multi-agent simulation environment

Computational Social Science (CSS) involves interdisciplinary fields and exploits computational methods, such as social network analysis as well as computer simulation with the goal of better understanding social phenomena.Agent-Based Models (ABMs) represent an effective research tool for CSS and consist of a class of models, which, aim to emulate or predict complex phenomena through a set of simple rules (i.e., independent actions, interactions and adaptation), performed by multiple agents. The efficiency and scalability of ABMs systems are typically obtained distributing the overall computation on several machines, which interact with each other in order to simulate a specific model. Unfortunately, the design of a distributed simulation model is particularly challenging, especially for domain experts who sporadically are computer scientists and are not used to developing parallel code.D-MASON framework is a distributed version of the MASON library for designing and executing ABMs in a distributed environment ensuring scalability and easiness. D-MASON enable the developer to exploit the computing power of distributed environment in a transparent manner; the developer has to do simple incremental modifications to existing MASON models, without re-designing them.This paper presents several novel features and architectural improvements introduced in the D-MASON framework: an improved space partitioning strategy, a distributed 3D field, a distributed network field, a decentralized communication layer, a novel memory consistency mechanism and the integration to cloud environments.Full documentation, additional tutorials, and other material can be found at https://github.com/isislab-unisa/dmason where the framework can be downloaded.

An Accessibility Driven Evolutionary Transit Network Design Approach in the Multi-agent Simulation Environment

This work explores the feasibility of using the accessibility measurements as an input in the public transit network design optimisation process. An existing paratransit-inspired evolutionary approach to the transit network optimization is extended to internalize the social benefits in the form of the accessibility improvements.The extension for the multi-agent simulation environment (MATSim) is proposed and tested within the number of scenarios. The novel algorithm measures inequalities in the accessibility of different types of attractions or activities on the microscopic level and calculates the subsidy, which is then introduced in the system as an incentive for paratransit operators to serve the low-accessibility zones. The algorithm also provides operators guidance on where the low accessibility zones are located, by performing weighted draw from the possible end stops during the initiation of new subsidized routes. Furthermore authors propose an approach to gradually optimize the level of provided subsidy.The results of the tests show that the proposed algorithm consistently improves the accessibility of deprived zones, building the sustainable subsidized routes, which the profit-oriented paratransit algorithm is not able to find1.