Future System States Research Articles

Task-Importance-Oriented Task Selection and Allocation Scheme for Mobile Crowdsensing

In Mobile Crowdsensing (MCS), sensing tasks have different impacts and contributions to the whole system or specific targets, so the importance of the tasks is different. Since resources for performing tasks are usually limited, prioritizing the allocation of resources to more important tasks can ensure that key data or information can be collected promptly and accurately, thus improving overall efficiency and performance. Therefore, it is very important to consider the importance of tasks in the task selection and allocation of MCS. In this paper, a task queue is established, the importance of tasks, the ability of participants to perform tasks, and the stability of the task queue are considered, and a novel task selection and allocation scheme (TSAS) in the MCS system is designed. This scheme introduces the Lyapunov optimization method, which can be used to dynamically keep the task queue stable, balance the execution ability of participants and the system load, and perform more important tasks in different system states, even when the participants are limited. In addition, the Double Deep Q-Network (DDQN) method is introduced to improve on the traditional solution of the Lyapunov optimization problem, so this scheme has a certain predictive ability and foresight on the impact of future system states. This paper also proposes action-masking and iterative training methods for the MCS system, which can accelerate the training process of the neural network in the DDQN and improve the training effect. Experiments show that the TSAS based on the Lyapunov optimization method and DDQN performs better than other algorithms, considering the long-term stability of the queue, the number and importance of tasks to be executed, and the congestion degree of tasks.

Excuse me Sir/Madam, which Model?

AbstractDevelopment of complex systems through the application of Model‐Based Systems Engineering (MBSE) will require the creation and maintenance of multiple models, created using multiple languages. Hence, structuring the models such that there is efficient support for incremental development requires some foresight. Having models with multiple or unclear purposes may introduce situations where parts of the organisation will ask for modification for representing a desired future system state, whereas other parts of the organisation require it to remain unchanged in order to represent the present state. This paper introduces a structure with associated terms and characteristics of the different model types in order to provide guidance to an organisation. In particular, it extends previous work on model tenses, capturing different time perspectives/purposes on the models, to also include model usage for keeping track of the actual state of a realised system instance. For example, the concepts of Digital Twin and Digital Shadow are discussed and conceptualised. As such, the contributions made in the paper allows an organisation to reason about the completeness of their modelling methodology and its applicability to support incremental development. Above all, the paper spells out clearly that for a complex heterogeneous system, any request for an update to “the model” will prompt the response “Excuse me Sir/Madam, which model?”.

Neural network survivability approach of a wave energy converter considering uncertainties in the prediction of future knowledge

To tune the wave energy converter (WEC) controller parameters such as damping to reduce the line force during extreme wave conditions, future knowledge of the line force is required. To achieve this, the incoming wave and system state should be predicted for a few seconds in the future. It is rather an arduous task to predict the future knowledge of waves and the system’s dynamic when dealing with breaking and steep waves, and the system is subject to various nonlinear forces. The classical model-based control strategies often rely on linear assumptions to estimate the WEC dynamics for the sake of simplicity. Unlike the model-based, the data-driven approaches are free from modeling errors and the algorithms are trained over the true and noisy data to predict non-linear system behaviors. Using data-driven approaches, we are able to model nonlinear dynamics. However, new questions emerge on the accuracy of the future wave and system state predictions, and how this uncertainty propagates into the final prediction of the line force. As incorrect damping may lead to excessive line force and detrimental damage to the system, these are the knowledge gaps that need to be addressed. The main purpose of this paper is to answer these questions through a survivability strategy for wave energy converters by providing a realistic perspective on the implementation of the neural network approaches by accounting for the errors in the input data. For this purpose, a series of neural networks is designed that first predicts the surface elevation for 0.36 s ahead, i.e. corresponding to 2 s in the full-scale WEC. This future knowledge of the wave elevation is then used to predict the system state (i.e. power take-off (PTO) translator position) for the same prediction horizon based on the PTO damping. This information is then fed to a convolutional neural network (CNN) that predicts the peak line force 0.36 s ahead. This paper also evaluates the predictive capabilities of neural network (NN) models, comparing them to classical autoregressive (AR) and Kalman filter methods. While the AR model exhibits slightly higher accuracy in fixed PTO system configurations, it falters in providing real-time predictions when system parameters undergo variations. In contrast, the NN prediction model excels by establishing a robust relationship between system configuration and output signals. Especially noteworthy is its ability to outperform classical methods with minimal training on a selective set of system configurations. Then, the sensitivity of the peak line force prediction to the uncertainties in the input data from NN models and the prediction horizon is analyzed. The neural network models are trained over the experimental data subjected to extreme sea states for a point absorber wave energy converter. The results suggest that the accuracy of the surface elevation prediction has an insignificant direct effect on the peak force prediction model. However, these uncertainties are reflected in the PTO translator position prediction, and the model is considerably sensitive to the accuracy of this prediction. This sensitivity nonetheless is less notable for higher PTO damping values.

The Future of Developed Barrier Systems: 1. Pathways Toward Uninhabitability, Drowning, and Rebound

AbstractMany barrier islands and spits (collectively, “barriers”) throughout the world are highly developed. As low‐lying, sandy coastal landforms, barrier systems are naturally reshaped by processes associated with storms and sea‐level rise (SLR). The resulting landscape changes threaten development, and in response, humans employ defensive measures that physically modify barrier geometry to reduce relatively short‐term risk. These measures include the construction of large dunes, emplacement of beach nourishment, and removal of washover. Simulations conducted using a new coupled modeling framework show that, over decades to centuries, measures to protect roadways and communities alter the physical characteristics of barrier systems in ways that ultimately limit their habitability. We find that the pathway toward uninhabitability (via roadway drowning or community narrowing) and future system states (drowning or rebound) depends largely on dune management—because building dunes blocks overwash delivery to the barrier interior—and on initial conditions (barrier elevation and width). In the model, barriers can become lower and narrower with SLR to the point of drowning. The timing and occurrence of barrier drowning depends on randomness in the timing and intensity of storms and dune recovery processes. We find that under a constant rate of SLR, negative feedbacks involving storms can allow barriers that do not drown to rebound toward steady‐state geometries within decades after management practices cease.

CANNIBALIZATION POLICIES: ADOPTION IN THE MAINTENANCE OF FLEET SYSTEMS

Fleet systems are considered complex due to the interaction between their units and components. Maintenance management systems face various challenges to achieve acceptable availability and reliability rates at a reasonable cost. A critical task for making maintenance decisions is understanding the system requirements to select maintenance policies appropriate for the actual and future system state. When there is a replacement shortage in a fleet system, and it is impossible to supply new spare parts quickly, cannibalization policies can mitigate this scarcity via the interchange of components. However, this procedure presents the maintenance manager with different evaluation effects, such as increased maintenance hours, decreased system reliability rate, and unavailability in some units. Finding an equilibrium between the benefits and risks has caught the attention of researchers. This work gathers diverse proposals for applying cannibalization policies and the effects that arise from using them. Models, methods, tools, and identified gaps in understanding what parameters of the components and environments of the fleet systems favor cannibalization are discussed.

SafeCool: Safe and Energy-Efficient Cooling Management in Data Centers With Model-Based Reinforcement Learning

Optimizing the cooling system plays a central role for capping the data center power consumption. However, the performance of traditional cooling management strategies is not satisfactory due to the complexity of thermodynamic process. Recently, several works leveraged Reinforcement Learning (RL) to improve the energy efficiency of data center cooling system. While they demonstrated that it is possible to reduce the cooling power consumption via RL, there are still some key challenges that have to be addressed before field deployment, such as safe operation guarantee and sample complexity, etc. In this paper, we propose <monospace xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">SafeCool</monospace> , an actor-critic Model-Based Reinforcement Learning (MBRL) algorithm for data center cooling management. <monospace xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">SafeCool</monospace> incorporates two system models, i.e., a transition model to predict the future system state and a risk model to estimate the negative effect of executing an action. The safety of proposed algorithm is ensured by Model Predictive Control (MPC) and risk-guided exploration. In addition, by employing the MBRL framework, <monospace xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">SafeCool</monospace> achieves higher sample efficiency and accelerated convergence. Simulations using real-world workload trace reveal that <monospace xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">SafeCool</monospace> saves up to 13.18% cooling power compared with state-of-the-art MBRL data center cooling solutions.

Embedding communication concepts in forecasting training increases students' understanding of ecological uncertainty

AbstractCommunicating and interpreting uncertainty in ecological model predictions is notoriously challenging, motivating the need for new educational tools, which introduce ecology students to core concepts in uncertainty communication. Ecological forecasting, an emerging approach to estimate future states of ecological systems with uncertainty, provides a relevant and engaging framework for introducing uncertainty communication to undergraduate students, as forecasts can be used as decision support tools for addressing real‐world ecological problems and are inherently uncertain. To provide critical training on uncertainty communication and introduce undergraduate students to the use of ecological forecasts for guiding decision‐making, we developed a hands‐on teaching module within the Macrosystems Environmental Data‐Driven Inquiry and Exploration (EDDIE; MacrosystemsEDDIE.org) educational program. Our module used an active learning approach by embedding forecasting activities in an R Shiny application to engage ecology students in introductory data science, ecological modeling, and forecasting concepts without needing advanced computational or programming skills. Pre‐ and post‐module assessment data from more than 250 undergraduate students enrolled in ecology, freshwater ecology, and zoology courses indicate that the module significantly increased students' ability to interpret forecast visualizations with uncertainty, identify different ways to communicate forecast uncertainty for diverse users, and correctly define ecological forecasting terms. Specifically, students were more likely to describe visual, numeric, and probabilistic methods of uncertainty communication following module completion. Students were also able to identify more benefits of ecological forecasting following module completion, with the key benefits of using forecasts for prediction and decision‐making most commonly described. These results show promise for introducing ecological model uncertainty, data visualizations, and forecasting into undergraduate ecology curricula via software‐based learning, which can increase students' ability to engage and understand complex ecological concepts.

Students’ systems thinking while modeling a dynamic ecological system

The world is facing global ecological changes, making it essential to prepare the future generation with the necessary understanding to effectively navigate and address complex tasks. Previous research has shown that both systems thinking and scientific modeling are particularly relevant in investigating the comprehensive understanding of such complex phenomena. However, there has been little research on the interrelation between systems thinking and scientific modeling. To address this research gap, we conducted a thinking-aloud study with nine high school students by confronting them with a simulation of a dynamic ecological system. Our qualitative content analysis of the students’ statements indicates an interrelation between systems thinking and scientific modeling. The students infrequently show systems thinking during the exploration, whereas when developing a graphical model, the students are involved in identifying the system organization and analyzing the system behavior. When predicting future system states, students engage in modeling the system evolution. Furthermore, during verbalizing analogies and experiences, students refer to the system organization and behavior, whereas in mental modeling, students additionally model the system evolution. These results illustrate a central difference between the two perspectives. Thus, scientific modeling focuses on students’ activities during their understanding process, while systems thinking addresses students’ analysis of systems and their properties. While the phenomenon exploration may not require systems thinking, pattern recognition and model development are frequently associated with identifying the system organization and analyzing the system behavior. Systems thinking must also be applied when deriving possible future system states by modeling the system evolution, an activity that is closely related to the prediction phase of scientific modeling. Interestingly, in our study, the students also demonstrated the modeling of system evolution in their mental modeling. In conclusion, a complementary consideration of systems thinking and scientific modeling affords a deeper understanding of students’ cognitive processes in dealing with complex phenomena.

A fundamental matrix based approach to design predictor-based control laws for discrete-time systems with state and input delays

In this paper, the concept of fundamental matrices is generalized to discrete-time systems with state delays, and the commutativity-like property is derived. With the aid of fundamental matrices, an explicit expression of state responses is obtained for discrete-time systems with state delays. Then, a prediction scheme is constructed to predict the future states of discrete-time linear systems with delays in both states and inputs. Further, a predictor-based feedback law is presented to stabilize the considered delayed systems. It is revealed that the characteristic equation of the closed-loop system under the presented predictor-based feedback control law is the same as that of the closed-loop system of the corresponding system without input delays under the delayed state feedback. Two numerical examples show the effectiveness of the proposed approach. In this paper, it is illustrated that the proposed fundamental matrices of discrete-time systems with state delays are the basis of a prediction scheme for solving the problem in the presence of delays in both states and inputs.

Differentiable programming for Earth system modeling

Abstract. Earth system models (ESMs) are the primary tools for investigating future Earth system states at timescales from decades to centuries, especially in response to anthropogenic greenhouse gas release. State-of-the-art ESMs can reproduce the observational global mean temperature anomalies of the last 150 years. Nevertheless, ESMs need further improvements, most importantly regarding (i) the large spread in their estimates of climate sensitivity, i.e., the temperature response to increases in atmospheric greenhouse gases; (ii) the modeled spatial patterns of key variables such as temperature and precipitation; (iii) their representation of extreme weather events; and (iv) their representation of multistable Earth system components and the ability to predict associated abrupt transitions. Here, we argue that making ESMs automatically differentiable has a huge potential to advance ESMs, especially with respect to these key shortcomings. First, automatic differentiability would allow objective calibration of ESMs, i.e., the selection of optimal values with respect to a cost function for a large number of free parameters, which are currently tuned mostly manually. Second, recent advances in machine learning (ML) and in the number, accuracy, and resolution of observational data promise to be helpful with at least some of the above aspects because ML may be used to incorporate additional information from observations into ESMs. Automatic differentiability is an essential ingredient in the construction of such hybrid models, combining process-based ESMs with ML components. We document recent work showcasing the potential of automatic differentiation for a new generation of substantially improved, data-informed ESMs.

Effect of planning policies on land use dynamics and livelihood opportunities under global environmental change: Evidence from the Mekong Delta

The Mekong Delta faces significant challenges in supplying Vietnam and its export market countries with agricultural commodities, while ensuring livelihoods and providing living space to its growing population in the context of climate change and the country’s agrarian transition. Anthropogenic factors, such as the construction of dykes to control river flooding, river sand mining, the further development of triple-cropping rice production, and infrastructure development, together with climate change impacts on sediment and water availability, are all combining to threaten agricultural production. One of the key challenges in sustainable development is the need to identify plausible future states of agricultural-based socio-ecological systems which draw upon differing strategies of land management, and to characterise the impacts of these systems on both the landscape and employment. It was hypothesised from the literature and rapid rural appraisals that each land system can only provide a certain number of jobs, which was further demonstrated using binomial regressions. We show that the odds of being employed are lower for intensive agricultural systems (OR=0.78 for triple rice; 0.91 for intensive aquaculture) than for diversified systems (OR=1.16 for rice-aquaculture; OR=1.63 for mixed fruit trees). Drawing from workshops with local and national stakeholders, we then used Earth observation and national census data in a spatial land use systems dynamic framework to simulate two alternative Mekong Delta futures based upon the climate pathway RCP 4.5 in combination with two existing policies (i) Resolution No. 124 (Specialisation) which promotes triple crop rice and aquaculture intensification and (ii) Resolution No. 639 (Diversification), which states that there should be a development of sustainable rice aquaculture and crop diversification. Based on the quantitative objectives of each policy, we estimated likely changes of services provided by land use systems if either policy were to dominate. The estimated impacts of each future scenario on the provision of employment ultimately indicate that policies with a diversification development paradigm will provide more employment (+0.9%) than policies with a specialisation paradigm (−46%), and that current policies have potentially conflicting consequences. Decisions driving towards intensive farming risk triggering rural unemployment and outmigration, potentially exacerbating urban poverty in major cities such as Can Tho and Ho Chi Minh City. On the other hand, decisions aiming at increasing diversified agricultural systems can help secure more job opportunities. Our results indicate that spatial planning policies should rely on a broad-based assessment of land system services that include employment and environmental impacts to ensure a just transition towards resilient and environmentally sustainable rural territories.

Ensemble Kalman filter for GAN-ConvLSTM based long lead-time forecasting

Data-driven machine learning techniques have been increasingly utilized for accelerating nonlinear dynamic system prediction. However, machine learning-based models for long lead-time forecasts remain a significant challenge due to the accumulation of uncertainty along the time dimension in online deployment. To tackle this issue, the ensemble Kalman filter (EnKF) has been introduced to machine learning-based long-term forecast models to reduce the uncertainty of long lead-time forecasts of chaotic dynamic systems. Both the deep convolutional generative adversarial network (DCGAN) and convolutional long short term memory (ConvLSTM) are used for learning the complex nonlinear relationships between the past and future states of dynamic systems. Using an iterative Multi-Input Multi-Output (MIMO) algorithm, the two-hybrid forecast models (DCGAN-EnKF and ConvLSTM-EnKF) are able to yield long lead-time forecasts of dynamic states. The performance of the hybrid models has been demonstrated by one-level and two-level Lorenz 96 models. Our results show that the use of EnKF in ConvLSTM and DCGAN models successfully corrects online model errors and significantly improves the real-time forecasting of dynamic systems for a long lead-time.

Optimal Near Real-Time Control of Water Distribution System Operations

The scarcity of freshwater resources, combined with deteriorating infrastructure, pushes water utilities to employ optimal operational practices to control water distribution systems (WDSs) based on objectives such as minimizing operational costs or leakages. This paper demonstrates a metaheuristic optimization framework for controlling WDS operations in near real-time by minimizing the total energy consumption, while maintaining sustainable system conditions and operations, such as those of tanks. The proposed framework, at its core, comprises a water demand forecasting model, an optimization-based control model, and a hydraulic continuity model. The hypothesis is that WDS can be controlled more efficiently by forecasting and predicting the near future system conditions based on past and prevailing conditions. Operational time steps of 60, 30, and 15 min are considered, to evaluate the benefits of using shorter operational time steps than the conventional norm. The proposed framework is demonstrated using a small-sized benchmark WDS. The results revealed that real-time control schemes reduce the operational costs of the selected WDS by up to 17.8%, with the shortest time step scheme (15 min) offering the most reduction in operational expenses, at the cost of more computational expensiveness. This study and its findings would help utilities plan more reliable and sustainable operational schemes.

Multiobjective evolutionary discovery of equation-based analytical models for dynamical systems

In this article, an approach to modeling dynamical systems in case of unknown governing physical laws has been introduced. The systems of differential equations obtained by means of a data-driven algorithm are taken as the desired models. In this case, the problem of predicting the state of the process is solved by integrating the resulting differential equations. In contrast to classical data-driven approaches to dynamical systems representation, based on the general machine learning methods, the proposed approach is based on the principles, comparable to the analytical equation-based modeling. Models in forms of systems of differential equations, composed as combinations of elementary functions and operation with the structure, were determined by adapted multi-objective evolutionary optimization algorithm. Time-series describing the state of each element of the dynamic system are used as input data for the algorithm. To ensure the correct operation of the algorithm on data characterizing real-world processes, noise reduction mechanisms are introduced in the algorithm. The use of multicriteria optimization, held in the space of complexity and quality criteria for individual equations of the differential equation system, makes it possible to improve the diversity of proposed candidate solutions and, therefore, to improve the convergence of the algorithm to a model that best represents the dynamics of the process. The output of the algorithm is a set of Pareto-optimal solutions of the optimization problem where each individual of the set corresponds to one system of differential equations. In the course of the work, a library of data-driven modeling of dynamic systems based on differential equation systems was created. The behavior of the algorithm was studied on a synthetic validation dataset describing the state of the hunter-prey dynamic system given by the Lotka-Volterra equations. Finally, a toolset based on the solution of the generated equations was integrated into the algorithm for predicting future system states. The method is applicable to data-driven modeling of arbitrary dynamical systems (e.g. hydrometeorological systems) in cases where the processes can be described using differential equations. Models generated by the algorithm can be used as components of more complex composite models, or in an ensemble of methods as an interpretable component.

Improved mapping and understanding of land system states for achieving land degradation neutrality using remote sensing surface endmembers—temperature space: A case‐study in Minqin, China

AbstractThe identification and understanding of land system states are critical for achieving land degradation neutrality (LDN). However, present conventional approaches can hardly generate a state space with both complex linear and abrupt shift relationships. Therefore, with the remote sensing surface endmembers—temperature space, this study developed a framework to identify, map and understand land system states and their associated trajectories based on the alternative state theory. Combined with three variables of land cover and landscape, our proposed vegetation productivity index (VPI) can highly improve the mapping accuracy of land system states in Minqin, without considering the variable of soil type. We built the three‐dimensional state catastrophe space with variables of soil organic matter (SOM), VPI, and cover function index (LCI). Thanks to the ball‐and‐cup model, two alternative stable states and five unstable states were identified at the landscape level in Minqin; then, we further explored the desired threshold of VPI and LCI were 13.11 and 0.39, while their undesired threshold were 5.15 and 0.76, respectively. The state space can reveal the linear or non‐linear relationship between VPI/LCI and SOM; besides, it can also present the hysteresis of land system state. Moreover, our established model in 2015 proved to be stable and thus can be used to accurately estimate the states in 2018. Therefore, our proposed framework can be an effective way of mapping and understanding present and future land system states, which can provide useful information on proactive conservation and restorative interventions for LDN management.

Joint Application Placement and Request Routing Optimization for Dynamic Edge Computing Service Management

As mobile edge computing (MEC) hosting applications at the network edge with limited capacities, service providers are facing the new challenge of how to make full use of the scarce edge resources to maximize the system performance. Accommodating this challenge requires careful application placement and request routing to coordinate diverse MEC nodes. However, frequent application re-placement would greatly increase the system reconfiguration cost, indicating a performance-cost trade-off. In response, in this paper, we study the problem of joint optimization on application placement and request routing to maximize the system performance, under a long-term budget of the application reconfiguration cost. Solving this problem is non-trivial since the long-term budget is coupled with the future system states (e.g., user request arrivals) that are typically unpredictable. To address this challenge, we first advocate an approximated dynamic optimization framework to decompose the long-term optimization problem into a series of one-shot problems which do not require the future system states. Moreover, since the decomposed problem is a mixed integer linear program (MILP) which is proven to be NP-hard, we then devise an efficient dependent rounding based approximation algorithm, which can achieve the near-optimal performance in a fast manner. Both rigorous theoretical analysis and extensive trace-driven evaluations demonstrate the proposed framework can achieve superior performance gain over existing schemes.

Meta-Learning-Based Proactive Online Planning for UAVs Under Degraded Conditions

Changes in model dynamics due to factors like actuator faults, platform aging, and unexpected disturbances can challenge an autonomous robot during real-world operations affecting its intended behavior and safety. Under such circumstances, it becomes critical to improve tracking performance, predict future states of the system, and replan to maintain safety and liveness conditions. In this letter, we propose a meta-learning-based framework to learn a model to predict the future system's states and their uncertainties under unforeseen and untrained conditions. Meta-learning is considered for this problem thanks to its ability to easily adapt to new tasks with a few data points gathered at runtime. We use the predictions from the meta-learned model to detect unsafe situations and proactively replan the system's trajectory when an unsafe situation is detected (e.g., a collision with an object). The proposed framework is applied and validated with both simulations and experiments on a faulty UAV performing an infrastructure inspection mission, demonstrating safety improvements.

On the Importance of Embodiment for Artificial Design Intelligence in the Context of Sociotechnical Systems

Artificial intelligence methods are increasingly used in the design of sociotechnical systems. The design of such systems can be described as an ill-defined problem. The problem space often changes while working on a design solution. Further, an anticipation of future system states is challenging, because of possible changes of organisational, technological or other environmental aspects during the lifecycle of a sociotechnical system. Embodied artificial intelligence can help to structure the problem space. It can also act as a feedback mechanism of the inner workings of a system and its surroundings for a designer. Therefore, this short article discusses the emphasis of embodied artificial intelligence for the theory of design in the context of sociotechnical systems and its implications.

Remaining useful life prediction for lithium-ion batteries in later period based on a fusion model

Lithium-ion batteries are broadly used in many fields. Accurate remaining useful life (RUL) prediction ensures the reliable operation and the safety of battery systems. However, no single model can realize long-term prediction for RUL with the reliable uncertainty management in the later period. To this end, a competitive model based on an improved autoregressive (AR) and particle filter (PF) model is proposed. Specifically, the similarity capacity series is creatively employed in the AR model, while the underlying capacity is introduced as a new approach for the parameter estimation of the observation equation in PF. Then, average weight is used to update the state equation and describe the future system states. After that, the RUL and its probability density function are obtained by PF again. The effectiveness and robustness are verified by the National Aeronautics and Space Administration (NASA) dataset. Results illustrate that the fusion model outperforms others and accurately predicts RUL with narrow uncertainty representation in the later period.

Multi-agent deep reinforcement learning for adaptive coordinated metro service operations with flexible train composition

This paper presents an adaptive control system for coordinated metro operations with flexible train composition by using a multi-agent deep reinforcement learning (MADRL) approach. The control problem is formulated as a Markov decision process (MDP) with multiple agents regulating different service lines in a metro network with passenger transfer. To ensure the overall computational effectiveness and stability of the control system, we adopt an actor–critic reinforcement learning framework in which each control agent is associated with a critic function for estimating future system states and an actor function deriving local operational decisions. The critics and actors in the MADRL are represented by multi-layer artificial neural networks (ANNs). A multi-agent deep deterministic policy gradient (MADDPG) algorithm is developed for training the actor and critic ANNs through successive simulated transitions over the entire metro network. The developed framework is tested with a real-world scenario in Bakerloo and Victoria Lines of London Underground, UK. Experiment results demonstrate that the proposed method can outperform previous centralized optimization and distributed control approaches in terms of solution quality and performance achieved. Further analysis shows the merits of MADRL for coordinated service regulation with flexible train composition. This study contributes to real-time coordinated metro network services with flexible train composition and advanced optimization techniques.