Real-time Scheduling Research Articles

Privacy-preserving multi-level co-regulation of VPPs via hierarchical safe deep reinforcement learning

Large amounts of distributed energy sources have brought challenges in terms of the safe and stable operation of the power grid. As a key technology between user-side energy resources and the distribution network (DN), how to realize the online coordinated scheduling of VPP and DN as well as the real-time response strategy of distributed equipment (DE) within VPP is the focus of this study. Thus, the hierarchical deep reinforcement learning (DRL) Hierarchical-TD3 algorithm is designed based on the unified modeling of adjustable space to achieve the real-time economic scheduling of VPPs. The upper layer DN considers the network security constraints and solves the economic scheduling model of VPPs based on the single-agent TD3 algorithm. Based on the scheduling instructions from the upper layer, the lower layer VPPs consider the requirements of privacy protection and control autonomy and realize real-time response of the DE within VPP via multi-agent MATD3 algorithm. Numerical results in the modified 33-nodes system show that the proposed Hierarchical-TD3 algorithm can achieve privacy protection and the coordinated scheduling of VPP and DE to reduce the operating cost. It differs from the optimal value by only 1.46% but can achieve online decision-making on the millisecond scale. Compared with the traditional centralized and decentralized DRL algorithms, the total cost is reduced by 10.15% and 5.52% respectively. Compared with the traditional soft-constraint method, there is no constraint violation during the training and testing phases. Finally, the actual 116-nodes testing system validates the scalability of the proposed Hierarchical-TD3 algorithm.

A Web-based Ticketing and Booking Application for Enhancing Maritime Travel Operations and Services in Siargao Island

Siargao Island is one of the world-class and most popular tourist destinations in the Philippines. The island is still in its developmental stage in terms of the tourism industry, which oftentimes face difficulties on its maritime travel services because of inefficiencies on manual ticketing and booking processes. With this, the study aimed at investigating the current services offered by the small-scale local maritime providers. A mixed-methodology approach was employed in the study through online survey and interviews to gather data on respondents’ preferences, challenges, and recommendations. The respondents were selected by convenience sampling method. The gathered data were analyzed using frequency distribution, weighted mean score, standard deviation, and ranking. The findings revealed significant results such as thechallenges related to labour-intensive ticketing processes and lack of real-time information are identified, suggesting the need for digital solutions. There is also a high demand for improved travel services, with preferences for online booking, real-time scheduling, and user-friendly interfaces. The study contributes to the development of a web application by providing insights into passenger preferences and challenges, guiding the design and implementation of a system that enhances maritime travel services in Siargao Island

Algorithmic approach leveraging a real-time task scheduler with fan-out strategy

Algorithmic approach leveraging a real-time task scheduler with fan-out strategy

Short-term water demand prediction based on decomposition technique optimization and a multihead attention mechanism

ABSTRACT Short-term water demand prediction is crucial for real-time optimal scheduling and leakage control in water distribution systems. This paper proposes a new deep learning-based method for short-term water demand prediction. The proposed method consists of four main parts: the variational mode decomposition method, the golden jackal optimization algorithm, the multihead attention mechanism, and the bidirectional gated recurrent unit (BiGRU) model. Furthermore, a seq2seq strategy was adopted for multistep prediction to avoid the error accumulation problem. Hourly water demand data collected from a real-world water distribution system were applied to investigate the potential of the proposed method. The results show that the proposed method can yield remarkably accurate and stable forecasts in single-step prediction (i.e., the mean absolute percentage error (MAPE) reaches 0.45%, and the root mean squared error (RMSE) is 25 m3/h). Moreover, the proposed method still achieves credible performance in 24-step prediction (i.e., the MAPE reaches 2.12%, and the RMSE is 126 m3/h). In general, for both single-step prediction and multistep prediction, the proposed method consistently outperforms other BiGRU-based methods. These findings suggest that the proposed method can provide a reliable alternative for short-term water demand prediction.

IMC-PnG: Maximizing runtime performance and timing guarantee for imprecise mixed-criticality real-time scheduling

Mixed-Criticality (MC) systems have successfully overcome the limitation of traditional real-time systems based on pessimistic Worst-Case Execution Times (WCETs), by using different WCETs depending on different criticalities. One of the important yet unsolved problems of current MC systems is to achieve two goals (G1 and G2) for low-criticality tasks without compromising timing guarantees for high-criticality tasks: (G1) providing a certain (degraded) level of timing guarantees for all low-criticality tasks; (G2) while maximizing the fully-serviced (non-degraded) execution of low-criticality tasks. To address the problem, we propose IMC-PnG, an MC scheduling framework, which employs two salient features: a task-level virtual-deadline assignment under the imprecise computing model with efficient resource utilization (supporting G1), and an online scheduling algorithm that dynamically changes criticality levels of individual tasks at runtime (supporting G2). In simulation results with random workloads, we showed that IMC-PnG has up to 12.10% higher schedulability and up to 42.10% higher runtime performance (measured by the fully-serviced ratio of low-criticality tasks) than the existing approaches.

Real-time stochastic flexible flow shop scheduling in a credit factory with model-based reinforcement learning

There has been a significant increase in consumer credit worldwide in recent years. The scheduling of jobs in credit factories is essential for speeding up the loan application process, which can improve the efficiency of credit factories. In this study, we propose a reinforcement learning approach for addressing the scheduling problem in credit factories, which is a stochastic flexible flow shop scheduling problem (SFFSP). First, we propose a mathematical model for the credit factory stochastic flexible flow shop scheduling problem, which abstracts the decision-making process as a semi-Markov process. Then, a reinforcement learning reward mechanism is designed based on the proposed mathematical model. After that, a self-attention neural network is used to extract state information from global and local multidimensional data, enabling each decision to consider the state of the entire process and make a decision that aligns with the global goal. Meanwhile, Monte Carlo Tree Search (MCTS) is utilised to enhance the training effect and sample utilisation of reinforcement learning. Finally, we conduct extensive experiments and demonstrate that our method achieves better performance for SFFSP in credit factories compared to other approaches.

A POSIX/RTEMS Monitoring Tool and a Benchmark to Detect Real-Time Scheduling Anomalies

This article deals with scheduling anomalies in real-time systems. We present MONANO, a POSIX user-level library allowing applications to dynamically detect a preidentified set of real-time scheduling anomalies. The MONANO library is based on the modelling of architecture and runtime constraints. MONANO monitors during the runtime the timing behavior of the application and deduces properties needed to identify scheduling anomalies. We present also a benchmark to evaluate our approach. The benchmark is composed of several programs implementing the most frequent real-time scheduling anomalies.

Real-time scheduling for parallel tasks with resource reclamation

This paper considers the real-time scheduling of a parallel task with reclaiming computing resources, which can be utilized for soft real-time tasks or switching to low-energy mode to save energy. Existing works allocate a rectangular piece of computing resources based on the worst-case characterizations of the task to guarantee the deadline, which inherently incurs severe resource wasting due to coarse-grained resource allocation. To address this resource-wasting problem, this paper proposes the ladder-like resource allocation (i.e., a series of rectangular pieces of computing resources). To characterize the ladder-like resource allocation, we present two concepts called resource distribution and allocation vector, which serve as the interfaces between hard and soft real-time tasks. For the former, we derive schedulability tests under the given two interfaces; for the latter, we discuss the methods of determining the two interfaces to reclaim computing resources. This paper is the first work to fully explore the concept of ladder-like resource allocation and its potential consequences on computing resources, soft real-time tasks, and energy. Experiments demonstrate that the proposed approach can effectively reclaim more computing resources than existing approaches while maintaining hard real-time guarantees.

Optimization and scheduling scheme of park-integrated energy system based on multi-objective Beluga Whale Algorithm

To improve the consumption rate of renewable energy and ensure the stability of power and heat supply within the industrial park, a park-integrated energy system (PIES), which incorporates electric heating equipment, energy storage devices, and multiple micro sources, is established. Based on the established PIES, a price-based demand response (PDR) mechanism is introduced to establish a multi-objective optimization and scheduling model with two time scales: day-ahead scheduling and intra-day scheduling, aiming to minimize the impact on the main grid and ensure the flexibility of system operation and real-time scheduling. With the objectives of minimizing operating costs and maximizing environmental benefits for PIES, a hybrid algorithm named Non-Dominated Sorting Beluga Whale Optimization (NSBWO), which combines the Non-Dominated Sorting Genetic Algorithm II (NSGA-II) and the Beluga Whale Optimization (BWO) algorithm is proposed to solve the optimization problem of the model. Simultaneously, an adaptive penalty function is introduced in the algorithm to handle constraint problems in optimization and additional penalty term for wind and solar curtailment is included within the constraints to improve the consumption rate of sustainable energy. The proposed NSBWO algorithm exhibits more robust search capabilities and faster convergence speed than the NSGA-II and MOPSO algorithm. Simulation results show that the proposed NSBWO algorithm can achieve low-carbon economic scheduling of the system in both long-term and short-term time scales and improve renewable energy consumption. Due to its flexibility, intra-day scheduling has lower operating costs than day-ahead scheduling.

Deep reinforcement learning based energy management of a hybrid electricity-heat-hydrogen energy system with demand response

Hybrid electricity-heat-hydrogen energy system with demand response (DR) is promising in enhancing flexibility and energy efficiency. However, the multi-energy coupling and source-load uncertainties makes it challenging to efficiently schedule energy flows of electricity generation, storage and DR. To this end, this paper proposes a continues deep reinforcement learning algorithm, specifically the deep deterministic policy gradient (DDPG), for the energy management optimization. Different Markov decision processes are firstly employed to analyze and compare two kinds of incentive-based electro-thermal DR contracts, i.e., load curtailment and load shifting. Simulation results exemplify the superiority of the proposed DDPG-based scheduling incorporating electro-thermal DR in terms of economy and sustainability, leading to a 16.02 % reduction in scheduling costs for contract load curtailment and an 8.52 % reduction for contract load shifting when compared to that without DR consideration. Furthermore, the robustness of DDPG-based scheduling is verified under 60 random source-load scenarios compared with different algorithms. Compared to results obtained by DDPG, DDPG-LC and DDPG-LS reduce the mean cost by 22.15 % and 12.84 %, while their error from the theoretical optimum is only around 5 %. The results demonstrate that the approximate optimality and rapid decision-making illustrate DDPG's efficient real-time scheduling capability, thereby enhancing the system's adaptability to uncertain environments.

Fast Pareto set approximation for multi-objective flexible job shop scheduling via parallel preference-conditioned graph reinforcement learning

The Multi-Objective Flexible Job Shop Scheduling Problem (MOFJSP) is a complex challenge in manufacturing, requiring balancing multiple, often conflicting objectives. Traditional methods, such as Multi-Objective Evolutionary Algorithms (MOEA), can be time-consuming and unsuitable for real-time applications. This paper introduces a novel Graph Reinforcement Learning (GRL) approach, named Preference-Conditioned GRL, which efficiently approximates the Pareto set for MOFJSP in a parallelized manner. By decomposing the MOFJSP into distinct sub-problems based on preferences and leveraging a parallel multi-objective training algorithm, our method efficiently produces high-quality Pareto sets, significantly outperforming MOEA methods in both solution quality and speed, especially for large-scale problems. Extensive experiments demonstrate the superiority of our approach, with remarkable results on large instances, showcasing its potential for real-time scheduling in dynamic manufacturing environments. Notably, for large instances (50 × 20), our approach outperforms MOEA baselines with remarkably shorter computation time (less than 1% of that of MOEA baselines). The robust generalization performance across various instances also highlights the practical value of our method for decision-makers seeking optimized production resource utilization.

Energy Bus-Based Matrix Modeling and Optimal Scheduling for Integrated Energy Systems

Integrated energy systems (IESs) can easily accommodate renewable energy resources (RESs) and improve the utilization efficiency of fossil energy by integrating various energy production, conversion, and storage technologies. However, the coupled multi-energy flows and the uncertainty of RESs bring challenges regarding optimal scheduling. Therefore, this study proposes an energy bus-based matrix-modeling method and a coordinated scheduling strategy for the IES. The matrix-modeling method can be used to formulate the steady- and transient-state balances of the multi-energy flows, and the transient model can clearly express the multi-time-scale characteristics of the different energy flows. The model parameters are fitted with data from experiments and the literature. To address the inherent randomness of the RESs and loads, a coordinated scheduling strategy is designed that contains two components: day-ahead optimization and rolling optimization. Day-ahead optimization uses the system steady-state model and multiple scenarios from the RES and load forecast data to minimize the operation cost while rolling optimization is based on the system’s transient-state model and aims to achieve the optimal real-time scheduling of the energy flows. Finally, a case study is conducted to verify the advantages and effectiveness of the proposed model and optimization method. The results show that stochastic optimization reduces the total daily cost by 1.48% compared to deterministic optimization when considering the prediction errors associated with the RESs and loads, highlighting the stronger adaptability of stochastic optimization to prediction errors. Moreover, rolling optimization based on the system’s transient-state model can reduce the errors between day-ahead scheduling and rolling correction.

Deep reinforcement learning-based optimal scheduling of integrated energy systems for electricity, heat, and hydrogen storage

The increasing load demands and the extensive usage of renewable energy in integrated energy systems pose a challenge to the most efficient scheduling of integrated energy systems (IES) because of the unpredictability and volatility of both the load side and renewable energy.Integrating heat storage and hydrogen storage technologies into integrated energy systems can effectively alleviate the phenomena of wind and solar power desertion caused by the instability of renewable energy sources. This study introduces a real-time optimal scheduling method based on the Soft Actor-Critic (SAC) algorithm, aimed at reducing system operating costs and enhancing the consumption rate of renewable energy. First, the established mathematical model is converted into a Markov Decision Process (MDP), the objective function is converted by designing the action state space and the reward function, and the deep reinforcement learning framework is established. Finally, the decision-making outcomes of intelligence in various energy storage scenarios of renewable energy consumption and extreme cases are analyzed and compared, and the results show that the heat storage and hydrogen storage system significantly improve the rate of renewable energy consumption and the economy of the system. This method is also shown to significantly improve the rate of renewable energy consumption.

Real-Time Microgrid Energy Scheduling Using Meta-Reinforcement Learning

With the rapid development of renewable energy and the increasing maturity of energy storage technology, microgrids are quickly becoming popular worldwide. The stochastic scheduling problem of microgrids can increase operational costs and resource wastage. In order to reduce operational costs and optimize resource utilization efficiency, the real-time scheduling of microgrids becomes particularly important. After collecting extensive data, reinforcement learning (RL) can provide good strategies. However, it cannot make quick and rational decisions in different environments. As a method with generalization ability, meta-learning can compensate for this deficiency. Therefore, this paper introduces a microgrid scheduling strategy based on RL and meta-learning. This method can quickly adapt to different environments with a small amount of training data, enabling rapid energy scheduling policy generation in the early stages of microgrid operation. This paper first establishes a microgrid model, including components such as energy storage, load, and distributed generation (DG). Then, we use a meta-reinforcement learning framework to train the initial scheduling strategy, considering the various operational constraints of the microgrid. The experimental results show that the MAML-based RL strategy has advantages in improving energy utilization and reducing operational costs in the early stages of microgrid operation. This research provides a new intelligent solution for microgrids’ efficient, stable, and economical operation in their initial stages.

A robust real-time energy scheduling strategy of integrated energy system based on multi-step interval prediction of uncertainties

The uncertainties of renewable energies and loads in integrated energy system (IES) result in unstable operation and performance degradation. To consider the impact of the uncertainties, the scheduling strategies of IES with the prediction of uncertainties are required. This paper designs a hybrid solar and gas turbine IES, in which the thermal and electrical storage units are integrated to address the uncertainties. A robust real-time energy scheduling strategy is proposed according to the real-time load and the predicted loads in the future time, in which the errors of multi-step interval prediction of renewable sources and loads are combined. The multi-step prediction method based on gated recurrent unit and time classification is constructed for the interval prediction of uncertain sources and loads by analyzing the probability statistics of prediction errors. In the robust optimization, the penalty factors of time sequences are proposed and considered to decrease the influences of future predicted information on current real-time scheduling, in which the future uncertainties in larger interval with current time has less influence. The optimal results in the hybrid IES demonstrate that the daily operation cost is 13.67 % lower than the method that does not consider the prediction parameters. Compared to the traditional scheduling strategies, the proposed strategy declines the operation cost by 2.93 %. The analysis of the confidence level of predictions ranging from 60 % to 98 % illustrates that the operation cost under the penalty factor of 0.94 is the lowest when the confidence level is from 94 % to 96 %.

Energy-aware dynamic rescheduling of flexible manufacturing system using edge-cloud collaborative decision-making method

ABSTRACT With the development of cloud computing technology, energy-aware scheduling based on cloud computing is an essential means to achieve energy saving and carbon reduction in manufacturing systems. Due to various production disturbances in flexible production, the scheduling based on cloud computing has data security problems and poor real-time performance. How to use real-time information to improve the security and responsiveness of cloud scheduling is a research gap. This paper establishes a cloud-edge collaborative dynamic flexible job-shop energy-aware rescheduling decision-making model. According to the characteristics of dynamic production in a flexible manufacturing system, create a learning model for cloud scheduling with Deep Q Network (DQN) and send the training model to the edge for scheduling decisions. Based on real-time production interference data, edge scheduling model real-time update scheduling scheme. To improve the robustness of the cloud scheduling model based on DQN, Dynamic scheduling data at the edge will be uploaded to update the cloud model. In addition, the effectiveness and practicability of the model are verified in an extrusion workshop. The experimental results show that this method can improve the comprehensive objective evaluation of minimum energy consumption and completion time by 3.6% −49.3% compared with the traditional scheduling rules.

Multi-time scale economic regulation model of virtual power plant considering multiple uncertainties of source, load and storag

A novel multi-stage time scale economic dispatch scheme is proposed for virtual power plants, taking into account the uncertainties arising from the connection of distribution network sources. This research introduces specific scheduling schemes tailored to various time scales within distribution networks, including a fuzzy optimized day ahead scheduling scheme, an intra-day scheduling scheme combined with Deep Q Network, and an adaptive optimized real-time scheduling scheme. This plan mainly considers the impact of photovoltaic output and conducts scheduling one day in advance through fuzzy optimization. In the intraday scheduling, different strategies were adopted in the study. By combining with Deep Q Network, research on scheduling for intraday demand within the power system. The analysis is conducted through rigorous modeling. Experimental tests were conducted to evaluate the performance of the proposed schemes. The day ahead dispatching primarily considers the impact of photovoltaic output and calculates the cost associated with each link in the grid under three different meteorological conditions. In the intra-day scheduling, the total costs for Scenario 1, Scenario 2, and Scenario 3 are found to be 34,724.5 yuan, 36,296.5 yuan, and 33,275.8 yuan, respectively. Notably, strategies 1 and 2 demonstrate lower costs compared to the pre-day scheduling, with the exception of Scenario 3. In real-time scheduling, considering the matching between sources and sources, the matching rate between sources and sources can be maintained at over 95%, and the stability and cost of the power grid have significantly decreased. In summary, by proposing a multi-stage time scale economic scheduling scheme, this study fully considers the uncertainty of the power supply of the distribution network access, as well as the different needs of day, day and real-time scheduling, providing an effective solution for the power dispatching of virtual power plants and providing important technical support for the reliability and economy of the power system.

A new methodology of peak energy demand reduction using coordinated real-time scheduling of EVs

A new methodology of peak energy demand reduction using coordinated real-time scheduling of EVs

Real-time scheduling and routing of shared autonomous vehicles considering platooning in intermittent segregated lanes and priority at intersections in urban corridors

Anticipating the forthcoming integration of shared autonomous vehicles (SAVs) into urban networks, the imperative of devising an efficient real-time scheduling and routing strategy for these vehicles becomes evident if one is to maximize their potential in enhancing travel efficiency. In this study, we address the problem of jointly scheduling and routing SAVs across an urban network with the possibility of platooning the vehicles at intersections to reduce their travel time. We argue that this is especially useful in large urban areas. We introduce a novel vehicle scheduling and routing method that allows a specific number of SAVs to converge at the intersections of urban corridors within designated time intervals, facilitating the formation of SAV platoons. Dedicated lanes and signal priority control are activated to ensure that these platoons go through the corridors efficiently. Based on the above concept, we propose a linear integer programming model to minimize the total travel time of SAVs and the delays experienced by the conventional vehicles due to SAV priority, thereby optimizing the overall performance of the road network. For large instances, we develop a two-stage heuristic algorithm to solve it faster. In the first stage, leveraging an evaluation index that manifests the compatibility of each vehicle-to-request combination, we allocate passenger requests to a fleet of SAVs. In the second stage, a customized genetic algorithm is designed to coordinate the paths of various SAVs, thus achieving the desired vehicle platooning effect. The optimization method is tested on a real-world road network in Shanghai, China. The results display a remarkable reduction of 15.76 % in the total travel time of the SAVs that formed platoons. The overall performance of the road network could be improved with the total travel time increase of conventional vehicles significantly smaller than the reduction observed in SAVs’ total travel time.

Federated Multiagent Deep Reinforcement Learning Approach via Physics-Informed Reward for Multimicrogrid Energy Management.

The utilization of large-scale distributed renewable energy (RE) promotes the development of the multimicrogrid (MMG), which raises the need of developing an effective energy management method to minimize economic costs and keep self energy sufficiency. The multiagent deep reinforcement learning (MADRL) has been widely used for the energy management problem because of its real-time scheduling ability. However, its training requires massive energy operation data of microgrids (MGs), while gathering these data from different MGs would threaten their privacy and data security. Therefore, this article tackles this practical yet challenging issue by proposing a federated MADRL (F-MADRL) algorithm via the physics-informed reward. In this algorithm, the federated learning (FL) mechanism is introduced to train the F-MADRL algorithm, thus ensures the privacy and the security of data. In addition, a decentralized MMG model is built, and the energy of each participated MG is managed by an agent, which aims to minimize economic costs and keep self energy sufficiency according to the physics-informed reward. At first, MGs individually execute the self-training based on local energy operation data to train their local agent models. Then, these local models are periodically uploaded to a server and their parameters are aggregated to build a global agent, which will be broadcasted to MGs and replace their local agents. In this way, the experience of each MG agent can be shared and the energy operation data are not explicitly transmitted, thus protecting the privacy and ensuring data security. Finally, experiments are conducted on Oak Ridge National Laboratory distributed energy control communication laboratory MG (ORNL-MG) test system, and the comparisons are carried out to verify the effectiveness of introducing the FL mechanism and the outperformance of our proposed F-MADRL.