State-Action-Reward-State-Action Research Articles

Implementation of Hybrid Adaptive Learning Algorithm for Task Offloading

Objectives: Offloading tasks in edge computing (EC) plays an important role in optimizing resource utilization and enhancing the system performance. This paper studies various AI-based computation offloading (CO) strategies and proposes a hybrid Adaptive Learning Algorithm, that reduces the latency significantly compared to traditional RL-based on-policy and off-policy algorithms namely Q-Learning (QL) and State Action Reward State Action (SARSA) for CO in an EC environment. The paper evaluates and compares the efficiency of the proposed algorithms in optimizing dynamic offloading decisions, focusing on factors such as latency. Methods: The research is inspired by existing literature that has discovered the various applications of QL and SARSA in CO and mobile EC. This paper inspects how these algorithms perform in terms of reducing overall latency and proposes an adaptive hybrid algorithm. Novelty: Although other AI-based techniques exist in earlier research on CO in EC, the primary novelty proposed is the addition of a hybrid adaptive RL algorithm. Factors impacting the task offloading decision are crucial in the proposed algorithm. Moreover, the decision is not based only on static policies; the current state is the driving factor for decision-making. Findings: The exploration and exploitation handshake is very promising in reducing latency, as seen in the simulation results. The proposed algorithm outperforms Q-Learning and SARSA in terms of latency. Keywords: Computation offloading, Mobile Edge Computing, Reinforcement learning, Q­learning, SARSA

Design and analysis of adaptive control systems: Adaptive control algorithms using machine learning

A growing field of research is the use of adaptive control algorithms with machine learning techniques like Q-learning and SARSA. With prospective applications in robotics, healthcare, and other fields, this interdisciplinary method strives to combine the stability and robustness of conventional control systems with the self-learning skills of reinforcement learning. Making sure that systems are stable and that learning occurs effectively is the main research problem. This paper demonstrates the stability and convergence of existing adaptive control algorithms when integrating with machine learning. There are mainly four primary methods of control algorithms: reinforcement learning, neutral network, support vector machine and deep learning. Reinforcement learning is the main focus of this paper. Data efficiency, robustness, and generalization are the main problems with reinforcement learning. Q-learning and SARSA (State, Action, Reward, State, Action) are two algorithms for reinforcement learning. The research will be done by analyzing these two algorithms based on existing material and the actual application of these two algorithms. SARSA is believed to be more safe as its on-policy methodology, and Q-learning is believed to be more proactive as its off-policy methodology.

Research on Power Service Route Planning Scheme Based on SDN Architecture and Reinforcement Learning Algorithm

The power communication network carries various power services to ensure the safe operation of the power network, among which, the relay protection service is the most important service. Reasonable planning of the service route can improve the effectiveness and reliability of data transmission in the power communication network, thereby ensuring the reliable operation of the power grid. This paper constructs a route planning architecture for the power communication network based on a software-defined network. On this basis, parameters such as the power service and network-carrying service status are defined. With the goal of minimizing network risk variance and considering link bandwidth utilization and overload constraints of relay protection services, a service route allocation problem has been raised. To solve this problem, a power service route planning scheme based on a reinforcement learning algorithm is proposed. This algorithm uses the state–action–reward–state–action (SARSA) algorithm to complete service route planning. The simulation results show that using the route planning scheme proposed in this paper can avoid the overload of relay protection services, reduce network risk variance, and effectively balance network risk.

Time-Sensitive Network Simulation for In-Vehicle Ethernet Using SARSA Algorithm

In order to more accurately analyze the problem of time delay simulation and calculation in the time-sensitive network (TSN) of vehicular Ethernet, a TSN reservation class data delay analysis model improved based on the State–Action–Reward–State–Action (SARSA) reinforcement learning algorithm is proposed. Firstly, the TSN data queue forwarding delay model and reservation class data delay analysis intelligent body model are established, then the TSN traffic scheduling mechanism is improved by the SARSA reinforcement learning algorithm, and the improved TSN network reservation class data analysis model is established for the uncertainty of traffic scheduling in the network; finally, the fitting performance of the proposed method is verified by simulation and experimental validation. The results show that the deviation between the two is less than 5% under different BE loads, i.e., the established reservation class data delay analysis model is able to correctly fit the scheduling mechanism of the vehicle-mounted TSN network, which proves the reasonableness of the model simulation.

Breast cancer diagnosis through knowledge distillation of Swin transformer-based teacher–student models

Breast cancer is a significant global health concern, emphasizing the crucial need for a timely and accurate diagnosis to enhance survival rates. Traditional diagnostic methods rely on pathologists analyzing whole-slide images (WSIs) to identify and diagnose malignancies. However, this task is complex, demanding specialized expertise and imposing a substantial workload on pathologists. Additionally, existing deep learning models, commonly employed for classifying histopathology images, often need enhancements to ensure their suitability for real-time deployment on WSI, especially when trained for small regions of interest (ROIs). This article introduces two Swin transformer-based architectures: the teacher model, characterized by its moderate size, and the lightweight student model. Both models are trained using a publicly available dataset of breast cancer histopathology images, focusing on ROIs with varying magnification factors. Transfer learning is applied to train the teacher model, and knowledge distillation (KD) transfers its capabilities to the student model. To enhance validation accuracy and minimize the total loss in KD, we employ the state–action–reward–state–action (SARSA) reinforcement learning algorithm. The algorithm dynamically computes temperature and a weighting factor throughout the KD process to achieve high accuracy within a considerably shorter training timeframe. Additionally, the student model is deployed to analyze malignancies in WSI. Despite the student model being only one-third the size and flops of the teacher model, it achieves an impressive accuracy of 98.71%, slightly below the teacher’s accuracy of 98.91%. Experimental results demonstrate that the student model can process WSIs at a throughput of 1.67 samples s−1 with an accuracy of 82%. The proposed student model, trained using KD and the SARSA algorithm, exhibits promising breast cancer classification and WSI analysis performance. These findings indicate its potential for assisting pathologists in diagnosing breast cancer accurately and effectively.

A real-time automatic fire emergency evacuation route selection model based on decision-making processes of pedestrians

After a fire occurs, it is imperative that people in danger evacuate as soon as possible. However, the current emergency plan based on the pre-established static exiting route is unable to considering the real-time fire scenario. In addition, the selection of evacuation routes significantly relies on the decision-maker's experiences. These issues seriously affect evacuation efficiency, decreasing the likelihood of survival. This paper developed an effective real-time evacuation guidance method that can automatically select the evacuation route in accordance with real-time fire scenarios. The model is established based on the on-policy learning algorithm SARSA (State–action–reward–state–action), an algorithm for learning a Markov decision process policy, which could mimic the decision-making of pedestrian behaviors in an emergency. In addition, two types of radar (exit radar and fire radar) are introduced into the SARSA algorithm to facilitate the wayfinding process, which formulated the so-called Radar-assisted SARSA (RSARSA). The results have shown that RSARSA can swiftly decide a safer evacuation route for pedestrians or crowd at arbitrary location. The convergence time of initial successful route planning is between 0.05 and 4.5 s under the tests in this paper. The evacuation route determined by this algorithm can well consider the fire, and timely avoid routes with potential dangerous. Moreover, RSARSA can flexibly respond to different fires under various heat release rates and development speeds. By applying this technology, fire evacuation can be guided by routes that are more attuned to the mindset of pedestrians. It can provide a good basis for route selection of crowd evacuation.

DALMIG: Matching-Based Data Center Allocation and Dual Live VM Migration in Cluster-Based Federated Cloud

The modern era has been ruled by federated cloud extensively owing to the upward popularity of real-time applications. However, balancing the load among the different service providers in the federation is a challenging task. This is because of the enlarged load of each service provider and the demand for Service Level Agreement (SLA) requirements. Live VM migration is an effective mechanism to balance the load among the different Cloud Service Providers (CSP). Yet, none of the works has focused on both inter- and intra- CSP-based live VM migration in the federated cloud. Besides, they utilized traditional techniques (Pre and Post-copy) to perform live VM migration. This increases migration and downtime in a federated cloud environment. Motivated by this, we propose matching-based data center allocation and Duple live VM migration in clustered federated cloud environment (DALMIG). The proposed DALMIG approach tackles the downsides faced in the federated cloud as yet. In this regard, three processes are contributed that are Matching-based Data Center Allocation, Two-Fold Clustering, and Duple live VM migration. Initially, brokers allocate apt data center to each incoming task using Auction-based Bipartite One to N (A-BON) matching algorithm. And then, we execute two-fold clustering process: Here, the data center is clustered using the Kernel Neutrosophic C-Means (kernel NCM) algorithm. And, we cluster VMs in each data center using the Credal C-Means (CCM) algorithm. To maintain the load of each CSP, we maintain two entities, such as a load monitor and a migrator. Here, the load monitor estimates a load of each data center and the migrator performs a live VM migration process. Here, the live VM migration process is performed using the Hybrid Copy technique. We execute both intra- and inter-CSP live VM migration. Here, intra-live VM migration is performed using the State Action Reward State Action (SARSA) algorithm. Inter-CSP live VM migration is accomplished using the Black Widow Optimization (BWO) algorithm. We show our DALMIG performance in the federation using the Cloudsim tool. The simulation results acquired from Cloudsim are auspicious in the context of metrics including latency, response time, number of VM migrations, Migration time, load fairness, and communication overhead.

A Multi-Scaling Reinforcement Learning Trading System Based on Multi-Scaling Convolutional Neural Networks

Advancements in machine learning have led to an increased interest in applying deep reinforcement learning techniques to investment decision-making problems. Despite this, existing approaches often rely solely on single-scaling daily data, neglecting the importance of multi-scaling information, such as weekly or monthly data, in decision-making processes. To address this limitation, a multi-scaling convolutional neural network for reinforcement learning-based stock trading, termed multi-scaling convolutional neural network SARSA (state, action, reward, state, action), is proposed. Our method utilizes a multi-scaling convolutional neural network to obtain multi-scaling features of daily and weekly financial data automatically. This involves using a convolutional neural network with several filter sizes to perform a multi-scaling extraction of temporal features. Multiple-scaling feature mining allows agents to operate over longer time scaling, identifying low stock positions on the weekly line and avoiding daily fluctuations during continuous declines. This mimics the human approach of considering information at varying temporal and spatial scaling during stock trading. We further enhance the network’s robustness by adding an average pooling layer to the backbone convolutional neural network, reducing overfitting. State, action, reward, state, action, as an on-policy reinforcement learning method, generates dynamic trading strategies that combine multi-scaling information across different time scaling, while avoiding dangerous strategies. We evaluate the effectiveness of our proposed method on four real-world datasets (Dow Jones, NASDAQ, General Electric, and AAPLE) spanning from 1 January 2007 to 31 December 2020, and demonstrate its superior profits compared to several baseline methods. In addition, we perform various comparative and ablation tests in order to demonstrate the superiority of the proposed network architecture. Through these experiments, our proposed multi-scaling module yields better results compared to the single-scaling module.

Reinforcement-Learning-Based Robust Resource Management for Multi-Radio Systems

The advent of the Internet of Things (IoT) has triggered an increased demand for sensing devices with multiple integrated wireless transceivers. These platforms often support the advantageous use of multiple radio technologies to exploit their differing characteristics. Intelligent radio selection techniques allow these systems to become highly adaptive, ensuring more robust and reliable communications under dynamic channel conditions. In this paper, we focus on the wireless links between devices equipped by deployed operating personnel and intermediary access-point infrastructure. We use multi-radio platforms and wireless devices with multiple and diverse transceiver technologies to produce robust and reliable links through the adaptive control of available transceivers. In this work, the term ‘robust’ refers to communications that can be maintained despite changes in the environmental and radio conditions, i.e., during periods of interference caused by non-cooperative actors or multi-path or fading conditions in the physical environment. In this paper, a multi-objective reinforcement learning (MORL) framework is applied to address a multi-radio selection and power control problem. We propose independent reward functions to manage the trade-off between the conflicting objectives of minimised power consumption and maximised bit rate. We also adopt an adaptive exploration strategy for learning a robust behaviour policy and compare its online performance to conventional methods. An extension to the multi-objective state–action–reward–state–action (SARSA) algorithm is proposed to implement this adaptive exploration strategy. When applying adaptive exploration to the extended multi-objective SARSA algorithm, we achieve a 20% increase in the F1 score in comparison to one with decayed exploration policies.

A reinforcement learning-based cluster routing scheme with dynamic path planning for mutli-UAV network

Unmanned Aerial Vehicles (UAVs) with visual sensors are widely used for area mapping, management of crops and traffic, rescuing lives, and many other applications that need to cover a large area. The process of coverage can be improved with the use of efficient path planning and data transfer algorithms. Numerous research studies have been performed by concentrating on each of the aforementioned elements separately. However, to tackle the rapidly changing environmental situations, this paper proposes a cluster-based routing approach by incorporating a dynamic planning algorithm. The proposed model is composed of four modules, such as an online path planning algorithm, clustering-based network topology construction, reinforcement learning-based cluster management, and a data routing mechanism. Firstly, to maximise the coverage output, an optimal set of waypoints has been generated for all UAVs. For each UAV to complete the coverage task, it needs to completely cover its own set of waypoints. Since the environment is changing, a static path planning approach might fail to achieve complete coverage. Therefore, to drive the mission without getting stuck, a dynamic path planning approach is proposed that decides the next waypoint for a UAV based on the current waypoint. The main purpose of the algorithm is to cover all the waypoints in a polynomial amount of time. Secondly, the topology construction module consists of the initialization process, cluster head election, and cluster formation. Based on five parameters such as degree of centrality, surplus energy, link stability time, connectivity with the backbone UAV, and velocity, a set of cluster heads are selected. Then the cluster management process is performed by the optimal re-clustering policy determined using an approach of reinforcement learning called State Action Reward State Action (SARSA) in ground control station. Finally, the introduction of inter-cluster forwarders and selective flooding of route requests makes the routing scheme enhance the packet delivery ratio and reduce the delay. The result shows that the proposed work performs better than the existing result in terms of different generic performance metrics used for path planning and routing.

A hybrid cancer prediction based on multi-omics data and reinforcement learning state action reward state action (SARSA)

These days, the ratio of cancer diseases among patients has been growing day by day. Recently, many cancer cases have been reported in different clinical hospitals. Many machine learning algorithms have been suggested in the literature to predict cancer diseases with the same class types based on trained and test data. However, there are many research rooms available for further research. In this paper, the studies look into the different types of cancer by analyzing, classifying, and processing the multi-omics dataset in a fog cloud network. Based on SARSA on-policy and multi-omics workload learning, made possible by reinforcement learning, the study made new hybrid cancer detection schemes. It consists of different layers, such as clinical data collection via laboratories and tool processes (biopsy, colonoscopy, and mammography) at the distributed omics-based clinics in the network. The study considers the different cancer classes such as carcinomas, sarcomas, leukemias, and lymphomas with their types in work and processes them using the multi-omics distributed clinics in work. In order to solve the problem, the study presents omics cancer workload reinforcement learning state action reward state action “SARSA” (OCWLS) schemes, which are made up of an on-policy learning scheme on different parameters like states, actions, timestamps, reward, accuracy, and processing time constraints. The goal is to process multiple cancer classes and workload feature matching while reducing the time it takes to process in clinical hospitals that are spread out. Simulation results show that OCWLS is better than other machine learning methods regarding+ processing time, extracting features from multiple classes of cancer, and matching in the system.

Security Analysis of Cyber-Physical Systems Using Reinforcement Learning.

Future engineering systems with new capabilities that far exceed today's levels of autonomy, functionality, usability, dependability, and cyber security are predicted to be designed and developed using cyber-physical systems (CPSs). In this paper, the security of CPSs is investigated through a case study of a smart grid by using a reinforcement learning (RL) augmented attack graph to effectively highlight the subsystems' weaknesses. In particular, the state action reward state action (SARSA) RL technique is used, in which the agent is taken to be the attacker, and an attack graph created for the system is built to resemble the environment. SARSA uses rewards and penalties to identify the worst-case attack scenario; with the most cumulative reward, an attacker may carry out the most harm to the system with the fewest available actions. Results showed successfully the worst-case attack scenario with a total reward of 26.9 and identified the most severely damaged subsystems.

UAV Path Planning and Obstacle Avoidance Based on Reinforcement Learning in 3D Environments

This study proposes using unmanned aerial vehicles (UAVs) to carry out tasks involving path planning and obstacle avoidance, and to explore how to improve work efficiency and ensure the flight safety of drones. One of the applications under consideration is aquaculture cage detection; the net-cages used in sea-farming are usually numerous and are scattered widely over the sea. It is necessary to save energy consumption so that the drones can complete all cage detections and return to their base on land. In recent years, the application of reinforcement learning has become more and more extensive. In this study, the proposed method is mainly based on the Q-learning algorithm to enable improvements to path planning, and we compare it with a well-known reinforcement learning state–action–reward–state–action (SARSA) algorithm. For the obstacle avoidance control procedure, the same reinforcement learning method is used for training in the AirSim virtual environment; the parameters are changed, and the training results are compared.

Using Reinforcement Learning to Reduce Energy Consumption of Ultra-Dense Networks With 5G Use Cases Requirements

In mobile networks, 5G Ultra-Dense Networks (UDNs) have emerged as they effectively increase the network capacity due to cell splitting and densification. A Base Station (BS) is a fixed transceiver that is the main communication point for one or more wireless mobile client devices. As UDNs are densely deployed, the number of BSs and communication links is dense, raising concerns about resource management with regard to energy efficiency, since BSs consume much of the total cost of energy in a cellular network. It is expected that 6G next-generation mobile networks will include technologies such as artificial intelligence as a service and focus on energy efficiency. Using machine learning it is possible to optimize energy consumption with cognitive management of dormant, inactive and active states of network elements. Reinforcement learning enables policies that allow sleep mode techniques to gradually deactivate or activate components of BSs and decrease BS energy consumption. In this work, a sleep mode management based on State Action Reward State Action (SARSA) is proposed, which allows the use of specific metrics to find the best tradeoff between energy reduction and Quality of Service (QoS) constraints. The results of the simulations show that, depending on the target of the 5G use case, in low traffic load scenarios and when a reduction in energy consumption is preferred over QoS, it is possible to achieve energy savings up to 80% with 50 ms latency, 75% with 20 ms and 10 ms latencies and 20% with 1 ms latency. If the QoS is preferred, then the energy savings reach a maximum of 5% with minimal impact in terms of latency.

A first attempt to enhance Demand-Driven Material Requirements Planning through reinforcement learning

Production planning methods, which are meant to schedule efficiently production orders to meet costumer demands, offer the possibility to fix some parameters to better fit to a specific industrial context. These parameters can be fixed either according to the user's prior knowledge, or according to decision support tools. Optimization algorithms, which require prior knowledge of costumer demand, are most-commonly used for decision-support. Fewer decision-support tools based on reinforcement learning and requiring no prior knowledge of costumer demand have also been suggested in recent years to parameterize most common production planning systems. This paper investigates such a reinforcement-learning approach to parameterize Demand-Driven Materials Requirements Planning (DDMRP), which is a recent production planning method proven to be successful to avoid stockouts while minimizing the on-hand inventory. Despite approximate and exact optimization methods have been suggested in literature to parameterize DDMRP, a reinforcement learning approach remains to be investigated. We suggest a SARSA (State–action–reward–state–action) algorithm to the problem and test it on a randomly-generated instance. The first results are promising in regards to the use of a reinforcement learning approach for the dynamic parameterization of the DDMRP.

Inspired lightweight robust quantum Q-learning for smart generation control of power systems

With the continuous development of zero-carbon power systems, more and more models need to be considered in the system, and the dimension of data provided in the system will be higher and higher. The grid frequency will be difficult to be controlled stably. The existing smart generation control (SGC) methods like proportional–integral–derivative (PID), traditional Q-learning (QL), state–action–reward–state–action (SARSA), sliding mode control (SMC), and fuzzy logic control (FLC) have low-performance problems, the curse of dimensionality, and a slow convergence rate in the zero-carbon power system. This work proposes lightweight robust quantum Q-learning (LRQQL) methods to improve the performance of SGC methods and solve these problems. The LRQQL methods are divided into four methods. Two methods apply the idea of quantum normalization to update the action probability and different action selection methods to provide the action with higher performance. Besides, two lighter methods based on the idea of Grover iteration with the less curse of dimensionality and a faster convergence rate are proposed. The LRQQL methods are simulated in the two-area, four-area, and complex four-area systems. The evaluation indexes verify the feasibility of LRQQL algorithms. The LRQQL methods are more than 25% smaller than traditional QL in frequency error, more than 40% smaller than traditional QL in area control error, and more than 40% faster than the QL and SARSA in convergence rate. Meanwhile, the LRQQL methods possess higher adaptivity than PID, QL, SARSA, SMC, and FLC.

A Vision-Based Bio-Inspired Reinforcement Learning Algorithms for Manipulator Obstacle Avoidance

Path planning for robotic manipulators has proven to be a challenging issue in industrial applications. Despite providing precise waypoints, the traditional path planning algorithm requires a predefined map and is ineffective in complex, unknown environments. Reinforcement learning techniques can be used in cases where there is a no environmental map. For vision-based path planning and obstacle avoidance in assembly line operations, this study introduces various Reinforcement Learning (RL) algorithms based on discrete state-action space, such as Q-Learning, Deep Q Network (DQN), State-Action-Reward- State-Action (SARSA), and Double Deep Q Network (DDQN). By positioning the camera in an eye-to-hand position, this work used color-based segmentation to identify the locations of obstacles, start, and goal points. The homogeneous transformation technique was used to further convert the pixel values into robot coordinates. Furthermore, by adjusting the number of episodes, steps per episode, learning rate, and discount factor, a performance study of several RL algorithms was carried out. To further tune the training hyperparameters, genetic algorithms (GA) and particle swarm optimization (PSO) were employed. The length of the path travelled, the average reward, the average number of steps, and the time required to reach the objective point were all measured and compared for each of the test cases. Finally, the suggested methodology was evaluated using a live camera that recorded the robot workspace in real-time. The ideal path was then drawn using a TAL BRABO 5 DOF manipulator. It was concluded that waypoints obtained via Double DQN showed an improved performance and were able to avoid the obstacles and reach the goal point smoothly and efficiently.

A Real-Time Energy Consumption Minimization Framework for Electric Vehicles Routing Optimization Based on SARSA Reinforcement Learning

A real-time, metadata-driven electric vehicle routing optimization to reduce on-road energy requirements is proposed in this work. The proposed strategy employs the state–action–reward–state–action (SARSA) algorithm to learn the EV’s maximum travel policy as an agent. As a function of the received reward signal, the policy model evaluates the optimal behavior of the agent. Markov chain models (MCMs) are used to estimate the agent’s energy requirements on the road, in which a single Markov step represents the average energy consumption based on practical driving conditions, including driving patterns, road conditions, and restrictions that may apply. A real-time simulation in Python with TensorFlow, NumPy, and Pandas library requirements was run, considering real-life driving data for two EVs trips retrieved from Google’s API. The two trips started at 4.30 p.m. on 11 October 2021, in Los Angeles, California, and Miami, Florida, to reach EV charging stations six miles away from the starting locations. According to simulation results, the proposed AI-based energy minimization framework reduces the energy requirement by 11.04% and 5.72%, respectively. The results yield lower energy consumption compared with Google’s suggested routes and previous work reported in the literature using the DDQN algorithm.

Secure State Estimation of Cyber-Physical System under Cyber Attacks: Q-Learning vs. SARSA

This paper proposes a reinforcement learning (RL) algorithm for the security problem of state estimation of cyber-physical system (CPS) under denial-of-service (DoS) attacks. The security of CPS will inevitably decline when faced with malicious cyber attacks. In order to analyze the impact of cyber attacks on CPS performance, a Kalman filter, as an adaptive state estimation technology, is combined with an RL method to evaluate the issue of system security, where estimation performance is adopted as an evaluation criterion. Then, the transition of estimation error covariance under a DoS attack is described as a Markov decision process, and the RL algorithm could be applied to resolve the optimal countermeasures. Meanwhile, the interactive combat between defender and attacker could be regarded as a two-player zero-sum game, where the Nash equilibrium policy exists but needs to be solved. Considering the energy constraints, the action selection of both sides will be restricted by setting certain cost functions. The proposed RL approach is designed from three different perspectives, including the defender, the attacker and the interactive game of two opposite sides. In addition, the framework of Q-learning and state–action–reward–state–action (SARSA) methods are investigated separately in this paper to analyze the influence of different RL algorithms. The results show that both algorithms obtain the corresponding optimal policy and the Nash equilibrium policy of the zero-sum interactive game. Through comparative analysis of two algorithms, it is verified that the differences between Q-Learning and SARSA could be applied effectively into the secure state estimation in CPS.

Gray Wolf Optimization for Scheduling Irrigation Water

Although various optimization algorithms have been developed for operational purposes, the development of new optimization algorithms is still an open problem due to the complex system of irrigation canals that must be addressed. Recently, a new algorithm named gray wolf optimization (GWO) has been introduced and applied in different contexts. It mimics the social hierarchy and hunting behavior of gray wolves in nature. In this research, GWO was formulated, developed, and linked to irrigation canal system simulation (ICSS) to schedule water delivery. A fitness (optimization) function was defined according to the standard water delivery performance indicators. Normal and water shortage operational scenarios in the E1R1 Dez canal in Iran were tested and evaluated. The results revealed that GWO is a powerful optimization method and avoids local optimal points when normal conditions exist. However, it has relatively poor performance in water shortage conditions in which there is not enough water. Water depth variations remain inside acceptable margins. Its results were comparable to fuzzy state, action, reward, state, action (SARSA) learning (FSL) in the same canal, showing a value of maximum absolute error (MAE) and integral absolute error (IAE) of 10.7% and 9.2%, respectively, and it can distribute water between turnouts adequately, efficiently, equitably, and dependably in normal conditions.