Learning Automata Research Articles

Convolutional neural network (CNN) configuration using a learning automaton model for neonatal brain image segmentation.

CNN is considered an efficient tool in brain image segmentation. However, neonatal brain images require specific methods due to their nature and structural differences from adult brain images. Hence, it is necessary to determine the optimal structure and parameters for these models to achieve the desired results. In this article, an adaptive method for CNN automatic configuration for neonatal brain image segmentation is presented based on the encoder-decoder structure, in which the hyperparameters of this network, i.e., size, length, and width of the filter in each layer along with the type of pooling functions with a reinforcement learning approach and an LA model are determined. These LA models determine the optimal configuration for the CNN model by using DICE and ASD segmentation quality evaluation criteria, so that the segmentation quality can be maximized based on the goal criteria. The effectiveness of the proposed method has been evaluated using a database of infant MRI images and the results have been compared with previous methods. The results show that by using the proposed method, it is possible to segment NBI with higher quality and accuracy.

Congestion control in internet of things (IoT) using auction theory

The Internet of Things (IoT) facilitates data transmission through communication networks, preventing congestion when input data rate exceeds output, and congestion control in computer networks modulates traffic entry. This paper proposes a fusion of auction theory with reinforcement learning as a means of managing congestion in the IoT. The proposed technique seeks to enhance network performance by utilizing object trustworthiness evaluation and auction-based route selection to manage congestion during data routing. The suggested method calculates the believability of objects by analyzing their historical performance in data forwarding and congestion avoidance, utilizing a learning automaton. The auction approach is employed to determine the most efficient ways for transmitting data. The IoT topology is initially organized into a collection of dependable links known as the Connected Dominating Set (CDS). Active objects employ the learning automata model to assess the reliability of their neighbors. The auction model ultimately chooses the optimal route based on characteristics such as credibility, energy, and delay. The experimental results demonstrate that the proposed methodology surpasses existing comparison methods in the initial scenario, exhibiting a 24.13% reduction in energy usage.

An encrypted traffic classifier via combination of deep learning and automata learning

An encrypted traffic classifier via combination of deep learning and automata learning

Asymmetric variable depth learning automaton and its application in defending against selfish mining attacks on bitcoin

Learning Automaton (LA) theory, a branch of reinforcement learning, initially began with the Fixed Structure Learning Automaton (FSLA) family and was later expanded to include the Variable Structure Learning Automaton (VSLA) family. Tuning the depth of FSLA in complex environments has long been a challenging task, significantly limiting the ability to effectively navigate the exploration-exploitation dilemma. No solution has been found for this open problem yet. This study addresses this issue by introducing a novel hybrid learning automaton model called Asymmetric Variable Depth Hybrid LA (AVDHLA). The AVDHLA model intelligently learns the depth of fixed structure LA in an autonomous manner by combining LKN,K from FSLA class and Variable Action Set LA (VASLA) from VSLA class. Computer simulations are conducted to validate the proposed model in diverse environments, including both stationary and non-stationary (Markovian switching and State-dependent) scenarios. Performance evaluation is based on predefined metrics, such as total number of rewards (TNR) and action switching (TNAS). Statistical tests indicate that across both stationary and non-stationary environments, AVDHLA consistently outperforms LKN,K in terms of TNR and TNAS across the majority of experiments. Moreover, the AVDHLA model is applied in two key applications. Firstly, it is used to defend against the selfish mining attack in Bitcoin and is compared with the well-known tie-breaking mechanism. Simulation results consistently demonstrate that our proposed method increases the threshold for successful selfish mining attacks from 25% to 40%. Secondly, the AVDHLA model has been applied to develop a novel learning automaton-based recommendation system. The results demonstrate the superiority of the proposed method in terms of the Click-Through Rate (CTR) and Precision compared to previous approaches.

Efficient identification of maximum independent sets in stochastic multilayer graphs with learning automata

Investigating the maximum independent set in stochastic multilayer graphs provides critical insights into the structural and dynamical properties of complex networks. Recently, stochastic multilayer graphs effectively model the intricate interactions and interdependencies inherent in real-world systems, including social, biological, and transportation networks. The identification of a maximum independent set -comprising nodes without direct connections- offers a significant understanding of phenomena such as information diffusion, resource allocation, and epidemic spread within complex social networks. For instance, independent sets play a crucial role in identifying influencers -individuals who profoundly impact their peers, propagating information or opinions widely. In this paper, we introduce the stochastic version of the maximum independent set and propose five algorithms based on learning automata to identify maximum independent sets in the stochastic multilayer graphs. Our approach utilizes learning automata to provide a guided sampling from candidate independent sets of the stochastic multilayer graph, aiming to identify the independent set with the maximum expected value while utilizing fewer vertex samples than standard methods that do not incorporate the learning. In addition to proving several mathematical properties of the proposed approach, simulations conducted across diverse stochastic multilayer graphs demonstrate that our learning automata-based algorithms outperform traditional approaches, achieving higher convergence rates and requiring fewer samples.

Enhanced reinforcement learning-based two-way transmit-receive directional antennas neighbor discovery in wireless ad hoc networks

The utilization of directional antennas for neighbor discovery in wireless ad hoc networks brings notable benefits, such as extended transmission range, reduced transmission interference, and enhanced antenna gain. However, when nodes use directional antennas for neighbor discovery, the communication range is limited, resulting in a lack of knowledge of potential neighbors. Hence, it is necessary to design a special antenna direction switching strategy for neighbor discovery based on directional antennas. Traditional methods of switching antenna directions are often random or follow predefined sequences, overlooking the historical knowledge of sector exploration for antenna directions. In contrast, existing machine learning approaches aim to leverage observed historical knowledge to adjust antenna directions for faster neighbor discovery. Nonetheless, the latency of neighbor discovery is still high because the node cannot fully utilize the observed historical knowledge (i.e.., only using the knowledge observed by the node in transmission mode, ignoring the knowledge observed by the node in reception mode). Meanwhile, the corresponding reward and penalty mechanisms are still not detailed enough (i.e.., these reward and penalty mechanisms only consider the sectors of discovered and undiscovered neighboring nodes, ignoring the scenario of sectors that have been rewarded). In this paper, the neighbor discovery process is modeled as a reinforcement learning-based learning automaton. We propose an enhanced reinforcement learning-based two-way transmit-receive directional antennas neighbor discovery algorithm, called ERTTND. The algorithm consists of a two-way transmit-receive reinforcement learning mechanism (TTRL) and an enhanced reward-and-penalty mechanism (ERAP). This algorithm leverages insights from nodes in transmission and reception modes to refine their tactical decisions. Then, through an enriched reward-and-penalty framework, nodes optimize their strategies, thus expediting neighbor discovery based on directional antennas in wireless ad hoc networks. Simulation results demonstrate that compared to existing representative algorithms, the proposed ERTTND algorithm can achieve over 30% savings in terms of average discovery delay and energy consumption.

Algorithm for Target Coverage Problem Based on Deep Learning in Wireless Sensor Networks

Aiming at the uncertain mechanism of node activation strategies and redundancy of feasible solution sets in the process of solving target coverage problem in wireless sensor networks, we proposed a deep learning based target coverage algorithm to learn the scheduling strategies of nodes in wireless sensor networks. Firstly , the algorithm abstracted the construction of feasible solution sets into Markov decision process, and intelligently selected activated sensor nodes as discrete actions according to the network environment. Secondly, a reward function evaluated the performance of the intelligent agent in selecting actions based on the coverage capacity and its residual energy of the active node. The simulation experiment result shows that the algorithm is effective in different network environments, and the network lifecycle is superior to the three greedy algorithms, the maximum lifetime coverage algorithm and the adaptive learning automaton algorithm.

Chaotic libration and control of a space tether-sail system in Earth polar orbits with J2 perturbation

The recently proposed space tether-sail system could enable various interesting and important potential space missions thanks to its thin and long connecting tether and solar radiation pressure (SRP) force without consuming propellants. This paper focuses on nonlinear libration and its control of a tether-sail system in Earth polar orbits. Firstly, nonlinear coupled in-plane and out-of-plane dynamics of the system with a rigid mass-distributed tether and two point masses (a chief satellite and sail respectively) is established using the Lagrangian equation method. Secondly, the predicted occurrence of chaotic in- and out-of-plane librational motions is verified utilizing numerical tools such as presenting the time history of libration, the phase plane, the Poincaré section and power spectral density(PSD). Specifically, the paper investigates the sensitivity of nonlinear dynamical responses to initial values, focuses on the chaotic motion caused by the coupling between the in- and out-of-plane librations, the J2 perturbation of the Earth gravitational field and the small eccentricity of the orbits, also studies the nonlinear librational characteristics influenced by the initial mechanical energy (the Hamiltonian). Thirdly, chaotic librational motions will be controlled merely using modulatable SRP force by the sail propellantlessly. A sliding mode controller based on exponential approaching law is developed. To mitigate the effects of control torque saturation, an input compensator based on Radial Basis Function (RBF) neural network is designed. In addition, to address the cumbersome and inefficient manual tuning of control parameters for the sliding mode controller, a parameter tuning algorithm based on Learning Automata is designed. This algorithm is capable of tuning a high-quality set of controller parameters within 100 iterations. The effectiveness of the propellantless actuators for chaotic libration motion control and developed control algorithm is supported by numerical simulations.

LATA: learning automata-based task assignment on heterogeneous cloud computing platform

A cloud computing environment is a distributed system where idle resources are accessible across a wide area network, such as the Internet. Due to the diverse specifications of these resources, computational clouds exhibit high heterogeneity. Task scheduling, the process of dispatching cloud applications onto processing nodes, becomes a critical challenge in such environments. Ensuring high utilization in this heterogeneous environment entails identifying suitable machines or virtual machines capable of efficiently executing jobs, constituting a multi-objective optimization problem. This paper proposes a dynamic Learning Automata-based Task Assignment algorithm, named LATA, to address this challenge. In the algorithm, each application is represented as a Directed Acyclic Graph, with tasks as nodes and data dependencies as edges. Initially, tasks are grouped based on their data dependencies to consolidate independent tasks into one group. Subsequently, a variable-structure learning automaton is assigned to each group of tasks to identify appropriate task-machine combinations. The primary objectives of LATA include minimizing makespan and energy consumption by facilitating efficient task placement to achieve load balance and maximize resource utilization. Additionally, an enhancement is proposed, involving the use of a different grouping policy prior to task assignment to further improve performance. Computer simulation results demonstrate the superior performance of the proposed algorithms in highly heterogeneous environments compared to state-of-the-art algorithms. Notably, total execution time and energy consumption decrease by up to 50% and 37%, respectively.

Load-aware continuous-time optimization for multi-agent systems: toward dynamic resource allocation and real-time adaptability

In the realm of next-generation mobile communication networks, characterized by dynamic and evolving workloads, the efficient resource allocation becomes paramount for achieving optimal performance. This paper addresses the intricate challenges of multiagent resource management through the integration of stochastic learning automata and continuous-time optimization techniques, presenting an innovative solution for real-time adaptability and dynamic load balancing. The central aim is to achieve optimal values for local time-varying cost functions while taking into account resource constraints and the feasibility of local allocation. In the context of multi-agent resource allocation systems, where conditions vary over time, the algorithm exhibits a dynamic adaptability, continuously adjusting to the evolving optimal solutions tailored for each agent. Utilizing stochastic learning automata, the algorithm effectively addresses the time-varying load-balancing function in response to dynamic user demands. Furthermore, we introduced an scalable solution for mobility robustness optimization based on deep reinforcement learning (DRL-MRO). This method dynamically identifies network-wide optimal parameter values across the entire network to accommodate diverse mobility patterns, ensuring the effectiveness of load balancing parameters that seamlessly adapt to the dynamic configuration of the network. The overarching goal is to maintain a consistent quality of service level for each agent, thereby enhancing the overall system performance. Simulation outcomes unequivocally affirm the superior performance of the proposed load balancing approach, rooted in continuous-time optimization and stochastic learning automata, surpassing existing schemes across diverse system configurations.

The Hierarchical Discrete Pursuit Learning Automaton: A Novel Scheme With Fast Convergence and Epsilon-Optimality.

Since the early 1960s, the paradigm of learning automata (LA) has experienced abundant interest. Arguably, it has also served as the foundation for the phenomenon and field of reinforcement learning (RL). Over the decades, new concepts and fundamental principles have been introduced to increase the LA's speed and accuracy. These include using probability updating functions, discretizing the probability space, and using the "Pursuit" concept. Very recently, the concept of incorporating "structure" into the ordering of the LA's actions has improved both the speed and accuracy of the corresponding hierarchical machines, when the number of actions is large. This has led to the ϵ -optimal hierarchical continuous pursuit LA (HCPA). This article pioneers the inclusion of all the above-mentioned phenomena into a new single LA, leading to the novel hierarchical discretized pursuit LA (HDPA). Indeed, although the previously proposed HCPA is powerful, its speed has an impediment when any action probability is close to unity, because the updates of the components of the probability vector are correspondingly smaller when any action probability becomes closer to unity. We propose here, the novel HDPA, where we infuse the phenomenon of discretization into the action probability vector's updating functionality, and which is invoked recursively at every stage of the machine's hierarchical structure. This discretized functionality does not possess the same impediment, because discretization prohibits it. We demonstrate the HDPA's robustness and validity by formally proving the ϵ -optimality by utilizing the moderation property. We also invoke the submartingale characteristic at every level, to prove that the action probability of the optimal action converges to unity as time goes to infinity. Apart from the new machine being ϵ -optimal, the numerical results demonstrate that the number of iterations required for convergence is significantly reduced for the HDPA, when compared to the state-of-the-art HCPA scheme.

Assessment of American Bullfrog (Lithobates catesbeianus) spreading in the Republic of Korea using rule learning of elementary cellular automata

The spread of American Bullfrog has a significant impact on the surrounding ecosystem. It is important to study the mechanisms of their spreading so that proper mitigation can be applied when needed. This study analyzes data from national surveys on bullfrog distribution. We divided the data into 25 regional clusters. To assess the spread within each cluster, we constructed temporal sequences of spatial distribution using the agglomerative clustering method. We employed Elementary Cellular Automata (ECA) to identify rules governing the changes in spatial patterns. Each cell in the ECA grid represents either the presence or absence of bullfrogs based on observations. For each cluster, we counted the number of presence location in the sequence to quantify spreading intensity. We used a Convolutional Neural Network (CNN) to learn the ECA rules and predict future spreading intensity by estimating the expected number of presence locations over 400 simulated generations. We incorporated environmental factors by obtaining habitat suitability maps using Maxent. We multiplied spreading intensity by habitat suitability to create an overall assessment of bullfrog invasion risk. We estimated the relative spreading assessment and classified it into four categories: rapidly spreading, slowly spreading, stable populations, and declining populations.

Recognition of facial emotion based on SOAR model.

Expressing emotions play a special role in daily communication, and one of the most essential methods in detecting emotions is to detect facial emotional states. Therefore, one of the crucial aspects of the natural human-machine interaction is the recognition of facial expressions and the creation of feedback, according to the perceived emotion. To implement each part of this model, two main steps have been introduced. The first step is reading the video and converting it to images and preprocessing on them. The next step is to use the combination of 3D convolutional neural network (3DCNN) and learning automata (LA) to classify and detect the rate of facial emotional recognition. The reason for choosing 3DCNN in our model is that no dimension is removed from the images, and considering the temporal information in dynamic images leads to more efficient and better classification. In addition, the training of the 3DCNN network in calculating the backpropagation error is adjusted by LA so that both the efficiency of the proposed model is increased, and the working memory part of the SOAR model can be implemented. Due to the importance of the topic, this article presents an efficient method for recognizing emotional states from facial images based on a mixed deep learning and cognitive model called SOAR. Among the objectives of the proposed model, it is possible to mention providing a model for learning the time order of frames in the movie and providing a model for better display of visual features, increasing the recognition rate. The accuracy of recognition rate of facial emotional states in the proposed model is 85.3%. To compare the effectiveness of the proposed model with other models, this model has been compared with competing models. By examining the results, we found that the proposed model has a better performance than other models.

Harnessing collaborative learning automata to guide multi-objective optimization based inverse analysis for structural damage identification

Structural damage identification based on physical models is often transformed into an optimization problem that minimizes the difference between measurement information of structure being monitored and the model prediction in the parametric space. However, the objective function in this context often exhibits multimodality, involving high-dimensional variables due to the reliance on finite element models for damage identification. These features pose challenges to optimization algorithms, where entrapment in local solutions can lead to false positives and false negatives in damage identification. In this research, we propose a reinforcement learning based multi-swarm optimizer to tackle such challenges in pursuit of a small yet diverse solution set that can capture the true damage scenario as one of the solutions. The proposed method leverages the flexibility of the particle swarm optimizer and incorporates novel strategies of metaheuristics to realize targeted improvement. To enable the particle swarm to adaptively select the appropriate search strategy based on the current environment, we adopt the learning automata technique, which sidesteps the need for reward strategy selection that is usually ad hoc at each step of the search. The integration harnesses the automatic learning and self-adaptation capabilities of learning automata, enabling the particles to navigate based on environmental signals. This leads to accumulated probabilities tied to advantageous movements, fostering an adaptive exploration of particles in the search space. The proposed approach is first validated through implementing into benchmark test cases with comparisons. It is then applied to structural damage identification with piezoelectric admittance experimental signals. `The results highlight the capability of the algorithm to identify a small solution set with high accuracy to match the actual damage scenario.

ETFC: Energy-efficient and deadline-aware task scheduling in fog computing

The Internet of Things (IoT) is constantly evolving and expanding. However, due to the limited IoT resources, it is intertwined with fog computing to use their resources to compensate for the limitations of IoT resources. On the other hand, fog devices face challenges, such as resource heterogeneity, high distribution, dynamism, and limitations, so an efficient task scheduling approach is needed to deploy fog computing resources effectively and improve the quality of service (QoS). This work mathematically formulates the task scheduling problem to minimize energy consumption and cost and improve QoS by reducing response time and deadline violation times of IoT tasks. Then, it proposes an Energy-efficient and deadline-Aware Task scheduling in Fog Computing (ETFC) method that predicts the traffic of fog nodes by a Support Vector Machine (SVM) and divides them into low-traffic and high-traffic groups. Next, the ETFC method schedules the low-traffic part with an algorithm based on reinforcement learning using the proposed ICLA-SOA, which is an algorithm based on irregular cellular learning automata and schedules the tasks of the high-traffic part with a metaheuristic algorithm using the proposed Non-dominated Sorting Genetic Algorithm (NSGA-III). The simulation results demonstrate that the ETFC method exhibits up to an 84 % enhancement in response time, up to a 33 % reduction in energy consumption, up to a 30 % decrease in costs, and up to a 28 % advancement in meeting task deadlines compared to other methods.

Enhanced Clustering MAC Protocol Based on Learning Automata for UV Networks

Ultraviolet (UV) networks are widely applied in complex electromagnetic environments. Designing an efficient multi-node medium access control (MAC) protocol for these networks is important. In this study, we proposed an enhanced clustering time division multiple access (TDMA) MAC protocol based on clustering and learning automata (LA). Subsequently, the effects of the network topology, class of service, and number of cluster nodes on the network performance under the proposed protocol were analyzed. Then, the protocol was compared with the TDMA protocol and clustering system. Results revealed that it obtained a better network performance, proving its suitability for multi-node UV networking.

Adaptive Spatial Scheduling for Event Traffic in LoRaWAN Networks.

Low-Power Wide-Area Networks constitute a leading, emerging Internet-of-Things technology, with important applications in environmental and industrial monitoring and disaster prevention and management. In such sensor networks, external detectable events can trigger synchronized alarm report transmissions. In LoRaWANs, and more generally in networks with a random access-based medium access algorithm, this can lead to a cascade of frame collisions, temporarily resulting in degraded performance and diminished system operational capacity, despite LoRaWANs' physical layer interference and collision reduction techniques. In this paper, a novel scheduling algorithm is proposed that can increase system reliability in the case of such events. The new adaptive spatial scheduling algorithm is based on learning automata, as well as previous developments in scheduling over LoRaWANs, and it leverages network feedback information and traffic spatial correlation to increase network performance while maintaining high reliability. The proposed algorithm is investigated via an extensive simulation under a variety of network conditions and compared with a previously proposed scheduler for event-triggered traffic. The results show a decrease of up to 30% in average frame delay compared to the previous approach and an order of magnitude lower delay compared to the baseline algorithm. These findings highlight the importance of using spatial information in adaptive schemes for improving network performance, especially in location-sensitive applications.

ELA-RCP: An energy-efficient and load balanced algorithm for reliable controller placement in software-defined networks

Software-defined networking is a modern and popular network technology that utilizes controllers as network operating systems to manage the network and allow user applications to interact with network hardware. Determining the optimal number of controllers and the optimal place to install them are some of the multi-controller model challenges that are known as the Controller Placement Problem (CPP). CPP is an optimization problem that tries to minimize delay and achieve a balanced network. CPP becomes more important when we intend to increase reliability and reduce energy consumption. This paper proposes an Energy-efficient and Load-balanced optimization Algorithm for Reliable Controller Placement (ELA-RCP) in Software-Defined Networks (SDN). The proposed ELA-RCP is a three-step solution to CPPs. First, it determines the optimal number of controllers using the proposed cellular learning automata, taking into account network traffic status, scale, and budget. Second, an innovative meta-heuristic algorithm is proposed to identify proper controller locations based on factors like path reliability, controller processing capacity, propagation delay, and energy consumption. In the last step, load balancing and the reliability of the controllers are improved using queuing theory and a proposed greedy algorithm. ELA-RCP is evaluated with real-world topologies. The simulation demonstrates up to 25% improvement in energy consumption, up to 108% improvement in the network survivability, and up to 68% improvement in mean average delay rise compared to particle swarm optimization, firefly optimization algorithm, ant lion optimization, and seagull optimization algorithm.

Handling Sudoku puzzles with irregular learning cellular automata

The use of Cellular Automata (CA) in combination with Learning Automata (LA) has demonstrated effectiveness in handling hard-to-be-solved problems. Due to their capacity to learn and adapt, as well as their inherent parallelism, they can expedite the problem-solving process for a range of problems, such as challenging logic puzzles. One such puzzle is Sudoku, which poses a combinatorial optimization challenge of great difficulty and complexity. In this study, a Sudoku puzzle was represented as an Irregular Learning Cellular Automaton (ILCA), using a reward and penalty algorithm to resolve it. Simulations for an amount of 400 puzzles were performed, while the results demonstrate that the proposed algorithm operates effectively, highlighting the concurrent and learning capabilities of the ILCA structure. Furthermore, two different performance enhancement methods are investigated, namely learning rates method and selective probability reset rule, which are able to increase the initial performance by 26.8%\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$26.8\\%$$\\end{document} and to achieve an overall 99.3%\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$99.3\\%$$\\end{document} resolution rate.

An Intelligent Bayesian Inference Based Learning Automaton Approach for Traffic Management in Radio Access Network

An Intelligent Bayesian Inference Based Learning Automaton Approach for Traffic Management in Radio Access Network