Learning Automata-based Algorithm Research Articles

A new approach to solving target coverage problem in wireless sensor networks using an effective hybrid genetic algorithm and tabu search

Literature in recent years has introduced several studies conducted to solve the target coverage problem in wireless sensor networks (WSNs). Sensors are conventionally assumed as devices with only a single power level. However, real applications may involve sensors with multiple power levels (i.e., multiple sensing ranges each of which possesses a unique power consumption). Consequently, one of the key problems in WSNs is how to provide a full coverage on all targets distributed in a network containing sensors with multiple power levels and simultaneously prolong the network lifetime as much as possible. This problem is known as Maximum Network Lifetime With Adjustable Ranges (MNLAR) and its NP-completeness has been already proved. To solve this problem, we proposed an efficient hybrid algorithm containing Genetic Algorithm (GA) and Tabu Search (TS) aiming at constructing cover sets that consist of sensors with appropriate sensing ranges to provide a desirable coverage for all the targets in the network. In our hybrid model, GA as a robust global searching algorithm is used for exploration purposes, while TS with its already-proved local searching ability is utilized for exploitation purposes. As a result, the proposed algorithm is capable of creating a balance between intensification and diversification. To solve the MNLR problem in an efficient way, the proposed model was also enriched with an effective encoding method, genetic operators, and neighboring structure. In the present paper, different experiments were performed for the purpose of evaluating how the proposed algorithm performs the tasks defined. The results clearly confirmed the superiority of the proposed algorithm over the greedy-based algorithm and learning automata-based algorithm in terms of extending the network lifetime. Moreover, it was found that the use of multiple power levels altogether caused the extension of the network lifetime.

An Economic and Non-cooperative Load-balancing Framework among Federated Cloudlets

Edge computing servers like cloudlets from different service providers compensate scarce computational and storage resources of mobile devices, are distributed across access networks. However, the dynamically varying computational requirements of associated mobile devices make cloudlets either overloaded or under-loaded. Hence, load-balancing among neighboring cloudlets appears to be an essential research problem. Especially, the load-balancing problem among federated cloudlets from the same as well as different service providers for low-latency applications needs significant attention. Thus, in this paper, we propose a load-balancing framework among federated cloudlets for low-latency applications that focuses on latency bound rather than latency minimization. In this framework, we employ dynamic processor slicing for internal handling of job requests from heterogeneous classes by the cloudlets. We propose a continuous-action reinforcement learning automata-based algorithm that enables cloudlets to independently compute the load-balancing strategies against their neighboring cloudlets in a software-defined networking enabled distributed network setting without any exhaustive control message exchange. To capture the economic interaction among federated cloudlets, we model this load-balancing problem as an economic and non-cooperative game and by scaffolding the properties of the game formulation, we achieve faster convergence of the reinforcement learning automata. Furthermore, through extensive simulations, we study the impacts of exploration and exploitation on learning accuracy.

Finding Suitable Membership Functions for Mining Fuzzy Association Rules in Web Data Using Learning Automata

Transactions in web data are huge amounts of data, often consisting of fuzzy and quantitative values. Mining fuzzy association rules can help discover interesting relationships between web data. The quality of these rules depends on membership functions, and thus, it is essential to find the suitable number and position of membership functions. The time spent by users on each web page, which shows their level of interest in those web pages, can be considered as a trapezoidal membership function (TMF). In this paper, the optimization problem was finding the appropriate number and position of TMFs for each web page. To solve this optimization problem, a learning automata-based algorithm was proposed to optimize the number and position of TMFs (LA-ONPTMF). Experiments conducted on two real datasets confirmed that the proposed algorithm enhances the efficiency of mining fuzzy association rules by extracting the optimized TMFs.

Navigation in the social internet-of-things (SIoT) for discovering the influential service-providers using distributed learning automata

The integration of the social network concepts and IoT has led to the creation of a new concept called Social IoT, wherein objects or things can interact and provide services to one another. Efficient and distributed service navigation in such systems is essential due to the potentially huge number of services that are provided. This paper proposes a Distributed Learning Automata-based algorithm called DLA_N, which aims at finding service-providers (i.e., objects) with high popularity or influence. The main idea is that an object can use its friends or friends of its friends to search for the desired service provider. Taking into account the SIoT’s graph properties (i.e., the topology of the network), we define a new centrality metric that indicates the importance degree of a node in SIoT. Embedding a Learning Automata in each object, a distributed LA approach is proposed for the selection of the most influential nodes in the network. Starting from a requesting object, the proposed DLA_N algorithm learns to select a path containing objects with a high centrality metric. The distributed nature of our navigation results in high scalability and low navigation time. The results of performance evaluation indicate that the proposed method outperforms existing methods in the literature.

Cellular automata-based optimised routing for secure data transmission in wireless sensor networks

ABSTRACT Sensor nodes encompass the capability of controlling and monitoring the environment by measuring the factors, like temperature, pressure, and humidity. Ensuring secure routing in the network is an important and significant task in the wireless sensor network (WSN). Even though various routing protocols are adopted in the network, providing security and maintaining the energy is still a challenging issue in the research community. Thus, an effective particle-based spider monkey optimisation (P-SMO) algorithm is proposed in this research to select the optimal route to manage secure communication in WSNs. Accordingly, the network is simulated and the cluster heads (CHs) are chosen based on the efficient learning automata-based cell clustering algorithm (ELACCA) such that the routing path is established based on the chosen CHs. Thus, the proposed P-SMO algorithm establishes the secure routing path based on the factors, such as energy, delay, consistency factor, and trust. The proposed P-SMO algorithm is the integration of particle swarm optimisation (PSO) and spider monkey optimisation (SMO) algorithm in such a way that the optimal routes are defined effectively. The analysis of the proposed routing protocol proved the effective performance with the maximal number of nodes alive, coverage, energy balancing index, and the average remaining energy of 67, 93.407%, 0.7555, and 0.3402 J, respectively.

A variable action set cellular learning automata-based algorithm for link prediction in online social networks

Link prediction (LP) is a crucial issue in the online social network (OSN) evolution analysis. Since OSNs are growing in size on a daily basis, a growing need for scalable LP algorithms is being felt. OSNs are innately evolutionary, such that the characteristics, behavior, and activities of their components (including nodes and links) change over time. In analyzing social networks which are based on the time evolution model, LP helps us realize the logic of social network growth. Deriving time patterns of evolutionary changes according to the communities and neighbors of nodes in a network can be aptly used for LP. This article introduces a new algorithm based on irregular cellular learning automata (ICLAs) for LP in the near future in OSNs. The algorithm we propose here models the network as an ICLA. The ICLA weighs the real links in the network according to entities' participation in forming communities over consecutive time periods. This method lies in the premise that social networks include communities. Based on the communities formed over successive time periods, the presented method calculates the probability of link formation between every pair of nodes which are unconnected at the present time, estimating the chances of their connection in the near future. Experiments performed on real social networks show that the proposed algorithm produces good results in predicting link formation in OSNs.

Learning Automata-Based Multiagent Reinforcement Learning for Optimization of Cooperative Tasks.

Multiagent reinforcement learning (MARL) has been extensively used in many applications for its tractable implementation and task distribution. Learning automata, which can be classified under MARL in the category of independent learner, are used to obtain the optimal joint action or some type of equilibrium. Learning automata have the following advantages. First, learning automata do not require any agent to observe the action of any other agent. Second, learning automata are simple in structure and easy to be implemented. Learning automata have been applied to function optimization, image processing, data clustering, recommender systems, and wireless sensor networks. However, a few learning automata-based algorithms have been proposed for optimization of cooperative repeated games and stochastic games. We propose an algorithm known as learning automata for optimization of cooperative agents (LA-OCA). To make learning automata applicable to cooperative tasks, we transform the environment to a P-model by introducing an indicator variable whose value is one when the maximal reward is obtained and is zero otherwise. Theoretical analysis shows that all the strict optimal joint actions are stable critical points of the model of LA-OCA in cooperative repeated games with an arbitrary finite number of players and actions. Simulation results show that LA-OCA obtains the pure optimal joint strategy with a success rate of 100% in all of the three cooperative tasks and outperforms the other algorithms in terms of learning speed.

A learning automata-based algorithm to solve imbalanced k-coverage in visual sensor networks

One of the most important problems in directional sensor networks is k-coverage in which the orientation of a minimum number of directional sensors is determined in such a way that each target can be monitored at least k times. This problem has been already considered in two different environments: over provisioned where the number of sensors is enough to cover all targets, and under provisioned where there are not enough sensors to do the coverage task (known as imbalanced k-coverage problem). Due to the significance of solving the imbalanced k-coverage problem, this paper proposes a learning automata (LA)-based algorithm capable of selecting a minimum number of sensors in a way to provide k-coverage for all targets in a balanced way. To evaluate the efficiency of the proposed algorithm performance, several experiments were conducted and the obtained results were compared to those of two greedy-based algorithms. The results confirmed the efficiency of the proposed algorithm in terms of solving the problem.

A learning automata-based algorithm for solving the target [formula omitted]-coverage problem in directional sensor networks with adjustable sensing ranges

In order to find a solution for the target coverage problem in directional sensor networks (DSNs), some researchers have recently introduced several efficient algorithms. These sensors are conventionally supposed to have a single power level and a single coverage is just needed for targets. In other words, there are various sensing ranges and power consumptions for these sensors under real conditions and at least k times monitoring is required for each target. The present paper addresses this issue as the target k-coverage with adjustable sensing range, which has not been already studied in DSNs. To solve this problem, two learning automata-based algorithms (Algorithms 1 and 2) are proposed and equipped with a strong pruning rule that facilitates the selection of appropriate sensor directions capable of providing the targets with k-coverage. After evaluating the efficiency of the algorithms’ performance by conducting several experiments, the results were compared to those ones obtained by a greedy-based algorithm, which is discussed in the literature. Finding indicated that algorithms have superiority over their rivals regarding the prolonged lifetime of their network.

An iterative stochastic algorithm based on distributed learning automata for finding the stochastic shortest path in stochastic graphs

In this paper, we study the problem of finding the shortest path in stochastic graphs and propose an iterative algorithm for solving it. This algorithm is based on distributed learning automata (DLA), and its objective is to use a DLA for finding the shortest path from the given source node to the given destination node whose weight is minimal in expected sense. At each stage of this algorithm, DLA specifies edges needed to be sampled. We show that the given algorithm finds the shortest path with minimum expected weight in stochastic graphs with high probability which can be close to unity as much as possible. We compare the given algorithm with some distributed learning automata-based iterative algorithms, a particle swarm optimization-based algorithm, an ant colony-based algorithm, a Q-learning-based algorithm, and an actor–critic-based algorithm for finding the shortest path. Computer experiments show that the proposed algorithm requires fewer edge samples to find the shortest path than the previously introduced DLA-based algorithms.

A hybrid learning automata and case-based reasoning for fractured zone detection using petrophysical logs

Fractured zone detection and fracture density estimation in oil wells have significant effects on wellbore stability, reservoir modeling, drilling operations, and well production. Although high-resolution image logs facilitate the identification of fracture characteristics, they are not available for a large set of wells as they drilled before development of the enhanced well logging technologies. Petrophysical logs contain valuable information and can be used as an alternative approach in fracture identifications. These logs are almost available in all wells and can be used as useful inputs to shape a predictive model representing fracture characteristics where image logs are not accessible. This paper proposes a generalized case-based reasoning (CBR) method for fractured zone detection via petrophysical logs. To such aim, a set of train wells are used to beget a database composed of both petrophysical data and the image logs. A learning automata-based algorithm is conducted to find the optimal similarity relation between petrophysical logs and manual interpretation of the borehole image logs. Moreover, new decision parameters are introduced to enhance the applicability of the proposed method in real-life projects. The developed model is successfully tested on the Asmari reservoir through several oil wells from, followed by a discussion on results.

A learning automata-based algorithm for energy and SLA efficient consolidation of virtual machines in cloud data centers

Resource management in cloud computing consists of allocating processing resources, storage, and network to a set of software applications. Resource providers focus on performance and utilization of resources considering the constraints of service level agreement. Resource performance is achieved by virtualization techniques, which share infrastructure of the resource provider between different virtual machines. This study proposes a novel algorithm based on learning automata, which improves resource utilization and reduces energy consumption. The proposed algorithm considers changes in the user demanded resources to predict the PM, which may suffer from overload. Due to preventing server overload, the proposed algorithm improves PMs’ utilization, reduces the number of migrations, and shuts down idle servers to reduce the energy consumption of the data center. The proposed algorithm is simulated in CloudSim simulator; the 10-day processor information of a real PlanetLab cloud infrastructure system are used for workload data. Performance of the proposed algorithm is compared with existing algorithms such as DVFS, NPA, and the threshold algorithm in terms of energy consumption and the number of shut down PMs. Simulation results indicate that the proposed algorithm outperforms other algorithms with 175.48 Kwh, 0.00326 in energy consumption, SLA violation respectively.

A Learning Automata-based Algorithm for Area Coverage Problem in Directional Sensor Networks

Coverage problem is a research hot spot in directional sensor networks (DSNs). However, the major problem affecting the performance of the current coverage-enhancing strategies is that they just optimize the coverage of networks, but ignore the maximum number of sleep sensors to save more energy. Aiming to find an approximate optimal method that can cover maximum area with minimum number of active sensors, in this paper, a new scheduling algorithm based on learning automata is proposed to enhance area coverage, and shut off redundant sensors as many as possible. To evaluate the performance of the proposed algorithm, several experiments are conducted. Simulation results indicate that the proposed algorithm have effective performance in terms of coverage enhancement and sleeping sensors compared to the existing algorithms.

A new cellular learning automata-based algorithm for community detection in complex social networks

Community structure is one of the common and fundamental characteristics of many real-world networks such as information and social networks. The structure, function, evolution and dynamics of complex social networks can be explored through detecting the community structure of networks. In this paper, a new community detection algorithm based on cellular learning automata (CLA), in which a number of learning automata (LA) cooperate with each other, is proposed. The proposed algorithm taking advantage of irregular CLA finds a partial spanning tree and then forms the local communities on the found the partial spanning tree at each step in order to reduce the network size. As the proposed algorithm proceeds, LA are interacted with both local and global environments to modify the found communities that gradually yielded the near-optimal community structure of the network through the evolution of the CLA. To evaluate the efficiency of the proposed algorithm, several experiments are conducted on synthetic and real networks. Experimental results confirm the superiority and effectiveness of the proposed CLA-based algorithm in terms of various evaluation measures comprising Conductance, Modularity, Normalized Mutual Information, Purity and Rand-index.

Learning automata-based algorithms for solving the stochastic shortest path routing problems in 5G wireless communication

Stochastic Shortest Path Routing (SSPR) problem has always been a critical topic in communication area, especially after a boom in the 5G wireless communication emerges. In this paper, we consider the SSPR problem in a network, where the context is a dynamic, and simultaneously the probability distribution of the constantly changing edges is unknown beforehand. A hierarchical structurized learning automata applied to SSPR, namely SSPR-hieraStructure LRI, is proposed to solve the issue, providing a novel way to pick out the shortest path from source to destination in a stochastic graph. Two kinds of LA prototypes, i.e, variable action learning automata and hierarchical system of learning automata are assembled to exert their respective strengths, based on which we further makes adequate use of a hierarchical structure for modeling. The innovation lies in a convergence by layer rather than as a whole, thus determining the shortest path is converted to searching an optimal node in each layer. Moreover, the searching space is largely reduced by structural pruning, for the purpose of speeding up the algorithm progress. The experimental results involving various aspects demonstrate the effectiveness and efficiency of the proposed algorithm, which outperforms the state of arts in LA based SSPR problems, with a higher accuracy rate, a faster convergence speed, a lower sampling consumption but a higher sampling efficiency.

Finding the Shortest Path in Stochastic Graphs Using Learning Automata and Adaptive Stochastic Petri Nets

Shortest path problem in stochastic graphs has been recently studied in the literature and a number of algorithms has been provided to find it using varieties of learning automata models. However, all these algorithms suffer from two common drawbacks: low speed and lack of a clear termination condition. In this paper, we propose a novel learning automata-based algorithm for this problem which can speed up the process of finding the shortest path using parallelism. For this parallelism, several traverses are initiated, in parallel, from the source node towards the destination node in the graph. During each traverse, required times for traversing from the source node up to any visited node are estimated. The time estimation at each visited node is then given to the learning automaton residing in that node. Using different time estimations provided by different traverses, this learning automaton gradually learns which neighbor of the node is on the shortest path. To set a condition for the termination of the proposed algorithm, we analyze the algorithm using a recently introduced model, Adaptive Stochastic Petri Net (ASPN-LA). The results of this analysis enable us to establish a necessary condition for the termination of the algorithm. To evaluate the performance of the proposed algorithm in comparison to the existing algorithms, we apply it to find the shortest path in six different stochastic graphs. The results of this evaluation indicate that the time required for the proposed algorithm to find the shortest path in all graphs is substantially shorter than that required by similar existing algorithms.

Irregular cellular learning automata-based algorithm for sampling social networks

Since online social networks usually have quite huge size and limited access, smaller subgraphs of them are often produced and analysed as the representative samples of original graphs. Sampling algorithms proposed so far are categorized into three main classes: node sampling, edge sampling, and topology-based sampling. Classic node sampling algorithm, despite its simplicity, performs surprisingly well in many situations. But the problem with node sampling is that the connectivity in sampled subgraph is less likely to be preserved. This paper proposes a topology–based node sampling algorithm using irregular cellular learning automata (ICLA), called ICLA-NS. In this algorithm, at first an initial sample subgraph of the input graph is generated using the node sampling method and then an ICLA isomorphic to the input graph is utilized to improve the sample in such a way that the connectivity of the sample is ensured and at the same time the high degree nodes are also included in the sample. Experimental results on real–world social networks indicate that the proposed sampling algorithm ICLA-NS preserves more accurately the underlying properties of the original graph compared to existing sampling methods in terms of Kolmogorov-Smirnov (KS) test.

Distributed learning automata-based algorithm for community detection in complex networks

Community structure is an important and universal topological property of many complex networks such as social and information networks. The detection of communities of a network is a significant technique for understanding the structure and function of networks. In this paper, we propose an algorithm based on distributed learning automata for community detection (DLACD) in complex networks. In the proposed algorithm, each vertex of network is equipped with a learning automation. According to the cooperation among network of learning automata and updating action probabilities of each automaton, the algorithm interactively tries to identify high-density local communities. The performance of the proposed algorithm is investigated through a number of simulations on popular synthetic and real networks. Experimental results in comparison with popular community detection algorithms such as walk trap, Danon greedy optimization, Fuzzy community detection, Multi-resolution community detection and label propagation demonstrated the superiority of DLACD in terms of modularity, NMI, performance, min-max-cut and coverage.

BNC-VLA: bayesian network structure learning using a team of variable-action set learning automata

Bayesian Network (BN) is a probabilistic graphical model which describes the joint probability distribution over a set of random variables. One of the most important challenges in the field of BNs is to find an optimal network structure based on an available training dataset. Since the problem of searching the optimal BN structure belongs to the class of NP-hard problems, typically greedy algorithms are used to solve it. In this paper a learning automata-based algorithm has been proposed to solve the BNs structure learning problem. There is a learning automaton corresponding with each random variable and at each stage of the proposed algorithm, named BNC-VLA, a set of learning automata is randomly activated and determined the graph edges that must be appeared in that stage. Finally, the constructed network is evaluated using a scoring function. As BNC-VLA algorithm proceeds, the learning process focuses on the BN structure with higher scores. The convergence of this algorithm is theoretically proved; and also some experiments are designed to evaluate the performance of it. Experimental results show that BNC-VLA is capable of finding the optimal structure of BN in an acceptable execution time; and comparing against other search-based methods, it outperforms them.

Bayesian network structure training based on a game of learning automata

Bayesian network (BN) is a probabilistic graphical model which describes the joint probability distribution over a set of random variables. Finding an optimal network structure based on an available training dataset is one of the most important challenges in the field of BNs. Since the problem of searching the optimal BN structure belongs to the class of NP-hard problems, typically greedy algorithms are used to solve it. In this paper two novel learning automata-based algorithms are proposed to solve the BNs’ structure learning problem. In both, there is a learning automaton corresponding with each possible edge to determine the appearance and the direction of that edge in the constructed network; therefore, we have a game of learning automata, at each stage of the proposed algorithms. Two special cases of the game of the learning automata have been discussed, namely, the game with a common payoff and the competitive game. In the former, all the automata in the game receive a unique payoff from the environment, but in the latter, each automaton receives its own payoff. As the algorithms proceed, the learning processes focus on the BN structures with higher scores. The use of learning automata has led to design the algorithms with a guided search scheme, which can avoid getting stuck in local maxima. Experimental results show that the proposed algorithms are capable of finding the optimal structure of BN in an acceptable execution time; and compared with other search-based methods, they outperform them.