Learning Automata Research Articles

Ripplet-Transform-based Cycle Spinning Denoising and Fuzzy-CLA Segmentation of Retinal Images for Accurate Hard Exudates and Lesion Detection

Aims: Digital retinal images are commonly used for hard exudates and lesion detection. These images are rarely noiseless and therefore before any further processing they should be underwent noise removal. Background: An efficient segmentation method is then needed to detect and discern the lesions from the retinal area. Objective: In this paper, a hybrid method is presented for digital retinal image processing for diagnosis and screening purposes. The aim of this study is to present a supervised/semi-supervised approach for exudate detection in fundus images and also to analyze the method to find the optimum structure. Methods: Ripplet transform and cycle spinning method is first used to remove the noises and artifacts. Results: The noises may be normal or any other commonly occurring forms such as salt and pepper. The image is transformed into fuzzy domain after it is denoised. Conclusion: A cellular learning automata model is used to detect any abnormality on the image which is related to a lesion. The automaton is created with an extra term as the rule updating term to improve the adaptability and efficiency of the cellular automata.Three main statistical criteria are introduced as the sensitivity, specificity and accuracy. A number of 50 retinal images with visually detection hard exudates and lesions are the experimental dataset for evaluation and validation of the method.

A multi-population differential evolution algorithm based on cellular learning automata and evolutionary context information for optimization in dynamic environments

This paper presents a multi-population differential evolution algorithm to address dynamic optimization problems. In the proposed approach, a cellular learning automaton adjusts the behavior of each subpopulation by adaptively controlling its updating schemes. As the environment changes over time, an evolving population may go through a quite number of state transitions. Each state demands specific characteristics from an optimizer; hence, an adapted evolutionary scheme for one state may be unsuitable for the upcoming ones. Additionally, a learning approach may have limited time to adapt to a newly encountered state due to the frequentness of the environmental changes. Hence, it is infeasible for a dynamic optimizer to unlearn its existing beliefs to accommodate the practices required to embrace newly encountered states. In order to address this issue, we introduce a context dependent learning approach, which can adapt the behavior of each subpopulation according to the contexts of its different states. The performance of the proposed approach is compared with several state-of-the-art dynamic optimizers over the GDBG benchmark set. Comparison results indicate that the proposed method can achieve statistically superior performances on a wide range of tested instances.

Improved differential evolution for noisy optimization

Abstract A novel approach is proposed in this paper to improve the optimization proficiency of the differential evolution (DE) algorithm in the presence of stochastic noise in the objective surface by utilizing the composite benefit of four strategies. The first strategy is devised with an aim to employ reinforcement learning scheme of stochastic learning automata for autonomous selection of the sample size of a trial solution (for its repeated fitness evaluation) based on the characteristics of the fitness landscape in its local neighborhood. The second stratagem is proposed to estimate the effective fitness measure from multiple fitness samples of a trial solution, resulting from sampling. The novelty of the second policy lies in considering the distribution of noisy samples during effective fitness evaluation, instead of their direct averaging. The third strategy deals with amelioration of the DE/current-to-best/1 mutation scheme to judiciously direct the search in promising region, even in prevailing existence of noise in the objective surface. Finally, the greedy selection policy of the traditional DE is modified by introducing the principle of probabilistic crowding induced niching to ensure both the population quality and the population diversity. Comparative analysis performed on simulation results for diverse noisy benchmark functions reveal the statistically significant superiority of the proposed algorithm to its contenders with respect to function error value.

An iterative stochastic algorithm based on distributed learning automata for finding the stochastic shortest path in stochastic graphs

In this paper, we study the problem of finding the shortest path in stochastic graphs and propose an iterative algorithm for solving it. This algorithm is based on distributed learning automata (DLA), and its objective is to use a DLA for finding the shortest path from the given source node to the given destination node whose weight is minimal in expected sense. At each stage of this algorithm, DLA specifies edges needed to be sampled. We show that the given algorithm finds the shortest path with minimum expected weight in stochastic graphs with high probability which can be close to unity as much as possible. We compare the given algorithm with some distributed learning automata-based iterative algorithms, a particle swarm optimization-based algorithm, an ant colony-based algorithm, a Q-learning-based algorithm, and an actor–critic-based algorithm for finding the shortest path. Computer experiments show that the proposed algorithm requires fewer edge samples to find the shortest path than the previously introduced DLA-based algorithms.

Learning Moore machines from input–output traces

The problem of learning automata from example traces (but no equivalence or membership queries) is fundamental in automata learning theory and practice. In this paper, we study this problem for finite-state machines with inputs and outputs, and in particular for Moore machines. We develop three algorithms for solving this problem: (1) the PTAP algorithm, which transforms a set of input–output traces into an incomplete Moore machine and then completes the machine with self-loops; (2) the PRPNI algorithm, which uses the well-known RPNI algorithm for automata learning to learn a product of automata encoding a Moore machine; and (3) the MooreMI algorithm, which directly learns a Moore machine using PTAP extended with state merging. We prove that MooreMI has the fundamental identification in the limit property. We compare the algorithms experimentally in terms of the size of the learned machine and several notions of accuracy, introduced in this paper. We also carry out a performance comparison against two existing tools (LearnLib and flexfringe). Finally, we compare with OSTIA, an algorithm that learns a more general class of transducers and find that OSTIA generally does not learn a Moore machine, even when fed with a characteristic sample.

Research on Fuzzy Enhanced Learning Model of Multienhanced Signal Learning Automata

With the continuous iteration of information technology, information presents an explosive development trend, so how to effectively classify, retrieve, and summarize information is very important. Data classification and mining technology as the key technology to solve the aforementioned problems is of great significance. At present, in the fields of data processing, information classification and machine learning reinforcement learning, it is usually based on the learning automata, but the traditional learning automata algorithm cannot meet the development of information technology in terms of convergence and learning rate. In order to improve the convergence stability and learning efficiency of the learning automata algorithm in random environment and enhance its learning ability in dealing with large-scale space and action, this paper innovatively proposes an adaptive fuzzy reinforcement learning model of the learning automata, which makes full use of the randomness of gradient information to solve its stability problem, and abandons the original reinforcement learning method. For the linear function, an adaptive fuzzy numerical superposition algorithm is used to further enhance its learning rate and improve its stability. The experimental results show that the proposed learning automata fuzzy reinforcement learning model has practical application value, and can solve the problems of unstable convergence and poor learning efficiency of the current learning automata.

Stochastic Stability of Perturbed Learning Automata in Positive-Utility Games

This paper considers a class of reinforcement-based learning (namely, perturbed learning automata ) and provides a stochastic-stability analysis in repeatedly played, positive-utility, finite strategic-form games. Prior work in this class of learning dynamics primarily analyzes asymptotic convergence through stochastic approximations, where convergence can be associated with the limit points of an ordinary-differential equation (ODE). However, analyzing global convergence through an ODE-approximation requires the existence of a Lyapunov or a potential function, which naturally restricts the analysis to a fine class of games. To overcome these limitations, this paper introduces an alternative framework for analyzing asymptotic convergence that is based upon an explicit characterization of the invariant probability measure of the induced Markov chain. We further provide a methodology for computing the invariant probability measure in positive-utility games, together with an illustration in the context of coordination games.

Distributed Learning Automata Approach for Workflow Mining: Discovering Process Model Using Condensate Drops Method

An information system is a process of collecting, processing, storing and distributing information, which leads to efficient decision-making and control in organizations. Examples of information systems include classical management systems, systems for workflow management, systems for case handling and middleware. Information systems collect information concerning important people, locations and other important matters in an organization and store relevant events in some form of structure. Based on event logs, from information systems, the discovery of process models can be made automatically by process mining techniques, without having an a priori model. By learning from the event logs, process mining aims to discover, monitor and improve processes. This paper proposes a method to discover a process model based on distributed learning automata and the condensate approach. In this proposed method, each event in the log is called a drop, which had its condensate and can be combined with other condensates. Each drop is connected to other drops and become a larger drop. All of those drops would obtain reward if it represents sequence of an event log. The evaluation results demonstrated that the proposed method could detect various patterns in the event log and discover a more efficient process model in terms of fitness, total node and total path of the mined process model.

An adaptive multi-objective algorithm based on decomposition and large neighborhood search for a green machine scheduling problem

Abstract Green machine scheduling consists in the allocation of jobs in order to maximize production, in view of the sustainable use of energy. This work addresses the unrelated parallel machine scheduling problem with setup times, with the minimization of the makespan and the total energy consumption. The latter takes into account the power consumption of each machine in different operation modes. We propose multi-objective extensions of the Adaptive Large Neighborhood Search (ALNS) metaheuristic with Learning Automata (LA) to improve the search process and to solve the large scale instances efficiently. ALNS combines ad-hoc destroy and repair (also named removal and insertion) operators and a local search procedure. The LA is used to adapt the selection of insertion and removal operators within the framework of ALNS. Two new algorithms are developed: the MO-ALNS and the MO-ALNS/D. The first algorithm is a direct extension of single objective ALNS by using multi-objective local search. As this method does not offer much control of the diversification of the Pareto front approximation, a second strategy employs the decomposition approach similar to MOEA/D algorithm. The results show that the MO-ALNS/D algorithm has better performance than MO-ALNS and MOEA/D in all indicators. These findings show that the decomposition strategy is beneficial not only for evolutionary algorithms, but it is indeed an efficient way to extend ALNS to multi-objective problems.

Massive Random Access in Automobile to Automobile Communications using Honey Bee Algorithm

Automobile-to-Automobile correspondence (M2M) is an inside piece in the Internet of Things (IoT) vision. In context on the enormous number of devices expected, the Whole deal Advancement Advanced (LTA-A) structures may present stop up and over-burden issues. The colossal passionate access in M2M trades will cause radio access arrange discourage in the base station (BS), actuating sharp debilitating in access deferral and access probability. Access class in any case (ACB) that can direct control the advancement of Automobile-type correspondence (MTC) devices by an ACB factor is an earth shattering course of action to keep the BS from traffic over-bother. In remote cell manages, the enthusiastic access resources (i.e., preludes) are shared by M2M and human-to-human (H2H) contraptions, and research on ACB plot generally perceive that a predestined number of partners are named with M2M traffic. In any case, when encountering colossal access in M2M correspondences, it is connecting rapidly satisfy the path requests from MTC contraptions using each open presentation, especially in time-sensitive IoT conditions. In this paper, we study the gigantic access issue in M2M traffic entered conditions where M2M and H2H traffic can apply for each available prelude without segment. Utilizing the self-adaptable learning property of learning automata, we further propose dynamic Honey bee estimation to adjust the close to rate parameter. Duplication results exhibit that the Honey bee Computation achieves the presentation close to hypothetical optimality. The BS equipped with the Honey bee Count can effectively control the M2M traffic by capability adjusting the ACB factor under the counteractive action of H2H traffic and give quality relationship to both M2M and H2H traffic.

Visual saliency object detection using sparse learning

In many applications in order to recognise the relationship between user and computer, the position at which the user looks should be detected. To this end, a salient object should be extracted that is attracted to the attention of the viewer. In this study, a new method is proposed to extract the object saliency map, which is based on learning automata and sparse algorithms. In the proposed method, after decomposition of an image to its superpixels, eight features (namely three features in red–green–blue colour space, coalition, central bias, rotation feature, brightness, and colour difference) are extracted. Then the extracted features are normalised to zero mean and unit variance. In this study, K-means singular-value decomposition is used to integrate the extracted features. The performance of the proposed method is compared with that of 20 other methods by applying four new databases, including MSRA-100, ECSSD, MSRA-10K, and Pascal-S. The obtained results show that the proposed method has a better performance compared to the other methods with regard to the prediction of the salient object.

Distributed Learning Automata-based S-learning scheme for classification

This paper proposes a novel classifier based on the theory of Learning Automata (LA), reckoned to as PolyLA. The essence of our scheme is to search for a separator in the feature space by imposing an LA-based random walk in a grid system. To each node in the grid, we attach an LA whose actions are the choices of the edges forming a separator. The walk is self-enclosing, and a new random walk is started whenever the walker returns to the starting node forming a closed classification path yielding a many-edged polygon. In our approach, the different LA attached to the different nodes search for a polygon that best encircles and separates each class. Based on the obtained polygons, we perform classification by labeling items encircled by a polygon as part of a class using a ray casting function. From a methodological perspective, PolyLA has appealing properties compared to SVM. In fact, unlike PolyLA, the SVM performance is dependent on the right choice of the kernel function (e.g., linear kernel, Gaussian kernel)—which is considered a “black art.” PolyLA, on the other hand, can find arbitrarily complex separator in the feature space. We provide sound theoretical results that prove the optimality of the scheme. Furthermore, experimental results show that our scheme is able to perfectly separate both simple and complex patterns outperforming existing classifiers, such as polynomial and linear SVM, without the need to map the problem to many dimensions or to introduce a “kernel trick.” We believe that the results are impressive, given the simplicity of PolyLA compared to other approaches such as SVM.

An online-learning-based evolutionary many-objective algorithm

When optimizing many-objective problems (MaOP), the same strategy might behave differently when facing problems with different features. Therefore, obtaining problem features helps to obtain high-quality solutions. However, in practice, the problem features are unknown during the optimization process. In this case, learning to adjust strategies to match the problem features is a challenging work. In this paper, a learning-based algorithm is proposed, aimed to enhance the generalization ability. On the basis of a decomposition-based many-objective optimization framework, a learning automaton (LA) is included in the algorithm. The LA adjusts the evolutionary strategies of the algorithm to adapt to the problem characteristics, according to the feedback information during the optimizing procedure. An external archive is employed to store the Pareto non-dominant solutions. Based on the external archive, a reference vector adjustment strategy is designed to enhance the capability of solving problems with a degenerate or discrete Pareto front (PF). To validate the performance of the proposed algorithm, a comparison experiment is conducted on a novel authority test suite. Five state-of-the-art algorithms are selected as peer algorithms. The results of the experiment indicate that the proposed algorithm obtains satisfactory performance in determining the convergence and the approximation of the PF.

Grey wolf optimizer-based learning automata for solving block matching problem

Block matching problem is of great importance, and it is the basic element of many computer vision systems such as video compression, object tracking, motion analysis, and traffic control. This paper proposes a novel grey wolf optimizer (GWO) algorithm based on learning automata (LA) to solve block matching problem for motion estimation. Two main contributions are presented in this paper. Firstly, for improving the exploration and exploitation abilities of the GWO technique, an enhanced GWO method based on LA algorithm is proposed. LA is integrated in the GWO to learn the objective function and decide whether it is an unimodal or multimodal function. Unimodal function needs a good exploitation of promising area in the search space. However, multimodal function requires high exploration ability. The classification obtained using LA is then used to create new solutions in the appropriate areas. In the creation phase, two equations are used. The first one is based on a Gaussian distribution, to enrich the exploitation for the unimodal function, and the second is based on a random distribution to support the exploration in multimodal function. The second contribution of this paper consists of applying our enhanced GWO algorithm in block matching problem. The proposed algorithm is validated on two phases. Firstly, we evaluate our enhanced GWO algorithm on eight well-known benchmark functions. The reported results show that the enhanced GWO algorithm has the potential to improve the optimization abilities of the conventional GWOs. Then, the proposed enhanced GWO algorithm-based block matching is tested on six video sequences and compared with several state-of-the-art methods. Simulation results show the effectiveness of the proposed BM algorithm and prove the applicability of our enhanced GWO to real-world optimization problem.

Control of Black-Box Embedded Systems by Integrating Automaton Learning and Supervisory Control Theory of Discrete-Event Systems

The paper presents an approach to the control of black-box embedded systems by integrating automaton learning and supervisory control theory (SCT) of discrete-event systems (DES), where automaton models of both the system and requirements are unavailable or hard to obtain. First, the system is tested against the requirements. If all the requirements are satisfied, no supervisor is needed and the process terminates. Otherwise, a supervisor is synthesized to enforce the system to satisfy the requirements. To apply SCT and automaton learning technologies efficiently, the system is abstracted to be a finite-discrete model. Then, a $C^{*}$ learning algorithm is proposed based on the classical $L^{*}$ algorithm to infer a Moore automaton describing both the behavior of the system and the conjunctive behavior of the system and the requirements. Subsequently, a supervisor for the system is derived from the learned Moore automaton and patched on the system. Finally, the controlled system is tested again to check the correctness of the supervisor. If the requirements are still not satisfied, a larger Moore automaton is learned and a refined supervisor is synthesized. The whole process iterates until the requirements hold in the controlled system. The effectiveness of the proposed approach is manifested through two realistic case studies. Note to Practitioners —Supervisory control theory of DES can synthesize maximally permissive supervisory controllers to ensure the correctness of software-controlled processes. The application of supervisory control theory relies on automaton models of the plant and specifications; however, the required models are often unavailable and difficult to obtain for black-box embedded systems. Automaton learning is an effective method for inferring models of black-box systems. This paper integrates the two technologies so that the supervisory control theory is applicable to the development of black-box embedded software systems. The proposed approach is implemented in a toolchain that connects automaton learning algorithms, SCT, and testing algorithms via scripts. The obtained supervisor is implemented as a software patch to monitor and control the original system online.

CCCLA: A cognitive approach for congestion control in Internet of Things using a game of learning automata

Internet of Things (IoT) typically consists of lossy and low powered networks (LLN) of interconnected sensors. Due to low bandwidth and high scale of communication, congestion can occur among the sensor nodes in the LLN, during communicating to a border router, or when some other clients from the Internet access the resources in the LLN. So, having a proper congestion control mechanism is very important for IoT. In this paper we want to cope with congestion in IoT; however the current IoT, which is still based on traditional static architectures, lacks intelligence and cannot comply with the increasing application performance requirements. Adding cognition in IoT empowers it with a brain and high level intelligence. Therefore, firstly a learning automata-based cognitive framework has been proposed for integrating cognition into IoT. Then, based on the framework, we have presented a new cognitive approach for congestion control, named CCCLA (Cognitive Congestion Control in IoT using a game of Learning Automata). In the proposed approach, a team of LA has been assigned to a group of effective controllable parameters; for example parameters, whose values can affect the congestion control. Each automaton has a finite set of possible values of its corresponding parameter, and it tries to learn the best one, which maximize the whole network performance. Each node in the network has its own group of learning automata, which act independently; however, all nodes receive the same feedbacks from the environment. Using simulation, we test the proposed cognitive framework in a congestion control scenario. Based on our findings CCCLA significantly avoids congestion while improves desired QoS parameters such as delay, reliability and throughput, even in highly lossy networks.

Multivariable System Identification Method Based on Continuous Action Reinforcement Learning Automata

In this work, a closed-loop identification method based on a reinforcement learning algorithm is proposed for multiple-input multiple-output (MIMO) systems. This method could be an attractive alternative solution to the problem that the current frequency-domain identification algorithms are usually dependent on the attenuation factor. With this method, after continuously interacting with the environment, the optimal attenuation factor can be identified by the continuous action reinforcement learning automata (CARLA), and then the corresponding parameters could be estimated in the end. Moreover, the proposed method could be applied to time-varying systems online due to its online learning ability. The simulation results suggest that the presented approach can meet the requirement of identification accuracy in both square and non-square systems.

Detecting Sybil nodes in stationary wireless sensor networks using learning automaton and client puzzles

A well-known harmful attack against wireless sensor networks (WSNs) is the Sybil attack. In a Sybil attack, WSN is destabilised by a malicious node which forges a large number of fake identities to disrupt network protocols such as routing, data aggregation, and fair resource allocation. In this study, the authors suggest a new algorithm based on a composition of learning automaton (LA) model and client puzzles theory to identify Sybil nodes in stationary WSNs. In the proposed algorithm, each node sends puzzles to its neighbours periodically during the network lifetime and tries to identify Sybil nodes among them, considering their response time (puzzle solving time). In this algorithm, each node equipped with a LA to reduce the communication and computation overhead of sending and solving puzzles. The proposed algorithm has been simulated using J-SIM simulator and simulation results have shown that the proposed algorithm can detect 100% of Sybil nodes and the false detection rate is about 5% on average. Also, the performance of the proposed algorithm has been compared to a wellknown neighbour-based algorithm through experiments and the results have shown that the proposed algorithm is significantly better than this algorithm in terms of detection and false detection rates.

Permissive Supervisor Synthesis for Markov Decision Processes Through Learning

This paper considers the permissive supervisor synthesis for probabilistic systems modeled as Markov Decision Processes (MDP). Such systems are prevalent in power grids, transportation networks, communication networks, and robotics. We propose a novel supervisor synthesis framework using automata learning and compositional model checking to generate the permissive local supervisors in a distributed manner. With the recent advances in assume-guarantee reasoning verification for MDPs, constructing the composed system can be avoided to alleviate the state space explosion. Our framework learns the supervisors iteratively using counterexamples from the verification and is guaranteed to terminate in finite steps and to be correct.

Learning Automata-Based Access Class Barring Scheme for Massive Random Access in Machine-to-Machine Communications

The machine-to-machine (M2M) communications, which achieve the implementation of Internet of Things (IoT), can be carried over wireless cellular networks. The massive random access (RA) in M2M communications will cause radio access network congestion in the base station (BS), leading to sharp deterioration in access delay and access probability. Access class barring (ACB) that can directly control the flow of machine-type communication (MTC) devices by an ACB factor is an efficient scheme to prevent the BS from traffic overload. In wireless cellular networks, the RA resources (i.e., preambles) are shared by M2M and human-to-human (H2H) devices, and research on ACB scheme ordinarily assumes that a restricted number of preambles are assigned to M2M traffic. However, when suffering from massive access in M2M communications, it is desirable to rapidly satisfy the access requests from MTC devices using all available preambles, especially in time-sensitive IoT scenarios. In this paper, we study the massive access problem in M2M traffic centered scenarios where M2M and H2H traffic can apply for all available preambles without distinction. Utilizing the self-adaptive learning property of learning automata, we further propose a novel learning automata-based ACB (LA-ACB) scheme. Simulation results show that the LA-ACB scheme achieves the performance close to theoretical optimality. The BS equipped with the LA-ACB scheme can effectively control the M2M traffic by dynamically adjusting the ACB factor under the interference of H2H traffic and provide quality services for both M2M and H2H traffic.