Complex Learning Algorithms Research Articles

Idle Vehicle Detection and Traffic Symbol Analysis using Artificial Intelligence and IoT

The emerging development in smart vehicles has improved transportation seamlessly. This paper deals with incorporating the modern technologies like Artificial Intelligence and Internet of Things(IoT) with the traditional transportation system. It states how AI algorithms can be integrated with IoT powered vehicles to manage and avoid crashing with idle vehicles on a speedway and assists for safer driving using the traffic symbol analysis to reduce accidents. Several communication standards for data transmission between the vehicle and control system has been stated. It also provides additional features like emergency or assistance for vehicles in speedway. It uses complex learning and analysis algorithms to improve the efficiency of driving and reduce rear-end collisions. This paper hence gives a comprehensive survey of the enabling technologies, protocols, and architecture for IoT powered transport.

Learning to Predict Ischemic Stroke Growth on Acute CT Perfusion Data by Interpolating Low-Dimensional Shape Representations.

Cerebrovascular diseases, in particular ischemic stroke, are one of the leading global causes of death in developed countries. Perfusion CT and/or MRI are ideal imaging modalities for characterizing affected ischemic tissue in the hyper-acute phase. If infarct growth over time could be predicted accurately from functional acute imaging protocols together with advanced machine-learning based image analysis, the expected benefits of treatment options could be better weighted against potential risks. The quality of the outcome prediction by convolutional neural networks (CNNs) is so far limited, which indicates that even highly complex deep learning algorithms are not fully capable of directly learning physiological principles of tissue salvation through weak supervision due to a lack of data (e.g., follow-up segmentation). In this work, we address these current shortcomings by explicitly taking into account clinical expert knowledge in the form of segmentations of the core and its surrounding penumbra in acute CT perfusion images (CTP), that are trained to be represented in a low-dimensional non-linear shape space. Employing a multi-scale CNN (U-Net) together with a convolutional auto-encoder, we predict lesion tissue probabilities for new patients. The predictions are physiologically constrained to a shape embedding that encodes a continuous progression between the core and penumbra extents. The comparison to a simple interpolation in the original voxel space and an unconstrained CNN shows that the use of such a shape space can be advantageous to predict time-dependent growth of stroke lesions on acute perfusion data, yielding a Dice score overlap of 0.46 for predictions from expert segmentations of core and penumbra. Our interpolation method models monotone infarct growth robustly on a linear time scale to automatically predict clinically plausible tissue outcomes that may serve as a basis for more clinical measures such as the expected lesion volume increase and can support the decision making on treatment options and triage.

An Algorithm Implementing the Method of the Nearest Neighbor in a Multi-Layer Perceptron

It is known that the implementation technology of recognition problems, based on the classic neural network, has a number of difficulties such as the need to have a large training set; the duration and complexity of learning algorithms; difficulty with the choice of such network design parameters as the number of neurons, layers, links, as well as ways to connect neurons; there may be no successful learning, with the need to re-change the network settings and re-training. In this paper we consider the possibility of creating a multi-layer perceptron with a full system of connections and with a threshold activation function on the basis of algorithms metric methods of recognition and in particular the nearest neighbor algorithm. It is shown that this method allows you to create a fully connected multilayer perceptron, such parameters of which as the number of neurons, layers, as well as the value of the weights and thresholds, are determined analytically. The distribution of weight and threshold values for the second and third layer is also discussed. On this basis, we have proposed an algorithm for calculating the thresholds and weights of a multilayer perceptron and showed an example of its implementation. The possible applications of the network for different tasks are considered.

Non-invasive progressive optimization for in-memory databases

Progressive optimization introduces robustness for database workloads against wrong estimates, skewed data, correlated attributes, or outdated statistics. Previous work focuses on cardinality estimates and rely on expensive counting methods as well as complex learning algorithms. In this paper, we utilize performance counters to drive progressive optimization during query execution. The main advantages are that performance counters introduce virtually no costs on modern CPUs and their usage enables a non-invasive monitoring. We present fine-grained cost models to detect differences between estimates and actual costs which enables us to kick-start reoptimization. Based on our cost models, we implement an optimization approach that estimates the individual selectivities of a multi-selection query efficiently. Furthermore, we are able to learn properties like sortedness, skew, or correlation during run-time. In our evaluation we show, that the overhead of our approach is negligible, while performance improvements are convincing. Using progressive optimization, we improve runtime up to a factor of three compared to average run-times and up to a factor of 4,5 compared to worst case run-times. As a result, we avoid costly operator execution orders and; thus, making query execution highly robust.

Maximum power point tracking using a variable antecedent fuzzy logic controller

Maximum power point tracking (MPPT) techniques are popularly used for maximizing the output of solar panels by continuously tracking the maximum power point of their P-V curves, which depend both on the panel temperature and the input insolation. Various MPPT algorithms have already been studied in literature, including perturb and observe (P&O), hill climbing, incremental conductance and neural networks. In particular, fuzzy logic control (FLC) is an another popular technique which achieves a significantly improved performance in MPPT in terms of response speed and no oscillations about the maximum power point (MPP). Unfortunately, a major issue that arises in classical FLC based MPPT algorithms is the lack of versatility to rapidly changing environmental conditions such as the applied irradiance. This paper presents an alternative design of an adaptive MPPT fuzzy logic controller which utilizes simple formulae instead of complex learning algorithms to adjust the antecedents. To verify the proposed MPPT system, a customized off the shelf solar panel is connected to a SEPIC converter and the overall system is both simulated on Simulink and experimentally verified. The resulting response is shown to be fast and stable in comparison to previous designs which used fixed fuzzy logic antecedents that need to be manually modified whenever the environmental conditions change or if a different solar panel is used.

Wrapper Feature Selection based on Genetic Algorithm for Recognizing Objects from Satellite Imagery

Object recognition is a research area that aims to associate objects to categories or classes. The recognition of object specific geospatial features, such as roads, buildings and rivers, from high-resolution satellite imagery is a time consuming and expensive problem in the maintenance cycle of a Geographic Information System (GIS). Feature selection is the task of selecting a small subset from original features that can achieve maximum classification accuracy and reduce data dimensionality. This subset of features has some very important benefits like, it reduces computational complexity of learning algorithms, saves time, improve accuracy and the selected features can be insightful for the people involved in problem domain. This makes feature selection as an indispensable task in classification task. In this work, the authors propose a new approach that combines Genetic Algorithms (GA) with Correlation Ranking Filter (CRF) wrapper to eliminate unimportant features and obtain better features set that can show better results with various classifiers such as Neural Networks (NN), K-nearest neighbor (KNN), and Decision trees. The approach is based on GA as an optimization algorithm to search the space of all possible subsets related to object geospatial features set for the purpose of recognition. GA is wrapped with three different classifier algorithms namely neural network, k-nearest neighbor and decision tree J48 as subset evaluating mechanism. The GA-ANN, GA-KNN and GA-J48 methods are implemented using the WEKA software on dataset that contains 38 extracted features from satellite images using ENVI software. The proposed wrapper approach incorporated the Correlation Ranking Filter (CRF) for spatial features to remove unimportant features. Results suggest that GA based neural classifiers and using CRF for spatial features are robust and effective in finding optimal subsets of features from large data sets.

Label Transfer by Measuring Compactness

This paper presents a new automatic image annotation algorithm. First, we introduce a new similarity measure between images: compactness. This uses low level visual descriptors for determining the similarity between two images. Compactness shows how close test image features lie to training image feature cluster centers. The measure provides the core for a k-nearest neighbor type image annotation method. Afterward, a formalism for defining different transfer techniques is devised and several label transfer techniques are provided. The method as whole is evaluated on four image annotation benchmarks. The results on these sets validate the accuracy of the approach, which outperforms many state-of-the-art annotation methods. The method presented here requires a simple training process, efficiently combines different feature types and performs better than complex learning algorithms, even in this incipient form. The main contributions of this paper are the usage of compactness as a similarity measure that enables efficient low level feature comparison and an annotation algorithm based on label transfer.

Learning in brain and machine—complexity, Gödel, Aristotle

COMBINATORIAL COMPLEXITY OF LEARNING Brains learn much better than computers, this has been discussed in a number of reviews on artificial intelligence, pattern recognitions, and neural networks (Perlovsky, 2001, 2006a). But why? Is there a fundamental reason behind computers being slow learners? Often slow learning is discussed in terms of computational complexity (Perlovsky, 1998), which is usually measured by the number of operations. Scientists have thought that faster computers would be able to catch up with the brain. Still, this has not happened despite computers becoming increasingly faster. Reviews (Perlovsky, 2001, 2006a; Perlovsky et al., 2011) have explained why: computational complexity of learning algorithms grows as a combinatorial (exponential) function of the complexity of a problem to be learned. This means that a learning algorithm might look like it’s quite capable of learning, and indeed, it learns solutions to simple problems. However, slightly more complex problems require not just slightly more computations, but require significantly more. So much more, in fact that learning problems of average complexity require more learning examples and more computer operations than all of the interactions of all elementary particles in the entire life of the Universe (In this article such complexity is called “practically infinite.”). The reason for combinatorial complexity can be explained as follows: consider first, an example of a simple problem requiring no combinatorial complexity for learning: recognition of a single isolated object, which always appears exactly the same. Learning consists in storing in memory the object’s image. Recognition consists in matching the stored image to a newly presented image: match or no match. The complexity of this algorithm approximately equals the number of pixels in an image. But in a real situation the object is not always exactly same; the algorithm has to account for variations in viewing angles, distance, color, etc. In addition, other objects are present with their variabilities. Combinations of various objects with their variabilities lead to combinatorial complexity. Combinations of all pixels in the field of view should be considered. A human eye senses ∼10,000 pixels 10 times a second. Today, sensors measure millions of pixels each second (or more). The number of combinations of these pixels is “practically infinite”; combinations of 100 pixels (a relatively simple problem) are 100100; this number is close to all of the interactions of all elementary particles in the entire life of the Universe.

Multi-selection of instances: A straightforward way to improve evolutionary instance selection

Although many more complex learning algorithms exist, k-nearest neighbor is still one of the most successful classifiers in real-world applications. One of the ways of scaling up the k-nearest neighbors classifier to deal with large datasets is instance selection. Due to the constantly growing amount of data in almost any pattern recognition task, we need more efficient instance selection algorithms, which must achieve larger reductions while maintaining the accuracy of the selected subset.In this paper we present a way to improve instance selection by allowing the algorithms to select instances more than once. In this way, fewer instances can cover more portions of the space, and the same testing accuracy of the standard approach can be obtained faster and with fewer instances. Although the approach is general enough to be used in any instance selection algorithm, we focus on evolutionary instance selection due to its superior performance.An extensive comparison using 45 datasets from the UCI Machine Learning Repository shows the usefulness of our approach compared with the established method of evolutionary instance selection. Our method is able to, in the worst case, match the accuracy obtained by standard instance selection with a larger reduction and shorter execution time. In addition, the method is applied to class-imbalance problems with very good results.

Genetic Algorithm Based RNN Structure for Rayleigh Fading MIMO Channel Estimation

Multi-Input Multi-Output (MIMO) wireless communication system is an emerging area which offers substantial advantages for achieving high data rate with cost effective design. The spectral efficiency and reliability of MIMO systems greatly depends on the assumption that the transmitter and/or the receiver have perfect knowledge of channel state information (CSI). Hence, it is required to predict the CSI at the receiver and send the updated CSI back to the transmitter. In this paper particularly the Frequency-Flat Rayleigh i.i.d MIMO channel is considered, where as the Rayleigh fading channel provides complex and random coefficients. These complex channel parameters are estimated by using split complex real-time recurrent learning (SCRTRL) and fully complex real-time recurrent learning (FCRTRL) algorithm based recurrent neural network (RNN) structure with complex weights. The RNN with SCRTRL and RNN with FCRTRL algorithm produced a premature convergence which is overcome by the proposed genetic algorithm (GA) based learning process.

Coordination failure as a source of congestion in information networks

Coordination failure, or agents' uncertainty about the action of other agents, may be an important source of congestion in large decentralized systems. The El Farol problem provides a simple paradigm for congestion and coordination problems that may arise with over utilization of the Internet. This paper reviews the El Farol problem and surveys previous approaches, which typically involve complex deterministic learning algorithms that exhibit chaotic-like trajectories. This paper recasts the problem in a stochastic framework and derives a simple adaptive strategy that has intriguing optimization properties; a large collection of decentralized decision makers, each acting in their own best interests and with limited knowledge, converge to a solution that (optimally) solves a complex congestion and social coordination problem. A variation in which agents are allowed access to full information is not nearly as successful. The algorithm, which can be viewed as a kind of habit formation, is analyzed using a weak convergence approach, and simulations illustrate the major results.

Integration of Soft Computing Approaches for Feature Subset Selection

Feature subset selection basically depends on the design of a criterion function to measure the effectiveness of a particular feature or a feature subset and the selection of a search strategy to find out he best feature subset. Lots of techniques, mostly statistical, have been developed so far which are mainly categorized into classifier independent filter approaches and classifier dependant wrapper approaches. Wrapper approaches produce good results but are computationally unattractive specially when nonlinear neural classifiers with complex learning algorithms are used The present work proposes some hybrid algorithms for feature subset selection using individual tools from soft computing paradigm taking advantage of both the filter and wrapper approaches. Artificial neural network, fuzzy logic and genetic algorithm are used to design neuro fuzzy and fuzzy genetic algorithms. A fuzzy set theoretic measure for assessing the goodness of a feature is used in conjunction with a multilayer perceptron (MLP) or a fractal neural network (FNN), the proposed modification of MLP having a statistically fractal sparse architecture. Though the process does not guarantee absolute optimality, the selected feature subset produces near optimal results for practical purposes. The process is less time consuming and computationally light compared to any neural network classifier based sequential feature subset selection technique. The same measure in conjunction with genetic algorithm has been used and it is found that fuzzy genetic algorithm is better than neuro fuzzy algorithms for large feature set problems for finding out a near optimal solution. The proposed algorithms have been simulated with two different data sets to show their effectiveness.

A Multistrategy Approach to Relational Knowledge Discovery inDatabases

When learning from very large databases, the reduction of complexity is extremely important. Two extremes of making knowledge discovery in databases (KDD) feasible have been put forward. One extreme is to choose a very simple hypothesis language, thereby being capable of very fast learning on real-world databases. The opposite extreme is to select a small data set, thereby being able to learn very expressive (first-order logic) hypotheses. A multistrategy approach allows one to include most of these advantages and exclude most of the disadvantages. Simpler learning algorithms detect hierarchies which are used to structure the hypothesis space for a more complex learning algorithm. The better structured the hypothesis space is, the better learning can prune away uninteresting or losing hypotheses and the faster it becomes. We have combined inductive logic programming (ILP) directly with a relational database management system. The ILP algorithm is controlled in a model-driven way by the user and in a data-driven way by structures that are induced by three simple learning algorithms.

Operations for Learning with Graphical Models

This paper is a multidisciplinary review of empirical, statistical learning from a graphical model perspective. Well-known examples of graphical models include Bayesian networks, directed graphs representing a Markov chain, and undirected networks representing a Markov field. These graphical models are extended to model data analysis and empirical learning using the notation of plates. Graphical operations for simplifying and manipulating a problem are provided including decomposition, differentiation, andthe manipulation of probability models from the exponential family. Two standard algorithm schemas for learning are reviewed in a graphical framework: Gibbs sampling and the expectation maximizationalgorithm. Using these operations and schemas, some popular algorithms can be synthesized from their graphical specification. This includes versions of linear regression, techniques for feed-forward networks, and learning Gaussian and discrete Bayesian networks from data. The paper concludes by sketching some implications for data analysis and summarizing how some popular algorithms fall within the framework presented. The main original contributions here are the decompositiontechniques and the demonstration that graphical models provide a framework for understanding and developing complex learning algorithms.

Computational complexity, learning rules and storage capacities: A Monte Carlo study for the binary perceptron

We examine the storage capacity for the binary perceptron using simulated annealing. In particular, we clarify the connection between the computational complexity of learning algorithms and the attained storage capacity. From finite-size studies we obtain a critical storage capacity,α c (κ)=0.8331±0.0016, in good agreement with the replica analysis of Krauth and Mezard. However, we demonstrate that a polynomial time cooling schedule yields a vanishing storage capacity in the thermodynamic limit as predicted by the dynamical theory of Horner. Nonetheless, we show these two results may be reconciled by explicitly verifying that the learning problem for the binary perceptron is NP-complete. This investigation has been made possible by the development of an accelerated annealing algorithm.

Digital neural network system based in new concepts on the recall phase dynamics

In this paper we present an electronic system designed to emulate neural networks. Two major restrictions are assumed: discrete synapses (+1,0,−1) and threshold-type neurons. Drawbacks given by restrictions are solved with a more complex learning algorithm that maps real valued configurations for synapses into bipolar ones. The central unit of the system is an ASIC designed in accordance with a new sequential dynamics, which is faster and with better recall characteristics. As a result, the system designed has a fast recall phase and a large number of neurons. A handwritten OCR system is being designed.

Adaptive arrays using a complex learning algorithm

The demand for the mobile communications is increasing every year, and TV and automobile telephone by satellite communication are expected to find wider applications. If individuals are to carry communication equipment, then the equipment must be small size, lightweight and highly efficient. To realize antennas which must also be of small size, lightweight and highly efficient. For such equipment it is desirable to employ the adaptive array with phase control. Conventionally, the LMS algorithm has been applied frequently in the design of the adaptive array. However, the authors believe that the learning algorithm is better suited to the adaptive array because of its higher convergence speed and the clear specification of the convergence condition. This paper extends the real-type learning algorithm to the complex-type learning algorithm. The convergence condition for the algorithm is derived, and the result of the application of the method to the adaptive array is shown. A method of deriving the reference wave also is presented which is less affected by the input signal variation and is suited to the adaptive array with only the phase control. Using the proposed complex learning algorithm and the new reference wave, the phase-controlled adaptive array is designed. A satisfactory result is obtained by simulation.

Cognitive systems based on adaptive algorithms

The type of cognitive system (CS) studied here has four basic parts: (1) a set of interacting elementary productions, called classifiers , (2) a performance algorithm that directs the action of the system in the environment, (3) a simple learning algorithm that keeps a record of each classifier's success in bringing about rewards, and (4) a more complex learning algorithm, called the genetic algorithm , that modifies the set of classifiers so that variants of good classifiers persist and new, potentially better ones are created in a provably efficient manner.