Fast Learning Method Research Articles

A fast learning method for large scale and multi-class samples of SVM

A multi-class classification SVM(Support Vector Machine) fast learning method based on binary tree is presented to solve its low learning efficiency when SVM processing large scale multi-class samples. This paper adopts bottom-up method to set up binary tree hierarchy structure, according to achieved hierarchy structure, sub-classifier learns from corresponding samples of each node. During the learning, several class clusters are generated after the first clustering of the training samples. Firstly, central points are extracted from those class clusters which just have one type of samples. For those which have two types of samples, cluster numbers of their positive and negative samples are set respectively according to their mixture degree, secondary clustering undertaken afterwards, after which, central points are extracted from achieved sub-class clusters. By learning from the reduced samples formed by the integration of extracted central points above, sub-classifiers are obtained. Simulation experiment shows that, this fast learning method, which is based on multi-level clustering, can guarantee higher classification accuracy, greatly reduce sample numbers and effectively improve learning efficiency.

Fast Inverse-Free Sparse Bayesian Learning via Relaxed Evidence Lower Bound Maximization

Sparse Beyesian learning is a popular approach for sparse signal recovery, and has demonstrated superior performance in a series of experiments. Nevertheless, the sparse Bayesian learning algorithm involves a matrix inverse at each iteration. Its associated computational complexity grows significantly with the problem size, which hinders its application to many practical problems even with moderately large datasets. To address this issue, in this letter, we develop a fast inverse-free sparse Bayesian learning method. Specifically, by invoking a fundamental property for smooth functions, we obtain a relaxed evidence lower bound (relaxed-ELBO) that is computationally more amiable than the conventional ELBO used by sparse Bayesian learning. A variational expectation-maximization (EM) scheme is then employed to maximize the relaxed-ELBO, which leads to a computationally efficient inverse-free sparse Bayesian learning algorithm. Simulation results show that the proposed algorithm has a fast convergence rate and achieves lower reconstruction errors than other state-of-the-art fast sparse recovery methods in the presence of noise.

Fast unsupervised learning for visual pattern recognition using spike timing dependent plasticity

Real-time learning needs algorithms operating in a fast speed comparable to human or animal, however this is a huge challenge in processing visual inputs. Research shows a biological brain can process complicated real-life recognition scenarios at milliseconds scale. Inspired by biological system, in this paper, we proposed a novel real-time learning method by combing the spike timing-based feed-forward spiking neural network (SNN) and the fast unsupervised spike timing dependent plasticity learning method with dynamic post-synaptic thresholds. Fast cross-validated experiments using MNIST database showed the high efficiency of the proposed method at an acceptable accuracy.

A fast learning method for data-driven design of interval type-2 fuzzy logic system

A fast learning method for data-driven design of interval type-2 fuzzy logic system

Fast binary support vector machine learning method by samples reduction

Fast binary support vector machine learning method by samples reduction

Fast binary support vector machine learning method by samples reduction

Support vector machine is a well-known method of statistical learning by its good accuracy; however, its training time is very poor especially in case of huge databases. Many research works aim to reduce training samples to improve training time without significant loss in accuracy. In this paper, we propose a method called CB-SR, based on filtering and revision stages to eliminate samples that have little influence on learning results. Filtering stage uses a covering-based principle of samples to eliminate those faraway from decision boundaries and keep the closest ones. Revision stage allows to add after the first learning, samples eventually discarded by mistake. The results we obtain show the benefits of our approach over others existing ones.

Fast learning method for convolutional neural networks using extreme learning machine and its application to lane detection

Deep learning has received significant attention recently as a promising solution to many problems in the area of artificial intelligence. Among several deep learning architectures, convolutional neural networks (CNNs) demonstrate superior performance when compared to other machine learning methods in the applications of object detection and recognition. We use a CNN for image enhancement and the detection of driving lanes on motorways. In general, the process of lane detection consists of edge extraction and line detection. A CNN can be used to enhance the input images before lane detection by excluding noise and obstacles that are irrelevant to the edge detection result. However, training conventional CNNs requires considerable computation and a big dataset. Therefore, we suggest a new learning algorithm for CNNs using an extreme learning machine (ELM). The ELM is a fast learning method used to calculate network weights between output and hidden layers in a single iteration and thus, can dramatically reduce learning time while producing accurate results with minimal training data. A conventional ELM can be applied to networks with a single hidden layer; as such, we propose a stacked ELM architecture in the CNN framework. Further, we modify the backpropagation algorithm to find the targets of hidden layers and effectively learn network weights while maintaining performance. Experimental results confirm that the proposed method is effective in reducing learning time and improving performance.

Fast multilayer perceptron neural network‐based control algorithm for shunt compensator in distribution systems

In this study, a fast learning method of back-propagation (BP) multilayer perceptron neural network-based control algorithm for shunt compensator in three-phase distribution systems is presented. The proposed method comprises of quadratic linear and non-linear errors to determine optimisation criterion error function to train the BP algorithm while the existing methods have used only linear quadratic error term. The newly developed optimisation criterion error function accelerates the convergence efficiency of BP algorithm for performance improvement of shunt compensator at point of common coupling under non-linear loading conditions. With the help of the proposed algorithm, the weighted amplitude of fundamental active and reactive current components of the load current are extracted from which the reference source currents are estimated. The performance analysis of the proposed algorithm has been evaluated using two case studies for zero-voltage regulation and power factor correction. The total harmonic distortions are improved in comparison with standard BP algorithm which has been validated in above-mentioned two case studies. This is the quite important advantage of the proposed control algorithm to improve the power quality over existing control algorithms for shunt compensator.

A fast learning method for streaming and randomly ordered multi-class data chunks by using one-pass-throw-away class-wise learning concept

Presently, the amount of data occurring in several business and academic areas such as ATM transactions, web searches, and sensor data are tremendously and continuously increased. Classifying as well as recognizing patterns among these data in a limited memory space complexity are very challenging. Various incremental learning methods have proposed to achieve highly accurate results but both already learned data and new incoming data must be retained throughout the learning process, causing high space and time complexities. In this paper, a new neural learning method based on radial-shaped function and discard-after-learn concept in the data streaming environment was proposed to reduce the space and time complexities. The experimental results showed that the proposed method used 1 to 95 times fewer neurons and 1.2 to 2,700 times faster than the results produced by MLP, RBF, SVM, VEBF, ILVQ, ASC, and other incremental learning methods. It is also robust to the incoming order of data chunks.

Toward Interactive Relational Learning

This paper introduces the Interactive Relational Machine Learning (iRML) paradigm in which users interactively design relational models by specifying the various components, constraints, and relational data representation, as well as perform evaluation, analyze errors, and make adjustments and refinements in a closed-loop. iRML requires fast real-time learning and inference methods capable of interactive rates. Methods are investigated that enable direct manipulation of the various components of the RML method. Visual representation and interaction techniques are also developed for exploring the space of relational models and the trade-offs of the various components and design choices.

Design of Intangible Interface for Mouseless Computer Handling using Hand Gestures

In this paper, an approach for developing an intangible interface with the help of computer vision technology for human computer interaction using static hand gestures has been discussed. Mouse operations like left click, right click and double click are performed by hand gestures. An image of the human hand is read from the webcam and the hand is segmented to obtain a binary image. The contour of the binary hand image is extracted to draw the convex hull around it as an envelope. The static hand gestures developed make use of feature extraction method. The developed system is compatible with a diverse group of users. It is a fast learning method and does not require extensive training of data facilitating low computational speed and cost. The main aim is to improve the interaction of human with the computer i.e. to make it more natural but keeping the cost factor in mind.

Fast Multiclass Dictionaries Learning With Geometrical Directions in MRI Reconstruction

Improve the reconstructed image with fast and multiclass dictionaries learning when magnetic resonance imaging is accelerated by undersampling the k-space data. A fast orthogonal dictionary learning method is introduced into magnetic resonance image reconstruction to provide adaptive sparse representation of images. To enhance the sparsity, image is divided into classified patches according to the same geometrical direction and dictionary is trained within each class. A new sparse reconstruction model with the multiclass dictionaries is proposed and solved using a fast alternating direction method of multipliers. Experiments on phantom and brain imaging data with acceleration factor up to 10 and various undersampling patterns are conducted. The proposed method is compared with state-of-the-art magnetic resonance image reconstruction methods. Artifacts are better suppressed and image edges are better preserved than the compared methods. Besides, the computation of the proposed approach is much faster than the typical K-SVD dictionary learning method in magnetic resonance image reconstruction. The proposed method can be exploited in undersampled magnetic resonance imaging to reduce data acquisition time and reconstruct images with better image quality.

A novel algorithm with differential evolution and coral reef optimization for extreme learning machine training.

Extreme learning machine (ELM) is a novel and fast learning method to train single layer feed-forward networks. However due to the demand for larger number of hidden neurons, the prediction speed of ELM is not fast enough. An evolutionary based ELM with differential evolution (DE) has been proposed to reduce the prediction time of original ELM. But it may still get stuck at local optima. In this paper, a novel algorithm hybridizing DE and metaheuristic coral reef optimization (CRO), which is called differential evolution coral reef optimization (DECRO), is proposed to balance the explorative power and exploitive power to reach better performance. The thought and the implement of DECRO algorithm are discussed in this article with detail. DE, CRO and DECRO are applied to ELM training respectively. Experimental results show that DECRO-ELM can reduce the prediction time of original ELM, and obtain better performance for training ELM than both DE and CRO.

Sequential active learning using meta-cognitive extreme learning machine

This paper proposes a fast and effective sequential active learning method using meta-cognitive extreme learning machine (SEAL-ELM). The proposed algorithm consists of two components, namely the cognitive component and the meta-cognitive component. The cognitive component is an online sequential extreme learning machine while the meta-cognitive component controls the learning process of the cognitive component using a self-regulating mechanism to decide what to learn, when to learn and how to learn the arriving samples. Active learning is employed to select different strategies, namely sample deletion, sample reserve and sample learning strategy to determine whether the data will be deleted directly, reserved for later use or used immediately. This is the first time the similarity of meta-cognitive machine learning and active learning is exploited. The meta-cognition mechanism and active learning principle are utilized to reduce the labeling efforts and costs. The use of ELM greatly reduces the computation complexity of the learning process. The new algorithm is evaluated on a set of real-world benchmark classification problems. Simulation results demonstrate the usefulness and effectiveness of sequential active learning and show that the SEAL-ELM can achieve similar or better learning accuracy with a much faster learning speed compared with the state-of-the-arts algorithms.

A fast evaluation method for RTS game strategy using fuzzy extreme learning machine

This paper proposes a fast learning method for fuzzy measure determination named fuzzy extreme learning machine (FELM). Moreover, we apply it to a special application domain, which is known as unit combination strategy evaluation in real time strategy (RTS) game. The contribution of this paper includes three aspects. First, we describe feature interaction among different unit types by fuzzy theory. Second, we develop a new set selection algorithm to represent the complex relation between input and hidden layers in extreme learning machine, in order to enable it to learn different fuzzy integrals. Finally, based on the set selection algorithm, we propose the FELM model for feature interaction description, which has an extremely fast learning speed. Experimental results on artificial benchmarks and real RTS game data show the feasibility and effectiveness of the proposed method in both accuracy and efficiency.

A Fast Learning Method for Multilayer Perceptrons in Automatic Speech Recognition Systems

We propose a fast learning method for multilayer perceptrons (MLPs) on large vocabulary continuous speech recognition (LVCSR) tasks. A preadjusting strategy based on separation of training data and dynamic learning-rate with a cosine function is used to increase the accuracy of a stochastic initial MLP. Weight matrices of the preadjusted MLP are restructured by a method based on singular value decomposition (SVD), reducing the dimensionality of the MLP. A back propagation (BP) algorithm that fits the unfolded weight matrices is used to train the restructured MLP, reducing the time complexity of the learning process. Experimental results indicate that on LVCSR tasks, in comparison with the conventional learning method, this fast learning method can achieve a speedup of around 2.0 times with improvement on both the cross entropy loss and the frame accuracy. Moreover, it can achieve a speedup of approximately 3.5 times with only a little loss of the cross entropy loss and the frame accuracy. Since this method consumes less time and space than the conventional method, it is more suitable for robots which have limitations on hardware.

Extreme Learning Machine Assisted Adaptive Control of a Quadrotor Helicopter

Control of quadrotor helicopters is difficult because the problem is naturally nonlinear. The problem becomes more challenging for common model based controllers when unpredictable uncertainties and disturbances in physical control system are taken into account. This paper proposes a novel intelligent controller design based on a fast online learning method called extreme learning machine (ELM). Our neural controller does not require precise system modeling or prior knowledge of disturbances and well approximates the dynamics of the quadrotor at a fast speed. The proposed method also incorporates a sliding mode controller for further elimination of external disturbances. Simulation results demonstrate that the proposed controller can reliably stabilize a quadrotor helicopter in both agitated attitude and position control tasks.

Network traffic classification via non-convex multi-task feature learning

Machine learning has been used in network traffic classification and statistical features are used to represent flows. However, conventional feature selection may work out in face of dynamic and complex traffic data. Multi-Task Learning has obtained quite wide attention nowadays, and one important form of multi-task learning is to exploit the features shared by tasks by sparse models. We propose a fast multi-task sparse feature learning method, using a non-convex Capped-ℓ1,ℓ1 as the regularizer to learn a set of shared features in traffic data. Specifically, the non-convex multi-task feature learning model can learn features belonging to each task as well as the common features shared among tasks. We use the iterative shrinkage and thresholding (IST) algorithm to solve the problem, which has a closed-form solution for one of the crucial steps in the whole iteration. Experiment on real traffic data captured from backbone network as well as synthetic data and other popular real-world data show the effectiveness the method, compared with state-of-the-art methods.

MTC: A Fast and Robust Graph-Based Transductive Learning Method.

Despite the great success of graph-based transductive learning methods, most of them have serious problems in scalability and robustness. In this paper, we propose an efficient and robust graph-based transductive classification method, called minimum tree cut (MTC), which is suitable for large-scale data. Motivated from the sparse representation of graph, we approximate a graph by a spanning tree. Exploiting the simple structure, we develop a linear-time algorithm to label the tree such that the cut size of the tree is minimized. This significantly improves graph-based methods, which typically have a polynomial time complexity. Moreover, we theoretically and empirically show that the performance of MTC is robust to the graph construction, overcoming another big problem of traditional graph-based methods. Extensive experiments on public data sets and applications on web-spam detection and interactive image segmentation demonstrate our method's advantages in aspect of accuracy, speed, and robustness.

A fast learning method for feedforward neural networks

In order to circumvent the weakness of very slow convergence of most traditional learning algorithms for single layer feedforward neural networks, the extreme learning machines (ELM) has been recently developed to achieve extremely fast learning with good performance by training only for the output weights. However, it cannot be applied to multiple-hidden layer feedforward neural networks (MLFN), which is a challenging bottleneck of ELM. In this work, the novel fast learning method (FLM) for feedforward neural networks is proposed. Firstly, based on the existing ridge regression theories, the hidden-feature-space ridge regression (HFSR) and centered ridge regression Centered-ELM are presented. Their connection with ELM is also theoretically revealed. As special kernel methods, they can inherently be used to propagate the prominent advantages of ELM into MLFN. Then, a novel fast learning method FLM for feedforward neural networks is proposed as a unified framework for HFSR and Centered-ELM. FLM can be applied for both SLFN and MLFN with a single or multiple outputs. In FLM, only the parameters in the last hidden layer require being adjusted while all the parameters in other hidden layers can be randomly assigned. The proposed FLM was tested against state of the art methods on real-world datasets and it provides better and more reliable results.