Hybrid Learning Method Research Articles

Learning lateral interactions for feature binding and sensory segmentation from prototypic basis interactions.

We present a hybrid learning method bridging the fields of recurrent neural networks, unsupervised Hebbian learning, vector quantization, and supervised learning to implement a sophisticated image and feature segmentation architecture. This architecture is based on the competitive layer model (CLM), a dynamic feature binding model, which is applicable on a wide range of perceptual grouping and segmentation problems. A predefined target segmentation can be achieved as attractor states of this linear threshold recurrent network, if the lateral weights are chosen by Hebbian learning. The weight matrix is given by the correlation matrix of special pattern vectors with a structure dependent on the target labeling. Generalization is achieved by applying vector quantization on pair-wise feature relations, like proximity and similarity, defined by external knowledge. We show the successful application of the method to a number of artifical test examples and a medical image segmentation problem of fluorescence microscope cell images.

A neuro-fuzzy method for predicting the chaotic time series

A neuro-fuzzy approach based on a novel hybrid learning method is presented, which can generate the best fuzzy rule set automatically from the desired input-output data pairs only and can give the initial neuro-fuzzy system and the initial parameters of fuzzy membership functions. Then the parameters of fuzzy membership functions and the weights can be easily tuned by employing neural network's self-learning techniques. This approach reduces the rule matching time and accelerates the speed of the fuzzy logic referencing and improves the adaptability of the neuro-fuzzy system. Using the proposed neuro-fuzzy system and the learning algorithms we simulated the prediction of the Lorenz chaotic time series, the results demonstrate the effectiveness of the chaotic time series prediction approach.

Symbolic adaptive neuro-fuzzy inference for data mining of heterogenous data

The application of neuro-fuzzy systems to domains involving prediction and classification of symbolic data requires a reconsideration and a careful definition of the concept of distance between patterns. Traditional distances are inadequate to provide information about the proximity between the symbolic patterns. This work proposes a new architecture of neurofuzzy systems, the Symbolic Adaptive Neuro Fuzzy Inference System (SANFIS) that utilizes effectively a statistically extracted distance measure. The learning approach is a hybrid one and consists of a sequence of steps some of which are essential and some are used in order to optimize further the performance. Initially, a Statistical Distance Metric space is computed from the information provided with the training set. The premise parameters are subsequently evaluated with a three-phase Instance Based Learning (IBL) scheme that estimates the input membership function centers and spreads and constructs the corresponding fuzzy rules. The first phase of this scheme explores heuristic approaches that can uncover information for the relative importance and the reliability of the examples. The second phase exploits this information and extracts an adequate subset of the training patterns for the construction of the fuzzy rules. The concept of fuzzy adaptive subsethood is used at the third phase, for the reduction of the number of the fuzzy sets used as input membership functions. The consequent parameters are estimated with an efficient linear least squares formulation. The obtained performances from the SANFIS trained with the hybrid learning methods are significantly better than the traditional nearest neighbour Instance Based Learning schemes and compares well with advanced neural designs. At the same time SANFIS provides an enhanced explanation ability with the construction of a few interpretable rules.

Immunity-based hybrid learning methods for approximator structure and parameter adjustment

From the point of view of information processing the immune system is a highly parallel and distributed intelligent system which has learning, memory, and associative retrieval capabilities. In this paper we present two immunity-based hybrid learning approaches for function approximation (or regression) problems that involve adjusting the structure and parameters of spatially localized models (e.g., radial basis function networks). The number and centers of the receptive fields for local models are specified by immunity-based structure adaptation algorithms, while the parameters of the local models, which enter in a linear fashion, are tuned separately using a least-squares method. The effectiveness of the procedure is demonstrated through a nonlinear function approximation problem and a nonlinear dynamical system modeling problem.

Color image compression using PCA and backpropagation learning

The RGB components of a color image contain redundant information that can be reduced using a new hybrid neural-network model based upon Sanger's algorithm for representing an image in terms of principal components and a backpropagation algorithm for restoring the original representation. The PCA method produces a black and white image with the same number of pixels as the original color image, but with each pixel represented by a scalar value instead of a three-dimensional vector of RGB components. Experimental results show that as our hybrid learning method adapts to local (spatial) image characteristics it outperforms the YIQ and YUV standard compression methods. Our experiments also show that it is feasible to apply training results from one image to previously unseen images.

Analysis of input-output clustering for determining centers of RBFN

The key point in design of radial basis function networks is to specify the number and the locations of the centers. Several heuristic hybrid learning methods, which apply a clustering algorithm for locating the centers and subsequently a linear leastsquares method for the linear weights, have been previously suggested. These hybrid methods can be put into two groups, which will be called as input clustering (IC) and input-output clustering (IOC), depending on whether the output vector is also involved in the clustering process. The idea of concatenating the output vector to the input vector in the clustering process has independently been proposed by several papers in the literature although none of them presented a theoretical analysis on such procedures, but rather demonstrated their effectiveness in several applications. The main contribution of this paper is to present an approach for investigating the relationship between clustering process on input-output training samples and the mean squared output error in the context of a radial basis function netowork (RBFN). We may summarize our investigations in that matter as follows: 1) A weighted mean squared input-output quantization error, which is to be minimized by IOC, yields an upper bound to the mean squared output error. 2) This upper bound and consequently the output error can be made arbitrarily small (zero in the limit case) by decreasing the quantization error which can be accomplished through increasing the number of hidden units.

Hybrid Learning for Tool Wear Monitoring

In automated manufacturing systems such as flexible manufacturing systems (FMSs), one of the most important issues is the detection of tool wear during the cutting process. This paper presents a hybrid learning method to map the relationship between the features of cutting vibration and the tool wear condition. The experimental results show that it can be used effectively to monitor the tool wear in drilling. First, a neural network model with fuzzy logic (FNN), responding to learning algorithms, is presented. It has many advantageous features, compared to a backpropagation neural network, such as less computation. Secondly, the experimental results show that the frequency distribution of vibration changes as the tool wears, so the r.m.s. of the different frequency bands measured indicates the tool wear condition. Finally, FNN is used to describe the relationship between the characteristics of vibration and the tool wear condition. The experimental results demonstrate the feasibility of using vibration signals to monitor the drill wear condition.

Learning Bayesian networks with a hybrid convergent method

During the past few years, a variety of methods have been developed for learning probabilistic networks from data, among which the heuristic single link forward or backward searches are widely adopted to reduce the search space. A major drawback of these search heuristics is that they can not guarantee to converge to the right networks even if a sufficiently large data set is available. This motivates us to explore an algorithm that will not suffer from this problem. We first identify an asymptotic property of different score metrics, based on which we then present a hybrid learning method that can be proved to be asymptotically convergent. We show that the algorithm, when employing the information criterion and the Bayesian metric, guarantees to converge in a very general way and is computationally feasible. Evaluation of the algorithm with simulated data is given to demonstrate the capability of the algorithm.

Knowledge-based artificial neural networks

Hybrid learning methods use theoretical knowledge of a domain and a set of classified examples to develop a method for accurately classifying examples not seen during training. The challenge of hybrid learning systems is to use the information provided by one source of information to offset information missing from the other source. By so doing, a hybrid learning system should learn more effectively than systems that use only one of the information sources. KBANN ( Knowledge-Based Artificial Neural Networks) is a hybrid learning system built on top of connectionist learning techniques. It maps problem-specific “domain theories”, represented in propositional logic, into neural networks and then refines this reformulated knowledge using backpropagation. KBANN is evaluated by extensive empirical tests on two problems from molecular biology. Among other results, these tests show that the networks created by KBANN generalize better than a wide variety of learning systems, as well as several techniques proposed by biologists.

Hybrid learning algorithm for Gaussian potential function networks

A new hybrid learning algorithm is proposed for use in the parametric estimation of Gaussian potential function networks (GPFNs). In the new algorithm, the number of network inputs is augmented by using target output values in the learning centres of Gaussian nodes in the network's hidden layer. This augmentation of input leads to a more reasonable distribution of centres in the hidden layer of a GPFN. A critical angle technique is then used to determine those nodes in which the shape factors will need further tuning by optimisation techniques. Two numerical examples are supplied to show the superior performance of this new algorithm as compared to that achieved through a traditional hybrid learning method, or to the optimised-only method of Lee and Kil. The capability of the GPFN as a dynamical model for continually tracking dynamics of non-stationary and time-varying systems is also illustrated.

Fast Learning in Networks of Locally-Tuned Processing Units

We propose a network architecture which uses a single internal layer of locally-tuned processing units to learn both classification tasks and real-valued function approximations (Moody and Darken 1988). We consider training such networks in a completely supervised manner, but abandon this approach in favor of a more computationally efficient hybrid learning method which combines self-organized and supervised learning. Our networks learn faster than backpropagation for two reasons: the local representations ensure that only a few units respond to any given input, thus reducing computational overhead, and the hybrid learning rules are linear rather than nonlinear, thus leading to faster convergence. Unlike many existing methods for data analysis, our network architecture and learning rules are truly adaptive and are thus appropriate for real-time use.