Local Learning Method Research Articles

Local value difference metric

Value difference metric (VDM) is one of the widely used distance functions designed to work with nominal attributes. Research has indicated that the definition of VDM follows naturally from a simple probabilistic model called a naive Bayes (NB). NB assumes that all the attributes are independent given the class. To further improve the performance of NB, several techniques have been proposed. Among these, an effective technique is local learning. Because VDM has a close relationship with NB, in this paper, we propose a local learning method for VDM. The improved distance function is called local value difference metric (LVDM). When LVDM computes the distance between a test instance and each training instance, the conditional probabilities in VDM are estimated by counting from the neighborhood of the test instance only instead of from all the training data. A modified decision tree algorithm is proposed to determine the neighborhood of the test instance. The experimental results on 43 datasets downloaded from the University of California at Irvine (UCI) show that the proposed LVDM significantly outperforms VDM in terms of the class-probability estimation performance of distance-based learning algorithms.

Rapid Oscillation Fault Detection and Isolation for Distributed Systems via Deterministic Learning

In this paper, a rapid detection and isolation scheme for oscillation faults in a distributed nonlinear system is proposed. The distributed nonlinear system considered is modeled as a set of interconnected subsystems. First, a local learning and merging method based on deterministic learning theory is proposed to obtain knowledge of the unknown interconnections and the fault functions. Second, using learned knowledge, a bank of consensus-based dynamical estimators are constructed for each subsystem, and average L <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> norms of the residuals are generated to make the detection and isolation decisions. Third, a rigorous analysis for characterizing the detection and isolation capabilities of the proposed scheme is given. The attraction of the intelligence fault diagnosis approach is to give a fast response to faults using the learned knowledge and processing huge data in a dynamical and distributed manner. Simulation studies are included to demonstrate the effectiveness of the approach.

Estimation of acute oral toxicity in rat using local lazy learning.

BackgroundAcute toxicity means the ability of a substance to cause adverse effects within a short period following dosing or exposure, which is usually the first step in the toxicological investigations of unknown substances. The median lethal dose, LD50, is frequently used as a general indicator of a substance’s acute toxicity, and there is a high demand on developing non-animal-based prediction of LD50. Unfortunately, it is difficult to accurately predict compound LD50 using a single QSAR model, because the acute toxicity may involve complex mechanisms and multiple biochemical processes.ResultsIn this study, we reported the use of local lazy learning (LLL) methods, which could capture subtle local structure-toxicity relationships around each query compound, to develop LD50 prediction models: (a) local lazy regression (LLR): a linear regression model built using k neighbors; (b) SA: the arithmetical mean of the activities of k nearest neighbors; (c) SR: the weighted mean of the activities of k nearest neighbors; (d) GP: the projection point of the compound on the line defined by its two nearest neighbors. We defined the applicability domain (AD) to decide to what an extent and under what circumstances the prediction is reliable. In the end, we developed a consensus model based on the predicted values of individual LLL models, yielding correlation coefficients R2 of 0.712 on a test set containing 2,896 compounds.ConclusionEncouraged by the promising results, we expect that our consensus LLL model of LD50 would become a useful tool for predicting acute toxicity. All models developed in this study are available via http://www.dddc.ac.cn/admetus.

Data-Driven Spatially-Adaptive Metric Adjustment for Visual Tracking

Matching visual appearances of the target over consecutive video frames is a fundamental yet challenging task in visual tracking. Its performance largely depends on the distance metric that determines the quality of visual matching. Rather than using fixed and predefined metric, recent attempts of integrating metric learning-based trackers have shown more robust and promising results, as the learned metric can be more discriminative. In general, these global metric adjustment methods are computationally demanding in real-time visual tracking tasks, and they tend to underfit the data when the target exhibits dynamic appearance variation. This paper presents a nonparametric data-driven local metric adjustment method. The proposed method finds a spatially adaptive metric that exhibits different properties at different locations in the feature space, due to the differences of the data distribution in a local neighborhood. It minimizes the deviation of the empirical misclassification probability to obtain the optimal metric such that the asymptotic error as if using an infinite set of training samples can be approximated. Moreover, by taking the data local distribution into consideration, it is spatially adaptive. Integrating this new local metric learning method into target tracking leads to efficient and robust tracking performance. Extensive experiments have demonstrated the superiority and effectiveness of the proposed tracking method in various tracking scenarios.

A Novel Eye Localization Method With Rotation Invariance

This paper presents a novel learning method for precise eye localization, a challenge to be solved in order to improve the performance of face processing algorithms. Few existing approaches can directly detect and localize eyes with arbitrary angels in predicted eye regions, face images, and original portraits at the same time. To preserve rotation invariant property throughout the entire eye localization framework, a codebook of invariant local features is proposed for the representation of eye patterns. A heat map is then generated by integrating a 2-class sparse representation classifier with a pyramid-like detecting and locating strategy to fulfill the task of discriminative classification and precise localization. Furthermore, a series of prior information is adopted to improve the localization precision and accuracy. Experimental results on three different databases show that our method is capable of effectively locating eyes in arbitrary rotation situations (360° in plane).

Fast Face Gender Recognition by Using Local Ternary Pattern and Extreme Learning Machine

Human face gender recognition requires fast image processing with high accuracy. Existing face gender recognition methods used traditional local features and machine learning methods have shortcomings of low accuracy or slow speed. In this paper, a new framework for face gender recognition to reach fast face gender recognition is proposed, which is based on Local Ternary Pattern (LTP) and Extreme Learning Machine (ELM). LTP is a generalization of Local Binary Pattern (LBP) that is in the presence of monotonic illumination variations on a face image, and has high discriminative power for texture classification. It is also more discriminate and less sensitive to noise in uniform regions. On the other hand, ELM is a new learning algorithm for generalizing single hidden layer feed forward networks without tuning parameters. The main advantages of ELM are the less stringent optimization constraints, faster operations, easy implementation, and usually improved generalization performance. The experimental results on public databases show that, in comparisons with existing algorithms, the proposed method has higher precision and better generalization performance at extremely fast learning speed.

Reinforced Similarity Integration in Image-Rich Information Networks

Social multimedia sharing and hosting websites, such as Flickr and Facebook, contain billions of user-submitted images. Popular Internet commerce websites such as Amazon.com are also furnished with tremendous amounts of product-related images. In addition, images in such social networks are also accompanied by annotations, comments, and other information, thus forming heterogeneous image-rich information networks. In this paper, we introduce the concept of (heterogeneous) image-rich information network and the problem of how to perform information retrieval and recommendation in such networks. We propose a fast algorithm heterogeneous minimum order k-SimRank (HMok-SimRank) to compute link-based similarity in weighted heterogeneous information networks. Then, we propose an algorithm Integrated Weighted Similarity Learning (IWSL) to account for both link-based and content-based similarities by considering the network structure and mutually reinforcing link similarity and feature weight learning. Both local and global feature learning methods are designed. Experimental results on Flickr and Amazon data sets show that our approach is significantly better than traditional methods in terms of both relevance and speed. A new product search and recommendation system for e-commerce has been implemented based on our algorithm.

Efficient kernel models for learning and approximate minimization problems

This paper investigates techniques for reducing the computational burden of local learning methods relying on kernel functions in the framework of approximate minimization, i.e., when they are employed to find the minimum of a given cost functional. The considered approach is based on an optimal choice of the kernel width parameters through the minimization of an empirical cost and can provide a solution to important problems, such as function approximation and multistage optimization. However, when the stored data are too many, the kernel model output evaluation can take a long time, making local learning unsuited to contexts where a fast function evaluation is required. At the same time, the training procedure to obtain the kernel widths can become too demanding as well. Here it is shown that a large saving in the computational effort can be achieved by considering subsets of the available data suitably chosen according to different criteria. An analysis of the performance of the new approach is provided. Then, simulation results show in practice the effectiveness of the proposed techniques when applied to learning and approximate minimization problems.

Feedback controlled particle swarm optimization and its application in time-series prediction

Particle swarm optimization (PSO) algorithm is an algorithmic technique for optimization by solving a wide range of optimization problems. This paper presents a new approach of extending PSO to solve optimization problems by using the feedback control mechanism (FCPSO). The proposed FCPSO consists of two major steps. First, by evaluating the fitness value of each particle, a simple particle evolutionary fitness function is designed to control parameters involving acceleration coefficient, refreshing gap, learning probabilities and number of the potential exemplars automatically. By such a simple particle evolutionary fitness function, each particle has its own search parameters in a search environment. Secondly, a local learning method using a competitive penalized method is developed to refine the solution. The FCPSO has been comprehensively evaluated on 18 unimodal, multimodal and composite benchmark functions with or without rotation. Compared with various state-of-the-art algorithms, including traditional PSO algorithms and representative variants of PSO algorithms, the performance of FCPSO is promising. The effects of parameter adaptation, parameter sensitivity and local search method are studied. Lastly, the proposed FCPSO is applied to constructing a radial basis neural network, together with the K-means method for time-series prediction.

HIERARCHICAL DISTANCE METRIC LEARNING FOR LARGE MARGIN NEAREST NEIGHBOR CLASSIFICATION

Distance metric learning is a powerful tool to improve performance in classification, clustering and regression tasks. Many techniques have been proposed for distance metric learning based on convex programming, kernel learning, dimension reduction and large margin. The recently proposed large margin nearest neighbor classification (LMNN) improves the performance of k-nearest neighbors classification (k-nn) by a learned global distance metric. However, it does not consider the locality of data distributions. We demonstrate a novel local distance metric learning method called hierarchical distance metric learning (HDM) which first builds a hierarchical structure by grouping data points according to the overlapping ratios defined by us and then learns distance metrics sequentially. In this paper, we combine HDM with LMNN and further propose a new method named hierarchical distance metric learning for large margin nearest neighbor classification (HLMNN). Experiments are performed on many artificial and real-world data sets. Comparisons with the traditional k-nn and the state-of-the-art LMNN show the effectiveness of the proposed HLMNN.

Single-image super-resolution via local learning

Nearest neighbor-based algorithms are popular in example-based super-resolution from a single image. The core idea behind such algorithms is that similar images are close in the sense of distance measurement. However, it is well known in the field of machine learning and statistical learning theory that the generalization of the nearest neighbor-based estimation is poor, when complex or high dimensional data are considered. To improve the power of the nearest neighbor-based algorithms in single-image based super-resolution, a local learning method is proposed in this paper. Similar to the nearest neighbor-based algorithms, a local training set is generated according to the similarity between the training samples and a given test sample. For super-resolving the given test sample, a local regression function is learned on the local training set. The generalization of nearest neighbor-based algorithms can be enhanced by the process of local regression. Based on such an idea, we propose a novel local-learning-based algorithm, where kernel ridge regression algorithm is used in local regression for its well generalization. Some experimental results verify the effectiveness and efficiency of the local learning algorithm in single-image based super-resolution.

Adaptive local kernel-based learning for soft sensor modeling of nonlinear processes

Soft sensor techniques have been widely used to estimate product quality or other key indices which cannot be measured online by hardware sensors. Unfortunately, their estimation performance would deteriorate under certain circumstances, e.g., the change of the process characteristics, especially for global learning approaches. Meanwhile, local learning methods always only utilize input information to select relevant instances, which may lead to a waste of output information and inaccurate sample selection. To overcome these disadvantages, a new local modeling algorithm, adaptive local kernel-based learning scheme (ALKL) is proposed. First, a new similarity measurement using both input and output information is proposed and utilized in a supervised locality preserving projection technique to select relevant samples. Second, an adaptive weighted least squares support vector regression (AW-LSSVR) is employed to establish a local model and predict output indices for each query data. In AW-LSSVR, instead of using traditional cross-validation methods, the trade-off parameters are adjusted iteratively and the local model is updated recursively, which reduces the computational complexity a lot. The proposed ALKL is applied to an online crude oil endpoint prediction in an industrial fluidized catalytic cracking unit (FCCU) process. The experimental results demonstrate the high precision of our ALKL approach.

Local Search with Probabilistic Modeling for Learning Multiple-Valued Logic Networks

This paper proposes a probabilistic modeling learning algorithm for the local search approach to the Multiple-Valued Logic (MVL) networks. The learning model (PMLS) has two phases: a local search (LS) phase, and a probabilistic modeling (PM) phase. The LS performs searches by updating the parameters of the MVL network. It is equivalent to a gradient decrease of the error measures, and leads to a local minimum of error that represents a good solution to the problem. Once the LS is trapped in local minima, the PM phase attempts to generate a new starting point for LS for further search. It is expected that the further search is guided to a promising area by the probability model. Thus, the proposed algorithm can escape from local minima and further search better results. We test the algorithm on many randomly generated MVL networks. Simulation results show that the proposed algorithm is better than the other improved local search learning methods, such as stochastic dynamic local search (SDLS) and chaotic dynamic local search (CDLS).

Local subspace learning by extended fuzzy c-medoids clustering

Linear fuzzy clustering is a technique for extracting linear-shape clusters, in which the fuzzy c-means (FCM)-like iterative procedure is performed with the prototypes of linear varieties, and is also regarded as a local subspace learning method. In fuzzy c-medoids (FCMdd), cluster prototypes are selected from data samples and clustering criteria are calculated by using only mutual distances among samples. Then, it is applicable to relational data clustering. This paper proposes an extended FCMdd approach for linear fuzzy clustering of relational data, which uses multiple representative objects (medoids) for representing prototypes. In the algorithm, new prototype is given by solving a combinatorial optimisation problem for searching medoids and the computational complexity is reduced by searching only from a subset of objects having large membership values. The information summarisation approach can be regarded as a multicluster-type multidimensional scaling for summarising data in multiple low-dimensional feature spaces.

An Improved Local Search Learning Method for Multiple-Valued Logic Network Minimization with Bi-objectives

SUMMARY This paper describes an improved local search method for synthesizing arbitrary Multiple-Valued Logic (MVL) function. In our approach, the MVL function is mapped from its algebraic presentation (sumof-products form) on a multiple-layered network based on the functional completeness property. The output of the network is evaluated based on two metrics of correctness and optimality. A local search embedded with chaotic dynamics is utilized to train the network in order to minimize the MVL functions. With the characteristics of pseudo-randomness, ergodicity and irregularity, both the search sequence and solution neighbourhood generated by chaotic variables enables the system to avoid local minimum settling and improves the solution quality. Simulation results based on 2variable 4-valued MVL functions and some other large instances also show that the improved local search learning algorithm outperforms the traditional methods in terms of the correctness and the average number of product terms required to realize a given MVL function.

Adaptive Local Linear Regression With Application to Printer Color Management

Local learning methods, such as local linear regression and nearest neighbor classifiers, base estimates on nearby training samples, neighbors. Usually, the number of neighbors used in estimation is fixed to be a global "optimal" value, chosen by cross validation. This paper proposes adapting the number of neighbors used for estimation to the local geometry of the data, without need for cross validation. The term enclosing neighborhood is introduced to describe a set of neighbors whose convex hull contains the test point when possible. It is proven that enclosing neighborhoods yield bounded estimation variance under some assumptions. Three such enclosing neighborhood definitions are presented: natural neighbors, natural neighbors inclusive, and enclosing k-NN. The effectiveness of these neighborhood definitions with local linear regression is tested for estimating lookup tables for color management. Significant improvements in error metrics are shown, indicating that enclosing neighborhoods may be a promising adaptive neighborhood definition for other local learning tasks as well, depending on the density of training samples.

Novel approaches to predict the retention of histidine‐containing peptides in immobilized metal‐affinity chromatography

The new method lazy learning method-local lazy regression (LLR) was first used to model the quantitative structure-retention relationship (QSRR) for predicting and explaining the retention behaviors of peptides in the nickel column in immobilized metal-affinity chromatography (IMAC). The best multilinear regression (BMLR) method implemented in the CODESSA was used to select the most appropriate molecular descriptors from a large set and build a linear regression model. Based on the selected five descriptors, another two approaches, projection pursuit regression (PPR) and LLR were used to build more accurate QSRR models. The coefficients of determination (R(2)) of the best model developed based on LLR were 0.9446 and 0.9252 for the training set and the test set, respectively. By comparison, it was proved that the novel local learning method LLR was a very promising tool for QSRR modeling with excellent predictive capability for the prediction of imidazole concentration (IMC) values of histidine-containing peptides in IMAC. It could be used in other chromatography research fields and that should facilitate the design and purification of peptides and proteins.

Direct Neighborhood Discriminant Analysis for Face Recognition

Face recognition is a challenging problem in computer vision and pattern recognition. Recently, many local geometrical structure‐based techiniques are presented to obtain the low‐dimensional representation of face images with enhanced discriminatory power. However, these methods suffer from the small simple size (SSS) problem or the high computation complexity of high‐dimensional data. To overcome these problems, we propose a novel local manifold structure learning method for face recognition, named direct neighborhood discriminant analysis (DNDA), which separates the nearby samples of interclass and preserves the local within‐class geometry in two steps, respectively. In addition, the PCA preprocessing to reduce dimension to a large extent is not needed in DNDA avoiding loss of discriminative information. Experiments conducted on ORL, Yale, and UMIST face databases show the effectiveness of the proposed method.

Local discriminative learning for pattern recognition

Local discriminative learning methods approximate a target function (a posteriori class probability function) directly by partitioning the feature space into a set of local regions, and appropriately modeling a simple input–output relationship (function) in each one. This paper presents a new method for judiciously partitioning the input feature space in order to accurately represent the target function. The method accomplishes this by approximating not only the target function itself but also its derivatives. As such, the method partitions the input feature space along those dimensions for which the class probability function changes most rapidly, thus minimizing bias. The efficacy of the method is validated using a variety of simulated and real-world data.

A new learning strategy for stereo matching derived from a fuzzy clustering method

This paper presents an approach to the local stereo correspondence problem. The primitives or features used are groups of collinear connected edge points called segments. Each segment has several associated attributes or properties. We have verified that the differences of the attributes for the true matches cluster in a cloud around a center. Then for each current pair of primitives we compute a distance between the difference of its attributes and the cluster center. The correspondence is established in the basis of the minimum distance criterion (similarity constraint). We have designed an image understanding system to learn the best representative cluster center. For such purpose a new learning method is derived from the Fuzzy c-Means (FcM) algorithm where the dispersion of the true samples in the cluster is taken into account through the Mahalanobis distance. This is the main contribution of this paper. A better performance of the proposed local stereo-matching learning method is illustrated with a comparative analysis between classical local methods without learning.