Nonlinear Data Structure Research Articles

Multiple Kernel Learning for Hyperspectral Image Classification: A Review

With the rapid development of spectral imaging techniques, classification of hyperspectral images (HSIs) has attracted great attention in various applications such as land survey and resource monitoring in the field of remote sensing. A key challenge in HSI classification is how to explore effective approaches to fully use the spatial–spectral information provided by the data cube. Multiple kernel learning (MKL) has been successfully applied to HSI classification due to its capacity to handle heterogeneous fusion of both spectral and spatial features. This approach can generate an adaptive kernel as an optimally weighted sum of a few fixed kernels to model a nonlinear data structure. In this way, the difficulty of kernel selection and the limitation of a fixed kernel can be alleviated. Various MKL algorithms have been developed in recent years, such as the general MKL, the subspace MKL, the nonlinear MKL, the sparse MKL, and the ensemble MKL. The goal of this paper is to provide a systematic review of MKL methods, which have been applied to HSI classification. We also analyze and evaluate different MKL algorithms and their respective characteristics in different cases of HSI classification cases. Finally, we discuss the future direction and trends of research in this area.

Optimal Couple Projections for Domain Adaptive Sparse Representation-Based Classification.

In recent years, sparse representation-based classification (SRC) is one of the most successful methods and has been shown impressive performance in various classification tasks. However, when the training data have a different distribution than the testing data, the learned sparse representation may not be optimal, and the performance of SRC will be degraded significantly. To address this problem, in this paper, we propose an optimal couple projections for domain-adaptive SRC (OCPD-SRC) method, in which the discriminative features of data in the two domains are simultaneously learned with the dictionary that can succinctly represent the training and testing data in the projected space. OCPD-SRC is designed based on the decision rule of SRC, with the objective to learn coupled projection matrices and a common discriminative dictionary such that the between-class sparse reconstruction residuals of data from both domains are maximized, and the within-class sparse reconstruction residuals of data are minimized in the projected low-dimensional space. Thus, the resulting representations can well fit SRC and simultaneously have a better discriminant ability. In addition, our method can be easily extended to multiple domains and can be kernelized to deal with the nonlinear structure of data. The optimal solution for the proposed method can be efficiently obtained following the alternative optimization method. Extensive experimental results on a series of benchmark databases show that our method is better or comparable to many state-of-the-art methods.

Using data to build a better EM: EM* for big data

Existing data mining techniques, more particularly iterative learning algorithms, become overwhelmed with big data. While parallelism is an obvious and, usually, necessary strategy, we observe that both (1) continually revisiting data and (2) visiting all data are two of the most prominent problems especially for iterative, unsupervised algorithms like expectation maximization algorithm for clustering (EM-T). Our strategy is to embed EM-T into a nonlinear hierarchical data structure (heap) that allows us to (1) separate data that needs to be revisited from data that does not and (2) narrow the iteration toward the data that is more difficult to cluster. We call this extended EM-T, EM*. We show our EM* algorithm outperform EM-T algorithm over large real-world and synthetic data sets. We lastly conclude with some theoretical underpinnings that explain why EM* is successful.

Multiple metric learning with query adaptive weights and multi-task re-weighting for person re-identification

Metric learning has been widely studied in person re-identification (re-id). However, most existing metric learning methods only learn one holistic Mahalanobis distance metric for the concatenated high dimensional feature. This single metric learning strategy cannot handle complex nonlinear data structure and may easily encounter overfitting. Besides, feature concatenation is incapable of exploring the discriminant capability of different features and low dimensional features tend to be dominated by high dimensional ones. Motivated by these problems, we propose a multiple metric learning method for the re-id problem, where individual sub-metrics are separately learned for each feature type and the final metric is formed as weighted sum of the sub-metrics. The sub-metrics are learned with the Cross-view Quadratic Discriminant Analysis (XQDA) algorithm and the weights to each sub-metric are assigned in a two-step procedure. First, the importance of each feature type is estimated according to its discriminative power, which is measured in a query adaptive manner as related to the partial Area Under Curve (pAUC) scores. Then, the weights of all feature types are learned simultaneously with a maximum-margin based multi-task structural SVM learning framework, in order to make sure that relevant gallery images are ranked before irrelevant ones within all feature spaces. Finally, the sub-metrics are integrated with the learned weights in an ensemble model, generating a sophisticated distance metric. Experiments on the challenging i-LIDS, VIPeR, CAVIAR and 3DPeS datasets demonstrate the effectiveness of the proposed method.

Keyframe Extraction for Human Motion Capture Data Based on Joint Kernel Sparse Representation

Human motion capture data, which are used to animate animation characters, have been widely used in many areas. To satisfy the high-precision requirement, human motion data are captured with a high frequency (120 frames/s) by a high-precision capture system. However, the high frequency and nonlinear structure make the storage, retrieval, and browsing of motion data challenging problems, which can be solved by keyframe extraction. Current keyframe extraction methods do not properly model two important characteristics of motion data, i.e., sparseness and Riemannian manifold structure. Therefore, we propose a new model called joint kernel sparse representation (SR), which is in marked contrast to all current keyframe extraction methods for motion data and can simultaneously model the sparseness and the Riemannian manifold structure. The proposed model completes the SR in a kernel-induced space with a geodesic exponential kernel, whereas the traditional SR cannot model the nonlinear structure of motion data in the Euclidean space. Meanwhile, because of several important modifications to traditional SR, our model can also exploit the relations between joints and solve two problems, i.e., the unreasonable distribution and redundancy of extracted keyframes, which current methods do not solve. Extensive experiments demonstrate the effectiveness of the proposed method.

Bilateral Two-Dimensional Neighborhood Preserving Discriminant Embedding for Face Recognition

In this paper, we propose a novel bilateral 2-D neighborhood preserving discriminant embedding for supervised linear dimensionality reduction for face recognition. It directly extracts discriminative face features from images based on graph embedding and Fisher’s criterion. The proposed method is a manifold learning algorithm based on graph embedding criterion, which can effectively discover the underlying nonlinear face data structure. Both within-neighboring and between-neighboring information are taken into account to seek an optimal projection matrix by minimizing the intra-class scatter and maximizing the inter-class scatter based on Fisher’s criterion. The performance of the proposed method is evaluated and compared with other face recognition schemes on the Yale, PICS, AR, and LFW databases. The experiment results demonstrate the effectiveness and superiority of the proposed method as compared with the state-of-the-art dimensionality reduction algorithms.

A robust background regression based score estimation algorithm for hyperspectral anomaly detection

Anomaly detection has become a hot topic in the hyperspectral image analysis and processing fields in recent years. The most important issue for hyperspectral anomaly detection is the background estimation and suppression. Unreasonable or non-robust background estimation usually leads to unsatisfactory anomaly detection results. Furthermore, the inherent nonlinearity of hyperspectral images may cover up the intrinsic data structure in the anomaly detection. In order to implement robust background estimation, as well as to explore the intrinsic data structure of the hyperspectral image, we propose a robust background regression based score estimation algorithm (RBRSE) for hyperspectral anomaly detection. The Robust Background Regression (RBR) is actually a label assignment procedure which segments the hyperspectral data into a robust background dataset and a potential anomaly dataset with an intersection boundary. In the RBR, a kernel expansion technique, which explores the nonlinear structure of the hyperspectral data in a reproducing kernel Hilbert space, is utilized to formulate the data as a density feature representation. A minimum squared loss relationship is constructed between the data density feature and the corresponding assigned labels of the hyperspectral data, to formulate the foundation of the regression. Furthermore, a manifold regularization term which explores the manifold smoothness of the hyperspectral data, and a maximization term of the robust background average density, which suppresses the bias caused by the potential anomalies, are jointly appended in the RBR procedure. After this, a paired-dataset based k-nn score estimation method is undertaken on the robust background and potential anomaly datasets, to implement the detection output. The experimental results show that RBRSE achieves superior ROC curves, AUC values, and background-anomaly separation than some of the other state-of-the-art anomaly detection methods, and is easy to implement in practice.

A New Approach for Optimization of Program Dependence Graph using Finite Automata

The Objective is to optimize the time in identification of similar code segment in a program, which is represented using PDG. The method adapted is minimization of finite automata, the states, which are having similar transitions, can be combined into a single state, through which we can remove the duplicate code in program. Methodology used in current study is detection of isomorphic sub graphs in a graph where the program segment has been represented using graph, this approach is very lengthy because finding of sub graphs and identifying isomorphicity between sub graphs. Findings are the demonstrated through an example in figure 1 to figure 4. The efficiency of suggested idea has been demonstrated in analysis part. This approach can be used in compiler optimization phase because it connects computationally related parts of a program; PDG is non-linear data structure in which the transformations can be performed uniformly for both data and control dependences.Keywords: DFA, Finite Automata, Isomorphic Graphs, Minimization of Finite Automata, Optimized PDG, PDG

Dynamic structure of the spot price of crude oil: does time aggregation matter?

This paper assesses nonlinear structures in the time series data generating mechanism of crude oil prices. We apply well-known univariate tests for nonlinearity, with distinct power functions over alternatives, but with different null hypotheses reflecting the existence of different concepts of linearity and nonlinearity in the time series literature. We utilize daily data on crude oil spot price for over 26years, as well as monthly data on crude oil spot price for 41years. Investigating the monthly price of crude oil along with the daily price distinguishes the approach of this paper from existing studies focusing on the time series structure of crude oil price. All the tests detect strong evidence of general nonlinear serial dependence, as well as nonlinearity in the mean, variance, and skewness functions in the daily spot price of crude oil. Since evidence of nonlinear dependence is less dramatic in monthly observations, nonlinear serial dependence is moderated by time aggregation in crude oil prices but not significantly.

Approximate pattern matching with gap constraints

Pattern matching is a key issue in sequential pattern mining. Many researchers now focus on pattern matching with gap constraints. However, most of these studies involve exact pattern matching problems, a special case of approximate pattern matching and a more challenging task. In this study, we introduce an approximate pattern matching problem with Hamming distance. Its objective is to compute the number of approximate occurrences of pattern P with gap constraints in sequence S under similarity constraint d. We propose an efficient algorithm named Single-rOot Nettree for approximate pattern matchinG with gap constraints (SONG) based on a new non-linear data structure Single-root Nettree to effectively solve the problem. Theoretical analysis and experiments demonstrate an interesting law that the ratio M( P, S, d)/ N( P, S, m) approximately follows a binomial distribution, where M( P, S, d) and N( P, S, m) are the numbers of the approximate occurrences whose distances to pattern P are d (0≤ d≤ m) and no more than m (the length of pattern P), respectively. Experimental results for real biological data validate the efficiency and effectiveness of SONG.

A Novel Pre-Processing Technique for Original Feature Matrix of Electronic Nose Based on Supervised Locality Preserving Projections.

An electronic nose (E-nose) consisting of 14 metal oxide gas sensors and one electronic chemical gas sensor has been constructed to identify four different classes of wound infection. However, the classification results of the E-nose are not ideal if the original feature matrix containing the maximum steady-state response value of sensors is processed by the classifier directly, so a novel pre-processing technique based on supervised locality preserving projections (SLPP) is proposed in this paper to process the original feature matrix before it is put into the classifier to improve the performance of the E-nose. SLPP is good at finding and keeping the nonlinear structure of data; furthermore, it can provide an explicit mapping expression which is unreachable by the traditional manifold learning methods. Additionally, some effective optimization methods are found by us to optimize the parameters of SLPP and the classifier. Experimental results prove that the classification accuracy of support vector machine (SVM combined with the data pre-processed by SLPP outperforms other considered methods. All results make it clear that SLPP has a better performance in processing the original feature matrix of the E-nose.

Geometric Preserving Local Fisher Discriminant Analysis for person re-identification

Recently, Local Fisher Discriminant Analysis (LFDA) has achieved impressive performance in person re-identification. However, the classic LFDA method pays little attention to the intrinsic geometrical structure of the complex person re-identification data. Due to large appearance variance, two images of the same person may be far away from each other in feature space while images of different people may be quite close to each other. The linear topology exploited in LFDA is not sufficient to describe this nonlinear data structure. In this paper, we assume that the data reside on a manifold and propose an effective method termed Geometric Preserving Local Fisher Discriminant Analysis (GeoPLFDA). The method integrates discriminative framework of LFDA with geometric preserving method which approximates local manifold utilizing a nearest neighbor graph. LFDA provides discriminative information by separating different labeled samples and pulling the same labeled samples together. The geometric preserving projection provides local manifold structure of the nonlinear data induced by graph topology. Taking advantage of the complementary between them, the proposed method achieves significant improvement over state-of-the-art approaches. Furthermore, a kernel extension of the GeoPLFDA method is proposed to handle the complex nonlinearity more effectively and to further improve re-identification accuracy. Experiments on the challenging iLIDS, VIPeR, CAVIAR and 3DPeS datasets demonstrate the effectiveness of the proposed method.

Prediction of Chinese word-formation patterns using the layer-weighted semantic graph-based KFP-MCO classifier

Nowadays natural language processing plays an important and critical role in the domain of intelligent computing, pattern recognition, semantic analysis and machine intelligence. For Chinese information processing, to construct the predictive models of different semantic word-formation patterns with a large-scale corpus can significantly improve the efficiency and accuracy of the paraphrase of the unregistered or new word, ambiguities elimination, automatic lexicography, machine translation and other applications. Therefore it is required to find the relationship between word-formation patterns and different influential factors, which can be denoted as a classification problem. However, due to noise, anomalies, imprecision, polysemy, ambiguity, nonlinear structure, and class-imbalance in semantic word-formation data, multi-criteria optimization classifier (MCOC), support vector machines (SVM) and other traditional classification approaches will give the poor predictive performance. In this paper, according to the characteristic analysis of Chinese word-formations, we firstly proposed a novel layered semantic graph of each disyllabic word, the layer-weighted graph edit distance (GED) and its similarity kernel embedded into a new vector space, then on the normalized data MCOC with kernel, fuzzification and penalty factors (KFP-MCOC) and SVM are employed to predict Chinese semantic word-formation patterns. Our experimental results and comparison with SVM show that KFP-MCOC based on the layer-weighted semantic graphs can increase the separation of different patterns, the predictive accuracy of target patterns and the generalization of semantic pattern classification on new compound words.

Efficient isometric multi-manifold learning based on the self-organizing method

Geodesic distance, as an essential measurement for data similarity, has been successfully used in manifold learning. However, many geodesic based isometric manifold learning algorithms, such as the isometric feature mapping (Isomap) and GeoNLM, fail to work on data that distribute on clusters or multiple manifolds. This limits their applications because practical data sets generally distribute on multiple manifolds. In this paper, we propose a new isometric multi-manifold learning method called Multi-manifold Proximity Embedding (MPE) which can be efficiently optimized using the gradient descent method or the self-organizing method. Compared with the previous methods, the proposed method has two steps which can isometrically learn data distributed on several manifolds and is more accurate in preserving both the intra-manifold and the inter-manifold geodesic distances. The effectiveness of the proposed method in recovering the nonlinear data structure and clustering is demonstrated through experiments on both synthetically and real data sets.

Spatial Euler kernel and its applications

Since the Principal Component Analysis (1D-PCA) extended to the image oriented 2D-PCA by Yang et al, a number of 1D methods have been extended to their corresponding 2D variants. Because of the use of natural spatial information of matrix-form data (e.g., an image), the 2D methods usually yield much better performance than their 1D versions, which is consistent with the theorem of No Free Lunch. However, the 2D methods still suffer from two main drawbacks: (1) they are almost all linear, which might not match the nonlinear structure of actual data; and (2) the spatial information of data is not fully used in existing 2D methods. To address the first drawback, although the kernel trick is theoretically feasible, it is practically difficult since the representation theorem cannot be straightforwardly extended for 2D-form data. To this end, we in this work propose to obtain a simple kernelizing method by changing the measurement without the representation theorem. In view of the second shortcoming, we aim to use the spatial information of data in the nonlinear mapping feature space (or say kernel space). Unfortunately, it usually requires to describe the data in the nonlinear feature space (i.e., kernel space), which is generally implemented through data dimension-increasing as well as implicit kernel mapping. To fulfill this goal, it usually refers to the implicit or explicit kernel mapping, the former, however, might distort the spatial structure of data while the latter leads to dimension risk. As a result, we can preferably preserve the spatial structure of data naturally if we employ the form of explicit kernels in which the dimension is identical and each component is uncoupled. Fortunately, many explicit kernel (e.g., Hellinger and Euler kernels) and approximate explicit mathematical formulations of some implicit additive kernels (e.g., Intersection, JS and χ 2 kernels) meet the requirements. Considering the conciseness and good generalization of the Euler kernel, we in this work attempt to kernelize matrix-form data by Euler kernel and then in the mapped kernel space to compensate the spatial information. Although there exist various ways to compensating the spatial information, e.g., spatial structure information constraints and image distance metric, we in this work take the Image Euclidean Distance as an example to carry out the study and then develop matrix/image-oriented Spatial Euler Kernel. Finally, through experiments, we demonstrate the effectiveness of the proposed strategies.

An Alternative Formulation of PCA for Process Monitoring Using Distance Correlation

Scale-invariant principal component analysis (PCA) is prevalent in process monitoring because of its simplicity and efficiency. However, a number of limitations are associated with this technique because of underlying assumptions. This article attempts to relax these limitations by introducing three key elements. First, a semiparametric Gaussian transformation is proposed to make the process data follow a multivariate Gaussian distribution, such that the standard PCA can be directly applied to explain the majority of the process data variance. The Gaussian transformation function preserves both important statistical information and the correlation structures of the process data. Second, eigenvectors spanning the feature space are extracted using the Spearman correlation coefficient and the distance correlation coefficient. This feature space is able to retain nonlinear and nonmonotonic correlation structures of the process data. Finally, this technique is computationally more efficient than KPCA, KICA, and improved KICA by avoiding expensive kernel mapping. Semiparametric PCA is tested on two industrial case studies and exhibits satisfactory performance.

Efficient Wu Manber String Matching Algorithm for Large Number of Patterns

String matching is one of the most important concept used in computer science in various real life applications like as Intrusion detection system, Data mining, Plagiarism detection system. There are many string matching algorithms which help to find pattern from the text. These algorithms are categorized in single string matching and multiple string matching. The Wu-Manber (WM) algorithm is multiple patterns algorithm which is the finest string matching algorithm. The performance of WM depends on various table build in pre processing phase these are prefix table, shift table and hase table. We introduce a new algorithm namely the Efficient Wu Manber algorithm (EWM) algorithm which is advance version of Wu Manber algorithm with respect to time. Efficient Wu-Manber Algorithm eliminate the prefix table which is unused most of the cases in wu manber, construct two shift table instead of single shift table and uses nonlinear data structure i.e. AVL tree instead of linear data structure i.e. linked list used in WM in Hash table, which reduce the traversed number of nodes to find exact match. The experimental results and analysis show that EWM algorithm has better performance as compare to WM and its existing improved algorithm and also better from various string matching tools.

Multi-mode separation and nonlinear feature extraction of hybrid gear failures in coal cutters using adaptive nonstationary vibration analysis

Reliable condition monitoring and fault diagnosis is an important issue for the normal operation of coal cutter gear systems. Intrinsic deterioration indicators are always hidden in the vibration response of the gearboxes, and it is often very difficult to correctly extract them due to nonlinear/chaotic nature of the vibration signal. Literature review suggests that hybrid gear faults diagnosis is a challenging task and how to extract quantitative indicators for hybrid faults detection is attracting considerable attentions. In order to address this issue, a new adaptive nonstationary vibration analysis method is proposed in this paper to extract useful quantitative indicators for hybrid gear faults decoupling detection. In this new technology, the center frequencies of the narrow bands of intrinsic modes contained in the vibration signal were adaptively estimated by the variational model decomposition (VMD) to determine the bandwidth of the modes. Hence, the hybrid gear faults were decoupled into single faults in the form of VMD modes. Then, the time and frequency features of each mode were calculated to obtain the feature space. Lastly, the feature space was projected into the reproducing kernel Hilbert space by the spectral regression-optimized kernel Fisher discrimination (SRKFD), where the instinct nonlinear structure in the original data can be identified and thus useful quantitative indicators can be extracted. Reliable hybrid faults decoupling detection was then achieved. Specially designed numerical simulations and experiments were conducted to evaluate the proposed VMD-SRKFD method on hybrid gear faults diagnosis of coal cutters. The performance was compared with existing techniques. The analysis results show high performance of the proposed method on quantitative hybrid faults detection in the coal cutter gear system.

A new nonlinear blind source separation method with chaos indicators for decoupling diagnosis of hybrid failures: A marine propulsion gearbox case with a large speed variation

The normal operation of propulsion gearboxes ensures the ship safety. Chaos indicators could efficiently indicate the state change of the gearboxes. However, accurate detection of gearbox hybrid faults using Chaos indicators is a challenging task and the detection under speed variation conditions is attracting considerable attentions. Literature review suggests that the gearbox vibration is a kind of nonlinear mixture of variant vibration sources and the blind source separation (BSS) is reported to be a promising technique for fault vibration analysis, but very limited work has addressed the nonlinear BSS approach for hybrid faults decoupling diagnosis. Aiming to enhance the fault detection performance of Chaos indicators, this work presents a new nonlinear BSS algorithm for gearbox hybrid faults detection under a speed variation condition. This new method appropriately introduces the kernel spectral regression (KSR) framework into the morphological component analysis (MCA). The original vibration data are projected into the reproducing kernel Hilbert space (RKHS) where the instinct nonlinear structure in the original data can be linearized by KSR. Thus the MCA is able to deal with nonlinear BSS in the KSR space. Reliable hybrid faults decoupling is then achieved by this new nonlinear MCA (NMCA). Subsequently, by calculating the Chaos indicators of the decoupled fault components and comparing them with benchmarks, the hybrid faults can be precisely identified. Two specially designed case studies were implemented to evaluate the proposed NMCA-Chaos method on hybrid gear faults decoupling diagnosis. The performance of the NMCA-Chaos was compared with state of art techniques. The analysis results show high performance of the proposed method on hybrid faults detection in a marine propulsion gearbox with large speed variations.

Optimal feature selection for nonlinear data using branch-and-bound in kernel space

Branch-and-bound (B&B) feature selection finds optimal feature subsets without performing an exhaustive search. However, the classification accuracy achievable with optimal B&B feature subsets is often inferior compared to the accuracy achievable with other algorithms that guarantee optimality. We argue this is due to the existing criterion functions that define the optimal feature subset but may not conceive inherent nonlinear data structures. Therefore, we propose B&B feature selection in Reproducing Kernel Hilbert Space (B&B-RKHS). This algorithm employs two existing criterion functions (Bhattacharyya distance, Kullback–Leibler divergence) and one new criterion function (mean class distance), however, all computed in RKHS. This enables B&B-RKHS to conceive inherent nonlinear data structures. The algorithm was experimentally compared to the popular wrapper approach that has to use an exhaustive search to guarantee optimality. The classification accuracy achieved with both methods was comparable. However, runtime of B&B-RKHS was superior using the two existing criterion functions and even completely out of reach using the new criterion function (about 60 times faster on average). Therefore, this paper proposes an efficient algorithm if feature subsets that guarantee optimality have to be selected in data sets with inherent nonlinear structures.