Feature Learning Algorithm Research Articles

Big data analytics enabled by feature extraction based on partial independence

Abstract Complex cells in primary visual cortex (V1) selectively respond to bars and edges at a particular location and orientation. Namely, they are relatively invariant to the phase as well as selective to the frequency and orientation emerging from natural images that are analogous to the characteristics of complex cells in V1 with the energy function of receptive fields (RFs) from tuning curve test with sinusoidal function in our related jobs. In this paper, we propose a feature learning algorithm based on the overcomplete AISA to apply on big data in parallel computing. In order to demonstrate the effectiveness of the overcomplete AISA features in the classification task, two feature representation architectures are evolved into the partial independent signal bases and partial independent factorial representation, respectively. Experiments on four datasets (Coil20, Extended YaleB, USPS, PIE), acquired conjunction with two classification architectures based on the overcomplete AISA features, show that the classification accuracy is mostly higher than those obtained from the other ICA related features and two other sparse representation features with a small number of training samples via nearest neighbor (NN) classification method.

CancerDiscover: an integrative pipeline for cancer biomarker and cancer class prediction from high-throughput sequencing data.

Accurate identification of cancer biomarkers and classification of cancer type and subtype from High Throughput Sequencing (HTS) data is a challenging problem because it requires manual processing of raw HTS data from various sequencing platforms, quality control, and normalization, which are both tedious and time-consuming. Machine learning techniques for cancer class prediction and biomarker discovery can hasten cancer detection and significantly improve prognosis. To date, great research efforts have been taken for cancer biomarker identification and cancer class prediction. However, currently available tools and pipelines lack flexibility in data preprocessing, running multiple feature selection methods and learning algorithms, therefore, developing a freely available and easy-to-use program is strongly demanded by researchers. Here, we propose CancerDiscover, an integrative open-source software pipeline that allows users to automatically and efficiently process large high-throughput raw datasets, normalize, and selects best performing features from multiple feature selection algorithms. Additionally, the integrative pipeline lets users apply different feature thresholds to identify cancer biomarkers and build various training models to distinguish different types and subtypes of cancer. The open-source software is available at https://github.com/HelikarLab/CancerDiscover and is free for use under the GPL3 license.

Efficient Multiple Kernel Classification Using Feature and Decision Level Fusion

Kernel methods for classification is a well-studied area in which data are implicitly mapped from a lower-dimensional space to a higher dimensional space to improve classification accuracy. However, for most kernel methods, one must still choose a kernel to use for the problem. Since there is, in general, no way of knowing which kernel is the best, multiple kernel learning (MKL) is a technique used to learn the aggregation of a set of valid kernels into a single (ideally) superior kernel. The aggregation can be done using weighted sums of the precomputed kernels, but determining the summation weights is not a trivial task. Furthermore, MKL does not work well with large datasets because of limited storage space and prediction speed. In this paper, we address all three of these multiple kernel challenges. First, we introduce a new linear feature level fusion technique and learning algorithm, GAMKLp. Second, we put forth three new algorithms, DeFIMKL, DeGAMKL, and DeLSMKL, for nonlinear fusion of kernels at the decision level. To address MKL's storage and speed drawbacks, we apply the Nystrom approximation to the kernel matrices. We compare our methods to a successful and state-of-the-art technique called MKL-group lasso (MKLGL), and experiments on several benchmark datasets show that some of our proposed algorithms outperform MKLGL when applied to support vector machine (SVM)-based classification. However, to no surprise, there does not seem to be a global winner but instead different strategies that a user can employ. Experiments with our kernel approximation method show that we can routinely discard most of the training data and at least double prediction speed without sacrificing classification accuracy. These results suggest that MKL-based classification techniques can be applied to big data efficiently, which is confirmed by an experiment using a large dataset.

Multi-graph embedding discriminative correlation feature learning for image recognition

Most of existing graph-based correlation analysis algorithms construct one graph for one view. However, only one graph is difficult to well reveal intrinsic geometry structure of the view. In this paper, we construct multiple graphs for each view by means of a dimensionality partitioning method, and then propose a novel multi-graph embedding discriminative correlation feature learning algorithm that focuses on multiple graph learning and label-based discriminating enhancement under the multi-view correlation analysis framework. In the algorithm, an effective combination of multiple graphs in each view can be automatically learned in order to better capture the intrinsic geometry structure of each view. Moreover, the algorithm can learn nonlinear correlation features with well discriminating power by maximizing multi-graph intrinsic correlations of different views and simultaneously minimizing intraclass scatter of each view. Extensive experiments on several real-world image datasets have demonstrated the superiority of the algorithm in image recognition.

An Extended Generative Feature Learning Algorithm for Image Recognition

An Extended Generative Feature Learning Algorithm for Image Recognition

Fuzzy Sparse Autoencoder Framework for Single Image Per Person Face Recognition.

The issue of single sample per person (SSPP) face recognition has attracted more and more attention in recent years. Patch/local-based algorithm is one of the most popular categories to address the issue, as patch/local features are robust to face image variations. However, the global discriminative information is ignored in patch/local-based algorithm, which is crucial to recognize the nondiscriminative region of face images. To make the best of the advantage of both local information and global information, a novel two-layer local-to-global feature learning framework is proposed to address SSPP face recognition. In the first layer, the objective-oriented local features are learned by a patch-based fuzzy rough set feature selection strategy. The obtained local features are not only robust to the image variations, but also usable to preserve the discrimination ability of original patches. Global structural information is extracted from local features by a sparse autoencoder in the second layer, which reduces the negative effect of nondiscriminative regions. Besides, the proposed framework is a shallow network, which avoids the over-fitting caused by using multilayer network to address SSPP problem. The experimental results have shown that the proposed local-to-global feature learning framework can achieve superior performance than other state-of-the-art feature learning algorithms for SSPP face recognition.

Mapping fractional landscape soils and vegetation components from Hyperion satellite imagery using an unsupervised machine-learning workflow

ABSTRACTAn unsupervised machine-learning workflow is proposed for estimating fractional landscape soils and vegetation components from remotely sensed hyperspectral imagery. The workflow is applied to EO-1 Hyperion satellite imagery collected near Ibirací, Minas Gerais, Brazil. The proposed workflow includes subset feature selection, learning, and estimation algorithms. Network training with landscape feature class realizations provide a hypersurface from which to estimate mixtures of soil (e.g. 0.5 exceedance for pixels: 75% clay-rich Nitisols, 15% iron-rich Latosols, and 1% quartz-rich Arenosols) and vegetation (e.g. 0.5 exceedance for pixels: 4% Aspen-like trees, 7% Blackberry-like trees, 0% live grass, and 2% dead grass). The process correctly maps forests and iron-rich Latosols as being coincident with existing drainages, and correctly classifies the clay-rich Nitisols and grasses on the intervening hills. These classifications are independently corroborated visually (Google Earth) and quantitatively (random soil samples and crossplots of field spectra). Some mapping challenges are the underestimation of forest fractions and overestimation of soil fractions where steep valley shadows exist, and the under representation of classified grass in some dry areas of the Hyperion image. These preliminary results provide impetus for future hyperspectral studies involving airborne and satellite sensors with higher signal-to-noise and smaller footprints.

Intelligent condition monitoring method for bearing faults from highly compressed measurements using sparse over-complete features

Abstract Condition classification of rolling element bearings in rotating machines is important to prevent the breakdown of industrial machinery. A considerable amount of literature has been published on bearing faults classification. These studies aim to determine automatically the current status of a roller element bearing. Of these studies, methods based on compressed sensing (CS) have received some attention recently due to their ability to allow one to sample below the Nyquist sampling rate. This technology has many possible uses in machine condition monitoring and has been investigated as a possible approach for fault detection and classification in the compressed domain, i.e., without reconstructing the original signal. However, previous CS based methods have been found to be too weak for highly compressed data. The present paper explores computationally, for the first time, the effects of sparse autoencoder based over-complete sparse representations on the classification performance of highly compressed measurements of bearing vibration signals. For this study, the CS method was used to produce highly compressed measurements of the original bearing dataset. Then, an effective deep neural network (DNN) with unsupervised feature learning algorithm based on sparse autoencoder is used for learning over-complete sparse representations of these compressed datasets. Finally, the fault classification is achieved using two stages, namely, pre-training classification based on stacked autoencoder and softmax regression layer form the deep net stage (the first stage), and re-training classification based on backpropagation (BP) algorithm forms the fine-tuning stage (the second stage). The experimental results show that the proposed method is able to achieve high levels of accuracy even with extremely compressed measurements compared with the existing techniques.

Multiview Feature Analysis via Structured Sparsity and Shared Subspace Discovery.

Since combining features from heterogeneous data sources can significantly boost classification performance in many applications, it has attracted much research attention over the past few years. Most of the existing multiview feature analysis approaches separately learn features in each view, ignoring knowledge shared by multiple views. Different views of features may have some intrinsic correlations that might be beneficial to feature learning. Therefore, it is assumed that multiviews share subspaces from which common knowledge can be discovered. In this letter, we propose a new multiview feature learning algorithm, aiming to exploit common features shared by different views. To achieve this goal, we propose a feature learning algorithm in a batch mode, by which the correlations among different views are taken into account. Multiple transformation matrices for different views are simultaneously learned in a joint framework. In this way, our algorithm can exploit potential correlations among views as supplementary information that further improves the performance result. Since the proposed objective function is nonsmooth and difficult to solve directly, we propose an iterative algorithm for effective optimization. Extensive experiments have been conducted on a number of real-world data sets. Experimental results demonstrate superior performance in terms of classification against all the compared approaches. Also, the convergence guarantee has been validated in the experiment.

Semi-supervised multi-label feature selection via label correlation analysis with l1-norm graph embedding

In this paper, we propose a novel semi-supervised multi-label feature selection algorithm and apply it to three different applications: natural scene classification, web page annotation, and yeast gene functional classification. Compared with the previous works, there are two advantages of our algorithm: (1) Manifold learning which leverages the underlying geometric structure of the training data is imposed to utilize both labeled and unlabeled data. Besides, the underlying manifold structure is guaranteed to be clear by using the l1-norm regularization. (2) Shared subspace learning which has shown its efficiency in multi-label learning scenarios, is also considered in our feature learning algorithm. The proposed objective function involves l2,1-norm and l1-norm, making it non-smooth and difficult to solve. We also design an efficient iterative algorithm to optimize it. Experimental results demonstrate the effectiveness of our algorithm compared with sate-of-the-art algorithms on different tasks.

Unsupervised Feature Learning for Land-Use Scene Recognition

This paper proposes a novel unsupervised feature learning algorithm for land-use scene recognition on very high resolution remote sensing imagery. The proposed technique utilizes a multipath sparse coding architecture in order to capture multiple aspects of discriminative structures within complex remote sensing sceneries. Unlike the previous sparse coding and bag-of-visual-words-based techniques that rely on the handcrafted feature descriptors such as scale-invariant feature transform, the proposed technique extracts dense low-level features from the raw data, including the visual (RGB) data and near-infrared (NIR) data, using image patches of varying sizes at different layers. The proposed technique has been evaluated on three data sets, including the 21-category UC Merced land-use $\texttt {RGB}$ data set with a 1-ft spatial resolution, the 9-category ground scene RGB-NIR data set, and the 10-category Singapore land-use $\texttt {RGB-NIR}$ data set with a 0.5-m spatial resolution. The experimental results show that the proposed technique outperforms the state-of-the-art methods.

A Skin Segmentation Algorithm Based on Stacked Autoencoders

A good skin detector that is capable of capturing skin tones under different conditions is important for human–machine interaction applications. In a general situation, skin detectors, such as skin probability maps or Gaussian mixture models, achieve acceptable skin segmentation results. However, the false positive rate increases significantly when the skin tones are in shadow or when skin-like background objects are under similar illumination. In this paper, we propose a novel skin feature learning algorithm based on stacked autoencoders, which are deep neural networks. To overcome the problems encountered in skin segmentation that are caused by different ethnicities and varying illumination conditions, the stacked autoencoders are utilized to learn more discriminative representations of the skin area in both the RGB color space and the HSV color space. Unlike traditional machine learning methods, instead of predicting each pixel individually, our algorithm utilizes blocks to learn the representations and detect the skin areas. The algorithm exploits the learning ability of deep neural networks to learn high-level representations of skin tones. Experiments on test images show that the proposed algorithm achieves acceptable results on several publicly available data sets. To reduce the difficulty of detecting skin pixels in these data sets, the ground truths of these data sets are commonly focused on foreground skin area detection. Our skin detector is also able to detect background areas, as shown in our experiments.

Multi-Task Feature Learning Algorithm Based on Preserving Classification Information

Multi-Task Feature Learning Algorithm Based on Preserving Classification Information

Multi-View Correlated Feature Learning by Uncovering Shared Component

Learning multiple heterogeneous features from different data sources is challenging. One research topic is how to exploit and utilize the correlations among various features across multiple views with the aim of improving the performance of learning tasks, such as classification. In this paper, we propose a new multi-view feature learning algorithm that simultaneously analyzes features from different views. Compared to most of the existing subspace learning methods that only focus on exploiting a shared latent subspace, our algorithm not only learns individual information in each view but also captures feature correlations among multiple views by learning a shared component. By assuming that such a component is shared by all views, we simultaneously exploit the shared component and individual information of each view in a batch mode. Since the objective function is non-smooth and difficult to solve, we propose an efficient iterative algorithm for optimization with guaranteed convergence. Extensive experiments are conducted on several benchmark datasets. The results demonstrate that our proposed algorithm performs better than all the compared multi-view learning algorithms.

Image Segmentation for Fruit Detection and Yield Estimation in Apple Orchards

Ground vehicles equipped with monocular vision systems are a valuable source of high‐resolution image data for precision agriculture applications in orchards. This paper presents an image processing framework for fruit detection and counting using orchard image data. A general‐purpose image segmentation approach is used, including two feature learning algorithms; multiscale multilayered perceptrons (MLP) and convolutional neural networks (CNN). These networks were extended by including contextual information about how the image data was captured (metadata), which correlates with some of the appearance variations and/or class distributions observed in the data. The pixel‐wise fruit segmentation output is processed using the watershed segmentation (WS) and circular Hough transform (CHT) algorithms to detect and count individual fruits. Experiments were conducted in a commercial apple orchard near Melbourne, Australia. The results show an improvement in fruit segmentation performance with the inclusion of metadata on the previously benchmarked MLP network. We extend this work with CNNs, bringing agrovision closer to the state‐of‐the‐art in computer vision, where although metadata had negligible influence, the best pixel‐wise F1‐score of 0.791 was achieved. The WS algorithm produced the best apple detection and counting results, with a detection F1‐score of 0.861. As a final step, image fruit counts were accumulated over multiple rows at the orchard and compared against the post‐harvest fruit counts that were obtained from a grading and counting machine. The count estimates using CNN and WS resulted in the best performance for this data set, with a squared correlation coefficient of .

Finding Abnormal Vessel Trajectories Using Feature Learning

Global Positioning System technology has been widely used in vehicle tracking and road planning applications. An enormous amount of data concerning the trajectories of vehicles has been collected and stored for tracking purposes. A trajectory contains not only the footprints of a moving object but also additional information, such as speed and stopping points. Therefore, the large-scale trajectory data sets provide rich information and are currently attracting considerable attention; there have been many successful studies of event detection based on trajectory data. However, most of these studies have focused only on vehicles traveling in a road network and have note considered maritime trajectories. A maritime trajectory also contains auxiliary data (e.g., speed and rotation) in addition to the movements of a ship. However, ships are not bound to road networks, and consequently, it is difficult to apply traditional mining algorithms based on road networks. In addition, even if the amount of maritime trajectory data is very large, these data are also spatially sparse, which will significantly reduce the effectiveness of most existing mining algorithms. In this paper, we propose a new method of abnormal trajectory detection to address this problem. This method can detect abnormal vessel trajectories from Automatic Identification System (AIS), records for vessels via our feature learning algorithm. To reduce the search space, we invoke reference points as well as the Piecewise Linear Segmentation (PLS), algorithm to compress the trajectories without losing important information. A time-aware and spatially correlated collaborative algorithm is proposed to increase the density of the trajectories to improve the accuracy of the detection algorithm, which is based on Dynamic Time Warping (DTW). Finally, we report experiments conducted on a real-world data set, which demonstrate that the proposed detection method can detect anomalous trajectories effectively.

An artificial neural network method for lumen and media-adventitia border detection in IVUS

Intravascular ultrasound (IVUS) has been well recognized as one powerful imaging technique to evaluate the stenosis inside the coronary arteries. The detection of lumen border and media-adventitia (MA) border in IVUS images is the key procedure to determine the plaque burden inside the coronary arteries, but this detection could be burdensome to the doctor because of large volume of the IVUS images. In this paper, we use the artificial neural network (ANN) method as the feature learning algorithm for the detection of the lumen and MA borders in IVUS images. Two types of imaging information including spatial, neighboring features were used as the input data to the ANN method, and then the different vascular layers were distinguished accordingly through two sparse auto-encoders and one softmax classifier. Another ANN was used to optimize the result of the first network. In the end, the active contour model was applied to smooth the lumen and MA borders detected by the ANN method. The performance of our approach was compared with the manual drawing method performed by two IVUS experts on 461 IVUS images from four subjects. Results showed that our approach had a high correlation and good agreement with the manual drawing results. The detection error of the ANN method close to the error between two groups of manual drawing result. All these results indicated that our proposed approach could efficiently and accurately handle the detection of lumen and MA borders in the IVUS images.

Multi-patch embedding canonical correlation analysis for multi-view feature learning

Locality-based feature learning for multi-view data has received intensive attention recently. As a result of only considering single-category local neighbor relationships, most of such the learning methods are difficult to well reveal intrinsic geometric structure information of raw high-dimensional data. To solve the problem, we propose a novel supervised multi-view correlation feature learning algorithm based on multi-category local neighbor relationships, called multi-patch embedding canonical correlation analysis (MPECCA). Our algorithm not only employs multiple local patches of each raw data to better capture the intrinsic geometric structure information, but also makes intraclass correlation features as close as possible by minimizing intraclass scatter of each view. Extensive experimental results on several real-world image datasets have demonstrated the effectiveness of our algorithm.

A novel mid-level distinctive feature learning for action recognition via diffusion map

Recent works have shown that mid-level feature is superior to low-level feature, which can not only improve discriminative power, but also enhance descriptive capability. In this paper, the classical STIP, spatial star-graph and temporal star-graph are first extracted to represent human action from multi-perspectives. Then a principled feature learning algorithm is proposed to embed these multi-cues into a unified space and enhance all low-level features using diffusion map. Unlike treating spatio-temporal patch as mid-level primitive, we use a graph to model different types of primitives, then apply graph partitioning to co-cluster them into visual-word clusters called mid-level distinctive feature, which can bridge the semantic gap across low-level features. Experimental results show that our approach can successfully classify human activities with higher accuracies both on single-person actions (KTH and UCF) and complex interactional activities (UT-Interaction and HMDB51).

Sparse feature learning for instrument identification: Effects of sampling and pooling methods.

Feature learning for music applications has recently received considerable attention from many researchers. This paper reports on the sparse feature learning algorithm for musical instrument identification, and in particular, focuses on the effects of the frame sampling techniques for dictionary learning and the pooling methods for feature aggregation. To this end, two frame sampling techniques are examined that are fixed and proportional random sampling. Furthermore, the effect of using onset frame was analyzed for both of proposed sampling methods. Regarding summarization of the feature activation, a standard deviation pooling method is used and compared with the commonly used max- and average-pooling techniques. Using more than 47 000 recordings of 24 instruments from various performers, playing styles, and dynamics, a number of tuning parameters are experimented including the analysis frame size, the dictionary size, and the type of frequency scaling as well as the different sampling and pooling methods. The results show that the combination of proportional sampling and standard deviation pooling achieve the best overall performance of 95.62% while the optimal parameter set varies among the instrument classes.