Feature Combination Method Research Articles

Extraction of multimodal features from depth and RGB images for recognising hand gestures

In this study, we propose a multimodal feature based framework for recognising hand gestures from RGB and depth images. In addition to the features from the RGB image, the depth image features are explored into constructing the discriminative feature labels of various gestures. Depth maps having powerful source of information, increases the performance level of various computer vision problems. A newly refined Gradient-Local Binary Pattern (G-LBP) is applied to extract the features from depth images and histogram of gradients (HOG) features are extracted from RGB images. The components from both RGB and depth channels, are concatenated to form a multimodal feature vector. In the final process, classification is performed using K-Nearest Neighbour and multi-class Support Vector Machines. The designed system is invariant to scale, rotation and illumination. The newly developed feature combination method is helpful to achieve superior recognition rates for future innovations.

Mapping paddy rice by the object-based random forest method using time series Sentinel-1/Sentinel-2 data

Rice is one of the world’s major staple foods, especially in China. In this study, we proposed an object-based random forest (RF) method for paddy rice mapping using time series Sentinel-1 and Sentinel-2 data. Firstly, the Robust Adaptive Spatial Temporal Fusion Model (RASTFM) was used to blend MODIS and Sentinel-2 data for achieving multi-temporal Sentinel-2 data. Subsequently, the Savitzky-Golay filter (S-G) was applied to smooth the time series Sentinel-2 NDVI data. And the phenological parameters were derived from the filtered time series NDVI using the threshold method. Then, the optimum feature combination for paddy mapping was formed on the basis of Sentinel-2 MSI images, time series Sentinel-2 NDVI, phenology data and time series Sentinel-1 SAR backscattering images by using the JBh distance. Finally, an object-based Random Forest classifier was used to extract paddy rice with the optimum feature combination. The result showed that fused Sentinel-2 NDVI time series using RASTFM has a high correlation with the original Sentinel-2 image. The overall accuracy and Kappa coefficient of the classification results are higher than 95% and 0.93, respectively, when use the optimum feature combination and object-based RF method. The proposed method can provide technology support for rice mapping in areas with a lot of cloudy and rainy weathers.

Improved Movie Recommendations Based on a Hybrid Feature Combination Method

Recommender systems help users find relevant items efficiently based on their interests and historical interactions with other users. They are beneficial to businesses by promoting the sale of products and to user by reducing the search burden. Recommender systems can be developed by employing different approaches, including collaborative filtering (CF), demographic filtering (DF), content-based filtering (CBF) and knowledge-based filtering (KBF). However, large amounts of data can produce recommendations that are limited in accuracy because of diversity and sparsity issues. In this paper, we propose a novel hybrid method that combines user–user CF with the attributes of DF to indicate the nearest users, and compare four classifiers against each other. This method has been developed through an investigation of ways to reduce the errors in rating predictions based on users’ past interactions, which leads to improved prediction accuracy in all four classification algorithms. We applied a feature combination method that improves the prediction accuracy and to test our approach, we ran an offline evaluation using the 1M MovieLens dataset, well-known evaluation metrics and comparisons between methods with the results validating our proposed method.

Bone- and air-conduction speech combination method for speaker recognition

In this paper, first, we report speaker recognition performance using bone-conduction speech based on an i-vector-based speaker recognition system, which is the current state-of-the-art method. In addition, we propose three speaker recognition methods combining bone-conduction speech and air-conduction speech: a feature combination method, a speaker model combination method, and a similarity score combination method. To evaluate the proposed methods, we conducted speaker recognition experiments using a part of a large speech corpus constructed by the National Research Institute of Police Science, Japan. Experimental results show the bone-conduction speech performs almost the same as the air-conduction speech when the enrolment data and evaluation data are collected in the same session. In addition, all proposed methods improved the speaker recognition performance of air- and bone-conduction speech in the experiments. From these results, we conclude that fusing air- and bone-conduction speech improves the speaker recognition performance.

An adaptive feature combination method based on ranking order for 3D model retrieval

Directly combining several complementary features may increase the retrieval precision for 3D models. However, in most cases, we need to set the weights manually and empirically. In this paper, we propose a new schema for automatically choosing the proper weights for different features on each database. The proposed schema utilises the ranking order of the retrieval results, and it is invariant to the magnitude scaling. We choose the best feature as the standard one, and the relevance values between the standard and other features are the weights for feature combination. Furthermore, we propose an improved re-ranking algorithm for further improving the retrieval performance. Experiment shows the proposed method can automatically choose the proper weights for different features, and the experiment results on the existing features exceed the benchmark.

An adaptive feature combination method based on ranking order for 3D model retrieval

Directly combining several complementary features may increase the retrieval precision for 3D models. However, in most cases, we need to set the weights manually and empirically. In this paper, we propose a new schema for automatically choosing the proper weights for different features on each database. The proposed schema utilises the ranking order of the retrieval results, and it is invariant to the magnitude scaling. We choose the best feature as the standard one, and the relevance values between the standard and other features are the weights for feature combination. Furthermore, we propose an improved re-ranking algorithm for further improving the retrieval performance. Experiment shows the proposed method can automatically choose the proper weights for different features, and the experiment results on the existing features exceed the benchmark.

3D-FHNet: Three-Dimensional Fusion Hierarchical Reconstruction Method for Any Number of Views

The research field of reconstructing 3D models from 2D images is becoming more and more important. Existing methods typically perform single-view reconstruction or multi-view reconstruction utilizing the properties of recurrent neural networks. Due to the self-occlusion of the model and the special nature of the recurrent neural network, these methods have some problems. We propose a novel three-dimensional fusion hierarchical reconstruction method that utilizes a multi-view feature combination method and a hierarchical prediction strategy to unify the single view and any number of multiple views 3D reconstructions. Experiments show that our method can effectively combine features between different views and obtain better reconstruction results than the baseline, especially in the thin parts of the object. Our source code is available at https://github.com/VIM-Lab/3D-FHNet.

Deep Hierarchical Strategy Model for Multi-Source Driven Quantitative Investment

Using deep learning to maximize the benefits of a series of risks in the securities market was a very interesting and widely concerned problem. From the perspective of multi-source driving, we proposed a feature combination method based on prior knowledge and designed a deep hierarchical strategy model to solve this challenge. This model includes a pre-judging module, which implements trend judgment of the time series, and an operation module that performs the trading actions. For the pre-judging module, we designed a regression constraint Wasserstein generative adversarial network (WGAN) to complete the mission instead of the common discriminant methods in the past. As for the operation module, in order to make the machine learning more compatible with human learning characteristics and having the ability to control risks, we designed a network structure based on deep deterministic policy gradient (DDPG), which is one of the reinforcement learning algorithms to decide continuous trading position. We used the Dow Jones Industrial Index and the Shanghai Stock Exchange Index to conduct training and testing. The results show that the proposed model has a good performance in terms of return on investment and robustness.

A Novel Adaptive Weighted Feature Combination Method for Pile Classification

A Novel Adaptive Weighted Feature Combination Method for Pile Classification

Complete three‐phase detection framework for identifying abnormal cervical cells

Automatic identification of abnormal cervical cells, including feature representation, feature combination and classification strategy, is highly demanded in women's annual cervical cancer screenings. However, previous methods only deal with one or two of these three phases, and currently there is few complete framework for this problem. A novel three-phrase boosting framework is proposed for the detection of abnormal cells from cervical smear images. First, the authors extract 160 dimensional features with respect to each cervical cell from three aspects, including cytology morphology, chromatin pathology and region intensity. In particular, 106 dimensional chromatin pathology features are newly adopted to describe the nucleus textural transformation. Second, an adaptive feature combination method is introduced to select the optimal feature patterns, which can combine all features using a reinforced margin-based approach with the heuristic knowledge. Finally, a two-stage classification strategy is presented to reduce erroneous classification abnormal cells using two different classifiers. Experimental results achieve state-of-the-art performance and the proposed framework outperforms the other 16 compared detection methods.

Content based Feature Combination Method for Face Image Retrieval using Neural Network and SVM Classifier for Face Recognition

Objectives: To propose a CBIR based face image retrieval and identification model. Method/Analysis: A model of hybrid face recognition system based on CBIR and SVM is proposed. The feature vectors from the face image database are generated by using Gabor wavelet (GW), wavelet Transformation (WT), and Principal Component Analysis (PCA). For the face image retrieval purpose, Artificial Neural Network (ANN) is adopted and its performance on the retrieval process is evaluated with PCA, WT, GW and their fusion as a feature vector. The query image is recognized from the faces returned by retrieval process by using Support Vector Machine (SVM). The experimental results indicate that the fusion of PCA, WT and GW features as a feature vector performs reasonably well for retrieval and recognition process. The experimental results also demonstrate the efficiency of the proposed approach for face recognition over existing methods when considering different performance measures such as system running time, Receiver Operating Characteristic (ROC) curve and recognition accuracy. The proposed model was evaluated with two face databases viz. Unconstrained Facial Image (UFI) and Oracle Research Laboratory (ORL) face database with a recognition accuracy of 95.42% and 98.75% respectively. Finding: The proposed system has been tested for retrieval and recognition and found with reasonable retrieval time and high recognition rate. Novelty/Improvement: The conventional model-based face recognition systems are limited in several aspects, like (1) It is usually time-consuming and expensive to collect a large amount of training facial images, (2) It is usually difficult to generalize the models when new training data or new persons are added, in which an intensive retraining process is usually required and (3) The recognition performance often scales poorly when the number of persons/ classes are very large. The proposed CBIR based retrieval and the recognition system is intended to take care of the above limitation.

RGBD Human Action Recognition using Multi-Features Combination and K-Nearest Neighbors Classification

In this paper, we present a novel system to analyze human body motions for action recognition task from two sets of features using RGBD videos. The Bag-of-Features approach is used for recognizing human action by extracting local spatialtemporal features and shape invariant features from all video frames. These feature vectors are computed in four steps: Firstly, detecting all interest keypoints from RGB video frames using Speed-Up Robust Features and filters motion points using Motion History Image and Optical Flow, then aligned these motion points to the depth frame sequences. Secondly, using a Histogram of orientation gradient descriptor for computing the features vector around these points from both RGB and depth channels, then combined these feature values in one RGBD feature vector. Thirdly, computing Hu-Moment shape features from RGBD frames; fourthly, combining the HOG features with Hu-moments features in one feature vector for each video action. Finally, the k-means clustering and the multi-class K-Nearest Neighbor is used for the classification task. This system is invariant to scale, rotation, translation, and illumination. All tested, are utilized on a dataset that is available to the public and used often in the community. By using this new feature combination method improves performance on actions with low movement and reach recognition rates superior to other publications of the dataset.

Combination method air and bone conducted speech for speaker recognition in i-vector space

Recently, some new sensors, such as bone-conductive microphones, throat microphones, and non-audible murmur (NAM) microphones, besides conventional condenser microphones have been developed for collecting speech data. Accordingly, some researchers began to study speaker and speech recognition using speech data collected by these new sensors. From these new sensors, we focus on a bone-conductive microphone. Recently, the speaker recognition method based on i-vector will be state-of-the-art. Hence, in this paper, first, we report the speaker recognition performance using bone-conducted speech based on i-vector-based speaker recognition system. In addition, we propose a speaker recognition method combined bone-conducted speech with air-conducted speech. In this paper, we investigate three combination methods, which are a distance combination method, an i-vector combination method, and a feature combination method. To evaluate the proposed methods, we conducted the speaker identification experiments. From experimental results, performance of the bone-conducted speech is almost same as that of the air-conducted speech under the condition of the enrolment and the evaluation speech collected on same session. In addition, the experimental results show that all proposed methods are able to improve the speaker recognition performance of air- and bone-conducted speech.

Improving the Accuracy of Land Cover Classification Using Fusion of Polarimetric SAR and Hyperspectral Images

In this study, various fusion methods based on feature-level and decision-level fusion of Polarimetric SAR (PolSAR) and hyperspectral images are investigated for the classification. In feature fusion, parallel feature combination method is used for classification. In decision fusion, first the individual data sources are classified separately and then several decision fusion methods are applied to fuse the classified images from each data. The results show that feature fusion has a better performance than the various decision fusion methods. The overall accuracy of feature fusion was 98.92 % which has illustrated improvement of 8 % compared to the obtained using hyperspectral data and 10 % compared to the acquired using PolSAR data.

Improve scene classification by using feature and kernel combination

Scene classification is an important issue in the computer vision field. In this paper, we propose an improved approach for scene classification. Compared with the previous work, the proposed approach has two processes to improve the performance of scene classification. First, to represent scene images more effectively, feature combination is conducted to extract more effective information to describe characteristics of each category decreasing the influence of scale, rotation and illumination. Second, to extract more discriminative information for building a multi-category classifier, a kernel fusion method is proposed. Experimental results on four commonly used data sets show that the use of the feature and kernel combination method can improve the classification accuracy effectively.

A Pedestrian-Detection Method Based on Heterogeneous Features and Ensemble of Multi-View–Pose Parts

Vision-based pedestrian detection remains a challenging task, so far. The detection performance often suffers from the various appearances of pedestrians, the illumination changes, and the possible partial occlusions. Aiming at resolving these challenges, in this paper, a new linear kernel function is proposed to effectively combine two heterogeneous features, i.e., histogram of oriented gradient and local binary pattern, which enhances the pedestrian description ability to illumination conditions and cluttered background. Then, a novel multi-view–pose part ensemble (MVPPE) detector is proposed, in order to better handle pedestrian variability, views, and partial occlusions. Experimental results in public data sets demonstrate that the proposed feature combination method significantly improves the description capabilities of pedestrian features. Compared with the existing multipart ensemble approaches, the proposed MVPPE detector boosts higher detection accuracy.

Discriminative fusion of shape and appearance features for human pose estimation

This paper presents a method for combining the shape and appearance feature types in a discriminative learning framework for human pose estimation. We first present a new appearance descriptor that is distinctive and resilient to noise for 3D human pose estimation. We then combine the proposed appearance descriptor with a shape descriptor computed from the silhouette of the human subject using discriminative learning. Our method, which we refer to as a localized decision level fusion technique, is based on clustering the output pose space into several partitions and learning a decision level fusion model for the shape and appearance descriptors in each region. The combined shape and appearance descriptor allows complementary information of the individual feature types to be exploited, leading to improved performance of the pose estimation system. We evaluate our proposed fusion method with feature level fusion and kernel level fusion methods using a synchronized video and 3D motion dataset. Our experimental results show that the proposed feature combination method gives more accurate pose estimation than the one obtained from each individual feature type. Among the three fusion methods, our localized decision level fusion method is demonstrated to perform the best for 3D pose estimation.

A simple feature combination method based on dominant sets

Feature combination is a popular method for improving object classification performances. In this paper we present a simple and effective weighting scheme for feature combination based on the dominant-set notion of a cluster. Specifically, we use dominant sets clustering to evaluate how accurate a kernel matrix is expected to be for a SVM classifier. This expected kernel accuracy reflects the discriminative power of the kernel matrix and thus used in weighting the kernel matrix in feature combination. Our method is simple, intuitive, memory and computation efficient, and performs comparably to the popular and sophisticated optimization based methods. We conduct experiments with several datasets of diverse object types and validate the effectiveness of the proposed method. In fact, in five out of the six datasets used in our experiments, we obtained the best results until now in our knowledge.

COMBINING MULTIPLE SENSOR FEATURES FOR STRESS DETECTION USING COMBINATORIAL FUSION

Physiological sensors have been used to detect different stress levels in order to improve human health and well-being. When analyzing these sensor data, sensor features are generated in the experiment and a subset of the features are selected and then combined using a host of informatics techniques (machine learning, data mining, or information fusion). Our previous work studied feature selection using correlation and diversity as well as feature combination using five methods C4.5, Naïve Bayes, Linear Discriminant Function, Support Vector Machine, and k-Nearest Neighbors. In this paper, we use combinatorial fusion, based on performance criterion (CF-P) and cognitive diversity (CF-CD), to combine those multiple sensor features. Our results showed that: (a) sensor feature combination method is distinctly much better than CF-CD and other algorithms, and (b) CF-CD is as good as other five feature combination methods, and is better in most of the cases.

Phase AutoCorrelation (PAC) features for noise robust speech recognition

In this paper, we introduce a new class of noise robust features derived from an alternative measure of autocorrelation representing the phase variation of speech signal frame over time. These features, referred to as Phase AutoCorrelation (PAC) features include PAC-spectrum and PAC-MFCC, among others. In traditional autocorrelation, correlation between two time delayed signal vectors is computed as their dot product. Whereas in PAC, angle between the vectors in the signal vector space is used to compute the correlation. PAC features are more noise robust because the angle is typically less affected by noise than the dot product. However, the use of angle as correlation estimate makes the PAC features inferior in clean speech. In this paper, we circumvent this problem by introducing another set of features where complementary information among the PAC features and the traditional features are combined adaptively to retain the best of both. An entropy based feature combination method in a multi-layer perceptron (MLP) based multi-stream framework is used to derive an adaptively combined representation of the component feature streams. An evaluation of the combined features using OGI Numbers95 database and Aurora-2 database under various noise conditions and noise levels show significant improvements in recognition accuracies in clean as well as noisy conditions.