Multi-channel Feature Research Articles

Blind image quality assessment in multiple bandpass and redundancy domains

Blind image quality assessment (BIQA) aims to automatically evaluate image quality without any prior knowledge of reference images and distortion types. Most of existing BIQA methods use certain probability distribution models to capture the natural scene statistics features of images in bandpass domains which can be viewed as a process of removing redundancy. There also exist methods which apply NSS features in redundancy domain. In this paper, we propose a novel method that employs both bandpass and redundancy domains to acquire the complementary features in multiple color spaces. Furthermore, hierarchical feature extraction strategy is adopted to make the image representation more powerful. Then we stack them as a multi-channel feature maps group, and use Gaussian mixture model to fit them. Finally, Fisher Vectors are used to encode them and a support vector regression model is trained as the quality predictor. Extensive experiments on four commonly evaluated image quality assessment benchmark databases show the proposed method is very competitive against other BIQA methods and has good generalization ability.

Pedestrian Detection by Using Weighted Channel Features with Hierarchical Region Reduction

Pedestrian detection in real time to avoid collision for unmanned vehicles is an interesting and challenging problem in computer vision. This paper proposes a new pedestrian detection method by using an appearance-based multi-channel features. The method involves only the monocular environment since most cameras have only a single lens. A pedestrian is represented by the combination of channel features partly weighted according to the clearly visual appearance and occlusion. Handling partial occlusion is carried out by constructing a hierarchical region reduction structure. A full pedestrian image is disintegrated into several horizontal and vertical regions. Each region captures the outstanding appearance of pedestrian’s body. The features extracted from all regions are hierarchically combined to perform the concurrent detection of pedestrian’s occurrence. The experiment yielded good results using standard benchmark dataset. The performance evaluations on miss rate, average false positive per image, and the trade-off between running time and performance show that the proposed detection framework is a reasonable option for real world applications.

Robust facial expression recognition using RGB-D images and multichannel features

Traditional and classical methods of facial expression recognition are mainly based on intensity image and are prone to be disturbed by illumination, poses, and disguise. This research aims to develop a robust facial expression recognition method using RGB-D images and multichannel features. Based on image entropy and visual saliency, facial texture features are firstly extracted from RGB images and depth images to construct the Histogram of Oriented Gradient (HOG) descriptors. And then, we extract geometric features of RGB images using Active Appearance Model (AAM). Combining the HOG texture features with the AAM geometric feature, we build a robust multichannel feature vector for facial expression recognition. On this basis, an improved Support Vector Machine (SVM) algorithm, namely GS-SVM, is used to classify facial expression recognition. The proposed GS-SVM algorithm applies Grid Search method to optimize the best parameters for SVM classifier and estimate the accuracy of each parameter combination in specified range. Finally, the proposed methods are tested and evaluated on the merged RGB-D database. Experimental results show that the proposed algorithm not only achieves a higher average recognition rate but also is robust to uncontrolled environments.

Learning Support Correlation Filters for Visual Tracking.

For visual tracking methods based on kernel support vector machines (SVMs), data sampling is usually adopted to reduce the computational cost in training. In addition, budgeting of support vectors is required for computational efficiency. Instead of sampling and budgeting, recently the circulant matrix formed by dense sampling of translated image patches has been utilized in kernel correlation filters for fast tracking. In this paper, we derive an equivalent formulation of a SVM model with the circulant matrix expression and present an efficient alternating optimization method for visual tracking. We incorporate the discrete Fourier transform with the proposed alternating optimization process, and pose the tracking problem as an iterative learning of support correlation filters (SCFs). In the fully-supervision setting, our SCF can find the globally optimal solution with real-time performance. For a given circulant data matrix with n2 samples of n ×n pixels, the computational complexity of the proposed algorithm is O(n2 logn) whereas that of the standard SVM-based approaches is at least O(n4). In addition, we extend the SCF-based tracking algorithm with multi-channel features, kernel functions, and scale-adaptive approaches to further improve the tracking performance. Experimental results on a large benchmark dataset show that the proposed SCF-based algorithms perform favorably against the state-of-the-art tracking methods in terms of accuracy and speed.

Efficient collaborative filtering recommendations with multi-channel feature vectors

Feature vectors and similarity measures are the two key issues of most existing collaborative filtering (CF) algorithms. In item-based CF algorithms, the feature vector is often defined as the ratings of all users for a given item. For a recommender system with n users, m items, and c ratings, the length of the feature vector is n; hence, the time complexity of the similarity computation is O(n). Consequently, the overall time complexity is $$O(m^2n^2)$$ , which may be computationally prohibitive for recommender systems with millions of users. In this paper, we define the multi-channel feature vector (MCFV), which is a vector of channel length c, and calculate the similarity between items using the respective MCFVs. Each element of an MCFV corresponds to the number of users with respective ratings for the item. The time complexity for the similarity computation is O(c), and the overall time complexity is $$O(m^2nc)$$ when the k-nearest neighbors and weighted average algorithms are used. Experiments were conducted on four movie recommender systems, where n ranges from a few hundred to half a million, and c is five. Results show that the recommendation algorithms using our new similarity measure are significantly faster than their counterparts without sacrificing prediction accuracy in terms of mean absolute error and root mean square error.

Electroencephalographic feature evaluation for improving personal authentication performance

Abstract Electroencephalography (EEG), a method of continuously recording the electrical activity of the brain, provides signals that are among the most promising types of information usable in vital biometrics. However, reliable biometrics based on EEG are still under development since it remains unclear how to extract EEG features that can be used to identify individuals the most effectively. In this study, new EEG features for use in biometrics were proposed and their effectiveness for personal authentication was demonstrated using an open-access EEG database containing 109 personal EEG datasets. From the EEG signals, we extracted 10 single-channel features (seven spectral and three nonlinear) by performing spectral and nonlinear analyses and 10 multichannel features by conducting network analysis based on phase synchronization. A distance-based classifier was built based on the extracted features to distinguish the self from the others. The performance of the proposed personal authentication scheme was assessed in terms of the equal error rate (EER) and false rejection rate (FRR) when the false acceptance rate (FAR) was fixed at 1%. The EER was 0.73% with the eyes open (REO) and 1.80% with the eyes closed (REC), and the FRR with a 1% FAR was 1.10% (REO) and 2.20% (REC). These results are superior to those of previous studies in which the same database was used. In addition, the nonlinear and network features appeared more important than the spectral features for authentication. This method of utilizing EEG features for personal authentication is expected to facilitate the advancement of EEG-based biometric systems.

Prediction of protein function using a deep convolutional neural network ensemble

BackgroundThe availability of large databases containing high resolution three-dimensional (3D) models of proteins in conjunction with functional annotation allows the exploitation of advanced supervised machine learning techniques for automatic protein function prediction.MethodsIn this work, novel shape features are extracted representing protein structure in the form of local (per amino acid) distribution of angles and amino acid distances, respectively. Each of the multi-channel feature maps is introduced into a deep convolutional neural network (CNN) for function prediction and the outputs are fused through support vector machines or a correlation-based k-nearest neighbor classifier. Two different architectures are investigated employing either one CNN per multi-channel feature set, or one CNN per image channel.ResultsCross validation experiments on single-functional enzymes (n = 44,661) from the PDB database achieved 90.1% correct classification, demonstrating an improvement over previous results on the same dataset when sequence similarity was not considered.DiscussionThe automatic prediction of protein function can provide quick annotations on extensive datasets opening the path for relevant applications, such as pharmacological target identification. The proposed method shows promise for structure-based protein function prediction, but sufficient data may not yet be available to properly assess the method’s performance on non-homologous proteins and thus reduce the confounding factor of evolutionary relationships.

Vehicle Detection by a Context-Aware Multichannel Feature Pyramid

The majority of the existing appearance-based vehicle-detection systems make use of local features for detection purposes, such as Haar-like, histograms of oriented gradients, scale-invariant feature transform, and so forth. However, these local features have limitations when dealing with illuminations, scale, shape variations, and complex background situations. It is desirable for a vehicle to have discriminative and robust features. For this purpose, a novel context-aware multichannel feature pyramid has been proposed in this paper. The main contribution of this paper is proposing two context-aware structural descriptors, termed as a context-aware difference sign transform feature and context-aware difference magnitude transform feature. An image has been tiled with a dense grid of the cells, and each cell is described by both local details and context-aware structural descriptors. The context-aware structural descriptors have the ability to capture the context-aware structural information of cells. The proposed context-aware multichannel feature pyramid is able to provide more effective features for vehicle detection. The results of the two public traffic analysis datasets show that the proposed approach leads to better performances when compared with the current state-of-the-art methods. Moreover, the experimental results are determined to be satisfactory, even for the images containing vehicles that have undergone scale variations and camera viewpoint changes, as well as for images that were photographed with complex backgrounds.

Visual tracking utilizing robust complementary learner and adaptive refiner

In recent years, several methods have been developed to utilize complementary cues for robust visual tracking. However, the robustness of these trackers is still limited in challenging scenarios such as dramatic illumination variation and heavy occlusion, since the corrupted cues are extracted in these cases. In this paper, we propose a visual tracking algorithm by utilizing robust Complementary Learner and adaptive Refiner (CLR). The complementary learners consist of a correlation filter and a statistical color model. To improve the robustness of the complementary learners, we introduce an accurate correlation filter with the proposed multi-channel feature. The devised effective feature is computed on both brightness channel and nonparametric local rank transformed channel at different resolutions of local cells. We further adopt a feature dimension reduction strategy to boost the tracking speed. Moreover, we investigate the problem of heavy occlusion and train an online SVM as a tracking result refiner to deal with this problem. The refiner model updates an incorrect prediction to a reliable position in case of low reliability of the current tracking result, thereby significantly reducing the risk of model drift risk as well as realizing robust tracking. We extensively evaluate the proposed CLR tracker on the 50 sequences of the OTB-2013 benchmark dataset. The experimental results demonstrate that the proposed tracker outperforms other state-of-the-art methods with real-time speed.

Room-localized spoken command recognition in multi-room, multi-microphone environments

The paper focuses on the design of a practical system pipeline for always-listening, far-field spoken command recognition in everyday smart indoor environments that consist of multiple rooms equipped with sparsely distributed microphone arrays. Such environments, for example domestic and multi-room offices, present challenging acoustic scenes to state-of-the-art speech recognizers, especially under always-listening operation, due to low signal-to-noise ratios, frequent overlaps of target speech, acoustic events, and background noise, as well as inter-room interference and reverberation. In addition, recognition of target commands often needs to be accompanied by their spatial localization, at least at the room level, to account for users in different rooms, providing command disambiguation and room-localized feedback. To address the above requirements, the use of parallel recognition pipelines is proposed, one per room of interest. The approach is enabled by a room-dependent speech activity detection module that employs appropriate multichannel features to determine speech segments and their room of origin, feeding them to the corresponding room-dependent pipelines for further processing. These consist of the traditional cascade of far-field spoken command detection and recognition, the former based on the detection of “activating” key-phrases. Robustness to the challenging environments is pursued by a number of multichannel combination and acoustic modeling techniques, thoroughly investigated in the paper. In particular, channel selection, beamforming, and decision fusion of single-channel results are considered, with the latter performing best. Additional gains are observed, when the employed acoustic models are trained on appropriately simulated reverberant and noisy speech data, and are channel-adapted to the target environments. Further issues investigated concern the inter-dependencies of the various system components, demonstrating the superiority of joint optimization of the component tunable parameters over their separate or sequential optimization. The proposed approach is developed for the Greek language, exhibiting promising performance in real recordings in a four-room apartment, as well as a two-room office. For example, in the latter, a 76.6% command recognition accuracy is achieved on a speaker-independent test, employing a 180-sentence decoding grammar. This result represents a 46% relative improvement over conventional beamforming.

Automated Detection of Red Lesions Using Superpixel Multichannel Multifeature.

Red lesions can be regarded as one of the earliest lesions in diabetic retinopathy (DR) and automatic detection of red lesions plays a critical role in diabetic retinopathy diagnosis. In this paper, a novel superpixel Multichannel Multifeature (MCMF) classification approach is proposed for red lesion detection. In this paper, firstly, a new candidate extraction method based on superpixel is proposed. Then, these candidates are characterized by multichannel features, as well as the contextual feature. Next, FDA classifier is introduced to classify the red lesions among the candidates. Finally, a postprocessing technique based on multiscale blood vessels detection is modified for removing nonlesions appearing as red. Experiments on publicly available DiaretDB1 database are conducted to verify the effectiveness of our proposed method.

Fire Detection Using Multi-Channel Information and Gray Level Co-occurrence Matrix Image Features

Fire Detection Using Multi-Channel Information and Gray Level Co-occurrence Matrix Image Features

Multi-Channel Features Spatio-Temporal Context Learning for Visual Tracking

Visual tracking is a challenging issue in surveillance, human–computer Interaction, and intelligent robotics, among others. Managing appearance changes of the target object, illumination changes, rotations, non-rigid deformations, partial or full occlusions, background clutter, fast motion, and so forth is generally difficult. Among the numerous existing trackers, the correlation-filter-based tracker can achieve appealing performance with a fast speed for fast Fourier transform. Motivated by this property, the spatio-temporal context (STC) learning algorithm was proposed with the consideration of the information from the context around the target, and this algorithm achieved good results. However, STC only utilizes the overall intensity information. In this paper, we propose a multi-channel features STC learning algorithm with an improved scale-adaptive scheme. Our algorithm integrates powerful features, including Histogram of Oriented Gradients and color naming, using kernel methods on the basis of the STC algorithm to further enhance the overall tracking performance. Extensive experimental results obtained from various benchmark data sets demonstrate that the proposed tracker is promising for various challenging scenarios and maintains real-time performance at an average speed of 78 frames/s. According to the test results, our algorithm outperforms the STC algorithm and achieves performance that is competitive with the state-of-the-art algorithms.

An information fusion framework with multi-channel feature concatenation and multi-perspective system combination for the deep-learning-based robust recognition of microphone array speech

We present an information fusion approach to the robust recognition of multi-microphone speech. It is based on a deep learning framework with a large deep neural network (DNN) consisting of subnets designed from different perspectives. Multiple knowledge sources are then reasonably integrated via an early fusion of normalized noisy features with multiple beamforming techniques, enhanced speech features, speaker-related features, and other auxiliary features concatenated as the input to each subnet to compensate for imperfect front-end processing. Furthermore, a late fusion strategy is utilized to leverage the complementary natures of the different subnets by combining the outputs of all subnets to produce a single output set. Testing on the CHiME-3 task of recognizing microphone array speech, we demonstrate in our empirical study that the different information sources complement each other and that both early and late fusions provide significant performance gains, with an overall word error rate of 10.55% when combining 12 systems. Furthermore, by utilizing an improved technique for beamforming and a powerful recurrent neural network (RNN)-based language model for rescoring, a WER of 9.08% can be achieved for the best single DNN system with one-pass decoding among all of the systems submitted to the CHiME-3 challenge.

A Low Cost VLSI Architecture for Spike Sorting Based on Feature Extraction with Peak Search.

The goal of this paper is to present a novel VLSI architecture for spike sorting with high classification accuracy, low area costs and low power consumption. A novel feature extraction algorithm with low computational complexities is proposed for the design of the architecture. In the feature extraction algorithm, a spike is separated into two portions based on its peak value. The area of each portion is then used as a feature. The algorithm is simple to implement and less susceptible to noise interference. Based on the algorithm, a novel architecture capable of identifying peak values and computing spike areas concurrently is proposed. To further accelerate the computation, a spike can be divided into a number of segments for the local feature computation. The local features are subsequently merged with the global ones by a simple hardware circuit. The architecture can also be easily operated in conjunction with the circuits for commonly-used spike detection algorithms, such as the Non-linear Energy Operator (NEO). The architecture has been implemented by an Application-Specific Integrated Circuit (ASIC) with 90-nm technology. Comparisons to the existing works show that the proposed architecture is well suited for real-time multi-channel spike detection and feature extraction requiring low hardware area costs, low power consumption and high classification accuracy.

Monocular visual-IMU odometry using multi-channel image patch exemplars

In this paper, we propose three sets of multi-channel image patch features for monocular visual-IMU (Inertial Measurement Unit) odometry. The proposed feature sets extract image patch exemplars from multiple feature maps of an image. We also modify an existing visual-IMU odometry framework by using different salient point detectors and feature sets and replacing the inlier selection approach with a self-adaptive scheme. The modified framework is used to examine the proposed feature sets. In addition to the Root Mean Square Error (RMSE) metric, we use the Hausdorff distance to measure the inconsistency between the estimated and ground-truth trajectories. Compared to the point-wise comparison used by RMSE, the Hausdorff distance takes the shape inconsistency of two trajectories into account and is hence more perceptually consistent. Experimental results show that the multi-channel feature sets outperform, or perform comparably to, the single gray level channel feature sets examined in this study. Particularly, the multi-channel feature set that uses integral channels, i.e., ICIMGP (Integral Channel Image Patches), outperforms two state-of-the-art feature sets: SIFT (Scale Invariant Feature Transform) and SURF (Speed Up Robust Features). Besides, ICIMGP performs better than the two multi-channel feature sets that are designed based on derivative channels and gradient channels respectively. These promising results are attributed to the fact that the multi-channel features encode richer image characteristics than their single gray level channel counterparts.

CLASSIFICATION OF URBAN AERIAL DATA BASED ON PIXEL LABELLING WITH DEEP CONVOLUTIONAL NEURAL NETWORKS AND LOGISTIC REGRESSION

The recent success of deep convolutional neural networks (CNN) on a large number of applications can be attributed to large amounts of available training data and increasing computing power. In this paper, a semantic pixel labelling scheme for urban areas using multi-resolution CNN and hand-crafted spatial-spectral features of airborne remotely sensed data is presented. Both CNN and hand-crafted features are applied to image/DSM patches to produce per-pixel class probabilities with a <i>L</i><sub>1</sub>-norm regularized logistical regression classifier. The evidence theory infers a degree of belief for pixel labelling from different sources to smooth regions by handling the conflicts present in the both classifiers while reducing the uncertainty. The aerial data used in this study were provided by ISPRS as benchmark datasets for 2D semantic labelling tasks in urban areas, which consists of two data sources from LiDAR and color infrared camera. The test sites are parts of a city in Germany which is assumed to consist of typical object classes including impervious surfaces, trees, buildings, low vegetation, vehicles and clutter. The evaluation is based on the computation of pixel-based confusion matrices by random sampling. The performance of the strategy with respect to scene characteristics and method combination strategies is analyzed and discussed. The competitive classification accuracy could be not only explained by the nature of input data sources: e.g. the above-ground height of nDSM highlight the vertical dimension of houses, trees even cars and the nearinfrared spectrum indicates vegetation, but also attributed to decision-level fusion of CNN’s texture-based approach with multichannel spatial-spectral hand-crafted features based on the evidence combination theory.

Blind Image Quality Assessment Based on Multichannel Feature Fusion and Label Transfer

In this paper, we propose an efficient blind image quality assessment (BIQA) algorithm, which is characterized by a new feature fusion scheme and a ${k}$ -nearest-neighbor (KNN)-based quality prediction model. Our goal is to predict the perceptual quality of an image without any prior information of its reference image and distortion type. Since the reference image is inaccessible in many applications, the BIQA is quite desirable in this context. In our method, a new feature fusion scheme is first introduced by combining an image’s statistical information from multiple domains (i.e., discrete cosine transform, wavelet, and spatial domains) and multiple color channels (i.e., Y, Cb, and Cr). Then, the predicted image quality is generated from a nonparametric model, which is referred to as the label transfer (LT). Based on the assumption that similar images share similar perceptual qualities, we implement the LT with an image retrieval procedure, where a query image’s KNNs are searched for from some annotated images. The weighted average of the KNN labels (e.g., difference mean opinion score or mean opinion score) is used as the predicted quality score. The proposed method is straightforward and computationally appealing. Experimental results on three publicly available databases (i.e., LIVE II, TID2008, and CSIQ) show that the proposed method is highly consistent with human perception and outperforms many representative BIQA metrics.

Pedestrian Detection Based on Informed Haar-like Features and Switchable Deep Network

As pedestrians usually appear up-right in image or video data, we therefore employ a statistical model of the up-right human body where the head, the upper body, and the lower body are treated as three distinct components. As we incorporate different kinds of low-level measurements, the resulting multi-modal & multi-channel Haar-like features represent characteristic differences between parts of the human body yet are robust against variations in clothing or environmental settings. Then we use a Switchable Deep Network(SDN) for pedestrian detection. The SDN automatically learns features of different body parts. Experimental results on many pedestrian datasets show that the proposed algorithm significantly improves the detection rates at 0.1FPPI compared with the state-of-the-art domain adaptation methods and that it is robust and accurate against cluttered dynamical background, occlusion and the object deformation.

Joint collaborative representation based sleep stage classification with multi-channel EEG signals.

Multi-channel electroencephalography (EEG) signals have been effectively used for sleet staging. However, it is still a challenge to effectively fuse and represent multi-channel EEG features. The coding based feature representation methods, such as sparse representation (SR), have achieved great success in computer vision and pattern recognition. Collaborative representation (CR) is a new coding method, which effectively works as a classifier. In this work, we first employ CR as a feature representation method. Moreover, a new joint CR (JCR) model is proposed for fusing multi-view data, which can represent not only the individual view information, but also the inner-correlative information between multi-views. JCR method is then applied to fuse and represent the features of multi-channel EEG signals for the classification of sleep stages. The experimental results indicate that CR feature outperforms SR feature, and JCR achieves best performance for sleep stage classification by effectively fusing multi-channel EEG signals.