Cross-view Quadratic Discriminant Analysis Research Articles

HAREDNet: A deep learning based architecture for autonomous video surveillance by recognizing human actions

Human Action Recognition (HAR) is still considered as a significant research area due to its emerging real-time applications like video surveillance, automated surveillance, real-time tracking and resecue missions. HAR domain still have gaps to cover, i.e., random changes in human variations, clothes, illumination, and backgrounds. Different camera settings, viewpoints and inter-class similarities have increased the complexity of this domain. The above-mentioned challenges in uncontrolled environment have ultimately reduced the performances of many well-designed models. The primary objective of this research is to propose and design an automated recognition system by overcoming these afore-mentioned issues. Redundant features and excessive computational time for the training and prediction process has also been a noteworthy problem. In this article, a hybrid recognition technique called HAREDNet is proposed, which has a) Encoder-Decoder Network (EDNet) to extract deep features; b) improved Scale-Invariant Feature Transform (iSIFT), improved Gabor (iGabor) and Local Maximal Occurrence (LOMO) techniques to extract local features; c) Cross-view Quadratic Discriminant Analysis (CvQDA) algorithm to reduce the feature redundancy; and d) weighted fusion strategy to merge properties of different essential features. The proposed technique is evaluated on three (3) publicly available datasets, including NTU RGB+D, HMDB51, and UCF-101, and achieved average recognition accuracy of 97.45%, 80.58%, and 97.48%, respectively, which is better than previously proposed methods.

Sleep Staging Using Plausibility Score: A Novel Feature Selection Method Based on Metric Learning.

As an effective method, feature selection can reduce computational complexity and improve classification performance. A number of criteria exist for feature selection using labeled data, unlabeled data and pairwise constraints, most of which are based on the Euclidean distance. In this paper, we propose a filter method for feature selection with pairwise constraints, aiming to jointly evaluate a feature subset based on metric learning. Two criteria are designed based on the well-known Kullback-Leibler divergence for measuring the difference between must-link constraints and cannot-link constraints that can indicate the feature subset discrimination based on Keep It Simple and Straightforward (KISS) metric learning and Cross-view Quadratic Discriminant Analysis (XQDA) metric learning. To address the challenging feature selection problem, we formulate a sequential search algorithm guided by indicators that are simplified from the proposed criteria. Furthermore, we conducted several experiments on sleep staging based on electroencephalogram (EEG) recordings from the Sleep-EDF Database Expanded. The experimental results demonstrate the effectiveness of the proposed method compared with nine representative feature selection methods. On the data set from healthy volunteers and the data set from volunteers that had mild difficulty falling asleep, the classification average accuracies achieve 97.66% and 93.57% by using the proposed method, respectively.

Robust Person Re-Identification Through the Combination of Metric Learning and Late Fusion Techniques

Fusion techniques with the aim to leverage the discriminative power of different appearance features for person representation have been widely applied in person re-identification. They are performed by concatenating all feature vectors (known as early fusion) or by combining matching scores of different classifiers (known as late fusion). Previous studies have proved that late fusion techniques achieve better results than early fusion ones. However, majority of the studies focus on determining the suitable weighting schemes that can reflect the role of each feature. The determined weights are then integrated in conventional similarity functions, such as Cosine [L. Zheng, S. Wang, L. Tian, F. He, Z. Liu and Q. Tian, Queryadaptive late fusion for image search and person reidentification, in Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2015, pp. 1741–1750]. The contribution of this paper is two-fold. First, a robust person re-identification method by combining the metric learning with late fusion techniques is proposed. The metric learning method Cross-view Quadratic Discriminant Analysis (XQDA) is employed to learn a discriminant low dimensional subspace to minimize the intra-person distance while maximize the inter-person distance. Moreover, product rule-based and sum rule-based late fusion techniques are applied on these distances. Second, concerning feature engineering, the ResNet extraction process has been modified in order to extract local features of different stripes in person images. To show the effectiveness of the proposed method, both single-shot and multi-shot scenarios are considered. Three state-of-the-art features that are Gaussians of Gaussians (GOG), Local Maximal Occurrence (LOMO) and deep-learned features extracted through a Residual network (ResNet) are extracted from person images. The experimental results on three benchmark datasets that are iLIDS-VID, PRID-2011 and VIPeR show that the proposed method [Formula: see text]% [Formula: see text]% of improvement over the best results obtained with the single feature. The proposed method that achieves the accuracy of 85.73%, 93.82% and 50.85% at rank-1 for iLIDS-VID, PRID-2011 and VIPeR, respectively, outperforms different SOTA methods including deep learning ones. Source code is publicly available to facilitate the development of person re-ID system.

A deep multi-feature distance metric learning method for pedestrian re-identification

Consider the problem that handcrafted features are limited by not being directly applicable to practical problems. Additionally, the deep convolution feature is a high-dimensional feature, and if it is directly used to match the image, it will consume considerable time and memory. Moreover, features from higher levels may be contaminated by dramatic variations in the human pose or background clutter. This paper proposes a method based on deep multi-feature distance metric learning. First, each spatial position and channel are weighted after extracting the deep convolution feature from the last layer of the CNN, and the final aggregation result, that is, the feature of the image, is obtained by sum-pooling. Second, a new method to improve and integrate the convolution feature of the region is proposed. The convolution features are processed by the sliding frame technique, and the low-dimensional eigenvector with dimensions equal to the number of convolution layer channels is obtained. Third, a distance learning algorithm is proposed by cross-view quadratic discriminant analysis metric learning. Finally, the weighted fusion strategy is used to accomplish the collaboration between the handcrafted and deep convolution features. On the Market-1501 and VIPeR datasets, the experimental results show that the rank 1 values of the proposed method on three experimental datasets reach 90.02% and 68.74%, respectively. Under the new classification rules of the CHUK03 dataset, the rank 1 performance of the proposed method reaches 34.2%. The experimental results show that the accuracy of pedestrian re-identification after distance-weighted fusion is higher than that obtained by the separate feature distance metric.

Multilinear subspace learning using handcrafted and deep features for face kinship verification in the wild

In this paper, we propose a new multilinear and multiview subspace learning method called Tensor Cross-view Quadratic Discriminant Analysis for face kinship verification in the wild. Most of the existing multilinear subspace learning methods straightforwardly focus on learning a single set of projection matrices, making it difficult to separate different classes. To address this issue, the proposed approach mutually learns multi-view representations for multidimensional cross-view matching. In order to decrease the effect of the within class variations for each mode of the tensor data, the proposed approach integrates the Within Class Covariance Normalization. Moreover, we propose a new tensor face descriptor based on the Gabor wavelets. Besides, we investigate the complementarity of handcrafted and deep face tensor features via their fusion at score level using the Logistic Regression method. Our extensive experiments demonstrate that the proposed kinship verification framework outperforms the state of the art, achieving 95.14%, 91.83% and 93.58% verification accuracies on Cornell KinFace, UB KinFace and TSKinFace face kinship datasets, respectively.

Person re-identification using prioritized chromatic texture (PCT) with deep learning

Person re-identification (re-ID) helps to identify a person’s attention in different cameras. But this is not an easy task, due to distance, illumination and lack of dataset. Nowadays, this field attracts many researchers because of its varied applications. Here, the information of both local texture and global color representations are concatenated with an original raw image. This concatenated information is gathered by finding the maximum value of chrominance in terms of HSV, texture in terms of Scale Invariant Local Ternary Pattern (SILTP) for each pixel and original raw image. SILTP is well known for its illumination invariant texture description. Convolutional Neural Network (CNN) is used in the proposed work to extract the features from the concatenated information. The proposed Prioritized Chromatic Texture Image (PCTimg) is concatenated with original raw image and fed into CNN. Here, finally a six dimensionalfeature is fed into CNN to extract the deep features. Cross-view Quadratic Discriminant Analysis (XQDA) similarity metric algorithm is employed to re-identify a person.Multiscale Retinex algorithm is used for pre-processing the images.To address the challenges in terms of view point deflection, a sliding window is formed for describing local details of a person in the SILTP feature extraction phase. The HSV helps to incorporate the human color perception. The triplet loss function is used to learn the similarity and the dissimilarity of the training images. The performance analysis of the proposed work is improvedwhen compared to the existingworks.

Research on the Pedestrian Re-identification Method Based on Local Features and Gait Energy Images

The appearance of pedestrians can vary greatly from image to image, and different pedestrians may look similar in a given image. Such similarities and variabilities in the appearance and clothing of individuals make the task of pedestrian re-identification very challenging. Here, a pedestrian re-identification method based on the fusion of local features and gait energy image (GEI) features is proposed. In this method, the human body is divided into four regions according to joint points. The color and texture of each region of the human body are extracted as local features, and GEI features of the pedestrian gait are also obtained. These features are then fused with the local and GEI features of the person. Independent distance measure learning using the cross-view quadratic discriminant analysis (XQDA) method is used to obtain the similarity of the metric function of the image pairs, and the final similarity is acquired by weight matching. Evaluation of experimental results by cumulative matching characteristic (CMC) curves reveals that, after fusion of local and GEI features, the pedestrian reidentification effect is improved compared with existing methods and is notably better than the recognition rate of pedestrian re-identification with a single feature.

Human action recognition based on convolutional neural network and spatial pyramid representation

Detecting and recognizing human action in natural scenarios, such as indoor and outdoor, is a significant technique in computer vision and intelligent systems, which is widely applied in video surveillance, pedestrian tracking and human-computer interaction. Conventional approaches have been proposed based on various features and achieved impressive performance. However, these methods failed to cope with partial occlusion and changes of posture. In order to address these limitations, we propose a novel human action recognition method. More specifically, in order to capture image spatial composition, we leverage a three-level spatial pyramid feature extraction scheme, where each pyramid is encoded by local features. Thereafter, regions generated by a proposal algorithm are fed into a dual-aggregation net for deep representation extraction. Afterwards, both local features and deep features are fused to describe each image. To describe human action category, we design a metric CXQDA based on Cosine measure and Cross-view Quadratic Discriminant Analysis (XQDA) to calculate the similarity among different action categories. Experimental results demonstrate that our proposed method can effectively cope with object scale variations, partial occlusion and achieve competitive performance.

Tensor cross-view quadratic discriminant analysis for kinship verification in the wild

This paper presents a new Tensor Cross-view Quadratic Discriminant Analysis (TXQDA) method based on the XQDA method for kinship verification in the wild. Many researchers used metric learning methods and have achieved reasonably good performance in kinship verification, none of these methods looks at the kinship verification as a cross-view matching problem. To tackle this issue, we propose a tensor cross-view method to train multilinear data using local histograms of local features descriptors. Therefore, we learn a hierarchical tensor transformation to project each pair face images into the same implicit feature space, in which the distance of each positive pair is minimized and that of each negative pair is maximized. Moreover, TXQDA was proposed to separate the multifactor structure of face images (i.e. kinship, age, gender, expression, illumination and pose) from different dimensions of the tensor. Thus, our TXQDA achieves better classification results through discovering a lowdimensional tensor subspace that enlarges the margin of different kin relation classes. Experimental evaluation on five challenging databases namely Cornell KinFace, UB KinFace, TSKinFace, KinFaceW-II and FIW databases, show that the proposed TXQDA significantly outperforms the current state of the art.

Effective multi-shot person re-identification through representative frames selection and temporal feature pooling

Multi-shot person re-identification (ReID) is a popular case of person ReID in which a set of images are processed for each person. However, using entire image set for person ReID as most experimented proposals is not always effective because of time and memory consuming. The main contribution of this work is the proposed strategies for (1) choosing representative image frames for each individual instead of entire set of frames, and (2) temporal feature pooling in multi-shot person ReID. These strategies are efficiently integrated in a person ReID framework which uses GoG (Gaussian of Gaussian) and XQDA (metric learning Cross-view Quadratic Discriminant Analysis) for person representation and matching. The effectiveness of the proposed framework on two benchmark datasets (PRID 2011 and iLIDS-VID) in terms of re-identification accuracy, computational time, and storage requirements are deeply investigated and analyzed. The experimental results allow to provide several recommendations on the use of these schemes based on the characteristics of the working dataset and the requirement of the applications. Furthermore, the study also offers a desktop-based application for person search and ReID. The implementation of the proposed framework will be made publicly available.

Minimum equalization for pedestrain re-identification

In order to solve the intelligent application of pedestrain re-identification technology in the actual monitoring scenes, the color and texture features are fused and some local blocks are used to extract image features by considering the changing shooting angle of pedestrian image. A method of histogram equalization before texture feature extraction is proposed to solve the problem of unclear pedestrian contour. By down sampling the image twice, the algorithm has better scale invariance. Compared with the existing re-identification method of cross-view quadratic discriminant analysis (XQDA) combined with local maximal occurrence (LOMO) characteristics, the experimental results show that the corresponding re-identification rate rank1 on the datasets of VIPeR, PUK-Reid and i-LIDS-VID is improved by 0.28%, 1.75% and 0.20%, respectively, which proves that the recognition rate of pedestrain re-identification with minimum equalization is improved.

An improved method for Person Re-Identification

In this paper, an effective feature representation named color pairs is proposed, which is robust to illumination and viewpoint changes. It was combined with Scale-invariant local ternary pattern (SILTP) texture features used for person re-identification. And we improved the method of Cross-view Quadratic Discriminant Analysis (XQDA) by adding a close function. It improves the performance of discriminant matric. This method were compared with other hand-crafted methods of feature extraction and measurement matching, then was proved to be advanced.

Deep Multi-View Feature Learning for Person Re-Identification

Person re-identification aims to identify the same pedestrians across different camera views at different locations. This important yet difficult intelligent video analysis problem remains a vigorous area of research due to demands for performance improvements. Person re-identification involves two main steps: feature representation and metric learning. Handcrafted features, such as color and texture histograms, are frequently used for person re-identification, but most handcrafted features are limited by not being directly applicable to practical problems. Deep learning methods have obtained the state-of-the-art performance in a wide variety of applications, including image annotation, face recognition, and speech recognition. However, deep learning features are heavily dependent on large-scale labeling of samples. In this paper, by utilizing the Cross-view Quadratic Discriminant Analysis (XQDA) metric learning, we propose a novel scheme called deep multi-view feature learning (DMVFL), which exploits the collaboration between handcrafted and deep learning features in a simple but effective way. Furthermore, we prove that the XQDA is a robust algorithm. Extensive experiments on two challenging person re-identification data sets (VIPeR and GRID) demonstrate that DMVFL improves on current state-of-the-art methods.

Normalized distance aggregation of discriminative features for person reidentification

We propose an effective person reidentification method based on normalized distance aggregation of discriminative features. Our framework is built on the integration of three high-performance discriminative feature extraction models, including local maximal occurrence (LOMO), feature fusion net (FFN), and a concatenation of LOMO and FFN called LOMO–FFN, through two fast and discriminant metric learning models, i.e., cross-view quadratic discriminant analysis (XQDA) and large-scale similarity learning (LSSL). More specifically, we first represent all the cross-view person images using LOMO, FFN, and LOMO–FFN, respectively, and then apply each extracted feature representation to train XQDA and LSSL, respectively, to obtain the optimized individual cross-view distance metric. Finally, the cross-view person matching is computed as the sum of the optimized individual cross-view distance metric through the min–max normalization. Experimental results have shown the effectiveness of the proposed algorithm on three challenging datasets (VIPeR, PRID450s, and CUHK01).

Spatial-temporal representatives selection and weighted patch descriptor for person re-identification

Abstract How to represent the sequential person images is a crucial issue in multi-shot person re-identification. In this paper, we propose to select the spatial-temporal informative representatives to describe the image sequence. Specifically, we address representatives selection as a row-sparsity regularized minimization problem which can be effectively solved via convex programming. The sparsity of the representatives is controlled by a regularization parameter based on both spatial and temporal dissimilarities. Furthermore, we design a weighted patch descriptor by employing the random walk with restart model to propagate the patch weights on the person image. Finally, we utilize the cross-view quadratic discriminant analysis as the metric learning to mitigate the cross-view gaps among different cameras. Extensive experiments on three benchmark datasets iLIDS-VID, PRID 2011 and SAIVT-SoftBio demonstrate the promising performance of the proposed method.

Person Re-Identification Using Hybrid Representation Reinforced by Metric Learning

Person Re-Identification (Re-Id) is among the main constituents of an automated visual surveillance system. It aims at finding out true matches of a given query person from a large repository of non-overlapping camera images/videos. In this paper, we have proposed an efficient Re-Id approach that is based on a highly discriminative hybrid person representation which combines the low-level hand-crafted appearance based features together with the mid-level attributes and semantic based deep features. The low-level hand crafted features are extracted by using hierarchical Gaussian and local histogram distributions in different color spaces. These features incorporate discriminative texture, shape and color information which is invariant to distractors, e.g., variations in pose, viewpoint and illumination, and so on. The mid-level attribute based deep features are extracted to incorporate contextual- and semantic-based information. The feature space is optimized and self-learned using cross-view quadratic discriminant analysis and multiple metric learning, with the aim to reduce the intra-class differences and increase the inter-class variations for robust person matching. The proposed framework is evaluated on publicly available small scale (VIPeR, PRID450s, and GRID) and large scale (CUHK01, Market1501, and DukeMTMC-ReID) person Re-Id datasets. The experimental results show that the hybrid hand-crafted and deep features outperformed the existing state-of-the-art in approaches in the unsupervised paradigm.

Multiple metric learning with query adaptive weights and multi-task re-weighting for person re-identification

Metric learning has been widely studied in person re-identification (re-id). However, most existing metric learning methods only learn one holistic Mahalanobis distance metric for the concatenated high dimensional feature. This single metric learning strategy cannot handle complex nonlinear data structure and may easily encounter overfitting. Besides, feature concatenation is incapable of exploring the discriminant capability of different features and low dimensional features tend to be dominated by high dimensional ones. Motivated by these problems, we propose a multiple metric learning method for the re-id problem, where individual sub-metrics are separately learned for each feature type and the final metric is formed as weighted sum of the sub-metrics. The sub-metrics are learned with the Cross-view Quadratic Discriminant Analysis (XQDA) algorithm and the weights to each sub-metric are assigned in a two-step procedure. First, the importance of each feature type is estimated according to its discriminative power, which is measured in a query adaptive manner as related to the partial Area Under Curve (pAUC) scores. Then, the weights of all feature types are learned simultaneously with a maximum-margin based multi-task structural SVM learning framework, in order to make sure that relevant gallery images are ranked before irrelevant ones within all feature spaces. Finally, the sub-metrics are integrated with the learned weights in an ensemble model, generating a sophisticated distance metric. Experiments on the challenging i-LIDS, VIPeR, CAVIAR and 3DPeS datasets demonstrate the effectiveness of the proposed method.

Person re-ID while Crossing Different Cameras: Combination of Salient-Gaussian Weighted BossaNova and Fisher Vector Encodings

Person re-identification (re-ID) is a challenging task in the camera surveillance field, since it addresses the problem of re-identifying people across multiple non-overlapping cameras. Most of existing approaches have been concentrated on: 1) achieving a robust and effective feature representation; and 2) enforcing discriminative metric learning to predict if two images represent the same identity. In this context, we present a new approach for person re-ID built upon multi-level descriptors. This is achieved by combining three complementary representations: salient-Gaussian Fisher Vector (SGFV) encoding method, salient-Gaussian BossaNova (SGBN) histogram encoding method and deep Convolutional Neural Network (CNN) features. The two first methods adapt the histogram encoding framework to the person re-ID task. This is achieved by integrating the pedestrian saliency map and the spatial location information, in the histogram encoding process. On one hand, human saliency is reliable and distinctive in the person re-ID task, since it can model the uniqueness of the identity. On the other hand, localizing a person in the image can effectively discard noisy background information. Finally, one of the most advanced metric learning in person re-ID: the Cross-view Quadratic Discriminant Analysis (XQDA) is applied on the top of the resulting description. The proposed method yields promising person re-ID results on two challenging image-based person re-ID benchmarks: CUHK03 and Market-1501.