Representation Fusion Research Articles

Multiple Features With Extreme Learning Machines For Clothing Image Recognition

Clothing image recognition has recently received considerable attention from many communities, such as multimedia information processing and computer vision, due to its commercial and social applications. However, the large variations in clothing images’ appearances and styles and their complicated formation conditions make the problem challenging. In addition, a generic treatment with convolutional neural networks (CNNs) cannot provide a satisfactory solution considering the training time and recognition performance. Therefore, how to balance those two factors for clothing image recognition is an interesting problem. Motivated by the fast training and straightforward solutions exhibited by extreme learning machines (ELMs), in this paper, we propose a recognition framework that is based on multiple sources of features and ELM neural networks. In this framework, three types of features are first extracted, including CNN features with pre-trained networks, histograms of oriented gradients and color histograms. Second, those low-level features are concatenated and taken as the inputs to an autoencoder version of the ELM for deep feature-level fusion. Third, we propose an ensemble of adaptive ELMs for decision-level fusion using the previously obtained feature-level fusion representations. Extensive experiments are conducted on an up-to-date large-scale clothing image data set. Those experimental results show that the proposed framework is competitive and efficient.

Prediction for Membrane Protein Types Based on Effective Fusion Representation and MIC-GA Feature Selection

Membrane proteins occupy an important position in the life activities of humans and other species. The elucidation of membrane protein types provides clues for understanding the structure and function of proteins. With the fusion of various protein information including amino acid classification, physicochemical property, and evolutionary information, this paper proposes a system for predicting membrane protein types. In this system, a new feature selection method called MIC-GA is proposed to deal with the curse of high-dimensional features. The findings show that this approach is effective in reducing feature dimensions and improves prediction accuracy. Ensemble method based on stacked generalization is also used to solve the problem of feature heterogeneity. The performance of the present method is evaluated on two benchmark datasets. The overall prediction accuracies of eight types are 89.23% and 93.49% using jackknife test and independent test, respectively. The final experimental results show that our method is more effective than the existing methods for prediction of membrane protein types.

Integration of GIS and Moving Objects in Surveillance Video

This paper discusses the integration of a geographic information system (GIS) and moving objects in surveillance videos (“moving objects” hereinafter) by using motion detection, spatial mapping, and fusion representation techniques. This integration aims to overcome the limitations of conventional video surveillance systems, such as low efficiency in video searching, redundancy in video data transmission, and insufficient capability to position video content in geographic space. Furthermore, a model for integrating GIS and moving objects is established. The model includes a moving object extraction method and a fusion pattern for GIS and moving objects. From the established integration model, a prototype of GIS and moving objects (GIS–MOV) system is constructed and used to analyze the possible applications of the integration of GIS and moving objects.

Sparse representation-based 3D model retrieval

In this study, we leveraged the sparse representation for multi-modal information fusion to handle 3D model retrieval problem. First, SIFT feature is extracted to represent the visual appearance of 2D view images for each 3D models. With this low-level feature representation, the Latent Dirichlet Allocation model is learned to generate the high-level & discriminative visual representation for individual 3D model. Then, we utilize the sparse representation framework to handle the key problem, the similarity measure between two different 3D models, for model retrieval. The performance of the proposed method is evaluated on the novel MV-RED 3D object dataset, which contains both RGB and depth 3D model data. The comparison experiments demonstrate the proposed sparse representation-based framework can benefit from multi-modal information fusion and consequently augment the performance.

Multimodal Sentiment Intensity Analysis in Videos: Facial Gestures and Verbal Messages

People share their opinions, stories, and reviews through online video sharing websites every day. The automatic analysis of these online opinion videos is bringing new or understudied research challenges to the field of computational linguistics and multimodal analysis. Among these challenges is the fundamental question of exploiting the dynamics between visual gestures and verbal messages to be able to better model sentiment. This article addresses this question in four ways: introducing the first multimodal dataset with opinion-level sentiment intensity annotations; studying the prototypical interaction patterns between facial gestures and spoken words when inferring sentiment intensity; proposing a new computational representation, called multimodal dictionary, based on a language-gesture study; and evaluating the authors' proposed approach in a speaker-independent paradigm for sentiment intensity prediction. The authors' study identifies four interaction types between facial gestures and verbal content: neutral, emphasizer, positive, and negative interactions. Experiments show statistically significant improvement when using multimodal dictionary representation over the conventional early fusion representation (that is, feature concatenation).

Feature Fusion Based SVM Classifier for Protein Subcellular Localization Prediction

Summary For the importance of protein subcellular localization in different branch of life science and drug discovery, researchers have focused their attentions on protein subcellular localization prediction. Effective representation of features from protein sequences plays most vital role in protein subcellular localization prediction specially in case of machine learning technique. Single feature representation like pseudo amino acid composition (PseAAC), physiochemical property model (PPM), amino acid index distribution (AAID) contains insufficient information from protein sequences. To deal with such problem, we have proposed two feature fusion representations AAIDPAAC and PPMPAAC to work with Support Vector Machine classifier, which fused PseAAC with PPM and AAID accordingly. We have evaluated performance for both single and fused feature representation of Gram-negative bacterial dataset. We have got at least 3% more actual accuracy by AAIDPAAC and 2% more locative accuracy by PPMPAAC than single feature representation.

Protein Sub-Nuclear Localization Based on Effective Fusion Representations and Dimension Reduction Algorithm LDA

An effective representation of a protein sequence plays a crucial role in protein sub-nuclear localization. The existing representations, such as dipeptide composition (DipC), pseudo-amino acid composition (PseAAC) and position specific scoring matrix (PSSM), are insufficient to represent protein sequence due to their single perspectives. Thus, this paper proposes two fusion feature representations of DipPSSM and PseAAPSSM to integrate PSSM with DipC and PseAAC, respectively. When constructing each fusion representation, we introduce the balance factors to value the importance of its components. The optimal values of the balance factors are sought by genetic algorithm. Due to the high dimensionality of the proposed representations, linear discriminant analysis (LDA) is used to find its important low dimensional structure, which is essential for classification and location prediction. The numerical experiments on two public datasets with KNN classifier and cross-validation tests showed that in terms of the common indexes of sensitivity, specificity, accuracy and MCC, the proposed fusing representations outperform the traditional representations in protein sub-nuclear localization, and the representation treated by LDA outperforms the untreated one.

Face recognition using part-based dense sampling local features

For years, researchers have made great efforts to find an appropriate face representation for face recognition. A fusion strategy of Local Binary Pattern (LBP) and Gabor filters yields great achievements. LBP is good at coding fine details of facial appearance and texture, whereas Gabor features can encode facial shape and appearance over a range of coarser scales. Despite the great performance, this fusion representation suffers from low effectiveness and resolution variance. In this paper, we propose a novel representation strategy of face images which is fast and robust to resolution variance. We apply dense sampling around each detected feature point, extract Local Difference Feature (LDF) for face representation, then utilize Principal Component Analysis (PCA)+Linear Discriminant Analysis (LDA) to reduce feature dimension and finally use cosine similarity evaluation for recognition. We have utilized our proposed face representation strategy on two databases, namely self-collected Second Generation ID Card of China and Driver׳s License (SGIDCDL) database and public Facial Recognition Technology (FERET) database. Our experimental results show that the proposed strategy has good performance on face recognition with fast speed.

Structure-Preserving Binary Representations for RGB-D Action Recognition.

In this paper, we propose a novel binary local representation for RGB-D video data fusion with a structure-preserving projection. Our contribution consists of two aspects. Toacquire a general feature for the video data, we convert the problem to describing the gradient fields of RGB and depth information of video sequences. With the local fluxes of the gradient fields, which include the orientation and the magnitude of the neighborhood of each point, a new kind of continuous local descriptor called Local Flux Feature(LFF) is obtained. Then the LFFs from RGB and depth channels are fused into a Hamming space via the Structure Preserving Projection (SPP). Specifically, an orthogonal projection matrix is applied to preserve the pairwise structure with a shape constraint to avoid the collapse of data structure in the projected space. Furthermore, a bipartite graph structure of data is taken into consideration, which is regarded as a higher level connection between samples and classes than the pairwise structure of local features. Theextensive experiments show not only the high efficiency of binary codes and the effectiveness of combining LFFs from RGB-D channels via SPP on various action recognition benchmarks of RGB-D data, but also the potential power of LFF for general action recognition.

A Spline-Based Trajectory Representation for Sensor Fusion and Rolling Shutter Cameras

The use of multiple sensors for ego-motion estimation is an approach often used to provide more accurate and robust results. However, when representing ego-motion as a discrete series of poses, fus...

Video object matching across multiple non-overlapping camera views based on multi-feature fusion and incremental learning

Matching objects across multiple cameras with non-overlapping views is a necessary but difficult task in the wide area video surveillance. Owing to the lack of spatio-temporal information, only the visual information can be used in some scenarios, especially when the cameras are widely separated. This paper proposes a novel framework based on multi-feature fusion and incremental learning to match the objects across disjoint views in the absence of space–time cues. We first develop a competitive major feature histogram fusion representation (CMFH11CMFH is the abbreviation of Competitive Major Feature Histogram fusion representation.) to formulate the appearance model for characterizing the potentially matching objects. The appearances of the objects can change over time and hence the models should be continuously updated. We then adopt an improved incremental general multicategory support vector machine algorithm (IGMSVM22IGMSVM is the abbreviation of Incremental General Multicategory Support Vector Machine learning algorithm.) to update the appearance models online and match the objects based on a classification method. Only a small amount of samples are needed for building an accurate classification model in our method. Several tests are performed on CAVIAR, ISCAPS and VIPeR databases where the objects change significantly due to variations in the viewpoint, illumination and poses. Experimental results demonstrate the advantages of the proposed methodology in terms of computational efficiency, computation storage, and matching accuracy over that of other state-of-the-art classification-based matching approaches. The system developed in this research can be used in real-time video surveillance applications.

Integrating Structural and Functional Imaging for Computer Assisted Detection of Prostate Cancer on Multi-Protocol In Vivo 3 Tesla MRI.

Screening and detection of prostate cancer (CaP) currently lacks an image-based protocol which is reflected in the high false negative rates currently associated with blinded sextant biopsies. Multi-protocol magnetic resonance imaging (MRI) offers high resolution functional and structural data about internal body structures (such as the prostate). In this paper we present a novel comprehensive computer-aided scheme for CaP detection from high resolution in vivo multi-protocol MRI by integrating functional and structural information obtained via dynamic-contrast enhanced (DCE) and T2-weighted (T2-w) MRI, respectively. Our scheme is fully-automated and comprises (a) prostate segmentation, (b) multimodal image registration, and (c) data representation and multi-classifier modules for information fusion. Following prostate boundary segmentation via an improved active shape model, the DCE/T2-w protocols and the T2-w/ex vivo histological prostatectomy specimens are brought into alignment via a deformable, multi-attribute registration scheme. T2-w/histology alignment allows for the mapping of true CaP extent onto the in vivo MRI, which is used for training and evaluation of a multi-protocol MRI CaP classifier. The meta-classifier used is a random forest constructed by bagging multiple decision tree classifiers, each trained individually on T2-w structural, textural and DCE functional attributes. 3-fold classifier cross validation was performed using a set of 18 images derived from 6 patient datasets on a per-pixel basis. Our results show that the results of CaP detection obtained from integration of T2-w structural textural data and DCE functional data (area under the ROC curve of 0.815) significantly outperforms detection based on either of the individual modalities (0.704 (T2-w) and 0.682 (DCE)). It was also found that a meta-classifier trained directly on integrated T2-w and DCE data (data-level integration) significantly outperformed a decision-level meta-classifier, constructed by combining the classifier outputs from the individual T2-w and DCE channels.

QuickBird Image Fusion by a Multirresolution-Multidirectional Joint Image Representation

A new fusion methodology for Multiespectral and Pancromatic images, has been proposed. This methodology is based on a joint multiresolution-multidirectional representation of the source images. For that an unique directional filters bank of low computacional complexity has been used.This kind of image representation allows an appropriated selection of the information extracted from the source images, avoiding some of the limitations inherent to other multiresolution fusion methods. The final aim is to obtain fused images with a high spectral and spatial quality simultaneously. The source images corresponds to the captured by Quickbird satellite (panchromatic and multispectral). The high quality of the obtained results shows the potential of the joint multiresolution-multidirectional representation for images fusion.

YinYang bipolar logic and bipolar fuzzy logic

It is observed that equilibrium (including quasi- or non-equilibrium) is natural reality or bipolar truth. It is asserted that a multiple valued logic is a finite-valued extension of Boolean logic; a fuzzy logic is a real-valued extension of Boolean logic; Boolean logic and its extensions are unipolar systems that cannot be directly used to represent bipolar truth for visualization. To circumvent the representational limitations of unipolar systems, a zero-order (propositional) bipolar combinational logic BCL 1 in the bipolar space B 1={−1,0}×{0,1} is upgraded to a first-order (predicate) bipolar logic. BCL 1 is then extended to an ( n+1) 2-valued crisp bipolar combinational logic BCL n in the bipolar space B n ={− n,…,−2,−1,0}×{0,1,2,…, n} and a real-valued bipolar fuzzy logic BCL F in the bipolar space B F=[−1,0]×[0,1]. A bipolar counterpart of unipolar axioms and rules of inference is introduced with the addition of bipolar augmentation. First-order consistency and completeness are proved. Depolarization functions are identified for the recovery of BCL 1, BCL n , and BCL F to Boolean logic, a ( n+1)-valued logic, and fuzzy logic, respectively. Thus, BCL 1, BCL n , and BCL F are bipolar generalizations or fusions of Boolean logic, multiple valued logic, and fuzzy logic, respectively. The bipolar family of systems provides a unique representation for bipolar knowledge fusion and visualization in an equilibrium world. The semantics of the bipolar systems are established, justified, and compared with unipolar systems. A redress is presented for the ancient paradox of the liar that leads to a few comments on Gödel's incompleteness theorem.

Barrier distributions and scattering

The extraction of representations of the fusion barrier distribution from backward-angle, quasi-elastic, elastic and transfer excitation functions is discussed. Such excitation functions have been measured for , and projectiles incident on a variety of targets. The results are compared with representations obtained from fusion excitation functions. Varying in their sensitivity, all representations show evidence of the barrier structure. Differences between the scattering and the fusion representations can be related to the effects of coupling to residual, weak reaction channels.