Single Feature Representation Research Articles

Pulmonary CT Registration Network Based on Deformable Cross Attention.

Current Transformer structure utilizes the self-attention mechanism to model global contextual relevance within image, which makes an impact on medical image registration. However, the use of Transformer in handling large deformation lung CT registration is relatively straightforwardly. These models only focus on single image feature representation neglecting to employ attention mechanism to capture the across image correspondence. This hinders further improvement in registration performance. To address the above limitations, we propose a novel registration method in a cascaded manner, Cascaded Swin Deformable Cross Attention Transformer based U-shape structure (SD-CATU), to address the challenge of large deformations in lung CT registration. In SD-CATU, we introduce a Cross Attention-based Transformer (CAT) block that incorporates the Shifted Regions Multihead Cross-attention (SR-MCA) mechanism to flexibly exchange feature information and thus reduce the computational complexity. Besides, a consistency constraint in the loss function is used to ensure the preservation of topology and inverse consistency of the transformations. Experiments with public lung datasets demonstrate that the Cascaded SD-CATU outperforms current state-of-the-art registration methods (Dice Similarity Coefficient of 93.19% and Target registration error of 0.98mm). The results further highlight the potential for obtaining excellent registration accuracy while assuring desirable smoothness and consistency in the deformed images.

Multi-level feature fusion network for neuronal morphology classification.

Neuronal morphology can be represented using various feature representations, such as hand-crafted morphometrics and deep features. These features are complementary to each other, contributing to improving performance. However, existing classification methods only utilize a single feature representation or simply concatenate different features without fully considering their complementarity. Therefore, their performance is limited and can be further improved. In this paper, we propose a multi-level feature fusion network that fully utilizes diverse feature representations and their complementarity to effectively describe neuronal morphology and improve performance. Specifically, we devise a Multi-Level Fusion Module (MLFM) and incorporate it into each feature extraction block. It can facilitate the interaction between different features and achieve effective feature fusion at multiple levels. The MLFM comprises a channel attention-based Feature Enhancement Module (FEM) and a cross-attention-based Feature Interaction Module (FIM). The FEM is used to enhance robust morphological feature presentations, while the FIM mines and propagates complementary information across different feature presentations. In this way, our feature fusion network ultimately yields a more distinctive neuronal morphology descriptor that can effectively characterize neurons than any singular morphological representation. Experimental results show that our method effectively depicts neuronal morphology and correctly classifies 10-type neurons on the NeuronMorpho-10 dataset with an accuracy of 95.18%, outperforming other approaches. Moreover, our method performs well on the NeuronMorpho-12 and NeuronMorpho-17 datasets and possesses good generalization.

AbFTNet: An Efficient Transformer Network with Alignment before Fusion for Multimodal Automatic Modulation Recognition

Multimodal automatic modulation recognition (MAMR) has emerged as a prominent research area. The effective fusion of features from different modalities is crucial for MAMR tasks. An effective multimodal fusion mechanism should maximize the extraction and integration of complementary information. Recently, fusion methods based on cross-modal attention have shown high performance. However, they overlook the differences in information intensity between different modalities, suffering from quadratic complexity. To this end, we propose an efficient Alignment before Fusion Transformer Network (AbFTNet) based on an in-phase quadrature (I/Q) and Fractional Fourier Transform (FRFT). Specifically, we first align and correlate the feature representations of different single modalities to achieve mutual information maximization. The single modality feature representations are obtained using the self-attention mechanism of the Transformer. Then, we design an efficient cross-modal aggregation promoting (CAP) module. By designing the aggregation center, we integrate two modalities to achieve the adaptive complementary learning of modal features. This operation bridges the gap in information intensity between different modalities, enabling fair interaction. To verify the effectiveness of the proposed methods, we conduct experiments on the RML2016.10a dataset. The experimental results show that multimodal fusion features significantly outperform single-modal features in classification accuracy across different signal-to-noise ratios (SNRs). Compared to other methods, AbFTNet achieves an average accuracy of 64.59%, with a 1.36% improvement over the TLDNN method, reaching the state of the art.

Fundus-DeepNet: Multi-label deep learning classification system for enhanced detection of multiple ocular diseases through data fusion of fundus images

Detecting multiple ocular diseases in fundus images is crucial in ophthalmic diagnosis. This study introduces the Fundus-DeepNet system, an automated multi-label deep learning classification system designed to identify multiple ocular diseases by integrating feature representations from pairs of fundus images (e.g., left and right eyes). The study initiates with a comprehensive image pre-processing procedure, including circular border cropping, image resizing, contrast enhancement, noise removal, and data augmentation. Subsequently, discriminative deep feature representations are extracted using multiple deep learning blocks, namely the High-Resolution Network (HRNet) and Attention Block, which serve as feature descriptors. The SENet Block is then applied to further enhance the quality and robustness of feature representations from a pair of fundus images, ultimately consolidating them into a single feature representation. Finally, a sophisticated classification model, known as a Discriminative Restricted Boltzmann Machine (DRBM), is employed. By incorporating a Softmax layer, this DRBM is adept at generating a probability distribution that specifically identifies eight different ocular diseases. Extensive experiments were conducted on the challenging Ophthalmic Image Analysis-Ocular Disease Intelligent Recognition (OIA-ODIR) dataset, comprising diverse fundus images depicting eight different ocular diseases. The Fundus-DeepNet system demonstrated F1-scores, Kappa scores, AUC, and final scores of 88.56 %, 88.92 %, 99.76 %, and 92.41 % in the off-site test set, and 89.13 %, 88.98 %, 99.86 %, and 92.66 % in the on-site test set.In summary, the Fundus-DeepNet system exhibits outstanding proficiency in accurately detecting multiple ocular diseases, offering a promising solution for early diagnosis and treatment in ophthalmology.

TCRNet: A Trifurcated Cascaded Refinement Network for Salient Object Detection

Due to the rapid development of deep learning, salient object detection has achieved promising results in recent years. However, how to construct a powerful saliency detection network to generate saliency maps with completely highlighted salient objects and effectively suppressed background noise still remains an open and challenging problem. In this paper, we propose a novel trifurcated cascaded refinement network (TCRNet) to explore multi-level feature fusion and global information representation. Specifically, all the side-output features from the backbone network are divided into three branches (i.e., global information, deep information and shallow information) according to feature similarities, and then the global contexts, semantic knowledge and fined details respectively generated by the three branches are integrated in a cascaded refinement way. This refinement strategy aims to guide the feature learning of important regions and suppress background noises. Moreover, we propose a cross-level feature fusion (CFF) module to fuse multi-level similar features within a same branch by exploring their interrelationship, thus capturing the important semantic information and structure details. To this end, the complementarity of similar features is utilized to refine feature at each level by sharing an encoded spatial weight map. Besides, we design a context-aware feature learning (CFL) module to capture global contexts in a single layer by taking account of both single-feature representation and multi-feature interaction, thus providing important location information of salient objects. Extensive experiments on five benchmarks are conducted to demonstrate that the proposed network TCRNet performs favorably against 20 state-of-the-art salient object detection methods on five benchmark datasets, indicating the effectiveness and superiority of our method.

Classification from Sky: A Robust Remote Sensing Time Series Image Classification Using Spatial Encoder and Multi-Fast Channel Attention

The unparalleled availability of Satellite Image Time Series (SITS) for crop phenology classification unravels agricultural parcel observation and monitoring with applications of both economic and ecological importance. Moreover, the need for distinct classification of agricultural parcels into individual crop types falls on state-of-the-art deep learning models for this extrinsic task. However, most existing approaches implemented are complex and ineffective attention incorporated models, which in turn lack the resilience to recognize useful bands in achieving greater accuracy. We propose a Multi-Fast Channel Attention module for deep CNNs based on a Spatial Encoder (SE-MFCA) that requires a few parameters while enhancing the performance-complexity trade-off dilemma. Hence, we leverage on spatial encoder module to extract the images as disorderly sets of pixels to enhance the coarse spatial resolution features. We empirically show that appropriate parameter sharing in the cross channel interaction can preserve performance while significantly reducing model complexity. The proposed multi-channel attention module can efficiently be implemented via an encoder-decoder network to prevent the loss of detailed spatial information. Again, we parallelly distributed the input channel into multiple heads in our network to recover the specialized input features, which will concatenate with the residual to form a rich single feature representation. The extensive experiment has shown that our model SE-MFCA is efficient and effective compared with the previous state-of-the-art time series classification algorithm on a publicly available dataset of Sentinel-2 images for agricultural parcels. Performance-wise SE-MFCA achieves the highest overall accuracy of 94.50% and the highest mean intersection over union score of 51.92%, besides the least trainable params of 131 K and fewer floating point operations of 0.16 M.

Multi-attention representation network partial domain adaptation for COVID-19 diagnosis

The outbreak of COVID-19 threatens the safety of all human beings. Rapid and accurate diagnosis of patients is the effective way to prevent the rapid spread of COVID-19. The current computer-aided diagnosis of COVID-19 requires extensive labeled data for training, and this undoubtedly increases human and material resources costs. Domain adaptation (DA), an existing promising approach, can transfer knowledge from rich labeled pneumonia datasets for COVID-19 diagnosis and classification. However, due to the differences in feature distribution and task semantic between pneumonia and COVID-19, negative transfer may reduce the performance in diagnosis COVID-19 and pneumonia. Furthermore, the training data is usually mixed with many noise samples in practice, and this also poses new challenges for domain adaptation. As a kind of domain adaptation, partial domain adaptation (PDA) can well avoid outlier samples in the source domain and achieve good classification performance in the target domain. However, the existing PDA methods all learn a single feature representation; this can only learn local information about the inputs and ignore other important information in the samples. Therefore multi-attention representation network partial domain adaptation (MARPDA) is proposed in this paper to overcome the above shortcomings of PDA. In MARPDA, we construct the multiple representation networks with attention to acquire the image representation and effectively learn knowledge from different feature spaces. We design the sample-weighted strategy to achieve partial data transfer and address the negative transfer of noise data during training. MARPDA adapts to complex application scenarios and learns fine-grained features of the image from multiple representations. We apply the model to classify pneumonia and COVID-19 respectively, and evaluate it in qualitative and quantitative manners. The experimental results show that our classification accuracy is higher than that of the existing state-of-the-art methods. The stability and reliability of the proposed method are validated by the confusion matrix and the performance curves experiments. In summary, our method has better performance for diagnosis COVID-19 compared to the existing state-of-the-art methods.

Partly Uncoupled Siamese Model for Change Detection from Heterogeneous Remote Sensing Imagery

This paper addresses the problematic of detecting changes in bitemporal heterogeneous remote sensing image pairs. In different disciplines, multimodality is the key solution for performance enhancement in a collaborative sensing context. Particularly, in remote sensing imagery there is still a research gap to fill with the multiplication of sensors, along with data sharing capabilities, and multitemporal data availability. This study is aiming to explore the multimodality in a multi-temporal set-up for a better understanding of the collaborative sensor wide information completion; we propose a pairwise learning approach consisting on a pseudo-Siamese network architecture based on two partly uncoupled parallel network streams. Each stream represents itself a Convolutional Neural Network (CNN) that encodes the input patches. The overall Change Detector (CD) model includes a fusion stage that concatenates the two encodings in a single multimodal feature representation which is then reduced to a lower dimension using fully connected layers and finally a loss function based on the binary cross entropy is used as a decision layer. The proposed pseudo-Siamese pairwise learning architecture allows to the CD model to capture the spatial and the temporal dependencies between multimodal input image pairs. The model processes the two multimodal input patches at one-time under different spatial resolutions. The evaluation performances on different real multimodal datasets reflecting a mixture of CD conditions with different spatial resolutions, confirm the effectiveness of the proposed CD architecture.

MalFCS: An effective malware classification framework with automated feature extraction based on deep convolutional neural networks

Identifying the family of malware can determine their malicious intent and attack patterns, which helps to efficiently analyze large numbers of malware variants. Methods based on traditional machine learning often require a lot of time and resources in feature engineering. Virtually all existing static analysis methods based on malware visualization are derived from grayscale images, while a single low-order feature representation may be detrimental to discovering hidden features in a malware family. Based on these problems, this paper proposes an effective malware classification framework (MalFCS) based on malware visualization and automated feature extraction. MalFCS includes mainly three modules: malware visualization, feature extraction, and classification. First, we visualize malware binaries as entropy graphs based on structural entropy. Second, we present a feature extractor based on deep convolutional neural networks to extract patterns shared by a family from entropy graphs automatically. Finally, we propose an SVM classifier to classify malware based on the extracted features. We evaluate the proposed MalFCS over two widely studied benchmark datasets, i.e., Malimg and Microsoft. Experimental results show that compared with the state-of-the-art methods, MalFCS can obtain excellent classification performance with accuracy of 0.997 and 1, respectively, achieving the state-of-the-art performance.

Vision tracking based on adaptive interactive fusion

Under the influence of COVID-19, detection and identification of moving targets are very important for personnel management. A lot of research work has improved the accuracy and robustness of the moving target tracking method, but the recognition accuracy of the traditional target tracking method in complex scenes (lighting changes, background interference, posture changes and other factors) is not satisfactory. In this paper, in view of the limitations of single feature representation of target objects, the method of fusion of HSV color features and edge direction features is used to identify and detect moving targets. In each frame of the tracking process, the weight of each feature is adjusted adaptively according to the proposed fusion strategy, and the position of the target is located by using the method of double template matching. Experiments show that the proposed tracking algorithm based on multi feature fusion can meet the requirements of moving target recognition in complex scenes. The method proposed in this paper has a certain reference value for personnel management under the influence of COVID-19.

Feature and Model Level Fusion of Pretrained CNN for Remote Sensing Scene Classification

Convolutional neural networks (CNN) have attracted tremendous attention in the remote sensing community due to its excellent performance in different domains. Especially for remote sensing scene classification, the CNN-based methods have brought a great breakthrough. However, it is not feasible to fully design and train a new CNN model for remote sensing scene classification, as this usually requires a large number of training samples and high computational costs. To alleviate these limitations of fully training a new model, some work attempts to use the pretrained CNN models as feature extractors to build feature representation of scene images for classification and has achieved impressive results. In this scheme, how to construct feature representation of scene image via the pretrained CNN model becomes the key process. Existing studies paid a little attention to build more discriminative feature representation by exploring the potential benefits of multilayer features from a single CNN model and different feature representations from multiple CNN models. To this end, this paper presents a fusion strategy to build the feature representation of the scene images by integrating multilayer features of a single pretrained CNN model, and extends it to a framework of multiple CNN models. For these purposes, a multiscale improved Fisher kernel coding method is used to build feature representation of the scene images on convolutional layers, and a feature fusion approach based on two feature subspace learning methods [principal component analysis (PCA)/spectral regression kernel discriminant analysis and PCA/spectral regression kernel locality preserving projection] is proposed to construct final fused features for scene classification. For validation and comparison purposes, the proposed approaches are evaluated with two challenging high-resolution remote sensing datasets and shows the competitive performance compared with existing state-of-the-art baselines such as fully trained CNN models, fine tuning CNN models, and other related works.

Group-Wise Learning for Aurora Image Classification With Multiple Representations.

In conventional aurora image classification methods, it is general to employ only one single feature representation to capture the morphological characteristics of aurora images, which is difficult to describe the complicated morphologies of different aurora categories. Although several studies have proposed to use multiple feature representations, the inherent correlation among these representations are usually neglected. To address this problem, we propose a group-wise learning (GWL) method for the automatic aurora image classification using multiple representations. Specifically, we first extract the multiple feature representations for aurora images, and then construct a graph in each of multiple feature spaces. To model the correlation among different representations, we partition multiple graphs into several groups via a clustering algorithm. We further propose a GWL model to automatically estimate class labels for aurora images and optimal weights for the multiple representations in a data-driven manner. Finally, we develop a label fusion approach to make a final classification decision for new testing samples. The proposed GWL method focuses on the diverse properties of multiple feature representations, by clustering the correlated representations into the same group. We evaluate our method on an aurora image data set that contains 12 682 aurora images from 19 days. The experimental results demonstrate that the proposed GWL method achieves approximately 6% improvement in terms of classification accuracy, compared to the methods using a single feature representation.

Local Large-Margin Multi-Metric Learning for Face and Kinship Verification

Metric learning has attracted wide attention in face and kinship verification, and a number of such algorithms have been presented over the past few years. However, most existing metric learning methods learn only one Mahalanobis distance metric from a single feature representation for each face image and cannot make use of multiple feature representations directly. In many face-related tasks, we can easily extract multiple features for a face image to extract more complementary information, and it is desirable to learn distance metrics from these multiple features, so that more discriminative information can be exploited than those learned from individual features. To achieve this, we present a large-margin multi-metric learning (LM3L) method for face and kinship verification, which jointly learns multiple global distance metrics under which the correlations of different feature representations of each sample are maximized, and the distance of each positive pair is less than a low threshold and that of each negative pair is greater than a high threshold. To better exploit the local structures of face images, we also propose a local metric learning and local LM3Lmethods to learn a set of local metrics. Experimental results on three face data sets show that the proposed methods achieve very competitive results compared with the state-of-the-art methods.

Protein subnuclear localization based on a new effective representation and intelligent kernel linear discriminant analysis by dichotomous greedy genetic algorithm.

A wide variety of methods have been proposed in protein subnuclear localization to improve the prediction accuracy. However, one important trend of these means is to treat fusion representation by fusing multiple feature representations, of which, the fusion process takes a lot of time. In view of this, this paper novelly proposed a method by combining a new single feature representation and a new algorithm to obtain good recognition rate. Specifically, based on the position-specific scoring matrix (PSSM), we proposed a new expression, correlation position-specific scoring matrix (CoPSSM) as the protein feature representation. Based on the classic nonlinear dimension reduction algorithm, kernel linear discriminant analysis (KLDA), we added a new discriminant criterion and proposed a dichotomous greedy genetic algorithm (DGGA) to intelligently select its kernel bandwidth parameter. Two public datasets with Jackknife test and KNN classifier were used for the numerical experiments. The results showed that the overall success rate (OSR) with single representation CoPSSM is larger than that with many relevant representations. The OSR of the proposed method can reach as high as 87.444% and 90.3361% for these two datasets, respectively, outperforming many current methods. To show the generalization of the proposed algorithm, two extra standard datasets of protein subcellular were chosen to conduct the expending experiment, and the prediction accuracy by Jackknife test and Independent test is still considerable.

Detection Potential of Multi-Features Representation of E-Nose Data in Classification of Moldy Maize Samples

In order to assess rapidly and timely the moldy degree of maize samples using electronic nose (E-nose) and improve the correct classification rate of E-nose, the different feature representation modes (DFRM) for E-nose data were explored in depth. A determining method for multi-features vector of E-nose based on Wilks Λ statistic was introduced so as to obtain the best multi-features vector for characterizing E-nose data. And then a selection method of representation features of each sensor signals based on elimination transform with pivoting of the Λ statistic was also introduced for the different excitation characteristic of each gas sensor. The research results show that the classification effect of multi-features representation mode (MFRM) is better than that of single feature representation mode (SFRM), and the MFRM is not a regular pattern, but the best multi-features vector of E-nose in MFRM can be obtained by the determining method. Moreover, it is necessary to select the representation features of each sensor signals in the MFRM using the selection method. The visual inspection results based on SFRM and MFRM were examined by Fisher discriminant analysis (FDA) and proved that the introduced methods were very effective, the highest correct discrimination rate based on SFRM is 80%, while the correct discrimination rate of the five features combination is 97%. As an outlook, we believe that the research findings may be universally applied for the classification of other food and agriculture products using E-nose.

Manifold regularized multi-view feature selection for social image annotation

The features used in many social media analysis-based applications are usually of very high dimension. Feature selection offers several advantages in highly dimensional cases. Recently, multi-task feature selection has attracted much attention, and has been shown to often outperform the traditional single-task feature selection. Current multi-task feature selection methods are either supervised or unsupervised. In this paper, we address the semi-supervised multi-task feature selection problem. We firstly introduce manifold regularization in multi-task feature selection to utilize the limited number of labeled samples and the relatively large amount of unlabeled samples. However, the graph constructed in manifold regularization from a single feature representation (view) may be unreliable. We thus propose to construct the graph using the heterogeneous feature representations from multiple views. The proposed method is called manifold regularized multi-view feature selection (MRMVFS), which can exploit the label information, label relationship, data distribution, as well as correlation among different kinds of features simultaneously to boost the feature selection performance. All these information are integrated into a unified learning framework to estimate feature selection matrix, as well as the adaptive view weights. Experimental results on three real-world image datasets, NUS-WIDE, Flickr and Animal, demonstrate the effectiveness and superiority of the proposed MRMVFS over other state-of-the-art feature selection methods.

Feature Fusion Based SVM Classifier for Protein Subcellular Localization Prediction

Summary For the importance of protein subcellular localization in different branch of life science and drug discovery, researchers have focused their attentions on protein subcellular localization prediction. Effective representation of features from protein sequences plays most vital role in protein subcellular localization prediction specially in case of machine learning technique. Single feature representation like pseudo amino acid composition (PseAAC), physiochemical property model (PPM), amino acid index distribution (AAID) contains insufficient information from protein sequences. To deal with such problem, we have proposed two feature fusion representations AAIDPAAC and PPMPAAC to work with Support Vector Machine classifier, which fused PseAAC with PPM and AAID accordingly. We have evaluated performance for both single and fused feature representation of Gram-negative bacterial dataset. We have got at least 3% more actual accuracy by AAIDPAAC and 2% more locative accuracy by PPMPAAC than single feature representation.

Understanding emotional impact of images using Bayesian multiple kernel learning

Affective classification and retrieval of multimedia such as audio, image, and video have become emerging research areas in recent years. The previous research focused on designing features and developing feature extraction methods. Generally, a multimedia content can be represented with different feature representations (i.e., views). However, the most suitable feature representation related to people׳s emotions is usually not known a priori. We propose here a novel Bayesian multiple kernel learning algorithm for affective classification and retrieval tasks. The proposed method can make use of different representations simultaneously (i.e., multiview learning) to obtain a better prediction performance than using a single feature representation (i.e., single-view learning) or a subset of features, with the advantage of automatic feature selections. In particular, our algorithm has been implemented within a multilabel setup to capture the correlation between emotions, and the Bayesian formulation enables our method to produce probabilistic outputs for measuring a set of emotions triggered by a single image. As a case study, we perform classification and retrieval experiments with our algorithm for predicting people׳s emotional states evoked by images, using generic low-level image features. The empirical results with our approach on the widely-used International Affective Picture System (IAPS) data set outperform several existing methods in terms of classification performance and results interpretability.

Robust object tracking via multi-feature adaptive fusion based on stability: contrast analysis

Object tracking under complex circumstances is a challenging task because of background interference, obstacle occlusion, object deformation, etc. Given such conditions, robustly detecting, locating, and analyzing a target through single-feature representation are difficult tasks. Global features, such as color, are widely used in tracking, but may cause the object to drift under complex circumstances. Local features, such as HOG and SIFT, can precisely represent rigid targets, but these features lack the robustness of an object in motion. An effective method is adaptive fusion of multiple features in representing targets. The process of adaptively fusing different features is the key to robust object tracking. This study uses a multi-feature joint descriptor (MFJD) and the distance between joint histograms to measure the similarity between a target and its candidate patches. Color and HOG features are fused as the tracked object of the joint representation. This study also proposes a self-adaptive multi-feature fusion strategy that can adaptively adjust the joint weight of the fused features based on their stability and contrast measure scores. The mean shift process is adopted as the object tracking framework with multi-feature representation. The experimental results demonstrate that the proposed MFJD tracking method effectively handles background clutter, partial occlusion by obstacles, scale changes, and deformations. The novel method performs better than several state-of-the-art methods in real surveillance scenarios.

A new discriminative coding method for image classification

The bag-of-words (BOW) based methods are widely used in image classification. However, huge number of visual information is omitted inevitably in the quantization step of the BOW. Recently, NBNN and its improved methods like Local NBNN were proposed to solve this problem. Nevertheless, these methods do not perform better than the state-of-the-art BOW based methods. In this paper, based on the advantages of BOW and Local NBNN, we introduce a novel locality discriminative coding (LDC) method. We convert each low level local feature, such as SIFT, into code vector using the Local Feature-to-Class distance other than by k-means quantization. After coding, sum-pooling combined with SPM is used to construct a single feature representation vector for each image. Extensive experimental results on several challenging benchmark datasets show that our LDC method outperforms six state-of-the-art image classification methods.