Feature Learning Network Research Articles

Differential-Augmented Current Feature Learning Network With Multi-Information Interaction for Fault Diagnosis in Electromechanical Drive System

Current-based fault diagnosis has become a promising solution for electromechanical systems due to the low cost and easy access. However, most of the existing studies require additional signal processing or diagnostic expertise to pre-process the signals due to the effects of the fundamental component and electrical noise. It is challenging to extract effective fault-related features from the raw current signals. To this end, this paper proposes a differential-augmented current feature learning network named CurrentNet for drive system fault diagnosis, which is an end-to-end model. Firstly, a differential-augmented strategy based on the raw current signal is introduced to generate complementary current representations. Furthermore, a multi-information interaction module (MIIM) is designed to adaptively capture complementary shared information between the original and enhanced signals through a parallel mechanism. Also, efficient channel attention is used to reduce the complexity of the model. Our proposed method is experimentally evaluated on two datasets and presents obvious superiority over existing fault diagnosis methods.

Multi-Scale Hybrid Spectral Network for Feature Learning and Hyperspectral Image Classification

Hyperspectral image (HSI) classification is an important concern in remote sensing, but it is complex since few numbers of labelled training samples and the high-dimensional space with many spectral bands. Hence, it is essential to develop a more efficient neural network architecture to improve performance in the HSI classification task. Deep learning models are contemporary techniques for pixel-based hyperspectral image (HSI) classification. Deep feature extraction from both spatial and spectral channels has led to high classification accuracy. Meanwhile, the effectiveness of these spatial-spectral methods relies on the spatial dimension of every patch, and there is no feasible method to determine the best spatial dimension to take into consideration. It makes better sense to retrieve spatial properties through examination at different neighborhood scales in spatial dimensions. In this context, this paper presents a multi-scale hybrid spectral convolutional neural network (MS-HybSN) model that uses three distinct multi-scale spectral-spatial patches to pull out properties in spectral and spatial domains. The presented deep learning framework uses three patches of different sizes in spatial dimension to find these possible features. The process of Hybrid convolution operation (3D-2D) is done on each selected patch and is repeated throughout the image. To assess the effectiveness of the presented model, three benchmark datasets that are openly accessible (Pavia University, Indian Pines, and Salinas) and new Indian datasets (Ahmedabad-1 and Ahmedabad-2) are being used in experimental studies. Empirically, it has been demonstrated that the presented model succeeds over the remaining state-of-the-art approaches in terms of classification performance.

Improving Face Anti-spoofing via Advanced Multi-perspective Feature Learning

Face anti-spoofing (FAS) plays a vital role in securing face recognition systems. Previous approaches usually learn spoofing features from a single perspective, in which only universal cues shared by all attack types are explored. However, such single-perspective-based approaches ignore the differences among various attacks and commonness between certain attacks and bona fides, thus tending to neglect some non-universal cues that contain strong discernibility against certain types. As a result, when dealing with multiple types of attacks, the above approaches may suffer from the uncomprehensive representation of bona fides and spoof faces. In this work, we propose a novel Advanced Multi-Perspective Feature Learning network (AMPFL), in which multiple perspectives are adopted to learn discriminative features, to improve the performance of FAS. Specifically, the proposed network first learns universal cues and several perspective-specific cues from multiple perspectives, then aggregates the above features and further enhances them to perform face anti-spoofing. In this way, AMPFL obtains features that are difficult to be captured by single-perspective-based methods and provides more comprehensive information on bona fides and spoof faces, thus achieving better performance for FAS. Experimental results show that our AMPFL achieves promising results in public databases, and it effectively solves the issues of single-perspective-based approaches.

Explore deep reinforcement learning for efficient task processing based on federated optimization in big data

In recent years, along with the extensive application of consumer electronics, the task execution with cloud computing for big data has become one of the research focuses. Nevertheless, the traditional theories and algorithms are still employed by existing research work to explore the feasible solutions, which takes a beating from low generalization performance, system load imbalance, more response delay, etc. To solve the matter, a task execution method called DROP (Deep Reinforcement network aided Optimization method aiming at task Processing) has been put forward, which is capable of completing task request allocation through virtual network embedding. The prominence of this method is explained by its effect in reducing load balancing degree, minimizing bandwidth resource overhead, and preserving electric energy as well as meeting customer demands. It makes use of Deep Deterministic Policy Gradient (DDPG) instead of depending on tons of iterations for better path selection schemes in previous methods, through continuous environment interaction and trial-and-error evaluation to get better strategy selection for virtual link embedding. To realize the virtual node embedding in the federated optimization based system architecture, the intentional deep feature learning network is applied. Compared with the cutting edge approaches, the performance benefits of DROP can be verified by the experimental results in terms of bringing down the extra cost on resources and energy of the substrate network during the task execution for big data.

DLGNet: A dual-branch lesion-aware network with the supervised Gaussian Mixture model for colon lesions classification in colonoscopy images.

Colorectal cancer is one of the malignant tumors with the highest mortality due to the lack of obvious early symptoms. It is usually in the advanced stage when it is discovered. Thus the automatic and accurate classification of early colon lesions is of great significance for clinically estimating the status of colon lesions and formulating appropriate diagnostic programs. However, it is challenging to classify full-stage colon lesions due to the large inter-class similarities and intra-class differences of the images. In this work, we propose a novel dual-branch lesion-aware neural network (DLGNet) to classify intestinal lesions by exploring the intrinsic relationship between diseases, composed of four modules: lesion location module, dual-branch classification module, attention guidance module, and inter-class Gaussian loss function. Specifically, the elaborate dual-branch module integrates the original image and the lesion patch obtained by the lesion localization module to explore and interact with lesion-specific features from a global and local perspective. Also, the feature-guided module guides the model to pay attention to the disease-specific features by learning remote dependencies through spatial and channel attention after network feature learning. Finally, the inter-class Gaussian loss function is proposed, which assumes that each feature extracted by the network is an independent Gaussian distribution, and the inter-class clustering is more compact, thereby improving the discriminative ability of the network. The extensive experiments on the collected 2568 colonoscopy images have an average accuracy of 91.50%, and the proposed method surpasses the state-of-the-art methods. This study is the first time that colon lesions are classified at each stage and achieves promising colon disease classification performance. To motivate the community, we have made our code publicly available via https://github.com/soleilssss/DLGNet.

TeethGNN: Semantic 3D Teeth Segmentation with Graph Neural Networks.

In this paper, we present TeethGNN, a novel 3D tooth segmentation method based on graph neural networks (GNNs). Given a mesh-represented 3D dental model in non-Euclidean domain, our method outputs accurate and fine-grained separation of each individual tooth robust to scanning noise, foreign matters (e.g., bubbles, dental accessories, etc.), and even severe malocclusion. Unlike previous CNN-based methods that bypass handling non-Euclidean mesh data by reshaping hand-crafted geometric features into regular grids, we explore the non-uniform and irregular structure of mesh itself in its dual space and exploit graph neural networks for effective geometric feature learning. To address the crowded teeth issues and incomplete segmentation that commonly exist in previous methods, we design a two-branch network, one of which predicts a segmentation label for each facet while the other regresses each facet an offset away from its tooth centroid. Clustering are later conducted on offset-shifted locations, enabling both the separation of adjoining teeth and the adjustment of incompletely segmented teeth. Exploiting GNN for directly processing mesh data frees us from extracting hand-crafted feature, and largely speeds up the inference procedure. Extensive experiments have shown that our method achieves the new state-of-the-art results for teeth segmentation and outperforms previous methods both quantitatively and qualitatively.

CGRS: Collaborative Knowledge Propagation Graph Attention Network for Recipes Recommendation

In the age of big data, recipe recommendation is of great significance. It can recommend recipes in line with the user's eating habits in massive data. Compared with other recommendation tasks, recipe recommendation is influenced by multiple aspects and requires fine-grained learning to obtain entity representations. Therefore, the traditional recommendation method cannot meet people's requirements. In this paper, we propose the Collaborative Knowledge Propagation Graph Attention Network for Recipes Recommendation (CGRS). This method designs collaborative information propagation to make full use of user interaction and recipe attribute information to meet the needs of multiple influencing factors. Use the graph attention feature learning network to obtain the high-order feature information of the entity to meet the demand for fine-grained representation. Specifically, the method first obtains the multi-hop triplet sets of users and recipes through a collaborative message propagation strategy. Then utilises a graph attention feature learning layer to learn the topological proximity structure features of the triplet sets. Obtain high-level semantic information of entities by superimposing network layers. Design an attention aggregator at the prediction layer to refine the embedding representation of entities. Finally predict the user-recipe interaction probability. Experimental results prove the advancement and effectiveness of CGRS.

Self Correspondence Distillation for End-to-End Weakly-Supervised Semantic Segmentation

Efficiently training accurate deep models for weakly supervised semantic segmentation (WSSS) with image-level labels is challenging and important. Recently, end-to-end WSSS methods have become the focus of research due to their high training efficiency. However, current methods suffer from insufficient extraction of comprehensive semantic information, resulting in low-quality pseudo-labels and sub-optimal solutions for end-to-end WSSS. To this end, we propose a simple and novel Self Correspondence Distillation (SCD) method to refine pseudo-labels without introducing external supervision. Our SCD enables the network to utilize feature correspondence derived from itself as a distillation target, which can enhance the network's feature learning process by complementing semantic information. In addition, to further improve the segmentation accuracy, we design a Variation-aware Refine Module to enhance the local consistency of pseudo-labels by computing pixel-level variation. Finally, we present an efficient end-to-end Transformer-based framework (TSCD) via SCD and Variation-aware Refine Module for the accurate WSSS task. Extensive experiments on the PASCAL VOC 2012 and MS COCO 2014 datasets demonstrate that our method significantly outperforms other state-of-the-art methods. Our code is available at https://github.com/Rongtao-Xu/RepresentationLearning/tree/main/SCD-AAAI2023.

Fine-Grained Visual Categorization: A Spatial–Frequency Feature Fusion Perspective

Fine-grained visual categorization is a challenging issue owing to high intra-class and low inter-class variances. Classical approaches rely on pre-trained models or many fine annotations. In this paper, we observe that spatial and frequency information provides distinct image views, and propose a new spatial–frequency feature fusion (SFFF) perspective to handle this challenging issue. Specifically, we design a heterogeneous feature extraction loss function, construct a global and local fusion SFFF network, and propose an importance-sparsity selection strategy. For feature extraction, we focus on the frequency domain feature learning network, extract fine-grained features, and achieve feature complementarity. For feature selection, we propose importance ranking and sparse regularity to constrain spatial–frequency features. For feature fusion, we design a spatial–frequency loss and an inter-layer switching strategy to achieve local-global collaboration. Comparative experiments were performed on popular fine-grained datasets and classic datasets such as CUB200-2011, Stanford Cars, Stanford Dogs, FGVC-Aircraft, and CIFAR100. The effectiveness and outstanding performance of SFFF are confirmed by comparisons with more than 40 state-of-the-art fine-grained categorization methods. Ablation studies and visualizations are provided to facilitate an understanding of our approach.

Dual-branch feature learning network for single image super-resolution

Dual-branch feature learning network for single image super-resolution

A novel sequential structure for lightweight multi-scale feature learning under limited available images

Although multi-scale feature learning can improve the performances of deep models, its parallel structure quadratically increases the model parameters and causes deep models to become larger and larger when enlarging the receptive fields (RFs). This leads to deep models easily suffering from over-fitting issue in many practical applications where the available training samples are always insufficient or limited. In addition, under this limited situation, although lightweight models (with fewer model parameters) can effectively reduce over-fitting, they may suffer from under-fitting because of insufficient training data for effective feature learning. In this work, a lightweight model called Sequential Multi-scale Feature Learning Network (SMF-Net) is proposed to alleviate these two issues simultaneously using a novel sequential structure of multi-scale feature learning. Compared to both deep and lightweight models, the proposed sequential structure in SMF-Net can easily extract features with larger RFs for multi-scale feature learning only with a few and linearly increased model parameters. The experimental results on both classification and segmentation tasks demonstrate that our SMF-Net only has 1.25M model parameters (5.3% of Res2Net50) with 0.7G FLOPS (14.6% of Res2Net50) for classification and 1.54M parameters (8.9% of UNet) with 3.35G FLOPs (10.9% of UNet) for segmentation but achieves higher accuracy than SOTA deep models and lightweight models, even when the training data is very limited available.

HSP-MFL: A High-level Semantic Property driven Multi-task Feature Learning Network for unsupervised person Re-ID

Unsupervised person re-identification aims to distinguish different pedestrians from discriminative representations on the basis of unlabeled data. Currently, most unsupervised Re-ID approaches explore visual representations to generate pseudo-labels for model’s training, which may suffer from background noise and semantic loss. To tackle this problem, this paper proposes a High-level Semantic Property driven Multi-task Feature Learning Network (HSP-MFL) to firstly introduce three high-level semantic properties for unsupervised person Re-ID. Technically, we design a novel Multiple Feature Fusion Module (MFFM) to deeply explore the complex correlation among multiple semantic and visual features to capture the discriminative feature cues, as well as a multi-task training scheme to generate robust fusion features. The architecture is quite simple and does not consume extra labeling costs. Extensive experiments on three datasets demonstrate that both high-level semantic properties and multi-task learning are effective in performance improvement, yielding SOTA mAPs for unsupervised person Re-ID.

Few-shot multi-modal registration with mono-modal knowledge transfer

Multi-modal registration is a key problem in many medical image analysis applications. Recent learning-based deformable image registration methods become attractive alternatives to traditional methods because of their great performance and fast run time. However, their success relies on large training datasets that are rarely available in multi-modal registration scenarios. To address this, we propose a novel knowledge transfer-based network (KT-Net) for few-shot multi-modal registration, which focuses on transferring knowledge of the mono-modal registration model to multi-modal registration. The contributions can be two-fold: (1) we propose model decoupling to disentangle the registration model into a feature learning network and an alignment learning network. The two networks are trained on large mono-modal datasets, making preparations for knowledge transfer. (2) a reverse teaching strategy is further designed to align the features of multi-modal images with a few samples, enabling the knowledge from mono-modal registration to transfer to multi-modal registration. Experimental results on multi-contrast brain MRI datasets demonstrate that our proposed method yields accurate and robust registration performance under the constraint of a few multi-modal samples. Compared with state-of-the-art registration methods, our proposed method achieves better registration performance with a high average Dice score of up to 83.5% and an average 95% percentile of Hausdorff distance as low as 1.26mm in various anatomical structures, showing great potential for mono-modal knowledge transfer to be applied in few-shot multi-modal registration.

Prediction of air pollutant concentrations based on TCN-BiLSTM-DMAttention with STL decomposition

A model with high accuracy and strong generalization performance is conducive to preventing serious pollution incidents and improving the decision-making ability of urban planning. This paper proposes a new neural network structure based on seasonal–trend decomposition using locally weighted scatterplot smoothing (Loess) (STL) and a dependency matrix attention mechanism (DMAttention) based on cosine similarity to predict the concentration of air pollutants. This method uses STL for series decomposition, temporal convolution, a bidirectional long short-term memory network (TCN-BiLSTM) for feature learning of the decomposed series, and DMAttention for interdependent moment feature emphasizing. In this paper, the long short-term memory network (LSTM) and the gated recurrent unit network (GRU) are set as the baseline models to design experiments. At the same time, to test the generalization performance of the model, short-term forecasts in hours were performed using PM2.5, PM10, SO2, NO2, CO, and O3 data. The experimental results show that the model proposed in this paper is superior to the comparison model in terms of root mean square error (RMSE) and mean absolute percentage error (MAPE). The MAPE values of the 6 kinds of pollutants are 6.800%, 10.492%, 9.900%, 6.299%, 4.178%, and 7.304%, respectively. Compared with the baseline LSTM and GRU models, the average reduction is 49.111% and 43.212%, respectively.

3-D Facial Feature Reconstruction and Learning Network for Facial Expression Recognition in the Wild

Facial expression recognition (FER) in the wild unavoidably suffers from the effects of face posture, illumination, and partial occlusion. In this article, we attempt to alleviate the above negative effects and improve the performance of FER in the wild based on 3-D face feature reconstruction and learning. Three-dimensional face reconstruction not only can effectively make up for the facial apparent information missing inform a 2-D face images but can also extract accurate 3-D facial geometric information in self-occlusion and extreme illumination scenarios. Therefore, we propose a novel end-to-end trainable 3-D face feature reconstruction and learning network (3-DF-RLN) is proposed for FER in the wild. In 3-DF-RLN, the 2-D implicitly frontalized face apparent data and 3-D facial landmarks are reconstructed by a 3-D face reconstruction module and input to two feature extraction pathways. The appearance pathway learns apparent features from the reconstructed 2-D face apparent data using a convolutional neural network. The geometry pathway learns the geometric features from the reconstructed 3-D facial landmarks using a graph convolutional network. Finally, FER is achieved via the fusion of the two pathways. Extensive experiments were conducted to evaluate the proposed method with three benchmark databases, including Multi-PIE, RAF-DB, and AffectNet. The results show that the proposed 3-DF-RLN model has better FER performance, both in the lab and in the wild. In addition, the face graph from the geometry pathway reveals the correlations between facial landmarks in FER.

NILRNN: A Neocortex-Inspired Locally Recurrent Neural Network for Unsupervised Feature Learning in Sequential Data

Unsupervised feature learning refers to the problem of learning useful feature extraction functions from unlabeled data. Despite the great success of deep learning networks in this task in recent years, both for static and for sequential data, these systems can in general still not compete with the high performance of our brain at learning to extract useful representations from its sensory input. We propose the Neocortex-Inspired Locally Recurrent Neural Network: a new neural network for unsupervised feature learning in sequential data that brings ideas from the structure and function of the neocortex to the well-established fields of machine learning and neural networks. By mimicking connection patterns in the feedforward circuits of the neocortex, our system tries to generalize some of the ideas behind the success of convolutional neural networks to types of data other than images. To evaluate the performance of our system at extracting useful features, we have trained different classifiers using those and other learnt features as input and we have compared the obtained accuracies. Our system has shown to outperform other shallow feature learning systems in this task, both in terms of the accuracies achieved and in terms of how fast the classification task is learnt. The results obtained confirm our system as a state-of-the-art shallow feature learning system for sequential data, and suggest that extending it to or integrating it into deep architectures may lead to new successful networks that are competent at dealing with complex sequential tasks.

Deep Learning-Based Intra Mode Derivation for Versatile Video Coding

In intra coding, Rate Distortion Optimization (RDO) is performed to achieve the optimal intra mode from a pre-defined candidate list. The optimal intra mode is also required to be encoded and transmitted to the decoder side besides the residual signal, where lots of coding bits are consumed. To further improve the performance of intra coding in Versatile Video Coding (VVC) , an intelligent intra mode derivation method is proposed in this paper, termed as Deep Learning based Intra Mode Derivation (DLIMD) . In specific, the process of intra mode derivation is formulated as a multi-class classification task, which aims to skip the module of intra mode signaling for coding bits reduction. The architecture of DLIMD is developed to adapt to different quantization parameter settings and variable coding blocks including non-square ones, where only one single trained model is required. Different from the existing deep learning based classification problems, the hand-crafted features are also fed into intra mode derivation network besides the learned features from feature learning network. To compete with traditional methods, one additional binary flag is utilized in the video codec to indicate the selected scheme with RDO. Extensive experimental results reveal that the proposed method can achieve 2.28%, 1.74%, and 2.18% bit rate reduction on average for Y, U, and V components on the platform of VVC test model, which outperforms the state-of-the-art works.

Adversarial Remote Sensing Scene Classification Based on Lie Group Feature Learning

Convolutional Neural Networks have been widely used in remote sensing scene classification. Since this kind of model needs a large number of training samples containing data category information, a Generative Adversarial Network (GAN) is usually used to address the problem of lack of samples. However, GAN mainly generates scene data samples that do not contain category information. To address this problem, a novel supervised adversarial Lie Group feature learning network is proposed. In the case of limited data samples, the model can effectively generate data samples with category information. There are two main differences between our method and the traditional GAN. First, our model takes category information and data samples as the input of the model and optimizes the constraint of category information in the loss function, so that data samples containing category information can be generated. Secondly, the object scale sample generation strategy is introduced, which can generate data samples of different scales and ensure that the generated data samples contain richer feature information. After large-scale experiments on two publicly available and challenging datasets, it is found that our method can achieve better scene classification accuracy even with limited data samples.

An Adaptive-Learning-Based Generative Adversarial Network for One-to-One Voice Conversion

Voice conversion (VC) emerged as a significant domain of research in the field of speech synthesis in recent years due to its emerging application in voice-assistive technologies, such as automated movie dubbing speech-to-singing conversion, to name a few. VC deals with the conversion of the vocal style of one speaker to another speaker while keeping the linguistic contents unchanged. Nowadays, generative adversarial network (GAN) models are widely used for speech feature mapping from the source speaker to the target speaker. In this article, we propose an adaptive-learning-based GAN model, called ALGAN-VC, to improve the one-to-one VC of speakers. Our ALGAN-VC framework consists of some approaches to improve the speech quality and voice similarity between the source and target speakers. We incorporate a dense residual network architecture into the generator network for efficient speech feature learning between source and target speakers. Our framework also includes an adaptive learning mechanism to compute the loss function for the proposed model. Moreover, a boosted learning rate approach is incorporated to enhance the learning capability of the proposed model. The proposed model is tested on Voice Conversion Challenge 2016, 2018, and 2020 datasets along with our self-prepared Indian regional-language-based speech dataset. In addition, an emotional speech dataset is also considered for evaluating the model’s performance. The objective and subjective evaluations of the generated speech samples indicated that the proposed model elegantly performed the voice conversion task by achieving high speaker similarity and good speech quality.

JNMR: Joint Non-linear Motion Regression for Video Frame Interpolation.

Video frame interpolation (VFI) aims to generate predictive frames by motion-warping from bidirectional references. Most examples of VFI utilize spatiotemporal semantic information to realize motion estimation and interpolation. However, due to variable acceleration, irregular movement trajectories, and camera movement in real-world cases, they can not be sufficient to deal with non-linear middle frame estimation. In this paper, we present a reformulation of the VFI as a joint non-linear motion regression (JNMR) strategy to model the complicated inter-frame motions. Specifically, the motion trajectory between the target frame and multiple reference frames is regressed by a temporal concatenation of multi-stage quadratic models. Then, a comprehensive joint distribution is constructed to connect all temporal motions. Moreover, to reserve more contextual details for joint regression, the feature learning network is devised to explore clarified feature expressions with dense skip-connection. Later, a coarse-to-fine synthesis enhancement module is utilized to learn visual dynamics at different resolutions with multi-scale textures. The experimental VFI results show the effectiveness and significant improvement of joint motion regression over the state-of-the-art methods.