Feature Representation Learning Research Articles

Multi-task contrastive learning for change detection in remote sensing images

ABSTRACT This letter proposes a novel multi-task contrastive learning (MTCL) approach for change detection of high-resolution remote sensing images. Current self-supervised learning approaches have limitations in utilizing multiview information of remote sensing images, and they cannot directly train the change detection backbone network for feature learning. The investigated method exploits multiview information contained in remote sensing images to directly train the change detection backbone network. Specifically, multiview views are constructed from high-resolution remote sensing images by handcrafted feature extraction approach, which contain complementary information from different domains. Then each constructed view is utilized for contrastive learning, and the multi-task learning strategy of parameter sharing is used to learn more rich and robust information. Furthermore, instead of learning feature representations from single remote sensing image, the proposed approach directly utilizes image pairs to train the change detection backbone, further improving the performance of change detection network. Two benchmark datasets are employed for comparative experiments, and experimental results certify the effectiveness and superiority of the investigated approach.

Deep learning-based automated high-accuracy location and identification of fresh vertebral compression fractures from spinal radiographs: a multicenter cohort study

BackgroundDigital radiography (DR) is a common and widely available examination. However, spinal DR cannot detect bone marrow edema, therefore, determining vertebral compression fractures (VCFs), especially fresh VCFs, remains challenging for clinicians.MethodsWe trained, validated, and externally tested the deep residual network (DRN) model that automated the detection and identification of fresh VCFs from spinal DR images. A total of 1,747 participants from five institutions were enrolled in this study and divided into the training cohort, validation cohort and external test cohorts (YHDH and BMUH cohorts). We evaluated the performance of DRN model based on the area under the receiver operating characteristic curve (AUC), feature attention maps, sensitivity, specificity, and accuracy. We compared it with five other deep learning models and validated and tested the model internally and externally and explored whether it remains highly accurate for an external test cohort. In addition, the influence of old VCFs on the performance of the DRN model was assessed.ResultsThe AUC was 0.99, 0.89, and 0.88 in the validation, YHDH, and BMUH cohorts, respectively, for the DRN model for detecting and discriminating fresh VCFs. The accuracies were 81.45% and 72.90%, sensitivities were 84.75% and 91.43%, and specificities were 80.25% and 63.89% in the YHDH and BMUH cohorts, respectively. The DRN model generated correct activation on the fresh VCFs and accurate peak responses on the area of the target vertebral body parts and demonstrated better feature representation learning and classification performance. The AUC was 0.90 (95% confidence interval [CI] 0.84–0.95) and 0.84 (95% CI 0.72–0.93) in the non-old VCFs and old VCFs groups, respectively, in the YHDH cohort (p = 0.067). The AUC was 0.89 (95% CI 0.84–0.94) and 0.85 (95% CI 0.72–0.95) in the non-old VCFs and old VCFs groups, respectively, in the BMUH cohort (p = 0.051).ConclusionIn present study, we developed the DRN model for automated diagnosis and identification of fresh VCFs from spinal DR images. The DRN model can provide interpretable attention maps to support the excellent prediction results, which is the key that most clinicians care about when using the model to assist decision-making.

Local spatial and temporal relation discovery model based on attention mechanism for traffic forecasting

Recognizing the evolution pattern of traffic condition and making accurate prediction play a vital role in intelligent transportation systems (ITS). With the massive increase of available traffic data, deep learning-based models have attracted considerable attention for their impressive performance in traffic forecasting. However, the majority of existing approaches neglect to model of asynchronously dynamic spatio-temporal correlation and fail to consider the impact of historical traffic data on future condition. Additionally, the attribute of deep learning method presents challenges in interpreting the explicit spatiotemporal relationships. In order to enhance the accuracy of traffic prediction as well as extract comprehensive and explainable spatial–temporal relevance in traffic networks, we propose a novel attention-based local spatial and temporal relation discovery (ALSTRD) model. Our model firstly implements feature representation learning to effectively express latent input traffic information. Then, a local attention mechanism structure is established to model asynchronous dependencies of historical input data. Finally, another attention network and the Pearson Correlation Coefficient method are introduced to extract the elaborate influence of the historical traffic condition of neighboring roads on the future condition of the target road. The experiment results on several datasets demonstrate that our model achieves significant improvements in prediction accuracy compared to other baseline methods, which can be attributed to its ability to extract the fine-grained correlation among historical traffic data and capture the dynamic association between past and future data. In addition, the incorporation of attention mechanism and Pearson Correlation Coefficient promotes the model’s ability to elucidate spatiotemporal correlations among traffic data, thereby providing a more robust explanation.

Analysis of Parenting Stress, Co-parenting Level, Family Intimacy and Adaptability on Children's Social Skills via self-attention Network

This paper uses the self-attention method to track family intimacy. We propose the Self-Attention Adversarial Deep Subspace Clustering Algorithm (SAADSC). The self-attention adversarial network is used to impose a prior distribution constraint in the feature learning of the autoencoder, which guides the learned feature representation to be more robust, thereby improving the clustering accuracy. The results show that the proposed algorithm outperforms the state-of-the-art methods in terms of accuracy (ACC) and standard mutual information (NMI). Therefore, we can effectively transfer this self-attention method to the analysis of the effects of adaptation on children's social skills.

A Multi-Level Relation-Aware Transformer model for occluded person re-identification

Occluded person re-identification (Re-ID) is a challenging task, as pedestrians are often obstructed by various occlusions, such as non-pedestrian objects or non-target pedestrians. Previous methods have heavily relied on auxiliary models to obtain information in unoccluded regions, such as human pose estimation. However, these auxiliary models fall short in accounting for pedestrian occlusions, thereby leading to potential misrepresentations. In addition, some previous works learned feature representations from single images, ignoring the potential relations among samples. To address these issues, this paper introduces a Multi-Level Relation-Aware Transformer (MLRAT) model for occluded person Re-ID. This model mainly encompasses two novel modules: Patch-Level Relation-Aware (PLRA) and Sample-Level Relation-Aware (SLRA). PLRA learns fine-grained local features by modeling the structural relations between key patches, bypassing the dependency on auxiliary models. It adopts a model-free method to select key patches that have high semantic correlation with the final pedestrian representation. In particular, to alleviate the interference of occlusion, PLRA captures the structural relations among key patches via a two-layer Graph Convolution Network (GCN), effectively guiding the local feature fusion and learning. SLRA is designed to facilitate the model to learn discriminative features by modeling the relations among samples. Specifically, to mitigate noisy relations of irrelevant samples, we present a Relation-Aware Transformer (RAT) block to capture the relations among neighbors. Furthermore, to bridge the gap between training and testing phases, a self-distillation method is employed to transfer the sample-level relations captured by SLRA to the backbone. Extensive experiments are conducted on four occluded datasets, two partial datasets and two holistic datasets. The results show that the proposed MLRAT model significantly outperforms existing baselines on four occluded datasets, while maintains top performance on two partial datasets and two holistic datasets.

Deep Adaptive Fuzzy Clustering for Evolutionary Unsupervised Representation Learning.

Cluster assignment of large and complex datasets is a crucial but challenging task in pattern recognition and computer vision. In this study, we explore the possibility of employing fuzzy clustering in a deep neural network framework. Thus, we present a novel evolutionary unsupervised learning representation model with iterative optimization. It implements the deep adaptive fuzzy clustering (DAFC) strategy that learns a convolutional neural network classifier from given only unlabeled data samples. DAFC consists of a deep feature quality-verifying model and a fuzzy clustering model, where deep feature representation learning loss function and embedded fuzzy clustering with the weighted adaptive entropy is implemented. We joint fuzzy clustering to the deep reconstruction model, in which fuzzy membership is utilized to represent a clear structure of deep cluster assignments and jointly optimize for the deep representation learning and clustering. Also, the joint model evaluates current clustering performance by inspecting whether the resampled data from estimated bottleneck space have consistent clustering properties to improve the deep clustering model progressively. Experiments on various datasets show that the proposed method obtains a substantially better performance for both reconstruction and clustering quality compared to the other state-of-the-art deep clustering methods, as demonstrated with the in-depth analysis in the extensive experiments.

An Optimal Edge-weighted Graph Semantic Correlation Framework for Multi-view Feature Representation Learning

In this article, we present an optimal edge-weighted graph semantic correlation (EWGSC) framework for multi-view feature representation learning. Different from most existing multi-view representation methods, local structural information and global correlation in multi-view feature spaces are exploited jointly in the EWGSC framework, leading to a new and high-quality multi-view feature representation. Specifically, a novel edge-weighted graph model is first conceptualized and developed to preserve local structural information in each of the multi-view feature spaces. Then, the explored structural information is integrated with a semantic correlation algorithm, labeled multiple canonical correlation analysis (LMCCA), to form a powerful platform for effectively exploiting local and global relations across multi-view feature spaces jointly. We then theoretically verified the relation between the upper limit on the number of projected dimensions and the optimal solution to the multi-view feature representation problem. To validate the effectiveness and generality of the proposed framework, we conducted experiments on five datasets of different scales, including visual-based (University of California Irvine (UCI) iris database, Olivetti Research Lab (ORL) face database, and Caltech 256 database), text-image-based (Wiki database), and video-based (Ryerson Multimedia Lab (RML) audio-visual emotion database) examples. The experimental results show the superiority of the proposed framework on multi-view feature representation over state-of-the-art algorithms.

Ac4C-AFL: A high-precision identification of human mRNA N4-acetylcytidine sites based on adaptive feature representation learning

RNA N4-acetylcytidine (ac4C) is a highly conserved RNA modification that plays a crucial role in controlling mRNA stability, processing, and translation. Consequently, accurate identification of ac4C sites across the genome is critical for understanding gene expression regulation mechanisms. In this study, we have developed ac4C-AFL, a bioinformatics tool that precisely identifies ac4C sites from primary RNA sequences. In ac4C-AFL, we identified the optimal sequence length for model building and implemented an adaptive feature representation strategy that is capable of extracting the most representative features from RNA. To identify the most relevant features, we proposed a novel ensemble feature importance scoring strategy to rank features effectively. We then used this information to conduct the sequential forward search, which individually determine the optimal feature set from the 16 sequence-derived feature descriptors. Utilizing these optimal feature descriptors, we constructed 176 baseline models using 11 popular classifiers. The most efficient baseline models were identified using the two-step feature selection approach, whose predicted scores were integrated and trained with the appropriate classifier to develop the final prediction model. Our rigorous cross-validations and independent tests demonstrate that ac4C-AFL surpasses contemporary tools in predicting ac4C sites. Moreover, we have developed a publicly accessible web server at https://balalab-skku.org/ac4C-AFL/.

An LSTM-based adversarial variational autoencoder framework for self-supervised neural decoding of behavioral choices.

Objective. This paper presents data-driven solutions to address two challenges in the problem of linking neural data and behavior: 1) unsupervised analysis of behavioral data and automatic label generation from behavioral observations, and 2) extraction of subject-invariant features for the development of generalized neural decoding models.

Approach. For behavioral analysis and label generation, an unsupervised method, which employs an autoencoder to transform behavior data into a cluster-friendly feature space is presented. The model iteratively refines the assigned clusters with soft clustering assignment loss, and gradually improves the learned feature representations. To address subject variability in decoding neural activity, adversarial learning in combination with a long short-term memory-based adversarial variational autoencoder (LSTM-AVAE) model is employed. By using an adversary network to constrain the latent representations, the model captures shared information among subjects' neural activity, making it proper for cross-subject transfer learning.

Main results. The proposed approach is evaluated using cortical recordings of Thy1-GCaMP6s transgenic mice obtained via widefield calcium imaging during a motivational licking behavioral experiment. The results show that the proposed model achieves an accuracy of 89.7% in cross-subject neural decoding, outperforming other well-known autoencoder-based feature learning models. These findings suggest that incorporating an adversary network eliminates subject dependency in representations, leading to improved cross-subject transfer learning performance, while also demonstrating the effectiveness of LSTM-based models in capturing the temporal dependencies within neural data.

Significance. Results demonstrate the feasibility of the proposed framework in unsupervised clustering and label generation of behavioral data, as well as achieving high accuracy in cross-subject neural decoding, indicating its potentials for relating neural activity to behavior.

Application of Image Segmentation Algorithms in Computer Vision

In the field of computer vision (CV), image segmentation technology, as a fundamental part, has a crucial impact on the accuracy of subsequent image processing tasks. Image segmentation is not only a crucial transitional step from image processing to image analysis, but also a hot and difficult research topic in the field of CV. Although significant progress has been made in the research of image segmentation algorithms, existing segmentation algorithms may still face challenges in certain specific scenarios due to the complexity and diversity of images, making it difficult to achieve ideal segmentation results. In recent years, the rapid development of deep learning (DL) technology has brought new breakthroughs to the field of image segmentation. DL models, especially Convolutional Neural Networks (CNNs), can capture semantic information of images more accurately by automatically learning feature representations in images, thereby achieving more precise image segmentation. This article delves into the research and application of image segmentation algorithms in CV, with a focus on the application of DL in the field of image segmentation. With the continuous development of advanced technologies such as DL, it is believed that image segmentation technology will play a greater role in more fields in the future.

TransPose: 6D object pose estimation with geometry-aware Transformer

Efficient and accurate estimation of objects’ pose is essential in numerous practical applications. Due to the depth data contains abundant geometric information, some existing methods devote to extract features from 3D point cloud. However, these depth-based methods focus on extracting the point cloud local features and consider less about the global information. How to extract and utilize the local and global geometry features in depth information is crucial to achieve accurate predictions. To this end, we propose TransPose, a novel 6D pose framework that exploits Transformer Encoder with geometry-aware module to develop better learning of point cloud feature representations. To better extract local geometry features, we finely design the graph convolution network-based feature extractor that first uniformly sample point cloud and extract point pair features of point cloud. To further improve robustness to occlusion, we adopt Transformer to perform the propagation of global information, making each local feature obtains global information. Moreover, we introduce geometry-aware module in Transformer Encoder, which to form an effective constrain for point cloud feature learning and makes the global information exchange more tightly coupled with point cloud tasks. Extensive experiments indicate the effectiveness of TransPose, our pose estimation pipeline achieves competitive results on three benchmark datasets.

CoReFace: Sample-guided Contrastive Regularization for Deep Face Recognition

The discriminability of the feature representation is crucial for face recognition. However, previous methods rely solely on the learnable weights of the classification layer, which represent the identities. This reliance could be problematic as the evaluation process depends on the similarity between pairs of face images and requires minimal identity information learned during training. As a result, there is an inconsistency between the training and evaluation processes, which can confuse the feature encoder and hinder the effectiveness of identity-based methods. To address this problem, we propose a novel approach namely Contrastive Regularization for Face Recognition (CoReFace), which applies sample-level regularization in feature representation learning. Specifically, we employ sample-guided contrastive learning to directly regularize the training based on the sample-sample relationship and thus align it with the evaluation process. To avoid image quality degradation, we augment the embeddings instead of the images in order to integrate contrastive learning into face recognition. Additionally, we introduce a new contrastive loss function for the regularization of representation distribution. This function incorporates an adaptive margin and a supervised contrastive mask to ensure stable loss values and prevent interference with the identity supervision signals. Finally, we explore new pair-coupling protocols in order to overcome the problem of semantically repetitive signals in contrastive learning. Extensive experiments demonstrate the efficacy and efficiency of our CoReFace approach, which achieves competitive results compared to state-of-the-art methods. Code could be found https://github.com/IsidoreSong/CoreFace here.

CAT-DTI: cross-attention and Transformer network with domain adaptation for drug-target interaction prediction

Accurate and efficient prediction of drug-target interaction (DTI) is critical to advance drug development and reduce the cost of drug discovery. Recently, the employment of deep learning methods has enhanced DTI prediction precision and efficacy, but it still encounters several challenges. The first challenge lies in the efficient learning of drug and protein feature representations alongside their interaction features to enhance DTI prediction. Another important challenge is to improve the generalization capability of the DTI model within real-world scenarios. To address these challenges, we propose CAT-DTI, a model based on cross-attention and Transformer, possessing domain adaptation capability. CAT-DTI effectively captures the drug-target interactions while adapting to out-of-distribution data. Specifically, we use a convolution neural network combined with a Transformer to encode the distance relationship between amino acids within protein sequences and employ a cross-attention module to capture the drug-target interaction features. Generalization to new DTI prediction scenarios is achieved by leveraging a conditional domain adversarial network, aligning DTI representations under diverse distributions. Experimental results within in-domain and cross-domain scenarios demonstrate that CAT-DTI model overall improves DTI prediction performance compared with previous methods.

Balancing Transferability and Discriminability for Unsupervised Domain Adaptation.

Unsupervised domain adaptation (UDA) aims to leverage a sufficiently labeled source domain to classify or represent the fully unlabeled target domain with a different distribution. Generally, the existing approaches try to learn a domain-invariant representation for feature transferability and add class discriminability constraints for feature discriminability. However, the feature transferability and discriminability are usually not synchronized, and there are even some contradictions between them, which is often ignored and, thus, reduces the accuracy of recognition. In this brief, we propose a deep multirepresentations adversarial learning (DMAL) method to explore and mitigate the inconsistency between feature transferability and discriminability in UDA task. Specifically, we consider feature representation learning at both the domain level and class level and explore four types of feature representations: domain-invariant, domain-specific, class-invariant, and class-specific. The first two types indicate the transferability of features, and the last two indicate the discriminability. We develop an adversarial learning strategy between the four representations to make the feature transferability and discriminability to be gradually synchronized. A series of experimental results verify that the proposed DMAL achieves comparable and promising results on six UDA datasets.

A novel generative–predictive data-driven approach for multi-objective optimization of horizontal axis tidal turbine

Tidal turbines play a critical role in converting the kinetic energy of water into electricity, contributing significantly to energy conversion. However, the current optimization design of these turbines involves computationally intensive simulations, leading to higher design costs. Additionally, traditional parameterized modeling methods, constrained by predefined design parameters, limit the exploration of innovative designs. In response, this study introduces an innovative data-driven “generative–predictive” design approach comprising a generative model and a predictive model. The generative model autonomously learns feature representations from existing turbines and leverages this knowledge to generate a novel set of turbines with superior hydrodynamic performance. Subsequently, an efficient performance evaluation is conducted using a predictive model for the generated turbines. Compared to the current parameterized modeling approaches, the proposed approach is combined with multi-objective optimization algorithm to optimize the tidal turbine hydrodynamic performance. Research results demonstrate that the generative model, trained on gradients, can generate highly complex turbines with minimal latent vectors. Through transfer learning, the predictive model exhibits robustness and accuracy, effectively guiding the design process. In the final optimization comparison, the proposed generative–predictive design approach requires only 4% of the optimization time while achieving results similar to or surpassing traditional design approaches. This approach proves to be a powerful tool for guiding the efficient and optimized design of turbines.

Deep learning-based estimation of ash content in coal: Unveiling the contributions of color and texture features

Determining coal ash content is paramount when evaluating coal quality and optimizing industrial processes. Conventional methods reliant on manual analysis prove exceedingly time-consuming and labor-intensive. The advent of deep learning has galvanized researchers to explore various models aimed at precision and efficiency in the coal industry. However, the selection of appropriate features assumes a pivotal role in achieving accurate estimation. This study meticulously scrutinizes the importance of color and texture features in estimating coal ash content using neural networks. The proposed framework for elucidating feature contributions encompasses the following steps: (1) A feature disentangled pipeline was employed to generate a color and texture dataset from the original dataset; (2) Harnessing convolutional neural network (CNN), vision Transformers(ViT), a hybrid model combing CNN and ViT, and graph convolution network (GCN) to learning feature representation for texture and color. 3) An interpretable decision module was employed to aggregate these two feature representations, thereby achieving an interpretable estimation of ash content. Experimental results and visualizations demonstrated the substantial importance of color in CNN, accounting for an impressive 64.77%, whereas the texture feature modestly contributed at 35.23%. The analysis of feature contributions assumes a crucial role in guiding the design of accuracy-driven models and in comprehending the inherent contributions or biases within deep learning architectures.

An Overview of Current Research on Computer-Aided Liver Diagnosis Based on Computed Tomography Employing Deep Learning Methods

Abstract: Compared to traditional X-rays, computed tomography (CT) scanning is a non-invasive diagnostic imaging method that yields more precise information about the liver. In contrast to ultrasonography (US) exams, the quality of a CT scan is not heavily operator dependent. Many studies have been conducted with improved outcomes utilizing traditional machine learning methods for computer-aided diagnosis (CAD) of the liver. Recent developments, particularly in the field of deep learning technology, have made it possible to identify, categorize, and segment patterns in medical images. These developments have also been used to other fields outside of medicine. The ability of deep learning to automatically learn feature representations from data, as opposed to feeding in manually created features based on application, is one of its fundamental capabilities. The fundamentals of deep learning are presented in this paper, along with their achievements in liver segmentation and lesion detection, classification utilizing CT imaging modalities, and discussion of their various network topologies. Another fascinating deep learning strategy that is covered is transfer learning. Thus, deep learning and the CAD system have had a significant influence and improved the performance of the healthcare sector.

GEFormerDTA: drug target affinity prediction based on transformer graph for early fusion

Predicting the interaction affinity between drugs and target proteins is crucial for rapid and accurate drug discovery and repositioning. Therefore, more accurate prediction of DTA has become a key area of research in the field of drug discovery and drug repositioning. However, traditional experimental methods have disadvantages such as long operation cycles, high manpower requirements, and high economic costs, making it difficult to predict specific interactions between drugs and target proteins quickly and accurately. Some methods mainly use the SMILES sequence of drugs and the primary structure of proteins as inputs, ignoring the graph information such as bond encoding, degree centrality encoding, spatial encoding of drug molecule graphs, and the structural information of proteins such as secondary structure and accessible surface area. Moreover, previous methods were based on protein sequences to learn feature representations, neglecting the completeness of information. To address the completeness of drug and protein structure information, we propose a Transformer graph-based early fusion research approach for drug-target affinity prediction (GEFormerDTA). Our method reduces prediction errors caused by insufficient feature learning. Experimental results on Davis and KIBA datasets showed a better prediction of drugtarget affinity than existing affinity prediction methods.

Combinatorial CNN-Transformer Learning with Manifold Constraints for Semi-supervised Medical Image Segmentation

Semi-supervised learning (SSL), as one of the dominant methods, aims at leveraging the unlabeled data to deal with the annotation dilemma of supervised learning, which has attracted much attentions in the medical image segmentation. Most of the existing approaches leverage a unitary network by convolutional neural networks (CNNs) with compulsory consistency of the predictions through small perturbations applied to inputs or models. The penalties of such a learning paradigm are that (1) CNN-based models place severe limitations on global learning; (2) rich and diverse class-level distributions are inhibited. In this paper, we present a novel CNN-Transformer learning framework in the manifold space for semi-supervised medical image segmentation. First, at intra-student level, we propose a novel class-wise consistency loss to facilitate the learning of both discriminative and compact target feature representations. Then, at inter-student level, we align the CNN and Transformer features using a prototype-based optimal transport method. Extensive experiments show that our method outperforms previous state-of-the-art methods on three public medical image segmentation benchmarks.

DCLP: Neural Architecture Predictor with Curriculum Contrastive Learning

Neural predictors have shown great potential in the evaluation process of neural architecture search (NAS). However, current predictor-based approaches overlook the fact that training a predictor necessitates a considerable number of trained neural networks as the labeled training set, which is costly to obtain. Therefore, the critical issue in utilizing predictors for NAS is to train a high-performance predictor using as few trained neural networks as possible. Although some methods attempt to address this problem through unsupervised learning, they often result in inaccurate predictions. We argue that the unsupervised tasks intended for the common graph data are too challenging for neural networks, causing unsupervised training to be susceptible to performance crashes in NAS. To address this issue, we propose a CurricuLum-guided Contrastive Learning framework for neural Predictor (DCLP). Our method simplifies the contrastive task by designing a novel curriculum to enhance the stability of unlabeled training data distribution during contrastive training. Specifically, we propose a scheduler that ranks the training data according to the contrastive difficulty of each data and then inputs them to the contrastive learner in order. This approach concentrates the training data distribution and makes contrastive training more efficient. By using our method, the contrastive learner incrementally learns feature representations via unsupervised data on a smooth learning curve, avoiding performance crashes that may occur with excessively variable training data distributions. We experimentally demonstrate that DCLP has high accuracy and efficiency compared with existing predictors, and shows promising potential to discover superior architectures in various search spaces when combined with search strategies. Our code is available at: https://github.com/Zhengsh123/DCLP.