Atrous Pooling Research Articles

NFSA-DTI: A Novel Drug–Target Interaction Prediction Model Using Neural Fingerprint and Self-Attention Mechanism

Existing deep learning methods have shown outstanding performance in predicting drug–target interactions. However, they still have limitations: (1) the over-reliance on locally extracted features by some single encoders, with insufficient consideration of global features, and (2) the inadequate modeling and learning of local crucial interaction sites in drug–target interaction pairs. In this study, we propose a novel drug–target interaction prediction model called the Neural Fingerprint and Self-Attention Mechanism (NFSA-DTI), which effectively integrates the local information of drug molecules and target sequences with their respective global features. The neural fingerprint method is used in this model to extract global features of drug molecules, while the self-attention mechanism is utilized to enhance CNN’s capability in capturing the long-distance dependencies between the subsequences in the target amino acid sequence. In the feature fusion module, we improve the bilinear attention network by incorporating attention pooling, which enhances the model’s ability to learn local crucial interaction sites in the drug–target pair. The experimental results on three benchmark datasets demonstrated that NFSA-DTI outperformed all baseline models in predictive performance. Furthermore, case studies illustrated that our model could provide valuable insights for drug discovery. Moreover, our model offers molecular-level interpretations.

Speech Emotion Recognition Using Transfer Learning: Integration of Advanced Speaker Embeddings and Image Recognition Models

Automatic Speech Emotion Recognition (SER) plays a vital role in making human–computer interactions more natural and effective. A significant challenge in SER development is the limited availability of diverse emotional speech datasets, which hinders the application of advanced deep learning models. Transfer learning is a machine learning technique that helps address this issue by utilizing knowledge from pre-trained models to improve performance on a new task in a target domain, even with limited data. This study investigates the use of transfer learning from various pre-trained networks, including speaker embedding models such as d-vector, x-vector, and r-vector, and image classification models like AlexNet, GoogLeNet, SqueezeNet, ResNet-18, and ResNet-50. We also propose enhanced versions of the x-vector and r-vector models incorporating Multi-Head Attention Pooling and Angular Margin Softmax, alongside other architectural improvements. Additionally, reverberation from the Room Impulse Response datasets was added to the speech utterances to diversify and augment the available data. Notably, the enhanced r-vector model achieved classification accuracies of 74.05% Unweighted Accuracy (UA) and 73.68% Weighted Accuracy (WA) on the IEMOCAP dataset, and 80.25% UA and 79.81% WA on the CREMA-D dataset, outperforming the existing state-of-the-art methods. This study shows that using cross-domain transfer learning is beneficial for low-resource emotion recognition. The enhanced models developed in other domains (for non-emotional tasks) can further improve the accuracy of SER.

GTransCYPs: an improved graph transformer neural network with attention pooling for reliably predicting CYP450 inhibitors

State‑of‑the‑art medical studies proved that predicting CYP450 enzyme inhibitors is beneficial in the early stage of drug discovery. However, accurate machine learning-based (ML) in silico methods for predicting CYP450 inhibitors remains challenging. Here, we introduce GTransCYPs, an improved graph neural network (GNN) with a transformer mechanism for predicting CYP450 inhibitors. This model significantly enhances the discrimination between inhibitors and non-inhibitors for five major CYP450 isozymes: 1A2, 2C9, 2C19, 2D6, and 3A4. GTransCYPs learns information patterns from molecular graphs by aggregating node and edge representations using a transformer. The GTransCYPs model utilizes transformer convolution layers to process features, followed by a global attention-pooling technique to synthesize the graph-level information. This information is then fed through successive linear layers for final output generation. Experimental results demonstrate that the GTransCYPs model achieved high performance, outperforming other state-of-the-art methods in CYP450 prediction.Scientific contributionThe prediction of CYP450 inhibition via computational techniques utilizing biological information has emerged as a cost-effective and highly efficient approach. Here, we presented a deep learning (DL) architecture based on GNN with transformer mechanism and attention pooling (GTransCYPs) to predict CYP450 inhibitors. Four GTransCYPs of different pooling technique were tested on an experimental tasks on the CYP450 prediction problem for the first time. Graph transformer with attention pooling algorithm achieved the best performances. Comparative and ablation experiments provide evidence of the efficacy of our proposed method in predicting CYP450 inhibitors. The source code is publicly available at https://github.com/zonwoo/GTransCYPs.

Chain-aware graph neural networks for molecular property prediction.

Predicting the properties of molecules is a fundamental problem in drug design and discovery, while how to learn effective feature representations lies at the core of modern deep-learning-based prediction methods. Recent progress shows expressive power of graph neural networks (GNNs) in capturing structural information for molecular graphs. However, we find that most molecular graphs exhibit low clustering along with dominating chains. Such topological characteristics can induce feature squashing during message passing and thus impair the expressivity of conventional GNNs. Aiming at improving node features' expressiveness, we develop a novel chain-aware graph neural network model, wherein the chain structures are captured by learning the representation of the center node along the shortest paths starting from it, and the redundancy between layers are mitigated via initial residual difference connection (IRDC). Then the molecular graph is represented by attentive pooling of all node representations. Compared to standard graph convolution, our chain-aware learning scheme offers a more straightforward feature interaction between distant nodes, thus it is able to capture the information about long-range dependency. We provide extensive empirical analysis on real-world datasets to show the outperformance of the proposed method. The MolPath code is publicly available at https://github.com/Assassinswhh/Molpath.

Ranking attention multiple instance learning for lymph node metastasis prediction on multicenter cervical cancer MRI.

In the current clinical diagnostic process, the gold standard for lymph node metastasis (LNM) diagnosis is histopathological examination following surgical lymphadenectomy. Developing a non-invasive and preoperative method for predicting LNM is necessary and holds significant clinical importance. We develop a ranking attention multiple instance learning (RA-MIL) model that integrates convolutional neural networks (CNNs) and ranking attention pooling to diagnose LNM from T2 MRI. Our RA-MIL model applies the CNNs to derive imaging features from 2D MRI slices and employs ranking attention pooling to create patient-level feature representation for diagnostic classification. Based on the MIL and attention theory, informative regions of top-ranking MRI slices from LNM-positive patients are visualized to enhance the interpretability of automatic LNM prediction. This retrospective study collected 300 female patients with cervical cancer who underwent T2-weighted magnetic resonance imaging (MRI) scanning and histopathological diagnosis from one hospital (289 patients) and one open-source dataset (11 patients). Our RA-MIL model delivers promising LNM prediction performance, achieving the area under the receiver operating characteristic curve (AUC) of 0.809 on the internal test set and 0.833 on the public dataset. Experiments show significant improvements in LNM status prediction using the proposed RA-MIL model compared with other state-of-the-art (SOTA) comparative deep learning models. The developed RA-MIL model has the potential to serve as a non-invasive auxiliary tool for preoperative LNM prediction, offering visual interpretability regarding informative MRI slices and regions in LNM-positive patients.

Semi-supervised sound event detection with dynamic convolution and confidence-aware mean teacher

Recently, sound event detection (SED) has made significant advancements through the application of deep learning, but there are still many difficulties and challenges to be addressed. One of the major challenges is the diversity of sound events, leading to substantial variations in time-frequency domain features. Additionally, most existing SED models can not effectively handle sound events of different scales, particularly those of short duration. Another challenge is the lack of well labeled dataset. The commonly used solution is mean teacher method, but inaccurate pseudo-labels could lead to confirmation bias and performance imbalance. In this paper, we introduce the multi-dimensional frequency dynamic convolution, which endows convolutional kernels with frequency-adaptive dynamic properties to enhance the feature representation capability. Moreover, we propose dual self attention pooling function to achieve more precise temporal localization. Finally, to solve the incorrect pseudo-labels problems, we propose the confidence-aware mean teacher to increase pseudo-labels accuracy and train the student model with high confidence labels. Experimental results on DCASE2017, DCASE2018 and DCASE2023 Task4 dataset validate the superior performance of proposed methods.

GGN-GO: geometric graph networks for predicting protein function by multi-scale structure features.

Recent advances in high-throughput sequencing have led to an explosion of genomic and transcriptomic data, offering a wealth of protein sequence information. However, the functions of most proteins remain unannotated. Traditional experimental methods for annotation of protein functions are costly and time-consuming. Current deep learning methods typically rely on Graph Convolutional Networks to propagate features between protein residues. However, these methods fail to capture fine atomic-level geometric structural features and cannot directly compute or propagate structural features (such as distances, directions, and angles) when transmitting features, often simplifying them to scalars. Additionally, difficulties in capturing long-range dependencies limit the model's ability to identify key nodes (residues). To address these challenges, we propose a geometric graph network (GGN-GO) for predicting protein function that enriches feature extraction by capturing multi-scale geometric structural features at the atomic and residue levels. We use a geometric vector perceptron to convert these features into vector representations and aggregate them with node features for better understanding and propagation in the network. Moreover, we introduce a graph attention pooling layer captures key node information by adaptively aggregating local functional motifs, while contrastive learning enhances graph representation discriminability through random noise and different views. The experimental results show that GGN-GO outperforms six comparative methods in tasks with the most labels for both experimentally validated and predicted protein structures. Furthermore, GGN-GO identifies functional residues corresponding to those experimentally confirmed, showcasing its interpretability and the ability to pinpoint key protein regions. The code and data are available at: https://github.com/MiJia-ID/GGN-GO.

Hierarchical Glocal Attention Pooling for Graph Classification

Graph pooling is an essential operation in Graph Neural Networks that reduces the size of an input graph while preserving its core structural properties. Existing pooling methods find a compressed representation considering the Global Topological Structures (e.g., cliques, stars, clusters) or Local information at node level (e.g., top-k informative nodes). However, an effective graph pooling method does not hierarchically integrate both Global and Local graph properties. To this end, we propose a dual-fold Hierarchical Global Local Attention Pooling (HGLA-Pool) layer that exploits the aforementioned graph properties, generating more robust graph representations. Exhaustive experiments on nine publicly available graph classification benchmarks under standard metrics show that HGLA-Pool significantly outperforms eleven state-of-the-art models on seven datasets while being on par for the remaining two.

N-BodyPat: Investigation on the dementia and Alzheimer's disorder detection using EEG signals

The N-body problem is a remarkable research topic in physics. We propose a new feature extraction model inspired by the N-body trajectory and test its feature extraction capability. In the first part of the research, an open-access electroencephalogram (EEG) dataset is used to test the proposed method. This dataset has three classes, namely (i) Alzheimer's Disorder (AD), (ii) frontal dementia (FD), and (iii) control groups. In the second step of the study, the EEG signals were divided into segments of 15 s in length, which resulted in 4,661 EEG signals. In the third part of the study, the proposed new self-organized feature engineering (SOFE) model is used to classify the EEG signals automatically. For this SOFE, two novel methods were presented: (i) a dynamic feature extraction function using a graph of the N-Body orbital, termed N-BodyPat, and (ii) an attention pooling function. A multileveled and combinational feature extraction method was proposed by deploying both methods. A feature selection function using ReliefF and Neighborhood Component Analysis (RFNCA) was used to choose the most informative features. An ensemble k-nearest neighbors (EkNN) classifier was employed in the classification phase. Our proposed N-BodyPat generates seven feature vectors for each channel, and the utilized EEG signal dataset contains 19 channels. In this aspect,133 (=19 × 7) EkNN-based outcomes were created. To attain higher classification performance by employing these 133 EkNN-based outcomes, an iterative majority voting (IMV)-based information fusion method was applied, and the most accurate outcomes were selected automatically. The recommended N-BodyPat-based SOFE achieved a classification accuracy of 99.64 %.

Deformation mechanism-assisted deep learning architecture for predicting step-like displacement of reservoir landslide

Reservoir landslides in the Three Gorges Reservoir, China, exhibit prolonged slow motion and the potential for catastrophic events due to fluctuations in reservoir levels and intense rainfall episodes. Their distinct step-like deformation characteristics, involving rapid transformation processes of different states, pose challenges for accurate early warning and prediction. Previous forecasting models have often struggled with limited accuracy. This study introduces a mechanism-assisted deep learning model, leveraging the Informer architecture, to predict prolonged step-like reservoir landslide displacement. Utilizing a 15-year continuous monitoring dataset of the Baishuihe landslide, this model investigates the landslide mechanism, identifies influencing conditions underlying the step-wise behavior, and customizes input features for the prediction model by integrating optimized variational mode decomposition and wavelet analysis. Additionally, the dynamic correlation and hysteresis analysis between triggering factors and displacement offer valuable physical insights into the model and enhance the interpretability of the model. The model is further tailored to accommodate features of the monitoring dataset associated with landslide evolution by integrating a global multi-head attention mechanism and pooling layers, enabling the capture of both globe dependencies and local critical features of the model inputs. Through rigorous model training, performance evaluation, and tuning, the proposed model efficiently predicts step-wise landslide displacement, particularly during short-term rapid transitions between creep-mutation states.

LESA-Net: Semantic segmentation of multi-type road point clouds in complex agroforestry environment

Point-cloud semantic segmentation is a visual task essential for agricultural robots to comprehend natural agroforestry environments. However, owing to the extremely large amount of point-cloud data in agroforestry environments, learning effective features for semantic segmentation from large-scale point clouds is challenging. Therefore, to address this issue and achieve accurate semantic segmentation of different types of road-surface point clouds in large-scale agroforestry environments, this study proposes a point-cloud semantic segmentation network framework based on double-distance self-attention. First, a point-cloud local feature enhancement module is proposed. This module primarily extends the receptive field and enhances the generalizability of multidimensional features by incorporating reflection intensity information and a spatial feature-encoding block that is enhanced with contextual semantic information. Second, we introduce a dual-distance attention pooling (DDAPS) block based on the self-attention mechanism. This block initially learns the feature representation of the local neighborhood of each point through the self-attention mechanism. Then, it uses the DDAPS block to aggregate more discriminative local neighborhood point features. Finally, extensive experimental results on large-scale point-cloud datasets, SemanticKITTI and RELLIS-3D, demonstrate that our algorithm outperforms similar algorithms in large-scale agroforestry environments.

A novel method for the classification of 3D point clouds based on the improved PointNet++

Abstract In deep learning, point clouds are used as the primary input format for 3D data, which can provide detailed geometric information about objects in the original 3D space. PointNet++ is a deep learning network that uses point cloud data as an input format, which avoids the losses associated with the previous conversion of point cloud into 3D voxelization and a collection of 2D images. Although PointNet++ can directly process point cloud data in various ways, due to the disordered, irregular, and unevenly distributed nature of point cloud data, the effect of extracting point cloud features could be better. The large amount of point cloud data also leads to the training model falling into the local optimal solution, which affects the training results. In recent years, some effective methods and strategies have emerged to address these problems. In this thesis, three methods are proposed based on the PointNet++ network: feature similarity-based attention pooling, small kernel convolution, and diverse branch block method to improve the performance of the PointNet++ network. Experiments show that the improvement methods proposed in this paper effectively improve the feature extraction accuracy, which improves the accuracy of the PointNet++ network for classification on the ModelNet40_Normal_Resampled dataset, with an overall improvement of 1% compared with PointNet++.

DSC-Net: learning discriminative spatial contextual features for semantic segmentation of large-scale ancient architecture point clouds

Semantic segmentation of point cloud data of architectural cultural heritage is of significant importance for HBIM modeling, disease extraction and analysis, and heritage restoration research fields. In the semantic segmentation task of architectural point cloud data, especially for the protection and analysis of architectural cultural heritage, the previous deep learning methods have poor segmentation effects due to the complexity and unevenness of the data, the high geometric feature similarity between different components, and the large scale changes. To this end, this paper proposes a novel encoder-decoder architecture called DSC-Net. It consists of an encoder-decoder structure based on point random sampling and several fully connected layers for semantic segmentation. To overcome the loss of key features caused by random downsampling, DSC-Net has developed two new feature aggregation schemes: the enhanced dual attention pooling module and the global context feature module, to learn discriminative features for the challenging scenes mentioned above. The former fully considers the topology and semantic similarity of neighboring points, generating attention features that can distinguish categories with similar structures. The latter uses spatial location and neighboring volume ratio to provide an overall view of different types of architectural scenes, helping the network understand the spatial relationships and hierarchical structures between different architectural elements. The proposed modules can be easily embedded into various network architectures for point cloud semantic segmentation. We conducted experiments on multiple datasets, including the ancient architecture dataset, the ArCH architectural cultural heritage dataset, and the publicly available architectural segmentation dataset S3DIS. The results show that the mIoU reached 63.56%, 55.84%, and 71.03% respectively. The experimental results prove that our method has the best segmentation effect in dealing with challenging architectural cultural heritage data and also demonstrates its practicality in a wider range of architectural point cloud segmentation applications.

Segmentation of void defects in X-ray images of chip solder joints based on PCB-DeepLabV3 algorithm

Defects within chip solder joints are usually inspected visually for defects using X-ray imaging to obtain images. The phenomenon of voids inside solder joints is one of the most likely types of defects in the soldering process, and accurate detection of voids becomes difficult due to their irregular shapes, varying sizes, and defocused edges. To address this problem, an X-ray void image segmentation algorithm based on improved PCB-DeepLabV3 is proposed. Firstly, to meet the demand for lightweight and easy deployment in industrial scenarios, mobilenetv2 is used as the feature extraction backbone network of the PCB-DeepLabV3 model; then, Attentional multi-scale two-space pyramid pooling network (AMTPNet) is designed to optimize the shallow feature edges and to improve the ability to capture detailed information; finally, image cropping and cleaning methods are designed to enhance the training dataset, and the improved PCB-DeepLabV3 is applied to the training dataset. The improved PCB-DeepLabV3 model is used to segment the void regions within the solder joints and compared with the classical semantic segmentation models such as Unet, SegNet, PSPNet, and DeeplabV3. The proposed new method enables the solder joint void inspection to get rid of the traditional way of visual inspection, realize intelligent upgrading, and effectively improve the problem of difficult segmentation of the target virtual edges, to obtain the inspection results with higher accuracy.

MAHyNet: Parallel Hybrid Network for RNA-Protein Binding Sites Prediction Based on Multi-Head Attention and Expectation Pooling.

RNA-binding proteins (RBPs) can regulate biological functions by interacting with specific RNAs, and play an important role in many life activities. Therefore, the rapid identification of RNA-protein binding sites is crucial for functional annotation and site-directed mutagenesis. In this work, a new parallel network that integrates the multi-head attention mechanism and the expectation pooling is proposed, named MAHyNet. The left-branch network of MAHyNet hybrids convolutional neural networks (CNNs) and gated recurrent neural network (GRU) to extract the features of one-hot. The right-branch network is a two-layer CNN network to analyze physicochemical properties of RNA base. Specifically, the multi-head attention mechanism is a computational collection of multiple independent layers of attention, which can extract feature information from multiple dimensions. The expectation pooling combines probabilistic thinking with global pooling. This approach helps to reduce model parameters and enhance the model performance. The combination of CNN and GRU enables further extraction of high-level features in sequences. In addition, the study shows that appropriate hyperparameters have a positive impact on the model performance. Physicochemical properties can be used to supplement characterization information to improving model performance. The experimental results show that MAHyNet has better performance than other models.

Efficient multiscale spatial attention 3D abdominal multiorgan segmentation model

AbstractAccurate and efficient 3D medical image segmentation models are critical for clinical applications. However, many models which performed well in recent years have consumed significant computational cost, especially those based on self‐attention. For more efficient and accurate segmentation of medical 3D images, we develop an efficient multiscale spatial attention model for 3D abdominal multiorgan segmentation. We use 1D and 2D kernels in the encoder and decoder modules to simulate 3D kernels to reduce the number of parameters and computational cost. Furthermore, we use the lightweight spatial attention and mixed pooling module to capture cross‐dimensional information, multiscale features, and long‐range contextual information. We achieved a Dice Similarity Coefficient (DSC) of 89.86 and a Normalized Surface Dice (NSD) of 78.2 on the FLARE dataset with only 1183 M GPU memory usage, 9 M parameters, and 193G floating point operations(FLOPs).

Improving speech depression detection using transfer learning with wav2vec 2.0 in low-resource environments

Depression, a pervasive global mental disorder, profoundly impacts daily lives. Despite numerous deep learning studies focused on depression detection through speech analysis, the shortage of annotated bulk samples hampers the development of effective models. In response to this challenge, our research introduces a transfer learning approach for detecting depression in speech, aiming to overcome constraints imposed by limited resources. In the context of feature representation, we obtain depression-related features by fine-tuning wav2vec 2.0. By integrating 1D-CNN and attention pooling structures, we generate advanced features at the segment level, thereby enhancing the model's capability to capture temporal relationships within audio frames. In the realm of prediction results, we integrate LSTM and self-attention mechanisms. This incorporation assigns greater weights to segments associated with depression, thereby augmenting the model's discernment of depression-related information. The experimental results indicate that our model has achieved impressive F1 scores, reaching 79% on the DAIC-WOZ dataset and 90.53% on the CMDC dataset. It outperforms recent baseline models in the field of speech-based depression detection. This provides a promising solution for effective depression detection in low-resource environments.

Ensuring spatial scalability with temporal-wise spatial attentive pooling for temporal action detection

Recent temporal action detection models have focused on end-to-end trainable approaches to utilize the representational power of backbone networks. Despite the advantages of end-to-end trainable methods, these models still employ a small spatial resolution (e.g., 96 × 96) due to the inefficient trade-off between computational cost and spatial resolution. In this study, we argue that a simple pooling method (e.g., adaptive average pooling) acts as a bottleneck at the spatial aggregation part, restricting representational power. To address this issue, we propose a temporal-wise spatial attentive pooling (TSAP), which alleviates the bottleneck between the backbone and the detection head using a temporal-wise attention mechanism. Our approach mitigates the inefficient trade-off between spatial resolution and computational cost, thereby enhancing spatial scalability in temporal action detection. Moreover, TSAP is adaptable to previous end-to-end approaches by simply replacing the spatial pooling part. Our experiments demonstrated the essential role of spatial aggregation, and consistent improvements are observed by incorporating TSAP into previous end-to-end methods.

EMPT: a sparsity Transformer for EEG-based motor imagery recognition.

Transformer network is widely emphasized and studied relying on its excellent performance. The self-attention mechanism finds a good solution for feature coding among multiple channels of electroencephalography (EEG) signals. However, using the self-attention mechanism to construct models on EEG data suffers from the problem of the large amount of data required and the complexity of the algorithm. We propose a Transformer neural network combined with the addition of Mixture of Experts (MoE) layer and ProbSparse Self-attention mechanism for decoding the time-frequency-spatial domain features from motor imagery (MI) EEG of spinal cord injury patients. The model is named as EEG MoE-Prob-Transformer (EMPT). The common spatial pattern and the modified s-transform method are employed for achieving the time-frequency-spatial features, which are used as feature embeddings to input the improved transformer neural network for feature reconstruction, and then rely on the expert model in the MoE layer for sparsity mapping, and finally output the results through the fully connected layer. EMPT achieves an accuracy of 95.24% on the MI EEG dataset for patients with spinal cord injury. EMPT has also achieved excellent results in comparative experiments with other state-of-the-art methods. The MoE layer and ProbSparse Self-attention inside the EMPT are subjected to visualisation experiments. The experiments prove that sparsity can be introduced to the Transformer neural network by introducing MoE and kullback-leibler divergence attention pooling mechanism, thereby enhancing its applicability on EEG datasets. A novel deep learning approach is presented for decoding EEG data based on MI.

A Lightweight 6D Pose Estimation Network Based on Improved Atrous Spatial Pyramid Pooling

It is difficult for lightweight neural networks to produce accurate 6DoF pose estimation effects due to their accuracy being affected by scale changes. To solve this problem, we propose a method with good performance and robustness based on previous research. The enhanced PVNet-based method uses depth-wise convolution to build a lightweight network. In addition, coordinate attention and atrous spatial pyramid pooling are used to ensure accuracy and robustness. This method effectively reduces the network size and computational complexity and is a lightweight 6DoF pose estimation method based on monocular RGB images. Experiments on public datasets and self-built datasets show that the average ADD(-S) estimation accuracy and 2D projection index of the improved method are improved. For datasets with large changes in object scale, the estimation accuracy of the average ADD(-S) is greatly improved.