Recent Deep Learning Methods Research Articles

Geometry-Aware Enhanced Mutual-Supervised Point Elimination with Overlapping Mask Contrastive Learning for Partitial Point Cloud Registration

Point cloud registration is one of the fundamental tasks in computer vision, but faces challenges under low overlap conditions. Recent approaches use transformers and overlapping masks to improve perception, but mask learning only considers Euclidean distances between features, ignores mismatches caused by fuzzy geometric structures, and is often computationally inefficient. To address these issues, we introduce a novel matching framework. Firstly, we fuse adaptive graph convolution with PPF features to obtain rich feature perception. Subsequently, we construct a PGT framework that uses GeoTransformer and combines it with location information encoding to enhance the geometry perception between source and target clouds. In addition, we improve the visibility of overlapping regions through information exchange and the AIS module, aiming at subsequent keypoint extraction, preserving points with distinct geometrical structures while suppressing the influence of non-overlapping regions to improve computational efficiency. Finally, the mask is refined through contrast learning to preserve geometric and distance similarity, which helps to compute the transformation parameters more accurately. We have conducted comprehensive experiments on synthetic and real-world scene datasets, demonstrating superior registration performance compared to recent deep learning methods. Our approach shows remarkable improvements of 68.21% in RRMSE and 76.31% in tRMSE on synthetic data, while also excelling in real-world scenarios with enhancements of 76.46% in RRMSE and 45.16% in tRMSE.

A novel image inpainting method based on a modified Lengyel–Epstein model

With the increasing popularity of digital images, developing advanced algorithms that can accurately reconstruct damaged images while maintaining high visual quality is crucial. Traditional image restoration algorithms often struggle with complex structures and details, while recent deep learning methods, though effective, face significant challenges related to high data dependency and computational costs. To resolve these challenges, we propose a novel image inpainting model, which is based on a modified Lengyel–Epstein (LE) model. We discretize the modified LE model by using an explicit Euler algorithm. A series of restoration experiments are conducted on various image types, including binary images, grayscale images, index images, and color images. The experimental results demonstrate the effectiveness and robustness of the method, and even under complex conditions of noise interference and local damage, the proposed method can exhibit excellent repair performance. To quantify the fidelity of these restored images, we use the peak signal-to-noise ratio (PSNR), a widely accepted metric in image processing. The calculation results further demonstrate the applicability of our model across different image types. Moreover, by evaluating CPU time, our method can achieve ideal repair results within a remarkably brief duration. The proposed method validates significant potential for real-world applications in diverse domains of image restoration and enhancement.

Advances in Protein-Ligand Binding Affinity Prediction via Deep Learning: A Comprehensive Study of Datasets, Data Preprocessing Techniques, and Model Architectures.

Drug discovery is a complex and expensive procedure involving several timely and costly phases through which new potential pharmaceutical compounds must pass to get approved. One of these critical steps is the identification and optimization of lead compounds, which has been made more accessible by the introduction of computational methods, including deep learning (DL) techniques. Diverse DL model architectures have been put forward to learn the vast landscape of interaction between proteins and ligands and predict their affinity, helping in the identification of lead compounds. This survey fills a gap in previous research by comprehensively analyzing the most commonly used datasets and discussing their quality and limitations. It also offers a comprehensive classification of the most recent DL methods in the context of protein-ligand binding affinity prediction, providing a fresh perspective on this evolving field. We thoroughly examine commonly used datasets for BAP and their inherent characteristics. Our exploration extends to various preprocessing steps and DL techniques, including graph neural networks, convolutional neural networks, and transformers, which are found in the literature. We conducted extensive literature research to ensure that the most recent deep learning approaches for BAP were included by the time of writing this manuscript. The systematic approach used for the present study highlighted inherent challenges to BAP via DL, such as data quality, model interpretability, and explainability, and proposed considerations for future research directions. We present valuable insights to accelerate the development of more effective and reliable DL models for BAP within the research community. The present study can considerably enhance future research on predicting affinity between protein and ligand molecules, hence further improving the overall drug development process.

LightingFormer: Transformer-CNN hybrid network for low-light image enhancement

Recent deep-learning methods have shown promising results in low-light image enhancement. However, current methods often suffer from noise and artifacts, and most are based on convolutional neural networks, which have limitations in capturing long-range dependencies resulting in insufficient recovery of extremely dark parts in low-light images. To tackle these issues, this paper proposes a novel Transformer-based low-light image enhancement network called LightingFormer. Specifically, we propose a novel Transformer-CNN hybrid block that captures global and local information via mixed attention. It combines the advantages of the Transformer in capturing long-range dependencies and the advantages of CNNs in extracting low-level features and enhancing locality to recover extremely dark parts and enhance local details in low-light images. Moreover, we adopt the U-Net discriminator to enhance different regions in low-light images adaptively, avoiding overexposure or underexposure, and suppressing noise and artifacts. Extensive experiments show that our method outperforms the state-of-the-art methods quantitatively and qualitatively. Furthermore, the application to object detection demonstrates the potential of our method in high-level vision tasks.

Advanced petrographic thin section segmentation through deep learning-integrated adaptive GLFIF

In geological research, precise segmentation of sandstone thin sections is crucial for detailed subsurface material analysis. Traditional methods often fall short in accurately capturing the complexities of these samples. This study presents an innovative segmentation approach that integrates an adaptive Global and Local Fuzzy Image Fitting (GLFIF) algorithm with Otsu's thresholding, significantly enhancing segmentation accuracy and efficiency. Our method combines deep learning and traditional image processing techniques. The adaptive GLFIF algorithm, powered by deep learning, automates parameter tuning, thereby reducing manual intervention and improving precision. Unlike conventional methods that learn fixed parameters, our model dynamically adjusts the segmentation process to achieve accurate results. The dual-phase segmentation strategy effectively isolates small features and handles intricate boundaries, ensuring high-quality outcomes. Experimental results demonstrate that our approach improves segmentation accuracy by 11.2% (from 82.6% to 93.8%), the Jaccard index by 15.4% (from 76.8% to 92.2%), and the Dice coefficient by 9% (from 86.9% to 95.9%) compared to traditional methods. This technique bridges the gap between conventional image analysis and deep learning, combining precise segmentation with the automation and computational power of advanced algorithms. Our segmentation algorithm represents a significant advancement in automated petrographic thin section analysis. Traditional image processing methods, such as thresholding and level sets, excel in handling small objects and complex boundaries but require significant manual intervention and cannot achieve full automation. Recent deep learning methods, particularly semantic segmentation, offer end-to-end automation but struggle with small targets and intricate boundaries. Our approach effectively combines the strengths of both methodologies, providing a comprehensive and efficient solution for geological image analysis that ensures both high accuracy and full automation.

Data-driven quantum chemical property prediction leveraging 3D conformations with Uni-Mol+

Quantum chemical (QC) property prediction is crucial for computational materials and drug design, but relies on expensive electronic structure calculations like density functional theory (DFT). Recent deep learning methods accelerate this process using 1D SMILES or 2D graphs as inputs but struggle to achieve high accuracy as most QC properties depend on refined 3D molecular equilibrium conformations. We introduce Uni-Mol+, a deep learning approach that leverages 3D conformations for accurate QC property prediction. Uni-Mol+ first generates a raw 3D conformation using RDKit then iteratively refines it towards DFT equilibrium conformation using neural networks, which is finally used to predict the QC properties. To effectively learn this conformation update process, we introduce a two-track Transformer model backbone and a novel training approach. Our benchmarking results demonstrate that the proposed Uni-Mol+ significantly improves the accuracy of QC property prediction in various datasets.

GalaxyDock-DL: Protein-Ligand Docking by Global Optimization and Neural Network Energy.

With the recent introduction of deep learning techniques into the prediction of biomolecular structures, structure prediction performance has significantly improved, and the potential for biomedical applications has increased considerably. The prediction of protein-ligand complex structures, applicable to the atomistic understanding of biomolecular functions and the effective design of drug molecules, has also improved with the introduction of deep learning. In this paper, it is demonstrated that docking performance can be greatly enhanced by training an energy function that encapsulates physical effects using deep learning within the framework of the traditional protein-ligand docking method. The advantage of this method, called GalaxyDock-DL, lies in its minimal overfitting to the training data compared to several existing deep learning-based protein-ligand docking methods. Unlike some recent deep learning methods, it does not use information about known binding pocket center positions. Instead, the results of this docking method show a systematic dependence on the physical properties of the target protein-ligand complexes such as atomic thermal fluctuations and binding affinity. GalaxyDock-DL utilizes the global optimization technique of the conventional protein-ligand docking method, GalaxyDock, and a neural network energy function trained to stabilize the native state compared to non-native states, just as physical free energy does. This physical principle-based approach suggests directions not only for future structure prediction involving structurally flexible biomolecular complexes but also for predicting binding affinity, thereby providing guidance for the effective design of biofunctional ligands.

Graph Transformer Network Incorporating Sparse Representation for Multivariate Time Series Anomaly Detection

Cyber–physical systems (CPSs) serve as the pivotal core of Internet of Things (IoT) infrastructures, such as smart grids and intelligent transportation, deploying interconnected sensing devices to monitor operating status. With increasing decentralization, the surge in sensor devices expands the potential vulnerability to cyber attacks. It is imperative to conduct anomaly detection research on the multivariate time series data that these sensors produce to bolster the security of distributed CPSs. However, the high dimensionality, absence of anomaly labels in real-world datasets, and intricate non-linear relationships among sensors present considerable challenges in formulating effective anomaly detection algorithms. Recent deep-learning methods have achieved progress in the field of anomaly detection. Yet, many methods either rely on statistical models that struggle to capture non-linear relationships or use conventional deep learning models like CNN and LSTM, which do not explicitly learn inter-variable correlations. In this study, we propose a novel unsupervised anomaly detection method that integrates Sparse Autoencoder with Graph Transformer network (SGTrans). SGTrans leverages Sparse Autoencoder for the dimensionality reduction and reconstruction of high-dimensional time series, thus extracting meaningful hidden representations. Then, the multivariate time series are mapped into a graph structure. We introduce a multi-head attention mechanism from Transformer into graph structure learning, constructing a Graph Transformer network forecasting module. This module performs attentive information propagation between long-distance sensor nodes and explicitly models the complex temporal dependencies among them to enhance the prediction of future behaviors. Extensive experiments and evaluations on three publicly available real-world datasets demonstrate the effectiveness of our approach.

Emotion Detection System for Malayalam Text using Deep Learning and Transformers

Emotion Detection System for Malayalam Text using Deep Learning and Transformers

Faithful Trip Recommender Using Diffusion Guidance (Student Abstract)

Trip recommendation aims to plan user’s travel based on their specified preferences. Traditional heuristic and statistical approaches often fail to capture the intricate nuances of user intentions, leading to subpar performance. Recent deep-learning methods show attractive accuracy but struggle to generate faithful trajectories that match user intentions. In this work, we propose a DDPM-based incremental knowledge injection module to ensure the faithfulness of the generated trajectories. Experiments on two datasets verify the effectiveness of our approach.

ConDENSE: Conditional Density Estimation for Time Series Anomaly Detection

In recent years deep learning methods, based on reconstruction errors, have facilitated huge improvements in unsupervised anomaly detection. These methods make the limiting assumption that the greater the distance between an observation and a prediction the lower the likelihood of that observation. In this paper we propose conDENSE, a novel anomaly detection algorithm, which does not use reconstruction errors but rather uses conditional density estimation in masked autoregressive flows. By directly estimating the likelihood of data, our model moves beyond approximating expected behaviour with a single point estimate, as is the case in reconstruction error models. We show how conditioning on a dense representation of the current trajectory, extracted from a variational autoencoder with a gated recurrent unit (GRU VAE), produces a model that is suitable for periodic datasets, while also improving performance on non-periodic datasets. Experiments on 31 time-series, including real-world anomaly detection benchmark datasets and synthetically generated data, show that the model can outperform state-of-the-art deep learning methods.

Nighttime color constancy using robust gray pixels.

Color constancy is a basic step for achieving stable color perception in both biological visual systems and the image signal processing (ISP) pipeline of cameras. So far, there have been numerous computational models of color constancy that focus on scenes under normal light conditions but are less concerned with nighttime scenes. Compared with daytime scenes, nighttime scenes usually suffer from relatively higher-level noise and insufficient lighting, which usually degrade the performance of color constancy methods designed for scenes under normal light. In addition, there is a lack of nighttime color constancy datasets, limiting the development of relevant methods. In this paper, based on the gray-pixel-based color constancy methods, we propose a robust gray pixel (RGP) detection method by carefully designing the computation of illuminant-invariant measures (IIMs) from a given color-biased nighttime image. In addition, to evaluate the proposed method, a new dataset that contains 513 nighttime images and corresponding ground-truth illuminants was collected. We believe this dataset is a useful supplement to the field of color constancy. Finally, experimental results show that the proposed method achieves superior performance to statistics-based methods. In addition, the proposed method was also compared with recent deep-learning methods for nighttime color constancy, and the results show the method's advantages in cross-validation among different datasets.

Software Tools for 2D Cell Segmentation.

Cell segmentation is an important task in the field of image processing, widely used in the life sciences and medical fields. Traditional methods are mainly based on pixel intensity and spatial relationships, but have limitations. In recent years, machine learning and deep learning methods have been widely used, providing more-accurate and efficient solutions for cell segmentation. The effort to develop efficient and accurate segmentation software tools has been one of the major focal points in the field of cell segmentation for years. However, each software tool has unique characteristics and adaptations, and no universal cell-segmentation software can achieve perfect results. In this review, we used three publicly available datasets containing multiple 2D cell-imaging modalities. Common segmentation metrics were used to evaluate the performance of eight segmentation tools to compare their generality and, thus, find the best-performing tool.

Exploiting Geometric Features via Hierarchical Graph Pyramid Transformer for Cancer Diagnosis Using Histopathological Images.

Cancer is widely recognized as the primary cause of mortality worldwide, and pathology analysis plays a pivotal role in achieving accurate cancer diagnosis. The intricate representation of features in histopathological images encompasses abundant information crucial for disease diagnosis, regarding cell appearance, tumor microenvironment, and geometric characteristics. However, recent deep learning methods have not adequately exploited geometric features for pathological image classification due to the absence of effective descriptors that can capture both cell distribution and gathering patterns, which often serve as potent indicators. In this paper, inspired by clinical practice, a Hierarchical Graph Pyramid Transformer (HGPT) is proposed to guide pathological image classification by effectively exploiting a geometric representation of tissue distribution which was ignored by existing state-of-the-art methods. First, a graph representation is constructed according to morphological feature of input pathological image and learn geometric representation through the proposed multi-head graph aggregator. Then, the image and its graph representation are feed into the transformer encoder layer to model long-range dependency. Finally, a locality feature enhancement block is designed to enhance the 2D local representation of feature embedding, which is not well explored in the existing vision transformers. An extensive experimental study is conducted on Kather-5K, MHIST, NCT-CRC-HE, and GasHisSDB for binary or multi-category classification of multiple cancer types. Results demonstrated that our method is capable of consistently reaching superior classification outcomes for histopathological images, which provide an effective diagnostic tool for malignant tumors in clinical practice.

A Machine Learning Model Based on GRU and LSTM to Predict the Environmental Parameters in a Layer House, Taking CO2 Concentration as an Example.

In a layer house, the CO2 (carbon dioxide) concentration above the upper limit can cause the oxygen concentration to be below the lower limit suitable for poultry. This leads to chronic CO2 poisoning in layers, which manifests as listlessness, reduced appetite, weak constitution, decreased production performance, and weakened resistance to disease. Regulating ventilation may ensure a suitable CO2 concentration in layer houses. Predicting the changes in CO2 concentration and regulating the CO2 concentration in advance are key to ensuring healthy large-scale breeding of layers. In recent years, machine learning and deep learning methods have been increasingly applied to this field. A CO2 prediction model for layer house is proposed based on a GRU (gated recurrent unit) and LSTM (long short-term memory). The temperature, humidity, and CO2 were determined as inputs to the model by the correlation coefficient. The datasets of the experimental layer house were continuously measured during June-July 2023, using a self-developed environmental monitor, and the monitored data were used as samples for model inputs. There were 22,000 time series data in the datasets. In this study, multivariate time series data were standardized via data pre-processing to improve model training. GRU and LSTM models were constructed. The models were trained using a training set. Then, these trained models were used to provide predictions on a test set. The prediction errors were calculated using the true values of the test set and the predicted values provided by the models. To test the performance of the model and accuracy of the predictions, predictions were made for different numbers of datasets. The results demonstrated that the combined prediction model had good generalization, stability, and convergence with high prediction accuracy. Due to the structure of the model, the stability of the LSTM model was higher than that of the GRU model, and its prediction accuracy and speed were lower than those of the GRU model. When the datasets of the GRU model were 15,000 to 17,000, The MAE of the GRU was 70.8077 to 126.7029 ppm, and the prediction time of the GRU is 16 to 24 ms. When the LSTM model's datasets were 15,000-20,000, the MAE of LSTM was 78.8596 to 136.0896 ppm, and the prediction time of the GRU was 17 to 26 ms.

Multi-cell type and multi-level graph aggregation network for cancer grading in pathology images.

In pathology, cancer grading is crucial for patient management and treatment. Recent deep learning methods, based upon convolutional neural networks (CNNs), have shown great potential for automated and accurate cancer diagnosis. However, these do not explicitly utilize tissue/cellular composition, and thus difficult to incorporate the existing knowledge of cancer pathology. In this study, we propose a multi-cell type and multi-level graph aggregation network (MMGA-Net) for cancer grading. Given a pathology image, MMGA-Net constructs multiple cell graphs at multiple levels to represent intra- and inter-cell type relationships and to incorporate global and local cell-to-cell interactions. In addition, it extracts tissue contextual information using a CNN. Then, the tissue and cellular information are fused to predict a cancer grade. The experimental results on two types of cancer datasets demonstrate the effectiveness of MMGA-Net, outperforming other competing models. The results also suggest that the information fusion of multiple cell types and multiple levels via graphs is critical for improved pathology image analysis.

ORRN: An ODE-Based Recursive Registration Network for Deformable Respiratory Motion Estimation With Lung 4DCT Images.

Deformable Image Registration (DIR) plays a significant role in quantifying deformation in medical data. Recent Deep Learning methods have shown promising accuracy and speedup for registering a pair of medical images. However, in 4D (3D + time) medical data, organ motion, such as respiratory motion and heart beating, can not be effectively modeled by pair-wise methods as they were optimized for image pairs but did not consider the organ motion patterns necessary when considering 4D data. This article presents ORRN, an Ordinary Differential Equations (ODE)-based recursive image registration network. Our network learns to estimate time-varying voxel velocities for an ODE that models deformation in 4D image data. It adopts a recursive registration strategy to progressively estimate a deformation field through ODE integration of voxel velocities. We evaluate the proposed method on two publicly available lung 4DCT datasets, DIRLab and CREATIS, for two tasks: 1) registering all images to the extreme inhale image for 3D+t deformation tracking and 2) registering extreme exhale to inhale phase images. Our method outperforms other learning-based methods in both tasks, producing the smallest Target Registration Error of 1.24 mm and 1.26 mm, respectively. Additionally, it produces less than 0.001% unrealistic image folding, and the computation speed is less than 1 s for each CT volume. ORRN demonstrates promising registration accuracy, deformation plausibility, and computation efficiency on group-wise and pair-wise registration tasks. It has significant implications in enabling fast and accurate respiratory motion estimation for treatment planning in radiation therapy or robot motion planning in thoracic needle insertion.

Speed Up of Volumetric Non-Local Transform-Domain Filter Utilising HPC Architecture.

This paper presents a parallel implementation of a non-local transform-domain filter (BM4D). The effectiveness of the parallel implementation is demonstrated by denoising image series from computed tomography (CT) and magnetic resonance imaging (MRI). The basic idea of the filter is based on grouping and filtering similar data within the image. Due to the high level of similarity and data redundancy, the filter can provide even better denoising quality than current extensively used approaches based on deep learning (DL). In BM4D, cubes of voxels named patches are the essential image elements for filtering. Using voxels instead of pixels means that the area for searching similar patches is large. Because of this and the application of multi-dimensional transformations, the computation time of the filter is exceptionally long. The original implementation of BM4D is only single-threaded. We provide a parallel version of the filter that supports multi-core and many-core processors and scales on such versatile hardware resources, typical for high-performance computing clusters, even if they are concurrently used for the task. Our algorithm uses hybrid parallelisation that combines open multi-processing (OpenMP) and message passing interface (MPI) technologies and provides up to 283× speedup, which is a 99.65% reduction in processing time compared to the sequential version of the algorithm. In denoising quality, the method performs considerably better than recent DL methods on the data type that these methods have yet to be trained on.

Abstract 18382: Clinical and Genetic Associations of Deep-Learning Estimated Peak Oxygen Consumption From the Resting 12-Lead Electrocardiogram

Introduction: Oxygen consumption at peak exercise (VO2 peak ) is the gold standard for cardiorespiratory fitness, but prior genome-wide association studies (GWAS) have been limited by the availability of cardiopulmonary exercise testing (CPET). Recent deep learning methods to estimate VO2 peak from the resting 12-lead electrocardiogram (ECG) may enhance genetic studies of cardiorespiratory fitness. Methods: We applied a validated deep learning model (Deep ECG-VO2) to estimate VO2 peak among UK Biobank participants with a 12-lead ECG. We assessed for associations between estimated VO2 peak and incident hypertension, diabetes, and atrial fibrillation (AF) using Cox proportional hazards models adjusted for age and sex, and plotted cumulative risk of each outcome stratified by tertile of estimated VO2 peak . We then performed a multi-ancestry GWAS of estimated VO2 peak using BOLT-LMM, adjusted for age, sex, array, and the first five principal components of ancestry. Candidate genes were prioritized based on proximity to the lead variant. Results: We applied Deep ECG-VO2 to estimate VO2 peak using the resting 12-lead ECG of 40,801 UK Biobank participants (age 65±8, 52% women). Greater estimated VO2 peak was associated with lower risks of hypertension (hazard ratio per 1-standard deviation 0.76, 95% CI 0.70-0.83), diabetes (0.60, 95% CI 0.53-0.67) and AF (0.82, 95% CI 0.74-0.90). Cumulative risk of each outcome was higher with decreasing estimated VO2 peak ( Figure ). In a GWAS of estimated VO2 peak within 39,716 participants with genetic data (age 65±8, 52% women, 90% European), we identified 10 novel genome-wide significant loci, including variants near genes involved in cardiac structure ( CCDC141/TTN, BAG3) , cardiac conduction ( SCN5A ), and adiposity ( FTO ). Conclusions: Leveraging artificial intelligence-enabled estimation of VO2 peak from the resting 12-lead ECG, we identify 10 novel common genetic variants associated with cardiorespiratory fitness.

Semi-supervised point consistency network for retinal artery/vein classification

Recent deep learning methods with convolutional neural networks (CNNs) have boosted advance prosperity of medical image analysis and expedited the automatic retinal artery/vein (A/V) classification. However, it is challenging for these CNN-based approaches in two aspects: (1) specific tubular structures and subtle variations in appearance, contrast, and geometry, which tend to be ignored in CNNs with network layer increasing; (2) limited well-labeled data for supervised segmentation of retinal vessels, which may hinder the effectiveness of deep learning methods. To address these issues, we propose a novel semi-supervised point consistency network (SPC-Net) for retinal A/V classification. SPC-Net consists of an A/V classification (AVC) module and a multi-class point consistency (MPC) module. The AVC module adopts an encoder–decoder segmentation network to generate the prediction probability map of A/V for supervised learning. The MPC module introduces point set representations to adaptively generate point set classification maps of the arteriovenous skeleton, which enjoys its prediction flexibility and consistency (i.e. point consistency) to effectively alleviate arteriovenous confusion. In addition, we propose a consistency regularization between the predicted A/V classification probability maps and point set representations maps for unlabeled data to explore the inherent segmentation perturbation of the point consistency, reducing the need for annotated data. We validate our method on two typical public datasets (DRIVE, HRF) and a private dataset (TR280) with different resolutions. Extensive qualitative and quantitative experimental results demonstrate the effectiveness of our proposed method for supervised and semi-supervised learning.