Recent Deep Learning Methods Research Articles

CCF-GNN: A Unified Model Aggregating Appearance, Microenvironment, and Topology for Pathology Image Classification.

Pathology images contain rich information of cell appearance, microenvironment, and topology features for cancer analysis and diagnosis. Among such features, topology becomes increasingly important in analysis for cancer immunotherapy. By analyzing geometric and hierarchically structured cell distribution topology, oncologists can identify densely-packed and cancer-relevant cell communities (CCs) for making decisions. Compared to commonly-used pixel-level Convolution Neural Network (CNN) features and cell-instance-level Graph Neural Network (GNN) features, CC topology features are at a higher level of granularity and geometry. However, topological features have not been well exploited by recent deep learning (DL) methods for pathology image classification due to lack of effective topological descriptors for cell distribution and gathering patterns. In this paper, inspired by clinical practice, we analyze and classify pathology images by comprehensively learning cell appearance, microenvironment, and topology in a fine-to-coarse manner. To describe and exploit topology, we design Cell Community Forest (CCF), a novel graph that represents the hierarchical formulation process of big-sparse CCs from small-dense CCs. Using CCF as a new geometric topological descriptor of tumor cells in pathology images, we propose CCF-GNN, a GNN model that successively aggregates heterogeneous features (e.g., appearance, microenvironment) from cell-instance-level, cell-community-level, into image-level for pathology image classification. Extensive cross-validation experiments show that our method significantly outperforms alternative methods on H&E-stained and immunofluorescence images for disease grading tasks with multiple cancer types. Our proposed CCF-GNN establishes a new topological data analysis (TDA) based method, which facilitates integrating multi-level heterogeneous features of point clouds (e.g., for cells) into a unified DL framework.

LoGE: an unsupervised local-global document extension generation in information retrieval for long documents

PurposeThis paper aims to manage the word gap in information retrieval (IR) especially for long documents belonging to specific domains. In fact, with the continuous growth of text data that modern IR systems have to manage, existing solutions are needed to efficiently find the best set of documents for a given request. The words used to describe a query can differ from those used in related documents. Despite meaning closeness, nonoverlapping words are challenging for IR systems. This word gap becomes significant for long documents from specific domains.Design/methodology/approachTo generate new words for a document, a deep learning (DL) masked language model is used to infer related words. Used DL models are pretrained on massive text data and carry common or specific domain knowledge to propose a better document representation.FindingsThe authors evaluate the approach of this study on specific IR domains with long documents to show the genericity of the proposed model and achieve encouraging results.Originality/valueIn this paper, to the best of the authors’ knowledge, an original unsupervised and modular IR system based on recent DL methods is introduced.

Identifying Laryngeal Neoplasms in Laryngoscope Images via Deep Learning Based Object Detection: A Case Study on an Extremely Small Data Set

ObjectivesLaryngoscopy is a medical procedure for obtaining a view of the human larynx. It is challenging for clinicians to distinguish laryngeal neoplasms by human visual observation. Recent deep learning methods can assist clinicians in improving the accuracy of distinguishing. However, existed methods are often trained on large-scale private datasets, while other researchers and hospitals can neither access these private datasets nor afford to build such large-scale datasets. In this paper, we focus on identifying laryngeal neoplasms under the “small data” regime, which is more important for many small hospitals to investigate deep learning models for diagnosis. Material and methodsWe build an extremely small dataset consisting of 279 laryngoscopic images of different categories. We found that traditional deep learning models for image classification cannot achieve satisfactory performance for small data, due to the great variability of recording laryngoscopic images and the small area of the neoplasms. To address these difficulties, we propose to employ object detection methods for this small data problem. Concretely, a Faster R-CNN is implemented here, which combines the DropBlock regularization technique to alleviate overfitting additionally. ResultsCompared to previous methods, our model is more robust to overfitting and can predict the location and category of detected neoplasms simultaneously. Our method achieves 73.00% overall accuracy, which is higher than the average of clinicians (65.05%) and the recent state-of-the-art classification method (65.00%). ConclusionThe proposed method shows great ability to detect both the category and location of neoplasms and can be served as a screening tool to help the final decisions of clinicians.

Structure-based prediction of nucleic acid binding residues by merging deep learning- and template-based approaches.

Accurate prediction of nucleic binding residues is essential for the understanding of transcription and translation processes. Integration of feature- and template-based strategies could improve the prediction of these key residues in proteins. Nevertheless, traditional hybrid algorithms have been surpassed by recently developed deep learning-based methods, and the possibility of integrating deep learning- and template-based approaches to improve performance remains to be explored. To address these issues, we developed a novel structure-based integrative algorithm called NABind that can accurately predict DNA- and RNA-binding residues. A deep learning module was built based on the diversified sequence and structural descriptors and edge aggregated graph attention networks, while a template module was constructed by transforming the alignments between the query and its multiple templates into features for supervised learning. Furthermore, the stacking strategy was adopted to integrate the above two modules for improving prediction performance. Finally, a post-processing module dependent on the random walk algorithm was proposed to further correct the integrative predictions. Extensive evaluations indicated that our approach could not only achieve excellent performance on both native and predicted structures but also outperformed existing hybrid algorithms and recent deep learning methods. The NABind server is available at http://liulab.hzau.edu.cn/NABind/.

Deep learning models for webcam eye tracking in online experiments

Eye tracking is prevalent in scientific and commercial applications. Recent computer vision and deep learning methods enable eye tracking with off-the-shelf webcams and reduce dependence on expensive, restrictive hardware. However, such deep learning methods have not yet been applied and evaluated for remote, online psychological experiments. In this study, we tackle critical challenges faced in remote eye tracking setups and systematically evaluate appearance-based deep learning methods of gaze tracking and blink detection. From their own homes and laptops, 65 participants performed a battery of eye tracking tasks including (i) fixation, (ii) zone classification, (iii) free viewing, (iv) smooth pursuit, and (v) blink detection. Webcam recordings of the participants performing these tasks were processed offline through appearance-based models of gaze and blink detection. The task battery required different eye movements that characterized gaze and blink prediction accuracy over a comprehensive list of measures. We find the best gaze accuracy to be 2.4° and precision of 0.47°, which outperforms previous online eye tracking studies and reduces the gap between laboratory-based and online eye tracking performance. We release the experiment template, recorded data, and analysis code with the motivation to escalate affordable, accessible, and scalable eye tracking that has the potential to accelerate research in the fields of psychological science, cognitive neuroscience, user experience design, and human–computer interfaces.

HGV4Risk: Hierarchical Global View-guided Sequence Representation Learning for Risk Prediction

Risk prediction, usually achieved by learning representations from patient’s physiological sequence or user’s behavioral sequence data, and has been widely applied in healthcare and finance. Despite that, some recent time-aware deep learning methods have led to superior performances in such sequence representation learning tasks, such improvement is limited due to a lack of guidance from hierarchical global view. To address this issue, we propose a novel end-to-end H ierarchical G lobal V iew-guided (HGV) sequence representation learning framework. Specifically, the Global Graph Embedding (GGE) module is proposed to learn sequential clip-aware representations from temporal correlation graph (TCG) at instance level. Furthermore, following the way of key-query attention, the harmonic β-attention (β-Attn) is also developed for making a global tradeoff between time-aware decay and observation significance at channel level adaptively. Moreover, the hierarchical representations at both instance level and channel level can be coordinated by the heterogeneous information aggregation under the guidance of global view. Experimental results on both healthcare risk prediction benchmark and SMEs credit overdue risk prediction task from the real-world industrial scenario in MYBank, Ant Group, have illustrated that the proposed model can achieve competitive prediction performance compared with other known baselines. The code has been released public available at: https://github.com/LiYouru0228/HGV.

Low-rank tensor assisted K-space generative model for parallel imaging reconstruction

Although recent deep learning methods, especially generative models, have shown good performance in magnetic resonance imaging, there is still much room for improvement. Considering that the sample number and internal dimension in score-based generative model have key influence on estimating the gradients of data distribution, we present a new idea for parallel imaging reconstruction, named low-rank tensor assisted k-space generative model (LR-KGM). It means that we transform low-rank information into high-dimensional prior information for learning. More specifically, the multi-channel data is constructed into a large Hankel matrix to reduce the number of training samples, which is subsequently collapsed into a tensor for the stage of prior learning. In the testing phase, the low-rank rotation strategy is utilized to impose low-rank constraints on the output tensors of the generative network. Furthermore, we alternate the reconstruction between traditional generative iterations and low-rank high-dimensional tensor iterations. Experimental comparisons with the state-of-the-arts demonstrated that the proposed LR-KGM method achieved better performance.

Research on early diagnosis of Alzheimer's disease based on dual fusion cluster graph convolutional network

ObjectiveMild Cognitive Impairment (MCI) is an early stage of Alzheimer's Disease (AD), often mistaken for natural aging. Early detection and treatment of MCI are crucial for effective treatment, but the condition can be difficult to diagnose. In recent years, multi-modal data and deep learning methods have shown promise in this field. The objective of this study is to develop a computer-aided MCI diagnosis system that effectively processes multi-modal data using deep learning methods. MethodWe proposed a Dual Fusion Cluster Graph Convolution Network (DFCGCN) model, which combines two channels of feature extraction, one adjacency matrix, and the Cluster GCN in series. Brain imaging is downsampled using graph pooling and flattened into sparse vectors, from which advanced features are extracted. Similarity between connectivity matrices is calculated using the Gaussian kernel function and combined with non-imaging details to construct a population graph that better represents inter-subject variability. Finally, features are assigned to subjects in the population graph, and node embeddings are learned using Cluster GCN to output diagnostic results. ResultWe tested the proposed algorithm on the public Alzheimer's Disease Neuroimaging Initiative (ADNI) dataset, achieving an accuracy, sensitivity, and specificity of 90.7%, 91.1%, and 94.0%, respectively. ConclusionThe DFCGCN model presented in this study enhances the diagnosis of MCI and outperforms other state-of-the-art algorithms. This approach has potential to be a valuable tool for early detection and treatment of MCI.

ShadowFormer: Global Context Helps Shadow Removal

Recent deep learning methods have achieved promising results in image shadow removal. However, most of the existing approaches focus on working locally within shadow and non-shadow regions, resulting in severe artifacts around the shadow boundaries as well as inconsistent illumination between shadow and non-shadow regions. It is still challenging for the deep shadow removal model to exploit the global contextual correlation between shadow and non-shadow regions. In this work, we first propose a Retinex-based shadow model, from which we derive a novel transformer-based network, dubbed ShandowFormer, to exploit non-shadow regions to help shadow region restoration. A multi-scale channel attention framework is employed to hierarchically capture the global information. Based on that, we propose a Shadow-Interaction Module (SIM) with Shadow-Interaction Attention (SIA) in the bottleneck stage to effectively model the context correlation between shadow and non-shadow regions. We conduct extensive experiments on three popular public datasets, including ISTD, ISTD+, and SRD, to evaluate the proposed method. Our method achieves state-of-the-art performance by using up to 150X fewer model parameters.

Advancements in Named Entity Recognition using Deep Learning Techniques: A Comprehensive Study on Emerging Trends

Because of the amount of computer-processable text material and processing hardware, deep learning architecture has been the focus of knowledge projects. The task of named entity recognition in natural language processing is no exception. Because of the volume of text material that computers can process and their processing capability, knowledge projects are now focusing on deep learning architecture. Named entity recognition, is a frequently occurring task in natural language processing. Deep learning models, particularly recurrent neural net- works (RNNs) and their variations such as gated recurrent units (GRUs) and long short-term memory (LSTM) , have revolutionised natural language pro- cessing (NER) by capturing intricate contextual connections. These models can accurately forecast the borders and types of named entities because they can effectively capture the sequential and hierarchical patterns inside text. However, using deep learning in NER is a challenging task in itself. Annotated training data is critical for building reliable models, but getting labelled data for all item kinds and domains can be time-consuming. To solve this difficulty, transfer learn- ing and domain adaptation strategies have arisen, using pre-trained models and adapting them to various domains or target tasks. This paper looks at recent deep learning methods for NER and how they evolved from older linear learning approaches. It also analyses the status of activities that are either upstream or downstream of NER, such as sequence tagging and entity linking, among other things

Deep Learning Methods for Identification of White Matter Fiber Tracts: Review of State-of-the-Art and Future Prospective.

Quantitative analysis of white matter fiber tracts from diffusion Magnetic Resonance Imaging (dMRI) data is of great significance in health and disease. For example, analysis of fiber tracts related to anatomically meaningful fiber bundles is highly demanded in pre-surgical and treatment planning, and the surgery outcome depends on accurate segmentation of the desired tracts. Currently, this process is mainly done through time-consuming manual identification performed by neuro-anatomical experts. However, there is a broad interest in automating the pipeline such that it is fast, accurate, and easy to apply in clinical settings and also eliminates the intra-reader variabilities. Following the advancements in medical image analysis using deep learning techniques, there has been a growing interest in using these techniques for the task of tract identification as well. Recent reports on this application show that deep learning-based tract identification approaches outperform existing state-of-the-art methods. This paper presents a review of current tract identification approaches based on deep neural networks. First, we review the recent deep learning methods for tract identification. Next, we compare them with respect to their performance, training process, and network properties. Finally, we end with a critical discussion of open challenges and possible directions for future works.

Evaluating established deep learning methods in constructing integrated remote sensing drought index: A case study in China

Agricultural drought seriously threatens the food and ecological security of most of the world’s developing countries. Data-driven integrated agricultural drought index with remote sensing provides an effective tool to monitor, evaluate, and predict the agricultural drought. However, there is still a lack of comprehensive analytical work on taking the most effective machine learning (ML) and deep learning (DL) methods to construct such integrated drought index. In other words, it is still unclear whether the recent DL methods can improve integrated drought monitoring as compared with the currently widely used ML methods. Therefore, we critically evaluated the performances of four representative DL methods (represents the four currently popular DL network types) i.e., Entity Embedding Deep Neural Network (EEDNN), One-dimensional Convolutional Neural Network (1D-CNN), Gated Recurrent Unit (GRU), and Self-Attention Mechanism (SAM) and three widely used tree-based ML methods i.e., Cubist, Random Forest (RF), and Light Gradient Boosting Machine (LGBM), through constructing a QuickDRI like integrated drought index (abbreviated as QuickDRI-China). About 30 years of meteorological data, 14 years of remote sensing data, and various biophysical variables in China such as land use/land cover, available water capacity, irrigated agriculture, elevation, and ecoregion were employed in this study. Results showed that the EEDNN performed best, followed by the RF and LGBM, and then the other methods including the currently wide used Cubist, according to the station accuracy evaluations, spatial description evaluations, and responses to specific drought event. The tree-based ML methods such as RF and LGBM are still competitive in constructing the integrated agricultural drought index at the current stage. However, the higher accuracy, the smoother spatial description, and the more responsive ability of the EEDNN demonstrate great potential of DL methods. The future integrated agricultural drought monitoring with remote sensing should develop a specialized DL network for heterogeneous agricultural drought features.

Recurrent Encoder–Decoder Networks for Vessel Trajectory Prediction With Uncertainty Estimation

Recent deep learning methods for vessel trajectory prediction are able to learn complex maritime patterns from historical Automatic Identification System (AIS) data and accurately predict sequences of future vessel positions with a prediction horizon of several hours. However, in maritime surveillance applications, reliably quantifying the prediction uncertainty can be as important as obtaining high accuracy. This paper extends deep learning frameworks for trajectory prediction tasks by exploring how recurrent encoder-decoder neural networks can be tasked not only to predict but also to yield a corresponding prediction uncertainty via Bayesian modeling of aleatoric and epistemic uncertainties. We compare the prediction performance of two different models based on labeled or unlabeled input data to highlight how uncertainty quantification and accuracy can be improved by using, if available, additional information on the intention of the ship (e.g., its planned destination).

A Systematic Study of Artificial Intelligence-Based Methods for Detecting Brain Tumors

The brain is regarded as one of the most effective body-controlling organs. The development of technology has enabled the early and accurate detection of brain tumors, which makes a significant difference in their treatment. The adoption of AI has grown substantially in the arena of neurology. This systematic review compares recent Deep Learning (DL), Machine Learning (ML), and hybrid methods for detecting brain cancers. This article evaluates 36 recent articles on these techniques, considering datasets, methodology, tools used, merits, and limitations. The articles contain comprehensible graphs and tables. The detection of brain tumors relies heavily on ML techniques such as Support Vector Machines (SVM) and Fuzzy C-Means (FCM). Recurrent Convolutional Neural Networks (RCNN), DenseNet, Convolutional Neural Networks (CNN), ResNet, and Deep Neural Networks (DNN) are DL techniques used to detect brain tumors more efficiently. DL and ML techniques are merged to develop hybrid techniques. In addition, a summary of the various image processing steps is provided. The systematic review identifies outstanding issues and future goals for DL and ML-based techniques for detecting brain tumors. Through a systematic review, the most effective method for detecting brain tumors can be identified and utilized for improvement.

Dual Channel Feature Attention-Based Approach for RUL Prediction Considering the Spatiotemporal Difference of Multisensor Data

The remaining useful life (RUL) prediction has always been the key technology to realize predictive maintenance. An accurate prediction can give decision-makers a reliable reference to develop maintenance schedules and adjust production planning. When dealing with the spatiotemporal data of multisensor system, recent deep learning (DL) methods, however, still remain unexplored to weigh the contributions from both spatial and temporal dimensions. In this article, we propose a novel DL-based approach with dual channel feature attention (DCFA) modules. First, the two-individual feature attention branches are used to automatically weigh the input on both time and spatial domain, which helps the model to focus more attention on the important elements. Then multilayer bidirectional long short-term memory (Bi-LSTM) and convolutional neural networks are used to extract the high-level features. Finally, a fusion network will combine the features to estimate the RUL. Evaluation experiments are conducted on the C-MAPSS dataset to verify the performance of the proposed model. The results show that the proposed model outperforms other state-of-the-art approaches.

CF3DNet: A learning-based approach for single-shot 3D reconstruction from circular fringes

3D reconstruction by fringe projection is an ill-posed problem that is of significance in many practical applications. Conventional algorithms employ multiple steps like fringe analysis, phase unwrapping, and calibration to reconstruct the 3D profile. Recent deep learning (DL) methods circumvent this multi-step approach by directly mapping the deformations in linear fringes caused by objects to the absolute phase shifts. However, this mapping is not one-to-one for the discontinuous objects due to 2π periodicity of linear fringes, and hence it is impossible to reconstruct using a single deformed fringe. In this work, we propose Circular Fringe-to-3D reconstruction Network (CF3DNet), a learning-based approach for 3D reconstruction from circular fringes, that can establish a one-to-one mapping between the phase deformations and the absolute phase shifts. This is viable because the radially symmetric circular fringes can distinctly record the lateral shifts caused by objects, and the learning-based approach can uniquely map them to absolute phase shifts. Thus, the proposed CF3DNet can reconstruct discontinuous object profiles, which is infeasible with linear fringe-based methods. Further, to evade the tedious process of real data collection and to bring generalization, the model is trained with synthetically generated smooth, discontinuous and Computer-Aided Design (CAD) object profiles. The results demonstrate that the proposed method led to tenfold improved performance over linear fringe-based methods in 3D reconstruction of various synthetic objects, and accurately estimate the real object profile compared to the other state-of-art methods. The extensive analysis presented in this work shows that the proposed method can work with carrier frequencies as low as 10 cycles/row to reconstruct dynamic ranges up to 100 radians under high noise, paving a way for the development of a cost-effective fringe projection system.

Recent deep learning methods for tabular data

Recent deep learning methods for tabular data

MWSR-YLCA: Improved YOLOv7 Embedded with Attention Mechanism for Nasopharyngeal Carcinoma Detection from MR Images

Nasopharyngeal carcinoma (NPC) is a malignant tumor, and early diagnosis and timely treatment are important for NPC patients. Accurate and reliable detection of NPC lesions in magnetic resonance (MR) images is very helpful for the disease diagnosis. However, recent deep learning methods need to be improved for NPC detection in MR images. Because NPC tumors are invasive and usually small in size, it is difficult to distinguish NPC tumors from the closely connected surrounding tissues in a huge and complex background. In this paper, we propose an automatic detection method, named MWSR-YLCA, to accurately detect NPC lesions in MR images. Specifically, we design two modules, the multi-window settings resampling (MWSR) module and an improved YOLOv7 embedded with a coordinate attention mechanism (YLCA) module, to detect NPC lesions more accurately. First, the MWSR generates a pseudo-color version of MR images based on a multi-window resampling method, which preserves richer information. Subsequently, the YLCA detects the NPC lesion areas more accurately by constructing a novel network based on an improved YOLOv7 framework embedded with the coordinate attention mechanism. The proposed method was validated on an MR image set of 800 NPC patients and obtained 80.1% mAP detection performance with only 4694 data samples. The experimental results show that the proposed MWSR-YLCA method can perform high-accuracy detection of NPC lesions and has superior performance.

Depression screening using hybrid neural network.

Depression is a common cause of increased suicides worldwide, and studies have shown that the number of patients suffering from major depressive disorder (MDD) increased several-fold during the COVID-19 pandemic, highlighting the importance of disease detection and depression management, while increasing the need for effective diagnostic tools. In recent years, machine learning and deep learning methods based on electroencephalography (EEG) have achieved significant results in the field of automatic depression detection. However, most current studies have focused on a small number of EEG signal channels, and experimental data require special processing by professionals. In this study, 128 channels of EEG signals were simply filtered and 24-fold leave-one-out cross-validation experiments were performed using 2DCNN-LSTM classifier, support vector machine, K-nearest neighbor and decision tree. The current results show that the proposed 2DCNN-LSTM model has an average classification accuracy of 95.1% with an AUC of 0.98 for depression detection of 6-second participant EEG signals, and the model is much better than 72.05%, 79.7% and 79.49% for support vector machine, K nearest neighbor and decision tree. In addition, we found that the model achieved a 100% probability of correctly classifying the EEG signals of 300-second participants.

EEG Reconstruction with a Dual-scale CNN-LSTM Model for Deep Artifact Removal.

Artifact removal has been an open critical issue for decades in tasks centering on EEG analysis. Recent deep learning methods mark a leap forward from the conventional signal processing routines; however, those in general still suffer from insufficient capabilities 1) to capture potential temporal dependencies embedded in EEG and 2) to adapt to scenarios without a priori knowledge of artifacts. This study proposes an approach (namely DuoCL) to deep artifact removal with a dual-scale CNN (Convolutional Neural Network)-LSTM (Long Short-Term Memory) model, operating on the raw EEG in three phases: 1) Morphological Feature Extraction, a dual-branch CNN utilizes convolution kernels of two different scales to learn morphological features (individual sample); 2) Feature Reinforcement, the dual-scale features are then reinforced with temporal dependencies (inter-sample) captured by LSTM; and 3) EEG Reconstruction, the resulting feature vectors are finally aggregated to reconstruct the artifact-free EEG via a terminal fully connected layer. Extensive experiments have been performed to compare DuoCL to six state-of-the-art counterparts (e.g., 1D-ResCNN and NovelCNN). DuoCL can reconstruct more accurate waveforms and achieve the highest SNR & correlation ( CC) as well as the lowest error ( RRMSEt & RRMSEf). In particular, DuoCL holds potentials in providing a high-quality removal of unknown and hybrid artifacts.