Introduction Of Deep Learning Research Articles

An Overview of Text to Visual Generation Using GAN

Text-to-visual generation was once a cumbersome task until the advent of deep learning networks. With the introduction of deep learning, both images and videos can now be generated from textual descriptions. Deep learning networks have revolutionized various fields, including computer vision and natural language processing, with the emergence of Generative Adversarial Networks (GANs). GANs have played a significant role in advancing these domains. A GAN typically comprises multiple deep networks combined with other machine learning techniques. In the context of text-to-visual generation, GANs have enabled the synthesis of images and videos based on textual input. This work aims to explore different variations of GANs for image and video synthesis and propose a general architecture for text- to-visual generation using GANs. Additionally, this study delves into the challenges associated with this task and discusses ongoing research and future prospects. By leveraging the power of deep learning networks and GANs, the process of generating visual content from text has become more accessible and efficient. This work will contribute to the understanding and advancement of text-to-visual generation, paving the way for numerous applica-tions across various industries.

A Multi-View learning approach to enhance automatic 12-Lead ECG diagnosis performance

ObjectivesThe electrocardiogram (ECG) has important clinical value for the early diagnosis of cardiovascular diseases. Recently, the performance of existing diagnosis models based on ECG data has improved with the introduction of deep learning (DL). However, the impact of various combinations of multiple DL components and/or the role of augmentation techniques on the diagnosis has not been sufficiently investigated in this field. In this sense, this study aims to design an integrated model consisting of diverse DL-based modules. MethodsHere, an ensemble-based multi-view learning approach with an ECG augmentation technique is proposed to achieve higher performance than traditional automatic 12-lead ECG diagnosis methods. ResultsAccordingly, several experiments have been conducted with CPSC2018 dataset for evaluation. The proposed model reports an F1 score of 0.840, which outperforms existing state-of-the-art methods in the literature. ConclusionThus, this study provides quantitative evidence demonstrating that the multi-view learning approach can be used as a unified algorithmic method in the field of automatic ECG diagnosis. SignificanceIn this work, we reviewed ensembles of models consisting of multiple DL components for automatic ECG diagnosis and proposed the ECG-specific augmentation technique.

Optical Encryption Using Attention-Inserted Physics-Driven Single-Pixel Imaging

Optical encryption based on single-pixel imaging (SPI) has made great advances with the introduction of deep learning. However, the use of deep neural networks usually requires a long training time, and the networks need to be retrained once the target scene changes. With this in mind, we propose an SPI encryption scheme based on an attention-inserted physics-driven neural network. Here, an attention module is used to encrypt the single-pixel measurement value sequences of two images, together with a sequence of cryptographic keys, into a one-dimensional ciphertext signal to complete image encryption. Then, the encrypted signal is fed into a physics-driven neural network for high-fidelity decoding (i.e., decryption). This scheme eliminates the need for pre-training the network and gives more freedom to spatial modulation. Both simulation and experimental results have demonstrated the feasibility and eavesdropping resistance of this scheme. Thus, it will lead SPI-based optical encryption closer to intelligent deep encryption.

Application of Behavior Recognition Technology Based on Deep Learning in Elderly Care

China is currently one of the countries with the largest elderly population in the world, and the issue of population aging has become a widespread concern. The behavior recognition algorithm based on deep learning is currently the main behavior recognition algorithm and one of the basic technologies in the field of computer vision. In existing research, the method of constructing complex classification models based on manual feature representation can no longer meet the requirements of high recognition accuracy and applicability, and the introduction of deep learning has brought new development directions for behavior recognition. Therefore, this article aims to study how to apply deep learning-based behavior recognition technology more accurately and effectively in the care of elderly people in the context of “artificial intelligence.”

An Efficient Single Image De-raining Model with Decoupled Deep Networks.

Single image de-raining is an emerging paradigm for many outdoor computer vision applications since rain streaks can significantly degrade the visibility and render the function compromised. The introduction of deep learning (DL) has brought about substantial advancement on de-raining methods. However, most existing DL-based methods use single homogeneous network architecture to generate de-rained images in a general image restoration manner, ignoring the discrepancy between rain location detection and rain intensity estimation. We find that this discrepancy would cause feature interference and representation ability degradation problems which significantly affect de-raining performance. In this paper, we propose a novel heterogeneous de-raining architecture aiming to decouple rain location detection and rain intensity estimation (DLINet). For these two subtasks, we provide dedicated network structures according to their differential properties to meet their respective performance requirements. To coordinate the decoupled subnetworks, we develop a high-order collaborative network learning the dynamic inter-layer interactions between rain location and intensity. To effectively supervise the decoupled subnetworks during training, we propose a novel training strategy that imposes task-oriented supervision using the label learned via joint training. Extensive experiments on synthetic datasets and real-world rainy scenes demonstrate that the proposed method has great advantages over existing state-of-the-art methods.

Adaptive Knowledge Distillation for High-Quality Unsupervised MRI Reconstruction with Model-Driven Priors.

Magnetic Resonance Imaging (MRI) reconstruction has made significant progress with the introduction of Deep Learning (DL) technology combined with Compressed Sensing (CS). However, most existing methods require large fully sampled training datasets to supervise the training process, which may be unavailable in many applications. Current unsupervised models also show limitations in performance or speed and may face unaligned distributions during testing. This paper proposes an unsupervised method to train competitive reconstruction models that can generate high-quality samples in an end-to-end style. Firstly teacher models are trained by filling the re-undersampled images and compared with the undersampled images in a self-supervised manner. The teacher models are then distilled to train another cascade model that can leverage the entire undersampled k-space during its training and testing. Additionally, we propose an adaptive distillation method to re-weight the samples based on the variance of teachers, which represents the confidence of the reconstruction results, to improve the quality of distillation. Experimental results on multiple datasets demonstrate that our method significantly accelerates the inference process while preserving or even improving the performance compared to the teacher model. In our tests, the distilled models show 5%-10% improvements in PSNR and SSIM compared with no distillation and are 10 times faster than the teacher. Source code of our work could be acquired at https://github.com/BITwzl/unsupervised_mri_reconstruction.

SlideTiler: A dataset creator software for boosting deep learning on histological whole slide images

The introduction of deep learning caused a significant breakthrough in digital pathology. Thanks to its capability of mining hidden data patterns in digitised histological slides to resolve diagnostic tasks and extract prognostic and predictive information. However, the high performance achieved in classification tasks depends on the availability of large datasets, whose collection and preprocessing are still time-consuming processes. Therefore, strategies to make these steps more efficient are worth investigation. This work introduces SlideTiler, an open-source software with a user-friendly graphical interface. SlideTiler can manage several image preprocessing phases through an intuitive workflow that does not require specific coding skills. The software was designed to provide direct access to virtual slides, allowing custom tiling of specific regions of interest drawn by the user, tile labelling, quality assessment, and direct export to dataset directories. To illustrate the functions and the scalability of SlideTiler, a deep learning-based classifier was implemented to classify 4 different tumour histotypes available in the TCGA repository. The results demonstrate the effectiveness of SlideTiler in facilitating data preprocessing and promoting accessibility to digitised pathology images for research purposes. Considering the increasing interest in deep learning applications of digital pathology, SlideTiler has a positive impact on this field. Moreover, SlideTiler has been conceived as a dynamic tool in constant evolution, and more updated and efficient versions will be released in the future.

Explainable End-to-End Supervised Learning Identifies Myelodysplastic Neoplasms in Bone Marrow Smears

Myelodysplastic neoplasms (MDS) encompass clonal myeloid malignancies that are characterized by ineffective hematopoiesis, cytopenia, myelodysplasia, and recurrent genetic events. Apart from genetic abnormalities and the assessment of peripheral blood for cytopenia, cytomorphologic evaluation of bone marrow morphology is key in initial diagnosis and response assessment as well as to detect disease transformation to acute myeloid leukemia (AML). However, the identification of aberrant morphologies in bone marrow cells is challenging and prone to inter-observer variability even for seasoned morphologists resulting in discrepancies between local site and central review. After the introduction of deep learning (DL) models for computer vision tasks, a wide variety of applications in imaging diagnostics has been addressed using convolutional neural networks including the assessment of bone marrow cell types and prediction of hematological malignancies. Bone marrow smear (BMS) images of manually selected regions of interest (ROI) from 483 MDS patients treated at the University Hospital Dresden were captured at 40-fold magnification with a resolution of 2560*1920 pixels and pre-processed. In parallel, BMS images from two control cohorts were taken: (i) 1226 AML patients registered in the German Study Alliance Leukemia (SAL) registry, and (ii) 236 healthy bone marrow donors. In addition, an external validation cohort of 50 MDS patients was obtained from the Munich Leukemia Laboratory (MLL). Image-level labels were provided denoting the corresponding diagnosis: “MDS”, “AML” and “healthy donor”. To balance different sample sizes between the datasets, simple image augmentation techniques such as random sized cropping, color shifting, and linear transformations were used. Training and testing were performed with 5-fold cross-validation, i.e. multiple train-test-splits of 80:20. Importantly, cell-level labels, e.g. specifically annotated cell types or features like distinct dysplastic morphologies were not used to train our model as we intended to abstract visual information exclusively through our neural networks instead of using expert-guided feature engineering. Six different recently introduced convolutional neural network (CNN) architectures were compared (ResNet-34/50/101/152, ResNeXt-50/101, ShuffleNet V2, DenseNet-121/169/201, Wide ResNet-50/101, and SqueezeNet 1.1). To evaluate individual model performance, we used accuracy, recall (=sensitivity), precision (=positive predictive value), and area-under-the-curve (AUC) of both the receiver-operating-characteristic (ROC) and precision-recall-curve (PRC). Binary classification tasks showed high accuracy of 0.956 and 0.981 for distinguishing MDS from healthy donors and AML, respectively (Table I). Results were validated in the external MDS cohort reaching an accuracy of 0.916 for MDS vs. donors and 0.968 for MDS vs. AML (Table I). For explainability, we used occlusion sensitivity mapping which iteratively blocks image areas from being evaluated by the CNNs, measures consecutive drops in prediction performance, and thus identifies image areas of high importance (highlighted in green). By reviewing images, we found the CNNs to target nuclei not only of dysplastic cells but rather of all available cells in a ROI (Figure 1). Using supervised end-to-end deep learning, we developed a software framework to distinguish between MDS, AML, and healthy controls with very high accuracy. Importantly, models do not require laborious feature engineering, i.e. manually drafted, cell-level labels alleviating the need for time-consuming and cost-ineffective manual labeling and thereby bypassing a key bottleneck for current standards of computer vision in microscopy.

Handwritten Character Recognition Using Deep Learning

Handwritten digit or character recognition in transforming the printed or handwritten text from an image. Optical character recognition plays an important role in documentation scanning ,text extractions from the image. Optical character recognition is used in different fields like postal services ,Ecommerce , Shipping ,Banking sector for character extraction from the images . However the existing character recognition system faces many challenges in extracting text from noisy and distortion images or complex layout and Extraction mostly limited to numbers and English alphabets . The introduction of Deep learning has changed Optical Character Recognition by using models like Recurrent Neural Networks,convolutional neural network .In this paper i am gonna compare the different models like CNN model and CRNN model with current State of art model Transformer based Optical Character Recognition KeyWords :Transformers ,Convolution Recurrent Neural Network, Handwritten ,Optical Character Recognition

A Comparative Study of Deep Learning and Machine Learning Techniques in Credit Score Classification

The field of credit score classification has experienced notable progress through the introduction of deep learning (DL) and machine learning (ML) techniques, empowering financial institutions to make well-informed decisions about assessing creditworthiness. However, existing research often focuses on just a few classifiers pertaining to either ML or DL techniques, lacking a comprehensive comparative analysis between the two. This gap calls for a thorough study that evaluates and compares a wide range of ML classifiers and DL models in the context of credit scoring. Our work aims to address this limitation by presenting an extensive comparative analysis between different ML and DL approaches. We provide novel insights into the strengths and weaknesses of each model, enabling financial institutions to select the most suitable approach for their specific needs. Through conducting extensive experiments on a credit records dataset, we evaluated the accuracy, precision, recall, and F1 score of various ML classifiers, such as logistic regression, decision trees, and random forests. Additionally, we delved into the capabilities of DL models, which included multi-layer perceptron (MLP), convolutional neural networks (CNN), recurrent neural networks (RNN), and hybrid models. Our findings revealed that Random Forest achieved the highest test accuracy of 90.27, while MLP and CNN closely followed with the second-highest accuracies at 87.08 and 87.16, respectively. These results also demonstrated the potential of both MLP and CNN in credit scoring assessment. MLP's strength lies in its capacity to handle non-linear relationships between features, providing a viable alternative to decision tree-based models. On the other hand, CNN excels in capturing spatial patterns and dependencies among features, presenting a distinct advantage in credit score classification. Overall, our study presents a broad-spectrum overview of the analysis, encompassing each model's performance and effectiveness in credit score classification. The findings empower financial institutions to leverage the benefits of DL and ML techniques, optimizing their decision-making processes and enhancing risk management strategies. By selecting the most suitable credit score classification model based on the insights gained from this comparative analysis, institutions can make informed choices and effectively evaluate creditworthiness, leading to improved risk assessment and lending decisions.

A Review of Recent Advances on Deep Learning Methods for Audio-Visual Speech Recognition

This article provides a detailed review of recent advances in audio-visual speech recognition (AVSR) methods that have been developed over the last decade (2013–2023). Despite the recent success of audio speech recognition systems, the problem of audio-visual (AV) speech decoding remains challenging. In comparison to the previous surveys, we mainly focus on the important progress brought with the introduction of deep learning (DL) to the field and skip the description of long-known traditional “hand-crafted” methods. In addition, we also discuss the recent application of DL toward AV speech fusion and recognition. We first discuss the main AV datasets used in the literature for AVSR experiments since we consider it a data-driven machine learning (ML) task. We then consider the methodology used for visual speech recognition (VSR). Subsequently, we also consider recent AV methodology advances. We then separately discuss the evolution of the core AVSR methods, pre-processing and augmentation techniques, and modality fusion strategies. We conclude the article with a discussion on the current state of AVSR and provide our vision for future research.

Echoformer: Transformer Architecture Based on Radar Echo Characteristics for UAV Detection

While recent years have witnessed an increasing number of commercial applications of unmanned aerial vehicles (UAVs), an imperative problem people have to face is the rapid growth of malicious use. So, it is imperative for security agencies to develop anti-UAV technology. The introduction of deep learning (DL) has a positive influence on radar signal processing, but DL-based methodologies have yet to be widespread in radar target detection because of the lack of unique architecture based on radar echo characteristics and the annotation method of radar data. In this article, a novel Transformer-based architecture is proposed, which transforms the problem of UAV detection into a binary classification task in each range cell. The complex encoder architecture and the Transformer-based extractor are designed to extract the Doppler frequency shift feature and the micro-Doppler signature (mDS) of a UAV simultaneously. The well-designed architecture based on radar echo characteristics can achieve a combination training of echoes with different coherent processing intervals (CPIs). In addition, we provide an annotation method and a data augmentation skill for our real measured dataset. The results of the experiment demonstrate that the proposed method has better detection performance and measuring accuracy under different SNRs in comparison with traditional radar target detection and other DL-based methods.

Multimodal data fusion for cancer biomarker discovery with deep learning.

Technological advances now make it possible to study a patient from multiple angles with high-dimensional, high-throughput multi-scale biomedical data. In oncology, massive amounts of data are being generated ranging from molecular, histopathology, radiology to clinical records. The introduction of deep learning has significantly advanced the analysis of biomedical data. However, most approaches focus on single data modalities leading to slow progress in methods to integrate complementary data types. Development of effective multimodal fusion approaches is becoming increasingly important as a single modality might not be consistent and sufficient to capture the heterogeneity of complex diseases to tailor medical care and improve personalised medicine. Many initiatives now focus on integrating these disparate modalities to unravel the biological processes involved in multifactorial diseases such as cancer. However, many obstacles remain, including lack of usable data as well as methods for clinical validation and interpretation. Here, we cover these current challenges and reflect on opportunities through deep learning to tackle data sparsity and scarcity, multimodal interpretability, and standardisation of datasets.

Recent Deep Learning Approaches for Object Detection

Object detection, a classic problem in computer vision, has been developed for more than 20 years. From the early traditional methods to today's deep learning methods, the accuracy is getting higher and higher, and the speed is getting faster and faster, which is benefit from deep learning and the continuous development of deep neural networks. Although the research on object detection is constantly developing, there are not many reviews on object detection, so this article will review the object detection after the introduction of deep learning. This article will first introduce the history of object detection, and then focus on a systematic introduction to the development of deep learning object detection in recent years, as well as one-stage detector and two-stage detector in anchor-free and anchor-based. The various methods applied in the stage detector will be sorted out, and the potential problems and future development of object detection will also be analyzed.

Automatic Modulation Recognition Based on Deep-Learning Features Fusion of Signal and Constellation Diagram

In signal communication based on a non-cooperative communication system, the receiver is an unlicensed third-party communication terminal, and the modulation parameters of the transmitter signal cannot be predicted in advance. After the RF signal passes through the RF band-pass filter, low noise amplifier, and image rejection filter, the intermediate frequency signal is obtained by down-conversion, and then the IQ signal is obtained in the baseband by using the intermediate frequency band-pass filter and down-conversion. In this process, noise and signal frequency offset are inevitably introduced. As the basis of subsequent analysis and interpretation, modulation recognition has important research value in this environment. The introduction of deep learning also brings new feature mining tools. Based on this, this paper proposes a signal modulation recognition method based on multi-feature fusion and constructs a deep learning network with a double-branch structure to extract the features of IQ signal and multi-channel constellation, respectively. It is found that through the complementary characteristics of different forms of signals, a more complete signal feature representation can be constructed. At the same time, it can better alleviate the influence of noise and frequency offset on recognition performance, and effectively improve the classification accuracy of modulation recognition.

ANALYSIS ON CAPSULE NETWORKS TO DETECT FORGED IMAGES AND VIDEOS

Modern day computer vision tasks requires efficient solution to problems such as image recognition, natural language processing, object detection, object segmentation and language translation. Symbolic Artificial Intelligence with its hard coding rules is incapable of solving these complex problems resulting in the introduction of Deep Learning (DL) models such as Recurrent Neural Networks and Convolutional Neural Networks (CNN). However, CNNs require lots of training data and are incapable of recognizing pose and deformation of objects leading to the introduction of Capsule Networks. And these are the new sensation in Deep Learning. They have lived to this expectation as their performance in relation to the above problems has been better than Convolutional Neural Networks. Even with this promise in performance, lack of architectural knowledge and inner workings of Capsules serves as a hindrance for researchers to take full advantage of this breakthrough. In this paper, we provide a comprehensive review of the state of the art architectures, tools and methodologies in existing implementations of capsule networks. We highlight the successes, failures and opportunities for further research to serve as a motivation to researchers and industry players to exploit the full potential of this new field. The main contribution of this survey article is that it explains and summarizes significant current state of the art Capsule Network architectures and implementations. The method introduced in this paper uses a capsule network to detect various kinds of spoofs, from replay attacks using printed images or recorded videos to computer generated videos using deep convolutional neural networks. It extends the application of capsule networks beyond their original intention to the solving of inverse graphics problems.

Review on Deep Learning Approaches for Anomaly Event Detection in Video Surveillance

In the last few years, due to the continuous advancement of technology, human behavior detection and recognition have become important scientific research in the field of computer vision (CV). However, one of the most challenging problems in CV is anomaly detection (AD) because of the complex environment and the difficulty in extracting a particular feature that correlates with a particular event. As the number of cameras monitoring a given area increases, it will become vital to have systems capable of learning from the vast amounts of available data to identify any potential suspicious behavior. Then, the introduction of deep learning (DL) has brought new development directions for AD. In particular, DL models such as convolution neural networks (CNNs) and recurrent neural networks (RNNs) have achieved excellent performance dealing with AD tasks, as well as other challenging domains like image classification, object detection, and speech processing. In this review, we aim to present a comprehensive overview of those research methods using DL to address the AD problem. Firstly, different classifications of anomalies are introduced, and then the DL methods and architectures used for video AD are discussed and analyzed, respectively. The revised contributions have been categorized by the network type, architecture model, datasets, and performance metrics that are used to evaluate these methodologies. Moreover, several applications of video AD have been discussed. Finally, we outlined the challenges and future directions for further research in the field.

Hierarchical Object-Focused and Grid-Based Deep Unsupervised Segmentation Method for High-Resolution Remote Sensing Images

Since the unsupervised segmentation of high-resolution remote sensing is a highly challenging task, the introduction of deep learning and processing may be a sensible choice to improve the quality of unsupervised segmentation. Unfortunately, any attempt to direct using unsupervised deep neural networks (UDNNs) to perform this task will be hindered by many obstacles: uncontrollable refinement processes, excessive fragmentation at the borders and excessive computing resource requirements. These obstacles can prevent us from obtaining acceptable results. To address this problem, this article proposes a hierarchical object-focused and grid-based deep unsupervised segmentation method for high-resolution remote sensing images (HOFG). Based on a grid approach, HOFG first adopt a lazy deep segmentation method (LDSM) to handle fragmentation and large image sizes. Then, a hierarchical and iterative segmentation strategy is introduced to reduce the accuracy expectation for the LDSM by means of a cascaded focus mechanism, making the entire segmentation process more controllable. HOFG can overcome all of the above obstacles while utilizing the high recognition ability of UDNNs. In experiments, HOFG are compared with shallow and deep unsupervised segmentation methods. The results show that HOFG can obtain fewer segments while maintaining a high accuracy. HOFG transform the unsupervised classification ability of UDNNs into a controllable and stable segmentation ability, making HOFG valuable for practical applications. The results show that on average, HOFG need only 81.73% as many segments as traditional shallow methods to achieve a high overall accuracy, and HOFG can obtain a 7.2% higher accuracy than a UDNN even when using only approximately 18% as many segments. HOFG can effectively and controllably utilize the recognition ability of UDNNs to achieve better unsupervised segmentation results.

Classification of the Human Protein Atlas Single Cell Using Deep Learning

Deep learning has made great progress in many fields. One of the most important fields is the medical field, where we can classify images, detect objects and so on. More specifically, deep learning algorithms entered the field of single-cell classification and revolutionized this field, by classifying the components of the cell and identifying the location of the proteins in it. Due to the presence of large numbers of cells in the human body of different types and sizes, it was difficult to carry out analysis of cells and detection of components using traditional methods, which indicated a research gap that was filled with the introduction of deep learning in this field. We used the Human Atlas dataset which contains 87,224 images of single cells. We applied three novel deep learning algorithms, which are CSPNet, BoTNet, and ResNet. The results of the algorithms were promising in terms of accuracy: 95%, 93%, and 91%, respectively.

Adversarial Attacks and Defenses for Deep-Learning-Based Unmanned Aerial Vehicles

The introduction of deep learning (DL) technology can improve the performance of cyber–physical systems (CPSs) in many ways. However, this also brings new security issues. To tackle these challenges, this article explores the vulnerabilities of DL-based unmanned aerial vehicles (UAVs), which are typical CPSs. Although many research works have been reported previously on adversarial attacks of DL models, only few of them are concerned about safety-critical CPSs, especially regression models in such systems. In this article, we analyze the problem of adversarial attacks against DL-based UAVs and propose two adversarial attack methods against regression models in UAVs. The experiments demonstrate that the proposed nontargeted and targeted attack methods both can craft imperceptible adversarial images and pose a considerable threat to the navigation and control of UAVs. To address this problem, adversarial training and defensive distillation methods are further investigated and evaluated, increasing the robustness of DL models in UAVs. To our knowledge, this is the first study on adversarial attacks and defenses against DL-based UAVs, which calls for more attention to the security and safety of such safety-critical applications.