Deep Learning-based Methodologies Research Articles

A Comprehensive Survey of Masked Faces: Recognition, Detection, and Unmasking

Masked face recognition (MFR) has emerged as a critical domain in biometric identification, especially with the global COVID-19 pandemic, which introduced widespread face masks. This survey paper presents a comprehensive analysis of the challenges and advancements in recognizing and detecting individuals with masked faces, which has seen innovative shifts due to the necessity of adapting to new societal norms. Advanced through deep learning techniques, MFR, along with face mask recognition (FMR) and face unmasking (FU), represents significant areas of focus. These methods address unique challenges posed by obscured facial features, from fully to partially covered faces. Our comprehensive review explores the various deep learning-based methodologies developed for MFR, FMR, and FU, highlighting their distinctive challenges and the solutions proposed to overcome them. Additionally, we explore benchmark datasets and evaluation metrics specifically tailored for assessing performance in MFR research. The survey also discusses the substantial obstacles still facing researchers in this field and proposes future directions for the ongoing development of more robust and effective masked face recognition systems. This paper serves as an invaluable resource for researchers and practitioners, offering insights into the evolving landscape of face recognition technologies in the face of global health crises and beyond.

An integrated computer-aided diagnosis BCanD model for detection, segmentation and classification of breast cancer

A fully integrated Computer-Aided Diagnosis (CAD) system involves the integration of detection, segmentation, and classification, which makes it very useful for medical applications, particularly while dealing with the detection of breast mass and its classification into malignant and benign. The carried-out research work is intended to propose a Breast Cancer Detection (BCanD) model that is an integrated CAD system, where the system is capable enough for mass detection, its segmentation, and for the classification using mammograms. The proposed integrated system utilizes deep learning based YOLO model to detect the abnormality (mass) in the mammogram, where U-net is used for segmentation of the mass, as it has the capability to produce pixel level segmentation map, and at last stage that is the classification stage deep CNN is used for the classification. The proposed system is evaluated on open-source MIAS database. For the performance evaluation of the proposed BCanD, a three-fold cross-validation test was utilized. The mass detection accuracy of the BCanD is 98.99%, MCC is 97.96%, and F1-score is 98.87%. The model is evaluated with and without automated mass segmentation to study the impact of segmentation on the suggested CAD system. The best results was observed with the segmentation with the overall accuracy of 94.20%, F1-score (Dice) of 93.60%, MCC of 88.33%, and Jaccard of 88.08%. The proposed BCanD model surpasses the latest existing deep learning-based methodologies like fuzzy classifier, CNNI-BCC etc Hence, the proposed CAD system can be implemented and used by radiologists for all the stages from detection to diagnosis of breast mass.

From Pixels to Precision: A Survey of Monocular Visual Odometry in Digital Twin Applications.

This survey provides a comprehensive overview of traditional techniques and deep learning-based methodologies for monocular visual odometry (VO), with a focus on displacement measurement applications. This paper outlines the fundamental concepts and general procedures for VO implementation, including feature detection, tracking, motion estimation, triangulation, and trajectory estimation. This paper also explores the research challenges inherent in VO implementation, including scale estimation and ground plane considerations. The scientific literature is rife with diverse methodologies aiming to overcome these challenges, particularly focusing on the problem of accurate scale estimation. This issue has been typically addressed through the reliance on knowledge regarding the height of the camera from the ground plane and the evaluation of feature movements on that plane. Alternatively, some approaches have utilized additional tools, such as LiDAR or depth sensors. This survey of approaches concludes with a discussion of future research challenges and opportunities in the field of monocular visual odometry.

Intelligent fault diagnosis of storage stacking machinery under variable working conditions using attention-based adaptive multimodal feature fusion networks

Due to the harsh working environment of storage stacking machinery, the fault information of important components is significantly complex, which leads to the problem of low classification accuracy and high computational complexity of existing deep learning-based fault diagnosis methods. To alleviate the problem, this paper presents a novel architecture named attention-based adaptive multimodal feature fusion networks for intelligent fault diagnosis of storage stacking machinery, which is aimed at improving the diagnostic precision and robustness of feature fusion network and learning the broader feature representation. Firstly, the long short-term memory layer is introduced to extract the feature information of multiple time steps to improve the self-extraction ability of multi-temporal features. Then, the maximum temporal feature fusion module is utilized to highlight the recognizability of deep fusion features. Finally, a residual layer with spanning connections is added to increase the utilization and characterization capability of deep fusion features. Experimental results demonstrate the effectiveness and superiority of the proposed method in intelligent fault diagnosis of storage stacking machinery under variable working conditions compared with some state-of-the-art deep learning-based methodologies.

Advancing glioma diagnosis: Integrating custom U-Net and VGG-16 for improved grading in MR imaging.

In the realm of medical imaging, the precise segmentation and classification of gliomas represent fundamental challenges with profound clinical implications. Leveraging the BraTS 2018 dataset as a standard benchmark, this study delves into the potential of advanced deep learning models for addressing these challenges. We propose a novel approach that integrates a customized U-Net for segmentation and VGG-16 for classification. The U-Net, with its tailored encoder-decoder pathways, accurately identifies glioma regions, thus improving tumor localization. The fine-tuned VGG-16, featuring a customized output layer, precisely differentiates between low-grade and high-grade gliomas. To ensure consistency in data pre-processing, a standardized methodology involving gamma correction, data augmentation, and normalization is introduced. This novel integration surpasses existing methods, offering significantly improved glioma diagnosis, validated by high segmentation dice scores (WT: 0.96, TC: 0.92, ET: 0.89), and a remarkable overall classification accuracy of 97.89%. The experimental findings underscore the potential of integrating deep learning-based methodologies for tumor segmentation and classification in enhancing glioma diagnosis and formulating subsequent treatment strategies.

Deep learning based retinal vessel segmentation and hypertensive retinopathy quantification using heterogeneous features cross-attention neural network.

Retinal vessels play a pivotal role as biomarkers in the detection of retinal diseases, including hypertensive retinopathy. The manual identification of these retinal vessels is both resource-intensive and time-consuming. The fidelity of vessel segmentation in automated methods directly depends on the fundus images' quality. In instances of sub-optimal image quality, applying deep learning-based methodologies emerges as a more effective approach for precise segmentation. We propose a heterogeneous neural network combining the benefit of local semantic information extraction of convolutional neural network and long-range spatial features mining of transformer network structures. Such cross-attention network structure boosts the model's ability to tackle vessel structures in the retinal images. Experiments on four publicly available datasets demonstrate our model's superior performance on vessel segmentation and the big potential of hypertensive retinopathy quantification.

Enhancing Deep Learning-Based Sentiment Analysis Using Static and Contextual Language Models

Sentiment Analysis (SA) is an essential task of Natural Language Processing and is used in various fields such as marketing, brand reputation control, and social media monitoring. The various scores generated by users in product reviews are essential feedback sources for businesses to discover their products' positive or negative aspects. However, it takes work for businesses facing a large user population to accurately assess the consistency of the scores. Recently, automated methodologies based on Deep Learning (DL), which utilize static and especially pre-trained contextual language models, have shown successful performances in SA tasks. To address the issues mentioned above, this paper proposes Multi-layer Convolutional Neural Network-based SA approaches using Static Language Models (SLMs) such as Word2Vec and GloVe and Contextual Language Models (CLMs) such as ELMo and BERT that can evaluate product reviews with ratings. Focusing on improving model inputs by using sentence representations that can store richer features, this study applied SLMs and CLMs to the inputs of DL models and evaluated their impact on SA performance. To test the performance of the proposed approaches, experimental studies were conducted on the Amazon dataset, which is publicly available and considered a benchmark dataset by most researchers. According to the results of the experimental studies, the highest classification performance was obtained by applying the BERT CLM with 82% test and 84% training accuracy scores. The proposed approaches can be applied to various domains' SA tasks and provide insightful decision-making information.

Deep Learning Techniques in Radar Emitter Identification

In the field of electronic warfare (EW), one of the crucial roles of electronic intelligence is the identification of radar signals. In an operational environment, it is very essential to identify radar emitters whether friend or foe so that appropriate radar countermeasures can be taken against them. With the electromagnetic environment becoming increasingly complex and the diversity of signal features, radar emitter identification with high recognition accuracy has become a significantly challenging task. Traditional radar identification methods have shown some limitations in this complex electromagnetic scenario. Several radar classification and identification methods based on artificial neural networks have emerged with the emergence of artificial neural networks, notably deep learning approaches. Machine learning and deep learning algorithms are now frequently utilized to extract various types of information from radar signals more accurately and robustly. This paper illustrates the use of Deep Neural Networks (DNN) in radar applications for emitter classification and identification. Since deep learning approaches are capable of accurately classifying complicated patterns in radar signals, they have demonstrated significant promise for identifying radar emitters. By offering a thorough literature analysis of deep learning-based methodologies, the study intends to assist researchers and practitioners in better understanding the application of deep learning techniques to challenges related to the classification and identification of radar emitters. The study demonstrates that DNN can be used successfully in applications for radar classification and identification.
 

Dual encoder–decoder-based deep polyp segmentation network for colonoscopy images

Detection of colorectal polyps through colonoscopy is an essential practice in prevention of colorectal cancers. However, the method itself is labor intensive and is subject to human error. With the advent of deep learning-based methodologies, and specifically convolutional neural networks, an opportunity to improve upon the prognosis of potential patients suffering with colorectal cancer has appeared with automated detection and segmentation of polyps. Polyp segmentation is subject to a number of problems such as model overfitting and generalization, poor definition of boundary pixels, as well as the model’s ability to capture the practical range in textures, sizes, and colors. In an effort to address these challenges, we propose a dual encoder–decoder solution named Polyp Segmentation Network (PSNet). Both the dual encoder and decoder were developed by the comprehensive combination of a variety of deep learning modules, including the PS encoder, transformer encoder, PS decoder, enhanced dilated transformer decoder, partial decoder, and merge module. PSNet outperforms state-of-the-art results through an extensive comparative study against 5 existing polyp datasets with respect to both mDice and mIoU at 0.863 and 0.797, respectively. With our new modified polyp dataset we obtain an mDice and mIoU of 0.941 and 0.897 respectively.

Recurrent Convolutional Neural Network-Based Assessment of Power System Transient Stability and Short-Term Voltage Stability

Transient stability (TS) and short-term voltage stability (STVS) assessment are of fundamental importance for the operation security of power systems. Both phenomena can be mutually influenced in weak power systems due to the proliferation of power electronic interface devices and the phase-out of conventional heavy machines (e.g., thermal power plants). There is little research on the assessment of both types of stability together, despite the fact that they develop over the same short-term period, and that they can have a major influence on the overall transient performance driven by large electrical disturbances (e.g., short circuits). This work addresses this open research challenge by proposing a methodology for the joint assessment of TS and STVS. The methodology aims at estimating the resulting short-term stability state (STSS) in stable, or unstable conditions, following critical events, such as the synchronism loss of synchronous generators (SG) or the stalling of induction motors (IM). The estimations capture the mechanisms responsible for the degradations of TS and STVS, respectively. The paper overviews the off-line design of the data-driven STSS classification methodology, which supports the design and training of a hybrid deep neural network RCNN (recurrent convolutional neural network). The RCNN can automatically capture spatial and temporal features from the power system through a time series of selected physical variables, which results in a high estimation degree for STSS in real-time applications. The methodology is tested on the New England 39-bus system, where the results demonstrate the superiority of the proposed methodology over other traditional and deep learning-based methodologies. For reference purposes, the numerical tests also illustrate the classification performance in special situations, when the training is performed by exclusively using measurements from generation and motor load buses, which constitute locations where the investigated stability can be observed.

Self-supervised transfer learning framework driven by visual attention for benign–malignant lung nodule classification on chest CT

Lung cancer is one of the most fatal malignant diseases, which poses an acute menace to human health and life. The accurate differential diagnosis of lung nodules is a vital step in the computed tomography (CT)-based noninvasive screening of lung cancer. Though deep learning-based methodologies have achieved good results in the task of nodule malignancy prediction, there are still two fundamental challenges that are required to be overcome, including insufficient labeled samples and the interferences of background tissues. Motivated by the above facts, a self-supervised transfer learning framework driven by visual attention (STLF-VA) is presented for benign–malignant identification of nodules on chest CT, which advocates using volumes containing the entire nodule objects as inputs to obtain discriminative features. Compared with traditional models that designed 2D natural image-based transfer learning models or learning from scratch 3D models, the proposed STLF-VA method can effectively alleviate the dependence on labeled samples by exploring the valuable information from 3D unlabeled CT scans in a coarse-to-fine self-supervised transfer learning fashion. Unlike the single attention mechanism, the multi-view aggregative attention (MVAA) module embedded in the STLF-VA architecture fully recalibrates the multi-layer feature maps from multiple attention angles, and can strengthen the anti-interference ability on background information. Moreover, a new dataset CQUCH-LND is constructed for evaluating the effectiveness of the STLF-VA model in clinical practice. Experimental results on the clinical dataset CQUCH-LND and the public dataset LIDC-IDRI indicate that the proposed STLF-VA framework achieves more competitive performance than some state-of-the-art nodule classification approaches.

Hyperspectral Pansharpening With Adaptive Feature Modulation-Based Detail Injection Network

Recently, deep learning-based methodologies have attained unprecedented performance in hyperspectral (HS) pansharpening, which aims to improve the spatial quality of HS images (HSIs) by making use of details extracted from the high-resolution panchromatic (HR-PAN) image. However, it remains challenging to incorporate the details into the pansharpened image effectively, while alleviating the spectral distortion simultaneously. To tackle this problem, in this article, we propose an adaptive feature modulation-based detail injection network (AFM-DIN) for HS pansharpening, which mainly consists of four phases: high-frequency details generation of the HR-PAN image, multiscale feature extraction of the upsampled HSI, AFM-based detail injection and reconstruction of the HR-HSI. First, a novel octave convolution unit is employed to decompose the HR-PAN image into high and low frequencies, and then merge the high-frequency features together to generate the comprehensive PAN-details. Second, the spatial and spectral separable 3D convolution units with multiple kernel sizes are designed to extract multiscale features of the upsampled HSI in a computationally efficient manner. Subsequently, by taking the critical PAN-details as prior, the proposed AFM module is able to not only incorporate the detail information effectively, but also adjust the injected details adaptively to ensure the spectral fidelity. Finally, the anticipated HR-HSI is obtained through adding the upsampled HSI to the predicted HSI-details reconstructed from informative modulated features. Extensive comparison experiments with several state-of-the-arts conducted on simulated and real HS data sets demonstrate that our proposed AFM-DIN can achieve superior pansharpening accuracy in both spatial and spectral aspects.

Augmentation of Contextualized Concatenated Word Representation and Dilated Convolution Neural Network for Sentiment Analysis

Deep learning‐based methodologies are significant to perform sentiment analysis on social media data. The valuable insights of social media data through sentiment analysis can be employed to develop intelligent applications. Among many networks, convolution neural networks (CNNs) are widely used in many conventional text classification tasks and perform a significant role. However, to capture long‐term contextual information and address the detail loss problem, CNNs require stacking multiple convolutional layers. Also, the stacking of convolutional layers has issues requiring massive computations and the tuning of additional parameters. To solve these problems, in this paper, a contextualized concatenated word representation (CCWRs) is initialized from social media data based on text which is essential to misspelled and out of vocabulary words (OOV). In CCWRs, different word representation models, for example, Word2Vec, its optimized version FastText and Global Vectors, and GloVe, collectively create contextualized representations upon the sequence of input. Second, a three‐layered dilated convolutional neural network (3D‐CNN) is proposed that places dilated convolution kernels instead of conventional CNN kernels. Incorporating the extension in the receptive field’s size successfully solves the detail loss problem and achieves long‐term context information with different dilation rates. Experiments on datasets demonstrate that the proposed framework achieves reliable results with the selection of numerous hyperparameter tuning and configurations for improved optimization leads to reduced computational resources and reliable accuracy.

Correction to: Benchmarking performance of machine and deep learning-based methodologies for Urdu text document classification

The article [Benchmarking performance of machine and deep learning-based methodologies for Urdu text document classification], written by [Muhammad Nabeel Asim, Muhammad Usman Ghani, Muhammad Ali Ibrahim, Waqar Mahmood, Andreas Dengel and Sheraz Ahmed], was originally published.

MAKEDONKA: Applied Deep Learning Model for Text-to-Speech Synthesis in Macedonian Language

This paper presents MAKEDONKA, the first open-source Macedonian language synthesizer that is based on the Deep Learning approach. The paper provides an overview of the numerous attempts to achieve a human-like reproducible speech, which has unfortunately shown to be unsuccessful due to the work invisibility and lack of integration examples with real software tools. The recent advances in Machine Learning, the Deep Learning-based methodologies, provide novel methods for feature engineering that allow for smooth transitions in the synthesized speech, making it sound natural and human-like. This paper presents a methodology for end-to-end speech synthesis that is based on a fully-convolutional sequence-to-sequence acoustic model with a position-augmented attention mechanism—Deep Voice 3. Our model directly synthesizes Macedonian speech from characters. We created a dataset that contains approximately 20 h of speech from a native Macedonian female speaker, and we use it to train the text-to-speech (TTS) model. The achieved MOS score of 3.93 makes our model appropriate for application in any kind of software that needs text-to-speech service in the Macedonian language. Our TTS platform is publicly available for use and ready for integration.

Benchmarking performance of machine and deep learning-based methodologies for Urdu text document classification

In order to provide benchmark performance for Urdu text document classification, the contribution of this paper is manifold. First, it provides a publicly available benchmark dataset manually tagged against 6 classes. Second, it investigates the performance impact of traditional machine learning-based Urdu text document classification methodologies by embedding 10 filter-based feature selection algorithms which have been widely used for other languages. Third, for the very first time, it assesses the performance of various deep learning-based methodologies for Urdu text document classification. In this regard, for experimentation, we adapt 10 deep learning classification methodologies which have produced best performance figures for English text classification. Fourth, it also investigates the performance impact of transfer learning by utilizing Bidirectional Encoder Representations from Transformers approach for Urdu language. Fifth, it evaluates the integrity of a hybrid approach which combines traditional machine learning-based feature engineering and deep learning-based automated feature engineering. Experimental results show that feature selection approach named as normalized difference measure along with support vector machine outshines state-of-the-art performance on two closed source benchmark datasets CLE Urdu Digest 1000k, and CLE Urdu Digest 1Million with a significant margin of 32% and 13%, respectively. Across all three datasets, normalized difference measure outperforms other filter-based feature selection algorithms as it significantly uplifts the performance of all adopted machine learning, deep learning, and hybrid approaches. The source code and presented dataset are available at Github repository https://github.com/minixain/Urdu-Text-Classification .

Zero-Shot Learning and its Applications from Autonomous Vehicles to COVID-19 Diagnosis: A Review

The challenge of learning a new concept, object, or a new medical disease recognition without receiving any examples beforehand is called Zero-Shot Learning (ZSL). One of the major issues in deep learning-based methodologies such as in Medical Imaging, Autonomous Systems and other real-world applications is the requirement of feeding a large annotated and labelled datasets, prepared by an expert human to train the network model. ZSL is known for having minimal human intervention by mainly relying only on previously known concepts and current auxiliary information. This is ever-growing research for the cases where we have very limited or no datasets available and at the same time, the detection/recognition system has human-like characteristics in learning new concepts. Therefore, it makes it applicable in real-world scenarios, from developing autonomous vehicles to medical imaging and COVID-19 Chest X-Ray (CXR) based diagnosis. In this review paper, we present the definition of the problem, we review over fundamentals, and the challenging steps of Zero-Shot Learning, including state-of-the-art categories of solutions as well as our recommended solution, motivations behind each approach, and their advantages over each category to guide the researchers to proceed with the best techniques and practices based on their applications. Inspired from different settings and extensions, we introduce a novel and broaden solution called one$\slash$few-shot learning. We then review through different image datasets inducing medical and non-medical images, the variety of splits, and the evaluation protocols proposed so far. Finally, we discuss the recent applications and future directions of ZSL. We aim to convey a useful intuition through this paper towards the goal of handling complex computer vision learning tasks more similar to the way humans learn. We mainly focus on two applications in the current modern yet challenging era: copping with an early and fast diagnosis of COVID-19 cases in the 2020 pandemic, as well as helping to develop safer autonomous systems, such as self-driving cars and ambulances.

COVID-19 detection in CT images with deep learning: A voting-based scheme and cross-datasets analysis.

Early detection and diagnosis are critical factors to control the COVID-19 spreading. A number of deep learning-based methodologies have been recently proposed for COVID-19 screening in CT scans as a tool to automate and help with the diagnosis. These approaches, however, suffer from at least one of the following problems: (i) they treat each CT scan slice independently and (ii) the methods are trained and tested with sets of images from the same dataset. Treating the slices independently means that the same patient may appear in the training and test sets at the same time which may produce misleading results. It also raises the question of whether the scans from the same patient should be evaluated as a group or not. Moreover, using a single dataset raises concerns about the generalization of the methods. Different datasets tend to present images of varying quality which may come from different types of CT machines reflecting the conditions of the countries and cities from where they come from. In order to address these two problems, in this work, we propose an Efficient Deep Learning Technique for the screening of COVID-19 with a voting-based approach. In this approach, the images from a given patient are classified as group in a voting system. The approach is tested in the two biggest datasets of COVID-19 CT analysis with a patient-based split. A cross dataset study is also presented to assess the robustness of the models in a more realistic scenario in which data comes from different distributions. The cross-dataset analysis has shown that the generalization power of deep learning models is far from acceptable for the task since accuracy drops from 87.68% to 56.16% on the best evaluation scenario. These results highlighted that the methods that aim at COVID-19 detection in CT-images have to improve significantly to be considered as a clinical option and larger and more diverse datasets are needed to evaluate the methods in a realistic scenario.

Improved Traffic Sign Detection and Recognition Algorithm for Intelligent Vehicles.

Traffic sign detection and recognition are crucial in the development of intelligent vehicles. An improved traffic sign detection and recognition algorithm for intelligent vehicles is proposed to address problems such as how easily affected traditional traffic sign detection is by the environment, and poor real-time performance of deep learning-based methodologies for traffic sign recognition. Firstly, the HSV color space is used for spatial threshold segmentation, and traffic signs are effectively detected based on the shape features. Secondly, the model is considerably improved on the basis of the classical LeNet-5 convolutional neural network model by using Gabor kernel as the initial convolutional kernel, adding the batch normalization processing after the pooling layer and selecting Adam method as the optimizer algorithm. Finally, the traffic sign classification and recognition experiments are conducted based on the German Traffic Sign Recognition Benchmark. The favorable prediction and accurate recognition of traffic signs are achieved through the continuous training and testing of the network model. Experimental results show that the accurate recognition rate of traffic signs reaches 99.75%, and the average processing time per frame is 5.4 ms. Compared with other algorithms, the proposed algorithm has remarkable accuracy and real-time performance, strong generalization ability and high training efficiency. The accurate recognition rate and average processing time are markedly improved. This improvement is of considerable importance to reduce the accident rate and enhance the road traffic safety situation, providing a strong technical guarantee for the steady development of intelligent vehicle driving assistance.

Lane Detection Algorithm for Intelligent Vehicles in Complex Road Conditions and Dynamic Environments.

Lane detection is an important foundation in the development of intelligent vehicles. To address problems such as low detection accuracy of traditional methods and poor real-time performance of deep learning-based methodologies, a lane detection algorithm for intelligent vehicles in complex road conditions and dynamic environments was proposed. Firstly, converting the distorted image and using the superposition threshold algorithm for edge detection, an aerial view of the lane was obtained via region of interest extraction and inverse perspective transformation. Secondly, the random sample consensus algorithm was adopted to fit the curves of lane lines based on the third-order B-spline curve model, and fitting evaluation and curvature radius calculation were then carried out on the curve. Lastly, by using the road driving video under complex road conditions and the Tusimple dataset, simulation test experiments for lane detection algorithm were performed. The experimental results show that the average detection accuracy based on road driving video reached 98.49%, and the average processing time reached 21.5 ms. The average detection accuracy based on the Tusimple dataset reached 98.42%, and the average processing time reached 22.2 ms. Compared with traditional methods and deep learning-based methodologies, this lane detection algorithm had excellent accuracy and real-time performance, a high detection efficiency and a strong anti-interference ability. The accurate recognition rate and average processing time were significantly improved. The proposed algorithm is crucial in promoting the technological level of intelligent vehicle driving assistance and conducive to the further improvement of the driving safety of intelligent vehicles.