Deep Learning-based Network Research Articles

A new architecture combining convolutional and transformer-based networks for automatic 3D multi-organ segmentation on CT images.

Deep learning-based networks have become increasingly popular in the field of medical image segmentation. The purpose of this research was to develop and optimize a new architecture for automatic segmentation of the prostate gland and normal organs in the pelvic, thoracic, and upper gastro-intestinal (GI) regions. We developed an architecture which combines a shifted-window (Swin) transformer with a convolutional U-Net. The network includes a parallel encoder, a cross-fusion block, and a CNN-based decoder to extract local and global information and merge related features on the same scale. A skip connection is applied between the cross-fusion block and decoder to integrate low-level semantic features. Attention gates (AGs) are integrated within the CNN to suppress features in image background regions. Our network is termed "SwinAttUNet." We optimized the architecture for automatic image segmentation. Training datasets consisted of planning-CT datasets from 300 prostate cancer patients from an institutional database and 100 CT datasets from a publicly available dataset (CT-ORG). Images were linearly interpolated and resampled to a spatial resolution of (1.0×1.0×1.5) mm3 . A volume patch (192×192×96) was used for training and inference, and the dataset was split into training (75%), validation (10%), and test (15%) cohorts. Data augmentation transforms were applied consisting of random flip, rotation, and intensity scaling. The loss function comprised Dice and cross-entropy equally weighted and summed. We evaluated Dice coefficients (DSC), 95th percentile Hausdorff Distances (HD95), and Average Surface Distances (ASD) between results of our network and ground truth data. SwinAttUNet, DSC values were 86.54±1.21, 94.15±1.17, and 87.15±1.68% and HD95 values were 5.06±1.42, 3.16±0.93, and 5.54±1.63mm for the prostate, bladder, and rectum, respectively. Respective ASD values were 1.45±0.57, 0.82±0.12, and 1.42±0.38mm. For the lung, liver, kidneys and pelvic bones, respective DSC values were: 97.90±0.80, 96.16±0.76, 93.74±2.25, and 89.31±3.87%. Respective HD95 values were: 5.13±4.11, 2.73±1.19, 2.29±1.47, and 5.31±1.25mm. Respective ASD values were: 1.88±1.45, 1.78±1.21, 0.71±0.43, and 1.21±1.11mm. Our network outperformed several existing deep learning approaches using only attention-based convolutional or Transformer-based feature strategies, as detailed in the results section. We have demonstrated that our new architecture combining Transformer- and convolution-based features is able to better learn the local and global context for automatic segmentation of multi-organ, CT-based anatomy.

Decentralized Task Assignment for Mobile Crowdsensing With Multi-Agent Deep Reinforcement Learning

Task assignment is a fundamental research problem in mobile crowdsensing (MCS) since it directly determines an MCS system’s practicality and economic value. Due to the complex dynamics of tasks and workers, task assignment problems are usually NP-hard, and approximation-based methods are preferred to impractical optimal methods. In the literature, a graph neural network-based deep reinforcement learning (GDRL) method is proposed in Xu and Song (2022) to solve routing problems in MCS and shows high performance and time efficiency. However, GDRL, as a centralized method, has to cope with the limitation in scalability and the challenge of privacy protection. In this paper, we propose a multi-agent deep reinforcement learning-based method named CQDRL to solve a task assignment problem in a decentralized fashion. The CQDRL method not only inherits the merits of GDRL over traditional heuristic and metaheuristic methods but also exploits computation potentials in mobile devices and protects workers’ privacy with a decentralized decision-making scheme. Our extensive experiments show that the CQDRL method can achieve significantly better performance than other traditional methods and performs fairly close to the centralized GDRL method.

A Comparative Study of Object Detection using YOLO and SSD Algorithms

One of the most significant and difficult areas of computer vision is object detection, which is used in many aspects of daily life, including security monitoring, autonomous vehicle operation, and so on, to find instances of semantic objects belonging to a particular class. In this survey, we first look at the current approaches of typical detection models and discuss the benchmark datasets in order to comprehensively and deeply comprehend the major development status of the object detection pipeline. This project explains how convolutional neural network-based deep learning techniques are used for object detection. In this study, deep learning methods for cutting-edge object identification systems are evaluated. Any kind of visual medium is represented by a computer as a collection of numerical numbers. In order to inspect the contents of images, they need image processing techniques. In order to determine which is the quickest and most effective object detection algorithm, this research analyzes two popular algorithms: Single Shot Detection (SSD) and You Only Look Once (YOLO). The COCO (Common Object in Context) dataset is used in this comparative research to assess the effectiveness of the two algorithms and to analyze their strengths and weaknesses in terms of factors like accuracy, precision, and speed. Based on the findings analysis, it can be said that YOLO-v3 outperforms SSD in the same testing scenario. Keywords- Object Detection, Deep learning, CNN, COCO, YOLO, SSD

MSCDNet-based multi-class classification of skin cancer using dermoscopy images.

Skin cancer is a life-threatening disease, and early detection of skin cancer improves the chances of recovery. Skin cancer detection based on deep learning algorithms has recently grown popular. In this research, a new deep learning-based network model for the multiple skin cancer classification including melanoma, benign keratosis, melanocytic nevi, and basal cell carcinoma is presented. We propose an automatic Multi-class Skin Cancer Detection Network (MSCD-Net) model in this research. The study proposes an efficient semantic segmentation deep learning model "DenseUNet" for skin lesion segmentation. The semantic skin lesions are segmented by using the DenseUNet model with a substantially deeper network and fewer trainable parameters. Some of the most relevant features are selected using Binary Dragonfly Algorithm (BDA). SqueezeNet-based classification can be made in the selected features. The performance of the proposed model is evaluated using the ISIC 2019 dataset. The DenseNet connections and UNet links are used by the proposed DenseUNet segmentation model, which produces low-level features and provides better segmentation results. The performance results of the proposed MSCD-Net model are superior to previous research in terms of effectiveness and efficiency on the standard ISIC 2019 dataset.

Channel and temporal-frequency attention UNet for monaural speech enhancement

The presence of noise and reverberation significantly impedes speech clarity and intelligibility. To mitigate these effects, numerous deep learning-based network models have been proposed for speech enhancement tasks aimed at improving speech quality. In this study, we propose a monaural speech enhancement model called the channel and temporal-frequency attention UNet (CTFUNet). CTFUNet takes the noisy spectrum as input and produces a complex ideal ratio mask (cIRM) as output. To improve the speech enhancement performance of CTFUNet, we employ multi-scale temporal-frequency processing to extract input speech spectrum features. We also utilize multi-conv head channel attention and residual channel attention to capture temporal-frequency and channel features. Moreover, we introduce the channel temporal-frequency skip connection to alleviate information loss between down-sampling and up-sampling. On the blind test set of the first deep noise suppression challenge, our proposed CTFUNet has better denoising performance than the champion models and the latest models. Furthermore, our model outperforms recent models such as Uformar and MTFAA in both denoising and dereverberation performance.

Evaluating Linear and Non-linear Dimensionality Reduction Approaches for Deep Learning-based Network Intrusion Detection Systems

Evaluating Linear and Non-linear Dimensionality Reduction Approaches for Deep Learning-based Network Intrusion Detection Systems

Data-Driven Soft Sensing for Batch Processes Using Neural Network-Based Deep Quality-Relevant Representation Learning

Soft sensors provide a means to reliably estimate unmeasurable variables, thereby playing a prevalent role in formulating closed-loop control in batch processes. In soft sensor development, enhancing quality-relevant information and eliminating quality-irrelevant information are important. This study proposes a neural network-based deep quality-relevant representation learning approach to improve the soft sensing performance in dynamic batch processes. The structure of a deep neural network is optimized in a layer-by-layer manner. First, given the generally abundant predictor variables, the maximal relevanceminimal redundancy criterion is used to optimize the input layer, select the most beneficial variables, and eliminate modeling redundancy. Second, mutual information-based quality-relevant representation selection is performed in the middle layers to enhance the quality-relevant information and eliminate the influence of irrelevant representations. Third, deep quality-relevant representations are extracted, and a soft sensor model is developed. The proposed method is tested on a fed-batch penicillin fermentation process and an industrial injection molding process. Lastly, it shows that the proposed method outperforms several state-of-the-art approaches, thereby confirming its effectiveness.

GLDM: Geo-location prediction of twitter users with deep learning methods1

Social networks like Twitter are extremely popular and widely used, which has increased interest in studying the information posted there. One such analytical application is extracting location information of users for real-time monitoring of the objects and events of interest, such as political and social events, disease surveillance, natural calamities, and crime prevention. Identifying geographic location is a nontrivial task, as user profiles contain outdated and inaccurate location information. Furthermore, extracting geographical information from Arabic tweets is challenging since they contain many nonstandard data (dialects), complex structures, abbreviations, grammatical and spelling mistakes, etc. This study focuses on the localization of Saudi Arabian users who tweet in Arabic. This study proposes a convolutional neural network-based deep learning model to predict a Twitter user’s region-level location using user profiles, text texts, place attachments, and historical tweets. The model was evaluated empirically on a dataset of 95,739 tweets written in Arabic and produced by 4,331 users from Saudi Arabia cities. Regarding classification accuracy, the proposed CNN model outperformed machine learning classifiers such as NB, LR, and SVM with a 60% accuracy on the test set. This study is the first of its kind, aimed at localizing Saudi users based on their tweets.

Surveillance audio-based rainfall observation: An enhanced strategy for extreme rainfall observation

The increased frequency of extreme rainfall events (EREs) causing disastrous effects on society has become an indisputable fact in recent years. The weak performance of the current observation network in terms of accuracy and spatiotemporal resolution makes the development of new rainfall estimation techniques essential to reduce the impact of ERE-driven disasters. The rainfall sounds collected by widespread surveillance cameras provide an opportunity to produce high-resolution, all-weather rainfall data. However, the complex structure of the ground surface and the random background noise make building an effective surveillance audio-based ERE estimation system challenging. In this study, from the viewpoint of surveillance sound space, a 3D printer was used to create a shelter for the surveillance camera to define the underlying surface of falling raindrops artificially. Combining the knowledge of meteorology, micro-physics, and acoustics of rainfall, the shelter structure was designed to standardize the acoustical behavior while enhancing the consistency and specificity of raindrop sound, especially in complex scenarios such as those disturbed by different levels of wind. After that, convolutional neural network-based deep learning algorithms were used to classify ERE levels, and an audio-based ERE classification system was built. The experimental results show that the shelter facilitates audio-based rainfall representation; moreover, with the help of shelter, our proposed system achieved performance with about 93.4% accuracy in complex rainfall scenarios. Our study supports high-resolution rainfall data production on existing surveillance resources, developing a novel and reliable alternative for the perception of ERE and the calibration of observations from current rainfall networks.

Optical and SAR Image Registration Based on Pseudo-SAR Image Generation Strategy

The registration of optical and SAR images has always been a challenging task due to the different imaging mechanisms of the corresponding sensors. To mitigate this difference, this paper proposes a registration algorithm based on a pseudo-SAR image generation strategy and an improved deep learning-based network. The method consists of two stages: a pseudo-SAR image generation strategy and an image registration network. In the pseudo-SAR image generation section, an improved Restormer network is used to convert optical images into pseudo-SAR images. An L2 loss function is adopted in the network, and the loss function fluctuates less at the optimal point, making it easier for the model to reach the fitting state. In the registration part, the ROEWA operator is used to construct the Harris scale space for pseudo-SAR and real SAR images, respectively, and each extreme point in the scale space is extracted and added to the keypoint set. The image patches around the keypoints are selected and fed into the network to obtain the feature descriptor. The pseudo-SAR and real SAR images are matched according to the descriptors, and outliers are removed by the RANSAC algorithm to obtain the final registration result. The proposed method is tested on a public dataset. The experimental analysis shows that the average value of NCM surpasses similar methods over 30%, and the average value of RMSE is lower than similar methods by more than 0.04. The results demonstrate that the proposed strategy is more robust than other state-of-the-art methods.

Convolutional neural network-based deep learning model for air quality prediction in October city of Egypt

Purpose Modern human society has continuous advancements that have a negative impact on the quality of the air. Daily transportation, industrial and residential operations churn up dangerous contaminants in our surroundings. Addressing air pollution issues is critical for human health and ecosystems, particularly in developing countries such as Egypt. Excessive levels of pollutants have been linked to a variety of circulatory, respiratory and nervous illnesses. To this end, the purpose of this research paper is to forecast air pollution concentrations in Egypt based on time series analysis. Design/methodology/approach Deep learning models are leveraged to analyze air quality time series in the 6th of October City, Egypt. In this regard, convolutional neural network (CNN), long short-term memory network and multilayer perceptron neural network models are used to forecast the overall concentrations of sulfur dioxide (SO2) and particulate matter 10 µm in diameter (PM10). The models are trained and validated by using monthly data available from the Egyptian Environmental Affairs Agency between December 2014 and July 2020. The performance measures such as determination coefficient, root mean square error and mean absolute error are used to evaluate the outcomes of models. Findings The CNN model exhibits the best performance in terms of forecasting pollutant concentrations 3, 6, 9 and 12 months ahead. Finally, using data from December 2014 to July 2021, the CNN model is used to anticipate the pollutant concentrations 12 months ahead. In July 2022, the overall concentrations of SO2 and PM10 are expected to reach 10 and 127 µg/m3, respectively. The developed model could aid decision-makers, practitioners and local authorities in planning and implementing various interventions to mitigate their negative influences on the population and environment. Originality/value This research introduces the development of an efficient time-series model that can project the future concentrations of particulate and gaseous air pollutants in Egypt. This research study offers the first time application of deep learning models to forecast the air quality in Egypt. This research study examines the performance of machine learning approaches and deep learning techniques to forecast sulfur dioxide and particular matter concentrations using standard performance metrics.

Deep Learning-Based Network Intrusion Detection System for Internet of Medical Things

This article presents a deep learning-based approach for network-based intrusion detection in the Internet of medical things (IoMT) systems using features of network flows and patient biometrics. The proposed approach effectively learns optimal feature representation by passing the information of network flows and patient biometrics into more than one hidden layer of deep learning. The network includes a global attention layer which helps to effectively extract the optimal features from the spatial and temporal features of deep learning. To avoid data imbalance, a cost-sensitive learning approach is integrated into the deep learning model. The proposed model showed a 10-fold cross-validation accuracy of 95 percent on network features, 89 percent on patient biometrics, and 99 percent on combined features. In addition to the IoMT environment, the robustness and generalization ability of the proposed model is shown by conducting experiments on other network-based intrusion datasets. The proposed approach outperformed the existing methods in all the test cases mainly showing a 3.9 percent higher accuracy on the IoMT intrusion dataset. The proposed model can be used as an IoMT network monitoring tool to safeguard the IoMT devices and networks from attackers inside the healthcare and medical environment.

Deep reinforcement learning-based network for optimized power flow in islanded DC microgrid

Deep reinforcement learning-based network for optimized power flow in islanded DC microgrid

An efficient feature selection and classification approach for an intrusion detection system using Optimal Neural Network

Network flaws are used by hackers to get access to private systems and data. This data and system access may be extremely destructive with losses. Therefore, this network intrusions detection is utmost significance. While investigating every feature set in the network, deep learning-based algorithms require certain inputs. That’s why, an Adaptive Artificial Neural Network Optimized with Oppositional Crow Search Algorithm is proposed for network intrusions detection (IDS-AANN-OCSA). The proposed method includes several phases, including feature selection, preprocessing, data acquisition, and classification. Here, the datas are gathered via CICIDS 2017 dataset. The datas are fed to pre-processing. During pre-processing, redundancy eradication and missing value replacement is carried out with the help of random forest along Local least squares for removing uncertainties. The pre-processed datas are fed to feature selection to select better features. The feature selection is accomplished under hybrid genetic algorithm together with particle swarm optimization technique (GPSO). The selected features are fed to adaptive artificial neural network (AANN) for categorization which categorizes the data as BENIGN, DOS Hulk, PortScan, DDoS, DoS Golden Eye. Finally, the hyper parameter of adaptive artificial neural network is tuned with Oppositional Crow Search Algorithm (OCSA) helps to gain better classification of network intrusions. The proposed approach is activated in Python, and its efficiency is evaluated with certain performance metrics, like accuracy, recall, specificity, precision, F score, sensitivity. The performance of proposed approach achieves better accuracy 99.75%, 97.85%, 95.13%, 98.79, better sensitivity 96.34%, 91.23%, 89.12%, 87.25%, compared with existing methods, like One-Dimensional Convolutional Neural Network Based Deep Learning for Network Intrusion Detection (IDS-CNN-GPSO), An innovative network intrusion detection scheme (IDS-CNN-LSTM) and Application of deep learning to real-time Web intrusion detection (IDS-CNN-ML-AIDS) methods respectively.

RGB-D Video based Hand Authentication using Deep Neural Network

In recent years, behavioral biometrics authentication, which uses the habit of behavioral characteristics for personal authentication, has attracted attention as an authentication method with higher security since behavioral biometrics cannot mimic fingerprint and face authentications. In behavioral biometrics, extensive research has been done on voiceprints and gait. However, there are few authentication technologies that utilize the habits of hand and finger movements during hand gestures. Only color images or depth images are used for hand gesture authentication in conventional methods. In this research, therefore, we propose to find individual habits from RGB-D images of finger movements and create a personal authentication system. 3D CNN, which is deep learning-based network, is used to extract individual habits. F-measure of 0.97 is achieved when rock-paper-scissors are used as the authentication operation. F-measure of 0.97 is achieved when the disinfection operation is used. These results show the effectiveness of using RGB-D video for personal authentication.

Two-stage voltage regulation in power distribution system using graph convolutional network-based deep reinforcement learning in real time

The model-based voltage control is widely used to mitigate quick voltage fluctuations caused by renewable energy uncertainties. However, the accurate and complete parameters of the distribution system are rarely available in practice. A two-stage voltage regulation framework based on a mixed-integer second order cone optimization programming (MISOCP) model and graph convolutional network-based deep reinforcement learning (GCN-DRL) is proposed for active distribution system voltage regulation. Specifically, in the day-ahead stage, a MISOCP is proposed for hourly voltage regulation optimization with capacitor banks (CBs), on-load tap changers (OLTC), and energy storage systems (ESS). Then, a GCN-DRL method is proposed in the real-time stage for dispatching reactive power from the intelligent inverters connected to the photovoltaic systems to alleviate the voltage fluctuations. The proposed grid topological graph convolutional network (GTGCN) leverages the distribution system’s graph structure information and the convolutional operation to capture and embed the graphical features among nodal measurements. Then, the deep deterministic policy gradient (DDPG) is innovatively proposed for GCN-DRL to learn the high-efficiency voltage regulation policies, which can be implemented in a real-time manner in practice. The proposed voltage regulation model is investigated on a modified IEEE 33-node distribution system and a 25-node unbalanced distribution system. The numerical results illustrate the high effectiveness and efficiency of the proposed adaptive robust operating model.

ICEGAN: inverse covariance estimating generative adversarial network

Owing to the recent explosive expansion of deep learning, several challenging problems in a variety of fields have been handled by deep learning, yet deep learning methods have been limited in their application to the network estimation problem. While network estimation has a possibility to be a useful method in various domains, deep learning-based network estimation has a limitation in that the number of variables must be fixed and the estimation cannot be performed by convolutional layers. In this study, we propose a Generative Adversarial Network (GAN) based method, called Inverse Covariance Estimating GAN (ICEGAN), which can alleviate these limitations. In ICEGAN, the concepts in Cycle-Consistent Adversarial Networks are modified for the problem and employed to adopt gene expression data. Additionally, the Monte Carlo approach is used to address the fixed size in the network estimation process. Thus, sub-networks are sampled from the entire network and estimated by ICEGAN; then, the Monte Carlo approach reconstructs the entire network with the estimations. In the simulation study, ICEGAN demonstrated superior performances compared to conventional models and the ordinary GAN model in estimating networks. Specifically, ICEGAN outperformed an ordinary GAN by 85.9% on average when the models were evaluated using the area under curve. In addition, ICEGAN performed gene network estimation of breast cancer using a gene expression dataset. Consequently, ICEGAN demonstrated promising results, considering the deep learning-based network estimation and the proposed Monte Carlo approach for GAN models, both of which can be expanded to other domains.

Deep learning-based network pharmacology for exploring the mechanism of licorice for the treatment of COVID-19

Licorice, a traditional Chinese medicine, has been widely used for the treatment of COVID-19, but all active compounds and corresponding targets are still not clear. Therefore, this study proposed a deep learning-based network pharmacology approach to identify more potential active compounds and targets of licorice. 4 compounds (quercetin, naringenin, liquiritigenin, and licoisoflavanone), 2 targets (SYK and JAK2) and the relevant pathways (P53, cAMP, and NF-kB) were predicted, which were confirmed by previous studies to be associated with SARS-CoV-2-infection. In addition, 2 new active compounds (glabrone and vestitol) and 2 new targets (PTEN and MAP3K8) were further validated by molecular docking and molecular dynamics simulations (simultaneous molecular dynamics), as well as the results showed that these active compounds bound well to COVID-19 related targets, including the main protease (Mpro), the spike protein (S-protein) and the angiotensin-converting enzyme 2 (ACE2). Overall, in this study, glabrone and vestitol from licorice were found to inhibit viral replication by inhibiting the activation of Mpro, S-protein and ACE2; related compounds in licorice may reduce the inflammatory response and inhibit apoptosis by acting on PTEN and MAP3K8. Therefore, licorice has been proposed as an effective candidate for the treatment of COVID-19 through PTEN, MAP3K8, Mpro, S-protein and ACE2.

The Application of Deep Learning for Network Traffic Classification

The classification, detection, and analysis of routine network traffic has been a hot topic for businesses and research institutions due to the proliferation of Internet of Things devices and the explosive development of networks. Traditional methods for categorizing network traffic primarily employ common machine learning algorithms e.g., decision trees and plain Bayes algorithms, but as deep learning technology advances, more and more traffic classifications are being successfully applied. This study examines existing deep learning-based network traffic classification techniques and focuses on the categorization of computer network traffic. Firstly, the research background of the topic is introduced, and then the traffic classification based on deep learning is mainly described, which includes traffic classification based on Stacked Autoencoder, traffic classification based on Convolutional Neural Network and traffic classification based on Recurrent Neural Networks. Following investigation, this paper comes to the conclusion that Long Short-Term Memory and Convolutional Neural Network models are the best deep learning models for traffic classification, with three-dimensional Convolutional Neural Network outperforming the others.

Hybrid gated recurrent unit and convolutional neural network-based deep learning mechanism for efficient shilling attack detection in social networks

The degree of openness of the socially aware recommendation systems and the possibility of the attackers injecting vast numbers of fake profiles biases the prediction of the system. Most classical shilling attack discovery mechanisms rely on artificially derived features, generally determined from user-generated data. Moreover, these baseline approaches cannot comprehensively capture the fine-grained relation between users and objects, thereby lagging in detection accuracy. This work introduces a Hybrid deep learning mechanism to address the Shilling Attack discovery with improved accuracy. The proposed mechanism highly enhances the detection accuracy by determining significant feature vectors managed by recurrent gated units and other high dimensional attribute vectors attained by using CNN. The GRU component is incorporated for the dimensionality transition process for existing weight classification data to learn the inherent significant features. The CNN mechanism is mainly used for analyzing the conditions of spatial-temporal data and transforming the data into a dormant feature vector. The experimental analysis of our proposed method is conducted using the datasets of Amazon, Netflix, and Movielens to demonstrate its predominance by enhancing accuracy, precision, and F-score.