Leveraging Nanosensor and Vision Transformer for Robust Anomaly Detection in Autonomous Vehicles

Efficiently forecasting the demands within a hospital’s Emergency Department (ED) is critical for optimal resource allocation and patient care management. This study focuses on leveraging deep learning techniques to predict various types of ED patient flows, facilitating informed decision-making by ED managers. The rising success of deep learning networks in modeling timeseries data makes them a compelling choice for patient flow forecasting. In this context, we investigate and compare seven deep learning models-Deep Belief Network (DBN), Restricted Boltzmann Machines (RBM), Long Short-Term Memory (LSTM), Gated Recurrent Unit (GRU), combined GRU and Convolutional Neural Networks (CNN-GRU), LSTM-CNN, and Generative Adversarial Network based on Recurrent Neural Networks (GAN-RNN)—to accurately forecast patient flow within a hospital’s emergency department. To enable traffic flow forecasting, a forecaster layer is introduced for each model. Real-world patient flow data spanning different ED services (biology, radiology, scanner, and echography) at Lille regional hospital in France serve as a case study to evaluate these models. Four effectiveness metrics are employed to assess and compare the forecasting methods. The outcomes demonstrate the superior performance of deep learning models in predicting ED patient flows compared to conventional shallow approaches like ridge regression and support vector regression. Significantly, the Deep Belief Network (DBN) stands out, achieving an averaged mean absolute percentage error of approximately 4.097.

In the recent era Deep learning is showing exemplary results in many application areas. Researches from different disciplines have incorporated deep learning in to their research to solve different interdisciplinary problems. Deep learning applications areas include Speech recognition, Natural Language Processing, Computer vision, Networking, Healthcare, IoT, Robotics, Agriculture, Remote sensing and many other areas. Thus, this paper contributes a review on various deep learning approaches which include Convolution Neural Network (CNN), Deep Neural Network (DNN), Auto-Encoder (AE), Recurrent Neural Network (RNN) enclosed with Gated Recurrent Units (GRU), Long Short-Term Memory (LSTM) and ConvLSTM, Deep Reinforcement Learning (DRL), Generative based Adversarial Network (GAN) and Deep Belief Network (DBN). For extraction of the most important features of an image various data extraction procedures have designed. CNN's tremendous learning capacity is based on the utilisation of several feature extraction stages that continuously learn from data. Some of the popular CNN architectures name as LeNet, AlexNet, ZFNet / Clarifai, VGGNET, GoogLeNet, ResNet, DenseNet, FractalNet and CapsuleNet. Recent works extended to a combination of two CNN models like Inception, ResNetV2 to attained different existing approaches consequences in deep learning. Identification of the significant objects, their parameters and relationships of an object images are required for image captioning. Syntactically and semantically correct sentences need to be generated. The intricacies and problems of picture captioning may be handled using deep learning approaches. A comparison between these models was also presented. We also discussed about different standard datasets which are utilized for executing and estimating the deep learning approaches.

Deep learning (also known as deep structured learning, hierarchical learning, or deep machine learning) is a branch of machine learning based on a set of algorithms that attempt to model high-level abstractions in data by using a deep graph with multiple processing layers, composed of multiple linear and nonlinear transformations. Deep learning has been characterized as a class of machine learning algorithms with the following characteristics [257]: • They use a cascade of many layers of nonlinear processing units for feature extraction and transformation. Each successive layer uses the output from the previous layer as input. The algorithms may be supervised or unsupervised and applications include pattern analysis (unsupervised) and classification (supervised). • They are based on the (unsupervised) learning of multiple levels of features or representations of the data. Higher-level features are derived from lower-level features to form a hierarchical representation. • They are part of the broader machine learning field of learning representations of data. • They learn multiple levels of representations that correspond to different levels of abstraction; the levels form a hierarchy of concepts. These definitions have in common: multiple layers of nonlinear processing units and the supervised or unsupervised learning of feature representations in each layer, with the layers forming a hierarchy from low-level to high-level features. Various deep learning architectures such as deep neural networks, convolutional deep neural networks, deep belief networks (DBN), and recurrent neural networks have been applied to fields like computer vision, automatic speech recognition, natural language processing, audio recognition, and bioinformatics where they have been shown to produce state-of-the-art results on various tasks. In this chapter we start in Section 7.1 with an introduction, giving a brief history of this field, the relevant literature, and its applications. Then we study some basic concepts of deep learning such as convolutional neural networks, recurrent neural networks, backpropagation algorithm, restricted Boltzmann machines, and deep learning networks in Section 7.2. Then we illustrate three examples for Apache Spark implementation for mobile big data (MBD), user moving pattern extraction, and combination with nonparametric Bayesian learning, respectively, in Sections 7.3 through 7.5. Finally, we have summary in Section 7.6.

Deep learning, which is originated from an artificial neural network (ANN), is one of the major technologies of today’s smart cybersecurity systems or policies to function in an intelligent manner. Popular deep learning techniques, such as multi-layer perceptron, convolutional neural network, recurrent neural network or long short-term memory, self-organizing map, auto-encoder, restricted Boltzmann machine, deep belief networks, generative adversarial network, deep transfer learning, as well as deep reinforcement learning, or their ensembles and hybrid approaches can be used to intelligently tackle the diverse cybersecurity issues. In this paper, we aim to present a comprehensive overview from the perspective of these neural networks and deep learning techniques according to today’s diverse needs. We also discuss the applicability of these techniques in various cybersecurity tasks such as intrusion detection, identification of malware or botnets, phishing, predicting cyberattacks, e.g. denial of service, fraud detection or cyberanomalies, etc. Finally, we highlight several research issues and future directions within the scope of our study in the field. Overall, the ultimate goal of this paper is to serve as a reference point and guidelines for the academia and professionals in the cyber industries, especially from the deep learning point of view.

Deep Learning Algorithms for Cybersecurity Applications: A Technological and Status Review

This article explores deep learning models in the field of malware detection in cyberspace, aiming to provide insights into their relevance and contributions. The primary objective of the study is to investigate the practical applications and effectiveness of deep learning models in detecting malware. By carefully analyzing the characteristics of malware samples, these models gain the ability to accurately categorize them into distinct families or types, enabling security researchers to swiftly identify and counter emerging threats. The PRISMA 2020 guidelines were used for paper selection and the time range of review study is January 2015 to Dec 2023. In the review, various deep learning models such as Recurrent Neural Networks, Deep Autoencoders, LSTM, Deep Neural Networks, Deep Belief Networks, Deep Convolutional Neural Networks, Deep Generative Models, Deep Boltzmann Machines, Deep Reinforcement Learning, Extreme Learning Machine, and others are thoroughly evaluated. It highlights their individual strengths and real-world applications in the domain of malware detection in cyberspace. The review also emphasizes that deep learning algorithms consistently demonstrate exceptional performance, exhibiting high accuracy and low false positive rates in real-world scenarios. Thus, this article aims to contribute to a better understanding of the capabilities and potential of deep learning models in enhancing cybersecurity efforts.

Effective forecasting of key features in hospital emergency department: Hybrid deep learning-driven methods

Deep learning (DL) architectures such as deep neural networks (DNN), deep belief networks (DBN), recurrent neural networks(RNN) and convolutional neural networks (CNN) have been applied to applications such as computer vision, speech recognition, natural language processing, audio recognition, social network filtering, machine translation, bioin-formatics, drug design, medical image analysis, material inspection and board game programs, in which has comparable performance than human experts. With the growing interest and research in the area of artificial neural network, deep neural network enable computers to get trained for error-free diagnosis to diseases like epilepsy. In literature, researchers carried out many mathematical models for pre-processing of EEG data and classification between seizure and seizure free signals or different 166types of network disorders. The introduction of various algorithms like machine learning deep learning, etc., in artificial intelligence, aids to classify the data with or without pre-processing and two class system. It is important to try multi-class time series classification of various brain activities (tumour, network disorders) using the sophisticated algorithms. In this chapter, different deep learning algorithms for multiclass, time series classification of different electrical activities in brain are discussed. The main focus is on the application of different RNN models in seizure classification of Electroencephalogram (EEG) signals. It is very important to interpret the 1D EEG signals and classify among different activities of brain for various diagnostic purpose. The fully interconnected hidden configuration of recurrent neural network (RNN) makes the model very dominant which enable to discover temporal correlations between far away events in the data. The training of RNN architecture when used in deep network is challenging because of vanishing/exploding gradient in deeper layer. This paper aims to perform multiclass time series classification of EEG signal using three different RNN techniques; simple Recurrent Neural Network, Long-Short Term Memory (LSTM) and GRUs. A comparative study between RNNs is done in terms of configuration, time taken and accuracy for EEG signals acquired from people having different pathological and physiological brain states. The accuracy and time taken for multilayer recurrent neural networks are determined for classification of EEG for five different classes using three different types of RNN networks, for 1 to 1024 units with 100 epochs and 5 different layers of 32 cells with 300 epochs, with a learning rate of 0.01. It has been observed that the number of layers increases the time complexity and provides constant accuracy for more than three layers. Further, it can be extended for the accuracy and time consumption for different batch sizes with different epochs to fix a proper network without over fitting the network.

Transportation Cyber-Physical Systems (TCPS) integrate the interaction between the connected transportation infrastructure, users, and computing and communication services to support the mobility of people and goods. TCPS is supported by many emerging technologies, among which Deep Learning (DL)-supported technology is a notable one. DL models closely mimic the functionality of human brains, and it can learn the relationship between inputs and outputs through data observations, thus facilitating different information extraction or decision-making functionalities of TCPS. In recent times, the availability of vast amounts of heterogeneous data from TCPS and the introduction of large-scale computing hardware have enabled the different applications of DL models. This chapter discusses early DL models, such as restricted Boltzmann machine, deep belief network, and deep Boltzmann machine, along with the recent ones including multilayer perceptron, autoencoder, convolutional neural network, recurrent neural networks, and deep reinforcement learning. The DL model development and testing considerations in the functioning of TCPS applications are discussed while pointing out the available hardware and software frameworks. Different types of cyberattacks that can occur on DL and the strategies protecting the DL against such cyberattacks are also presented. This chapter concludes with various applications of DL models in TCPS.

For data protection, the most vital factors are the statistics' safety, use of cryptographic controls during data transmission, an effective access management system, and powerful tracking. This paper seeks to provide a committed evaluation of the very current studies works on using Deep studying strategies to remedy computer security demanding situations. In this study, we analyzed and reviewed using deep learning algorithms for the Intruder detection system and Cybersecurity programs. Deep learning consists of system-mastering strategies that permit the network to learn from unsupervised data and solve complicated problems. Deep learning approaches such as Convolutional Neural Network (CNN), Auto Encoder (AE), Deep Belief Network (DBN), Recurrent Neural Network (RNN), Generative Adversal Network (GAN), and Deep Reinforcement Learning (DIL) are used to categorize the papers referred. This paper discusses various challenges, issues, and types of cyber-attacks and security measures

Advancements in Deep Learning Algorithms is a comprehensive exploration of the cutting-edge developments in deep learning, a subset of artificial intelligence that has revolutionized the way machines learn from data. This book starts with the basics, introducing the reader to the fundamental concepts and terminologies of deep learning, before delving into the core algorithms that form the backbone of this field, including neural networks, convolutional neural networks (CNNs), recurrent neural networks (RNNs), and generative adversarial networks (GANs). It further explores advanced architectures and techniques such as attention mechanisms, deep reinforcement learning, federated learning, and autoencoders, providing a deep dive into the mechanisms that enable machines to mimic human-like learning processes. The book also addresses critical aspects of data handling and preprocessing, optimization and regularization techniques, and the practical applications of deep learning in various industries, highlighting real-world case studies. Additionally, it discusses the challenges, ethical considerations, and future implications of deploying deep learning technologies. With an eye towards recent trends and the future directions of deep learning, this book aims to equip researchers, practitioners, and enthusiasts with the knowledge to understand and leverage the potential of deep learning in solving complex problems. Keywords: Deep Learning, Neural Networks, Convolutional Neural Networks (CNNs), Recurrent Neural Networks (RNNs), Generative Adversarial Networks (GANs), Attention Mechanisms, Deep Reinforcement Learning, Federated Learning, Autoencoders, Data Preprocessing, Optimization Techniques, Artificial Intelligence, Industry Applications, Ethical Considerations, Future Directions.

In recent years, deep learning has garnered tremendous success in a variety of application domains. This new field of machine learning has been growing rapidly and has been applied to most traditional application domains, as well as some new areas that present more opportunities. Different methods have been proposed based on different categories of learning, including supervised, semi-supervised, and un-supervised learning. Experimental results show state-of-the-art performance using deep learning when compared to traditional machine learning approaches in the fields of image processing, computer vision, speech recognition, machine translation, art, medical imaging, medical information processing, robotics and control, bioinformatics, natural language processing, cybersecurity, and many others. This survey presents a brief survey on the advances that have occurred in the area of Deep Learning (DL), starting with the Deep Neural Network (DNN). The survey goes on to cover Convolutional Neural Network (CNN), Recurrent Neural Network (RNN), including Long Short-Term Memory (LSTM) and Gated Recurrent Units (GRU), Auto-Encoder (AE), Deep Belief Network (DBN), Generative Adversarial Network (GAN), and Deep Reinforcement Learning (DRL). Additionally, we have discussed recent developments, such as advanced variant DL techniques based on these DL approaches. This work considers most of the papers published after 2012 from when the history of deep learning began. Furthermore, DL approaches that have been explored and evaluated in different application domains are also included in this survey. We also included recently developed frameworks, SDKs, and benchmark datasets that are used for implementing and evaluating deep learning approaches. There are some surveys that have been published on DL using neural networks and a survey on Reinforcement Learning (RL). However, those papers have not discussed individual advanced techniques for training large-scale deep learning models and the recently developed method of generative models.

This paper presents a detailed review of existing and emerging deep learning algorithms for time series forecasting in geotechnics and geoscience applications. Deep learning has shown promising results in addressing complex prediction problems involving large datasets and multiple interacting variables without requiring extensive feature extraction. This study provides an in-depth description of prominent deep learning methods, including recurrent neural networks (RNNs), convolutional neural networks (CNNs), generative adversarial network, deep belief network, reinforcement learning, attention and transformer algorithms as well as hybrid networks using a combination of these architectures. In addition, this paper summarizes the applications of these models in various fields, including mining and tunnelling, railway and road construction, seismology, slope stability, earth retaining and stabilizing structures, remote sensing, as well as scour and erosion. This review reveals that RNN-based models, particularly Long Short-Term Memory networks, are the most commonly used models for time series forecasting. The advantages of deep learning models over traditional machine learning, including their superior ability to handle complex patterns and process large-scale data more effectively, are discussed. Furthermore, in time series forecasting within the fields of geotechnics and geosciences, studies frequently reveal that deep learning methods tend to surpass traditional machine learning techniques in effectiveness.

The increasing complexity of modern smart power distribution systems (SPDSs) has made anomaly detection a significant challenge, as these systems generate vast amounts of heterogeneous and time-dependent data. Conventional detection methods often struggle with adaptability, generalization, and real-time decision-making, leading to high false alarm rates and inefficient fault detection. To address these challenges, this study proposes a novel deep reinforcement learning (DRL)-based framework, integrating a convolutional neural network (CNN) for hierarchical feature extraction and a recurrent neural network (RNN) for sequential pattern recognition and time-series modeling. To enhance model performance, we introduce a novel non-dominated sorting artificial bee colony (NSABC) algorithm, which fine-tunes the hyper-parameters of the CNN-RNN structure, including weights, biases, the number of layers, and neuron configurations. This optimization ensures improved accuracy, faster convergence, and better generalization to unseen data. The proposed DRL-NSABC model is evaluated using four benchmark datasets: smart grid, advanced metering infrastructure (AMI), smart meter, and Pecan Street, widely recognized in anomaly detection research. A comparative analysis against state-of-the-art deep learning (DL) models, including RL, CNN, RNN, the generative adversarial network (GAN), the time-series transformer (TST), and bidirectional encoder representations from transformers (BERT), demonstrates the superiority of the proposed DRL-NSABC. The proposed DRL-NSABC model achieved high accuracy across all benchmark datasets, including 95.83% on the smart grid dataset, 96.19% on AMI, 96.61% on the smart meter, and 96.45% on Pecan Street. Statistical t-tests confirm the superiority of DRL-NSABC over other algorithms, while achieving a variance of 0.00014. Moreover, DRL-NSABC demonstrates the fastest convergence, reaching near-optimal accuracy within the first 100 epochs. By significantly reducing false positives and ensuring rapid anomaly detection with low computational overhead, the proposed DRL-NSABC framework enables efficient real-world deployment in smart power distribution systems without major infrastructure upgrades and promotes cost-effective, resilient power grid operations.

Good representations of data eliminate irrelevant variability of the input data, while preserving the information that is useful for the ultimate task. Among the various ways for learning representation is using deep learning methods. Deep feature hierarchies are formed by stacking unsupervised modules on top of each other, forming multiple nonlinear transformations to produce better representations. In this talk, we will first show how deep learning is used for bioactivity prediction of chemical compounds. Molecules are represented as several convolutional neural networks to predict their bioactivity. In addition, a new concept of merging multiple convolutional neural networks and an automatic learning features representation for the chemical compounds was proposed using the values within neurons of the last layer of the CNN architecture. We will also show how the concepts of deep learning is adapted into a deep belief network (DBN) to enhance the molecular similarity searching. The DBN achieves feature abstraction by reconstruction weight for each feature and minimizing the reconstruction error over the whole feature set. The DBN is later enhanced using data fusion to obtain a lower detection error probability and a higher reliability by using data from multiple distributed descriptors. Secondly, we will show how we used deep learning for stock market prediction. Here, we developed a Deep Long Short Term Memory Network model that is able to forecast the crude palm oil price movement with combined factors such as other commodities prices, weather and news sentiments and price movement of crude palm oil. We also show how we combined stock markets price and financial news and deployed the Long Short Term Memory (LSTM), Recurrent Neural Network (RNN), and Word 2 Vector (Word2Vec) to project the stock prices for the following seven days. Finally, we will show how we exploited deep learning method for the opinion mining and later used it to extract the product's aspects from the user textual review for recommendation systems. Specifically, we employ a multichannel convolutional neural network (MCNN) for two different input layers, namely, word embedding layer and Part-of-speech (POS) tag embedding layer. We show effectiveness of the proposed model in terms of both aspect extraction and rating prediction performance.