Parameters Of Deep Neural Network Research Articles

XGrad: Boosting Gradient-Based Optimizers With Weight Prediction.

In this paper, we propose a general deep learning training framework XGrad which introduces weight prediction into the popular gradient-based optimizers to boost their convergence and generalization when training the deep neural network (DNN) models. In particular, ahead of each mini-batch training, the future weights are predicted according to the update rule of the used optimizer and are then applied to both the forward pass and backward propagation. In this way, during the whole training period, the optimizer always utilizes the gradients w.r.t. the future weights to update the DNN parameters, making the gradient-based optimizer achieve better convergence and generalization compared to the original optimizer without weight prediction. XGrad is rather straightforward to implement yet pretty effective in boosting the convergence of gradient-based optimizers and the accuracy of DNN models. Empirical results concerning five popular optimizers including SGD with momentum, Adam, AdamW, AdaBelief, and AdaM3 demonstrate the effectiveness of our proposal. The experimental results validate that XGrad can attain higher model accuracy than the baseline optimizers when training the DNN models.

Uncertainty-aware deep learning for monitoring and fault diagnosis from synthetic data

Deep neural networks (DNNs) are often coupled with physics-based and data-driven models to perform fault detection and health monitoring. The system models serve as digital surrogates that generate large quantities of data for training DNNs which would otherwise be difficult to obtain from the real-life system. In such a scenario, the uncertainty in the system model and in the DNN parameters will influence the predictions of the DNN. Here, we quantify the impact of this uncertainty on the performance of DNNs. The uncertainty from the system model is captured with two methods, namely assumed density filtering and heteroskedastic modelling. In addition to quantification, these methods allow training DNNs in an uncertainty-aware manner. The uncertainty in the DNN parameters is captured with Monte Carlo dropout. The proposed approach is demonstrated for fault diagnosis of electric power lines. Data generated from a physics-based model calibrated with real-life measurements is used to train three neural network architectures for fault diagnosis. The results reveal that uncertainty-aware models can provide 1% to 19% improvement in classification accuracy than their deterministic counterparts. The uncertainty-aware models also exhibit better robustness to uncertainty and, thus, offer more reliable models for deployment. Remarkably, the article provides a system-agnostic framework for uncertainty-aware training of DNN models for fault diagnosis and monitoring that explicitly accounts for the synthetic nature of training data.

Novel adaptive quantization methodology for 8-bit floating-point DNN training

There is a high energy cost associated with training Deep Neural Networks (DNNs). Off-chip memory access contributes a major portion to the overall energy consumption. Reduction in the number of off-chip memory transactions can be achieved by quantizing the data words to low data bit-width (E.g., 8-bit). However, low-bit-width data formats suffer from a limited dynamic range, resulting in reduced accuracy. In this paper, a novel 8-bit Floating Point (FP8) data format quantized DNN training methodology is presented, which adapts to the required dynamic range on-the-fly. Our methodology relies on varying the bias values of FP8 format to fit the dynamic range to the required range of DNN parameters and input feature maps. The range fitting during the training is adaptively performed by an online statistical analysis hardware unit without stalling the computation units or its data accesses. Our approach is compatible with any DNN compute cores without any major modifications to the architecture. We propose to integrate the new FP8 quantization unit in the memory controller. The FP32 data from the compute core are converted to FP8 in the memory controller before writing to the DRAM and converted back after reading the data from DRAM. Our results show that the DRAM access energy is reduced by 3.07×\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\ imes $$\\end{document} while using an 8-bit data format instead of using 32-bit. The accuracy loss of the proposed methodology with 8-bit quantized training is ≈1%\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\approx 1\\%$$\\end{document} for various networks with image and natural language processing datasets.

Variable Imaging Projection Cloud Scattering Tomography.

Scattering-based computed tomography (CT) recovers a heterogeneous volumetric scattering medium using images taken from multiple directions. It is a nonlinear problem. Prior art mainly approached it by explicit physics-based optimization of image-fitting, being slow and difficult to scale. Scale is particularly important when the objects constitute large cloud fields, where volumetric recovery is important for climate studies. Besides speed, imaging and recovery need to be flexible, to efficiently handle variable viewing geometries and resolutions. These can be caused by perturbation in camera poses or fusion of data from different types of observational sensors. There is a need for fast variable imaging projection scattering tomography of clouds (VIP-CT). We develop a learning-based solution, using a deep-neural network (DNN) which trains on a large physics-based labeled volumetric dataset. The DNN parameters are oblivious to the domain scale, hence the DNN can work with arbitrarily large domains. VIP-CT offers much better quality than the state of the art. The inference speed and flexibility of VIP-CT make it effectively real-time in the context of spaceborne observations. The paper is the first to demonstrate CT of a real cloud using empirical data directly in a DNN. VIP-CT may offer a model for a learning-based solution to nonlinear CT problems in other scientific domains. Our code is available online.

A Deep-Learning and Transfer-Learning Hybrid Aerosol Retrieval Algorithm for FY4-AGRI: Development and Verification over Asia

The Advanced Geosynchronous Radiation Imager (AGRI) is a mission-critical instrument for the Fengyun series of satellites. AGRI acquires full-disk images every 15 min and views East Asia every 5 min through 14 spectral bands, enabling the detection of highly variable aerosol optical depth (AOD). Quantitative retrieval of AOD has hitherto been challenging, especially over land. In this study, an AOD retrieval algorithm is proposed that combines deep learning and transfer learning. The algorithm uses core concepts from both the Dark Target (DT) and Deep Blue (DB) algorithms to select features for the machine-learning (ML) algorithm, allowing for AOD retrieval at 550 nm over both dark and bright surfaces. The algorithm consists of two steps: ① A baseline deep neural network (DNN) with skip connections is developed using 10 min Advanced Himawari Imager (AHI) AODs as the target variable, and ② sunphotometer AODs from 89 ground-based stations are used to fine-tune the DNN parameters. Out-of-station validation shows that the retrieved AOD attains high accuracy, characterized by a coefficient of determination (R2) of 0.70, a mean bias error (MBE) of 0.03, and a percentage of data within the expected error (EE) of 70.7%. A sensitivity study reveals that the top-of-atmosphere reflectance at 650 and 470 nm, as well as the surface reflectance at 650 nm, are the two largest sources of uncertainty impacting the retrieval. In a case study of monitoring an extreme aerosol event, the AGRI AOD is found to be able to capture the detailed temporal evolution of the event. This work demonstrates the superiority of the transfer-learning technique in satellite AOD retrievals and the applicability of the retrieved AGRI AOD in monitoring extreme pollution events.

Fragility, robustness and antifragility in deep learning

We propose a systematic analysis of deep neural networks (DNNs) based on a signal processing technique for network parameter removal, in the form of synaptic filters that identifies the fragility, robustness and antifragility characteristics of DNN parameters. Our proposed analysis investigates if the DNN performance is impacted negatively, invariantly, or positively on both clean and adversarially perturbed test datasets when the DNN undergoes synaptic filtering. We define three filtering scores for quantifying the fragility, robustness and antifragility characteristics of DNN parameters based on the performances for (i) clean dataset, (ii) adversarial dataset, and (iii) the difference in performances of clean and adversarial datasets. We validate the proposed systematic analysis on ResNet-18, ResNet-50, SqueezeNet-v1.1 and ShuffleNet V2 x1.0 network architectures for MNIST, CIFAR10 and Tiny ImageNet datasets. The filtering scores, for a given network architecture, identify network parameters that are invariant in characteristics across different datasets over learning epochs. Vice-versa, for a given dataset, the filtering scores identify the parameters that are invariant in characteristics across different network architectures. We show that our synaptic filtering method improves the test accuracy of ResNet and ShuffleNet models on adversarial dataset when only the robust and antifragile parameters are selectively retrained at any given epoch, thus demonstrating applications of the proposed strategy in improving model robustness.

Deep Neural Network and Predator Crow Optimization-Based Intelligent Healthcare System for Predicting Cardiac Diseases

Cardiovascular diseases (CVD) are amongst the leading causes of death worldwide. The Internet of Things (IoT) is an emerging technology that enables the healthcare system to identify cardiovascular diseases. In this article, a novel cardiovascular disease prediction framework combining Predator Crow Optimization (PCO) and Deep Neural Network (DNN) is designed. In the proposed PCO-DNN framework, DNN is used to predict cardiac disease, and the PCO is utilized to optimize the DNN parameters, thereby maximizing the prediction performances. The proposed framework aims to predict and classify cardiovascular diseases accurately. Further, an intensive comparative analysis is performed to validate the obtained results with the existing classification models. The results show that the proposed framework achieves an accuracy of 96.6665%, a precision of 97.5256%, a recall of 97.0953%, and an F1-measure of 96.4242% and can outperform the existing CVD predictors.

DNNShifter: An efficient DNN pruning system for edge computing

Deep neural networks (DNNs) underpin many machine learning applications. Production quality DNN models achieve high inference accuracy by training millions of DNN parameters which has a significant resource footprint. This presents a challenge for resources operating at the extreme edge of the network, such as mobile and embedded devices that have limited computational and memory resources. To address this, models are pruned to create lightweight, more suitable variants for these devices. Existing pruning methods are unable to provide similar quality models compared to their unpruned counterparts without significant time costs and overheads or are limited to offline use cases. Our work rapidly derives suitable model variants while maintaining the accuracy of the original model. The model variants can be swapped quickly when system and network conditions change to match workload demand. This paper presents DNNShifter, an end-to-end DNN training, spatial pruning, and model switching system that addresses the challenges mentioned above. At the heart of DNNShifter is a novel methodology that prunes sparse models using structured pruning - combining the accuracy-preserving benefits of unstructured pruning with runtime performance improvements of structured pruning. The pruned model variants generated by DNNShifter are smaller in size and thus faster than dense and sparse model predecessors, making them suitable for inference at the edge while retaining near similar accuracy as of the original dense model. DNNShifter generates a portfolio of model variants that can be swiftly interchanged depending on operational conditions. DNNShifter produces pruned model variants up to 93x faster than conventional training methods. Compared to sparse models, the pruned model variants are up to 5.14x smaller and have a 1.67x inference latency speedup, with no compromise to sparse model accuracy. In addition, DNNShifter has up to 11.9x lower overhead for switching models and up to 3.8x lower memory utilisation than existing approaches. DNNShifter is available for public use from https://github.com/blessonvar/DNNShifter.

An Electro-Photonic System for Accelerating Deep Neural Networks

The number of parameters in deep neural networks (DNNs) is scaling at about 5× the rate of Moore’s Law. To sustain this growth, photonic computing is a promising avenue, as it enables higher throughput in dominant general matrix-matrix multiplication (GEMM) operations in DNNs than their electrical counterpart. However, purely photonic systems face several challenges including lack of photonic memory and accumulation of noise. In this article, we present an electro-photonic accelerator, ADEPT, which leverages a photonic computing unit for performing GEMM operations, a vectorized digital electronic application-specific integrated circuits for performing non-GEMM operations, and SRAM arrays for storing DNN parameters and activations. In contrast to prior works in photonic DNN accelerators, we adopt a system-level perspective and show that the gains while large are tempered relative to prior expectations. Our goal is to encourage architects to explore photonic technology in a more pragmatic way considering the system as a whole to understand its general applicability in accelerating today’s DNNs. Our evaluation shows that ADEPT can provide, on average, 5.73× higher throughput per watt compared to the traditional systolic arrays in a full-system, and at least 6.8× and 2.5× better throughput per watt, compared to state-of-the-art electronic and photonic accelerators, respectively.

A graph-based interpretability method for deep neural networks

With the development of artificial intelligence, the most representative deep learning has been applied to various fields, which is greatly influencing human society. However, deep neural networks (DNNs) are still a black-box model, and the process how they make decisions internally is still difficult to understand and control. At the same time, DNNs take up more hardware resources, resulting in high energy consumption. Therefore, it is significant to study the characteristics of deep AI models and deeply understand the interactions between parameters within AI models so as to improve the interpretability of DNNs, optimize their structure and increase their computational efficiency. In this paper, we propose a graph-based interpretability method for deep neural networks (GIMDNN). The running parameters of DNNs are modeled as a graph by using a kernel function or the Graph Transformer Networks (GTN), where the nodes of the graph are obtained by dimensional mapping of the parameters of the DNNs, and the edges are calculated by the Gaussian kernel function. The generated graphs are classified by a graph convolutional network (GCN). The association relationship between the adjacent layers and the running mechanism of DNNs are analyzed, and the importance of the parameters of each layer in the DNNs for the final classification result can be obtained. Convolutional neural networks (CNNs) are one of the most representative models in DNNs. The proposed method is experimentally evaluated on the CNNs. The experimental results show that the proposed method can interpret the associations among the weight parameters as well as the correlation between two adjacent layers. Therefore, the DNNs for special tasks, such as portable applications, edge computing, and so on, can be customized, the number of parameters can be reduced. It is valuable to interpret the operation and principle of CNNs.

From Algorithm to Module: Adaptive and Energy-Efficient Quantization Method for Edge Artificial Intelligence in IoT Society

Next-generation industrial edge artificial intelligence (AI) applications will undoubtedly emerge on energy-efficient, highly-integrated platforms incorporating various sensors, processors, and functional modules. Effectively bridging sensor data captured from a microprocessor or central processing unit with dedicated processing modules is still being studied in academia and industry. Therefore, embedding the data quantization module onto edge AI chips, connecting sensors, processors, and functional modules, is critical to achieving adaptive transformation of diverse data representation formats. This paper proposes a novel adaptive and low-power quantization technique and systematically validates its effectiveness from algorithm to hardware module for industrial IoT applications, covering precise navigation for autonomous vehicles and accurate classification utilizing deep neural networks (DNNs). The proposed quantization method merges an adaptive conversion function from floating-point to fixed-point binaries and an adaptive radix-point determination function, ensuring adequate resolution and minimal error loss of the fixed-point inputs to the edge AI modules. The experimental results demonstrate that the quantization error in the proposed quantization technique contributes negligible errors to the navigation solutions of the strapdown inertial navigation system and the DNNs' top-1 and top-5 classification accuracy (on the order of 10 <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$^{-8}$</tex-math></inline-formula> and 10 <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$^{-7}$</tex-math></inline-formula> ). Moreover, a quantization-on-multiplier (QoM) hardware module is designed, synthesized, and routed in accordance with the proposed quantization technique. The simulation results indicate that the QoM's power consumption and area are 0.1 <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$mW$</tex-math></inline-formula> and 649.552 <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\mu m^{2}$</tex-math></inline-formula> , accounting for 5% and 14.15% of its total energy consumption and area, respectively. With our proposed on-chip quantization technique, the time required to quantize the parameters of DNNs is up to 1142 times shorter than with the existing benchmark off-chip quantization approaches.

Trouble-Shooting at GAN Point: Improving Functional Safety in Deep Learning Accelerators

The proliferation of Deep Neural Networks (DNNs) in real-time mission critical applications has promoted the implementation of custom-built DNN inference accelerators. These accelerators require a considerable amount of on-chip memory to store millions of trained DNN parameters for executing inference at the edge. Drastic technology scaling in recent years have made these memory circuits highly vulnerable to faults due to various reasons like aging, latent defects, single event upsets, etc. Such faults are highly detrimental to the classification accuracy of the DNN accelerator, leading to the crucial Functional Safety (FuSa) violation. This can eventuate to catastrophic circumstances, when used in mission-critical applications. In order to detect such violations in mission mode, we propose to generate a set of functional test patterns by leveraging the concept of Generative Adversarial Networks (GANs), that are independent of the DNN model and the accelerator characteristics. Our experimental results demonstrate that, the generated test patterns significantly improve FuSa violation detection coverage by up to 130.28%, compared to existing techniques. To the best of our knowledge, this is the first work that generates GAN-based test patterns in order to perform FuSa violation detection in mission-critical DNN accelerators.

Asynchronous Federated Learning for Improved Cardiovascular Disease Prediction Using Artificial Intelligence.

Healthcare professionals consider predicting heart disease an essential task and deep learning has proven to be a promising approach for achieving this goal. This research paper introduces a novel method called the asynchronous federated deep learning approach for cardiac prediction (AFLCP), which combines a heart disease dataset and deep neural networks (DNNs) with an asynchronous learning technique. The proposed approach employs a method for asynchronously updating the parameters of DNNs and incorporates a temporally weighted aggregation technique to enhance the accuracy and convergence of the central model. To evaluate the effectiveness of the proposed AFLCP method, two datasets with various DNN architectures are tested, and the results demonstrate that the AFLCP approach outperforms the baseline method in terms of both communication cost and model accuracy.

Physics-guided neural network for channeled spectropolarimeter spectral reconstruction.

A reconstruction method incorporates the complete physical model into a traditional deep neural network (DNN) is proposed for channeled spectropolarimeter (CSP). Unlike traditional DNN-based methods that need to employ training datasets, the method starts from randomly initialized parameters which are constrained by the CSP physical model. It iterates through the gradient descent algorithm to obtain the estimation of the DNN parameters and then to obtain the mapping relationship. As a result, it eliminates the need for thousands of sets of ground truth data, while also leveraging the physical model to achieve high-precision reconstruction. As seen, the physical model participates in the optimization process of DNN parameters, thus achieving physical guidance for the DNN output results. Based on the characteristic of the network, we designate this method as the physics-guided neural network (PGNN). Both simulations and experiments demonstrate the superior performance of the proposed method. Our approach will further promote the practical application of CSP in a wider range of fields.

Autoencoder and restricted Boltzmann machine for transfer learning in functional magnetic resonance imaging task classification

Deep neural networks (DNNs) have been adopted widely as classifiers for functional magnetic resonance imaging (fMRI) data, advancing beyond traditional machine learning models. Consequently, transfer learning of the pre-trained DNN becomes crucial to enhance DNN classification performance, specifically by alleviating an overfitting issue that occurs when a substantial number of DNN parameters are fitted to a relatively small number of fMRI samples. In this study, we first systematically compared the two most popularly used, unsupervised pretraining models for resting-state fMRI (rfMRI) volume data to pre-train the DNNs, namely autoencoder (AE) and restricted Boltzmann machine (RBM). The group in-brain mask used when training AE and RBM displayed a sizable overlap ratio with Yeo's seven functional brain networks (FNs). The parcellated FNs obtained from the RBM were fine-grained compared to those from the AE. The pre-trained AE and RBM served as the weight parameters of the first of the two hidden DNN layers, and the DNN fulfilled the task classifier role for fMRI (tfMRI) data in the Human Connectome Project (HCP). We tested two transfer learning schemes: (1) fixing and (2) fine-tuning the DNN's pre-trained AE or RBM weights. The DNN with transfer learning was compared to a baseline DNN, trained using random initial weights. Overall, DNN classification performance from the transfer learning proved superior when the pre-trained RBM weights were fixed and when the pre-trained AE weights were fine-tuned (average error rates: 14.8% for fixed RBM, 15.1% fine-tuned AE, and 15.5% for the baseline model) compared to the alternative scenarios of DNN transfer learning schemes. Moreover, the optimal transfer learning scheme between the fixed RBM and fine-tuned AE varied according to seven task conditions in the HCP. Nonetheless, the computational load reduced substantially for the fixed-weight-based transfer learning compared to the fine-tuning-based transfer learning (e.g., the number of weight parameters for the fixed-weight-based DNN model reduced to 1.9% compared with a baseline/fine-tuned DNN model). Our findings suggest that weight initialization at the DNN's first layer using RBM-based pre-trained weights provides the most promising approach when the whole-brain fMRI volume supports associated task classification. We believe that our proposed scheme could be applied to a variety of task conditions to improve their classification performance and to utilize computational resources efficiently using our AE/RBM-based pre-trained weights compared to random initial weights for DNN training.

Deep neural network-based synthetic image digital fluoroscopy using digitally reconstructed tomography.

We developed a deep neural network (DNN) to generate X-ray flat panel detector (FPD) images from digitally reconstructed radiographic (DRR) images. FPD and treatment planning CT images were acquired from patients with prostate and head and neck (H&N) malignancies. The DNN parameters were optimized for FPD image synthesis. The synthetic FPD images' features were evaluated to compare to the corresponding ground-truth FPD images using mean absolute error (MAE), peak signal-to-noise ratio (PSNR), and structural similarity index measure (SSIM). The image quality of the synthetic FPD image was also compared with that of the DRR image to understand the performance of our DNN. For the prostate cases, the MAE of the synthetic FPD image was improved (= 0.12 ± 0.02) from that of the input DRR image (= 0.35 ± 0.08). The synthetic FPD image showed higher PSNRs (= 16.81 ± 1.54 dB) than those of the DRR image (= 8.74 ± 1.56 dB), while SSIMs for both images (= 0.69) were almost the same. All metrics for the synthetic FPD images of the H&N cases were improved (MAE 0.08 ± 0.03, PSNR 19.40 ± 2.83 dB, and SSIM 0.80 ± 0.04) compared to those for the DRR image (MAE 0.48 ± 0.11, PSNR 5.74 ± 1.63 dB, and SSIM 0.52 ± 0.09). Our DNN successfully generated FPD images from DRR images. This technique would be useful to increase throughput when images from two different modalities are compared by visual inspection.

FastFusion: Deep stereo‐LiDAR fusion for real‐time high‐precision dense depth sensing

AbstractLight detection and ranging (LiDAR) and stereo cameras are two generally used solutions for perceiving 3D information. The complementary properties of these two sensor modalities motivate a fusion to derive practicable depth sensing toward real‐world applications. Promoted by deep neural network (DNN) techniques, recent works achieve superior performance on accuracy. However, the complex architecture and the sheer number of DNN parameters often lead to poor generalization capacity and non‐real‐time computing. In this paper, we present FastFusion, a three‐stage stereo‐LiDAR deep fusion scheme, which integrates the LiDAR priors into each step of classical stereo‐matching taxonomy, gaining high‐precision dense depth sensing in a real‐time manner. We integrate stereo‐LiDAR information by taking advantage of a compact binary neural network and utilize the proposed cross‐based LiDAR trust aggregation to further fuse the sparse LiDAR measurements in the back‐end of stereo matching. To align the photometrical of the input image and the depth of the estimation, we introduce a refinement network to guarantee consistency. More importantly, we present a graphic processing unit‐based acceleration framework for providing a low‐latency implementation of FastFusion, gaining both accuracy improvement and real‐time responsiveness. In the experiments, we demonstrate the effectiveness and practicability of FastFusion, which obtains a significant speedup over state‐of‐the‐art baselines while achieving comparable accuracy on depth sensing. The video demo for real‐time depth estimation of FastFusion on the real‐world driving scenario is available at https://youtu.be/nP7cls2BA8s.

Floyd–Warshalls algorithm and modified advanced encryption standard for secured communication in VANET

Vehicular Ad Hoc Networks (VANETs), which have the strategic goal of ensuring service delivery on roads and smart cities, are made up of a number of critical components, including the integration and communication between vehicles, sensors, and stationary road-side components. VANETs have many distinctive characteristics, including quick movable nodes, self-organization, scattered networks, and often changing topologies. Since attack prevention is still a contentious topic, security, data integrity, and user private information continue to be major concerns despite the recent development of VANETs. In particular, detecting Sybil attacks, in which one node attempts to appear as many, seemingly conflicts with the goal of privacy preservation, and existing schemes fail on either one or both accounts. In order to prevent Sybil attacks, VANETs assert that they have security measures in place. A new framework for VANET security and sybil attack detection was developed in this study. It groups the cars together using Kernel k-harmonic means (KKHM). Kernel functions are here included into the k-means problem's cost function via kernelizing in the feature space. Cluster centres are believed to exist in a higher level Hilbert space called the feature space. Using the Floyd-Warshall algorithm (FWA), cluster heads are chosen. The Deep Neural Network (DNN) is used to identify the malicious CH by extracting the necessary information from the CH. For the purpose of optimizing DNN parameters, a new technique known as gradient-based elephant herding optimization (GBEHO) is presented. To initialize the population in the search space, the gradient-based method GBO is paired with the elephant optimization algorithm (EHO). Second, by including Gaussian chaos mapping, the initialized population is enhanced in order to correct the disparity between the first exploration and utilization of the EHO. Additionally, two operators are programmed to modify the agents' position update strategy: the random wandering and variation operators. Last but not least, the Modified Advanced Encryption Standard (MAES) is used to send the data in the CH safely to the cloud. According on experimental findings, the suggested model increases the security rate of 96% and reduces the encryption time of 19(s) for 100 (kb) data which is better when compared to other existing models.

A novel hybrid autoencoder and modified particle swarm optimization feature selection for intrusion detection in the internet of things network

The Internet of Things (IoT) represents a paradigm shift in which the Internet is connected to real objects in a range of areas, including home automation, industrial processes, human health, and environmental monitoring. The global market for IoT devices is booming, and it is estimated that there will be 50 billion connected devices by the end of 2025. This explosion of IoT devices, which can be expanded more easily than desktop PCs, has led to an increase in cyber-attacks involving IoT devices. To address this issue, it is necessary to create novel approaches for identifying attacks launched by hacked IoT devices. Due to the possibility that these attacks would succeed, Intrusion Detection Systems (IDS) are required. IDS' feature selection stage is widely regarded as the most essential stage. This stage is extremely time-consuming and labor-intensive. However, numerous machine learning (ML) algorithms have been proposed to enhance this stage to boost an IDS's performance. These approaches, however, did not produce desirable results in terms of accuracy and detection rate (DR). In this paper, we propose a novel hybrid Autoencoder and Modified Particle Swarm Optimization (HAEMPSO) for feature selection and deep neural network (DNN) for classification. The PSO with modification of inertia weight was utilized to optimize the parameters of DNN. The experimental analysis was performed on two realistic UNSW-NB15 and BoT-IoT datasets that are suitable for IoT environment. The findings obtained by analyzing the proposed HAEMPSO against the Generic attack in the UNSW-NB15 dataset gave an accuracy of 98.8%, and a DR of 99.9%. While the benign class revealed an accuracy of 99.9% and DR of 99.7%. In the BoT-IoT dataset, the DDoS HTTP attack revealed an accuracy of 99.22% and DR of 97.79%. While the benign class gave an accuracy of 97.54% and DR of 97.92%. In comparison with the state-of-the-art machine learning schemes, our proposed HAEMPSO-DNN achieved a competitive feat in terms of DR and accuracy.

AoI-Aware Optimization of Service Caching-Assisted Offloading and Resource Allocation in Edge Cellular Networks

The rapid development of the Internet of Things (IoT) has led to computational offloading at the edge; this is a promising paradigm for achieving intelligence everywhere. As offloading can lead to more traffic in cellular networks, cache technology is used to alleviate the channel burden. For example, a deep neural network (DNN)-based inference task requires a computation service that involves running libraries and parameters. Thus, caching the service package is necessary for repeatedly running DNN-based inference tasks. On the other hand, as the DNN parameters are usually trained in distribution, IoT devices need to fetch up-to-date parameters for inference task execution. In this work, we consider the joint optimization of computation offloading, service caching, and the AoI metric. We formulate a problem to minimize the weighted sum of the average completion delay, energy consumption, and allocated bandwidth. Then, we propose the AoI-aware service caching-assisted offloading framework (ASCO) to solve it, which consists of the method of Lagrange multipliers with the KKT condition-based offloading module (LMKO), the Lyapunov optimization-based learning and update control module (LLUC), and the Kuhn-Munkres (KM) algorithm-based channel-division fetching module (KCDF). The simulation results demonstrate that our ASCO framework achieves superior performance in regard to time overhead, energy consumption, and allocated bandwidth. It is verified that our ASCO framework not only benefits the individual task but also the global bandwidth allocation.