Deep Learning Network Structure Research Articles

HP-LSTM: Hawkes Process–LSTM-Based Detection of DDoS Attack for In-Vehicle Network

Connected and autonomous vehicles (CAVs) are advancing at a fast speed with the improvement of the automotive industry, which opens up new possibilities for different attacks. A Distributed Denial-of-Service (DDoS) attacker floods the in-vehicle network with fake messages, resulting in the failure of driving assistance systems and impairment of vehicle control functionalities, seriously disrupting the normal operation of the vehicle. In this paper, we propose a novel DDoS attack detection method for in-vehicle Ethernet Scalable service-Oriented Middleware over IP (SOME/IP), which integrates the Hawkes process with Long Short-Term Memory networks (LSTMs) to capture the dynamic behavioral features of the attacker. Specifically, we employ the Hawkes process to capture features of the DDoS attack, with its parameters reflecting the dynamism and self-exciting properties of the attack events. Subsequently, we propose a novel deep learning network structure, an HP-LSTM block, inspired by the Hawkes process, while employing a residual attention block to enhance the model’s detection efficiency and accuracy. Additionally, due to the scarcity of publicly available datasets for SOME/IP, we employed a mature SOME/IP generator to create a dataset for evaluating the validity of the proposed detection model. Finally, extensive experiments were conducted to demonstrate the effectiveness of the proposed DDoS attack detection method.

Flight Technology Evaluation Based on Flight Parameters

Based on the flight safety, this paper develops a flight technology evaluation method based on the gradient boosting decision tree ( GBDT ) model by collecting and analyzing flight data. This method comprehensively considers flight parameters and provides a more accurate pilot flight technology assessment tool through the analysis of flight records. It is expected to improve the training plan and enhance the technical level of pilots to further improve the safety and sustainable development of air transportation. By introducing the deep learning network structure optimization evaluation method, the flight safety is further enhanced.

A Deep Learning Localization Method for Acoustic Source via Improved Input Features and Network Structure

Shallow water passive source localization is an essential problem in underwater detection and localization. Traditional matched-field processing (MFP) methods are sensitive to environment mismatches. Many neural network localization methods still have room for improvement in accuracy if they are further adjusted to underwater acoustic characteristics. To address these problems, we propose a deep learning localization method via improved input features and network structure, which can effectively estimate the depth and the closest point of approach (CPA) range of the acoustic source. Firstly, we put forward a feature preprocessing scheme to enhance the localization accuracy and robustness. Secondly, we design a deep learning network structure to improve the localization accuracy further. Finally, we propose a method of visualizing the network to optimize the estimated localization results. Simulations show that the accuracy of the proposed method is better than other compared features and network structures, and the robustness is significantly better than that of the MFP methods. Experimental results further prove the effectiveness of the proposed method.

BIOMEDICAL IMAGE SEGMENTATION BASED ON DEEP LEARNING ALGORITHMS

A new biomedical image segmentation method based on deep learning networks has been constructed. Under the framework of deep learning, a dual-branch deep learning network structure was designed to perform average pooling and maximum pooling on input image data, respectively. The results of pooling processing are sent to multi-layer convolution for further processing, where 2D convolution and dilated convolution are used, respectively, and the more flexible SoftMax function is selected for activation processing. Multiple fully connected processes were used between each link and between two branches to form better fusion criteria. The experimental results show that our method has a more ideal segmentation effect in both homologous and heterogeneous training segmentation experiments, with AC, SE, and SP indicators reaching over 95%, 84%, and 99%, respectively.

Evaluation of Soil Rocky Desertification in Karst Region Based on Deep Belief Network

Abstract Dynamic features from remote sensing photos may be successfully extracted using deep learning and symmetric network structure, which can then be used to direct them to carry out accurate classification. The DBN model can more effectively extract features from photos since it uses unsupervised learning. It can be reduced to the many symmetric Restricted Boltmann Machines (RBM) training problem. In this paper, a soil rocky desertification (RD) assessment model based on a deep belief network (DBN) is created in light of the complicated influencing aspects of Karst RD risk assessment encompassing several geographical elements. The model builds upon the conventional RBM framework and incorporates the influence layer of related elements as an auxiliary requirement for retrieving Geographic Information System (GIS) score data. Then, in order to forecast the level of soil rocky desertification, it learns the features of many elements. The experimental results show that the proposed model proposed in this paper has better prediction performance and faster convergence speed, and its classification results for different degrees of RD are more consistent with the actual risk assessment results.

MIU-Net: MIX-Attention and Inception U-Net for Histopathology Image Nuclei Segmentation

In the medical field, hematoxylin and eosin (H&E)-stained histopathology images of cell nuclei analysis represent an important measure for cancer diagnosis. The most valuable aspect of the nuclei analysis is the segmentation of the different nuclei morphologies of different organs and subsequent diagnosis of the type and severity of the disease based on pathology. In recent years, deep learning techniques have been widely used in digital histopathology analysis. Automated nuclear segmentation technology enables the rapid and efficient segmentation of tens of thousands of complex and variable nuclei in histopathology images. However, a challenging problem during nuclei segmentation is the blocking of cell nuclei, overlapping, and background complexity of the tissue fraction. To address this challenge, we present MIU-net, an efficient deep learning network structure for the nuclei segmentation of histopathology images. Our proposed structure includes two blocks with modified inception module and attention module. The advantage of the modified inception module is to balance the computation and network performance of the deeper layers of the network, combined with the convolutional layer using different sizes of kernels to learn effective features in a fast and efficient manner to complete kernel segmentation. The attention module allows us to extract small and fine irregular boundary features from the images, which can better segment cancer cells that appear disorganized and fragmented. We test our methodology on public kumar datasets and achieve the highest AUC score of 0.92. The experimental results show that the proposed method achieves better performance than other state-of-the-art methods.

Exploring nonlinear strengthening in polycrystalline metallic materials by machine learning methods and heterostructure design

To improve the strength and plasticity of structural materials, researchers often introduce various strengthening mechanisms such as second-phase strengthening, dislocation strengthening, and back stress strengthening (HDI). Due to the interaction of multiple mechanisms, the linear superposition relationship has a poor fitting effect and is only used for rough calculations of the strengthening mechanisms. In this study, the transfer learning data was used to optimize the deep learning network structure (Re-CNN) based on the residual algorithm, and the yield strength prediction physical neural informed model (PNIM) of polycrystalline metallic materials was established. To promote the industrial application of the heterostructure design method, a medium carbon steel heterostructure design strategy based on the existing equipment of the factory was proposed. Medium-carbon heterostructure materials (MHSM) with mixed strengthening mechanisms were successfully prepared. MHSM exhibits excellent comprehensive mechanical properties. When a linear relationship is used to describe the MHSM yield strength, there is a large error, while Re-CNN shows satisfactory prediction accuracy. The linear relationship is incompatible with homogeneous structure materials and heterogeneous structure materials, and its universality is lower than that of nonlinear Re-CNN. Re-CNN shows high cross-scale prediction ability and can be compatible with homogeneous microstructures and heterogeneous microstructures. Using the heterogeneity evolution characteristics of MHSM, the key factors deviating from the linear relationship were revealed. The overestimation and underestimation of the linear relation are demonstrated by Taylor factor and TEM analysis to be caused by the multiscale properties of ferrite, the behavior of the second phase particles, and the interaction of various mechanisms. This study provides a new idea for the cross-scale calculation of the mechanical properties of polycrystalline metallic materials.

Fuzzy neighbourhood neural network for high-resolution remote sensing image segmentation

ABSTRACT Remote sensing image segmentation plays an important role in many industrial-grade image processing applications. However, the problem of uncertainty caused by intraclass heterogeneity and interclass blurring is prevalent in high-resolution remote sensing images. Moreover, the complexity of information in high-resolution remote sensing images leads to a large amount of background information around objects. To solve this problem, a new fuzzy convolutional neural network is proposed in this paper. This network resolves the ambiguity and uncertainty of feature information by introducing a fuzzy neighbourhood module in the deep learning network structure. In addition, it adds a multi-attention gating module to highlight small object features and separate them from the complex background information to achieve fine segmentation of high-resolution remote sensing images. Experimental results on three different segmentation datasets suggest that the proposed method has higher segmentation accuracy and better performance than other deep learning networks, especially for complicated shadow information. Code will be provided in (https://github.com/tingtingqu/code).

Deep-learning based surrogate modeling for fast and accurate simulation in realistic 3D reservoir with varying well controls

For many field applications, the need to make production and/or injection allocation among hundreds of wells on a daily or weekly basis for a fixed geological model is not uncommon. In those problems, the computational cost could be very high using traditional numerical simulation. The embed-to-control (E2C) framework provides a new deep-learning-based reduced-order modeling (ROM) framework for much faster predictions of subsurface flow compared to traditional simulation. Existing research on E2C centers around conceptual reservoirs with a small number of wells in 2D Cartesian grids, and a direct application to realistic 3D reservoirs remain a challenge. This paper proposes an improved surrogate model with significant enhancements in the deep-learning network structure, well quantity calculation, and loss function specification. Firstly, we adopt the dense block structure to increase the width of the encoder, and apply the skip connection architecture to use the intermediate information in the decoding process to calibrate the model; secondly, we add dense structure to evolve production information directly in converter instead of using Peaceman's equations as a post-processing procedure that might cause significant errors; thirdly we add the oil and water rate losses to enhance the accuracy of well dynamics. Compared with original model in a complex 2D reservoir with 189 wells, our improved model can reduce the median error of production from 36% to 4%, saturation from 13% to 1.2% and pressure from 4.6% to 0.5%. Finally, our improved model is extended to 3D applied on an actual 3D reservoir with 105 wells. Compared to the simulation results, the improved E2C model is highly accurate with a median average error of 3.1% for production, 1.0% for saturation and 5.9% for pressure. In terms of time cost, our model can achieve approximately 700x speedups on a RTX2060 GPU after training.

Deep Learning-Based Classification of Raw Hydroacoustic Signal: A Review

Underwater target recognition is a research component that is crucial to realizing crewless underwater detection missions and has significant prospects in both civil and military applications. This paper provides a comprehensive description of the current stage of deep-learning methods with respect to raw hydroacoustic data classification, focusing mainly on the variety and recognition of vessels and environmental noise from raw hydroacoustic data. This work not only aims to describe the latest research progress in this field but also summarizes three main elements of the current stage of development: feature extraction in the time and frequency domains, data enhancement by neural networks, and feature classification based on deep learning. In this paper, we analyze and discuss the process of hydroacoustic signal processing; demonstrate that the method of feature fusion can be used in the pre-processing stage in classification and recognition algorithms based on raw hydroacoustic data, which can significantly improve target recognition accuracy; show that data enhancement algorithms can be used to improve the efficiency of recognition in complex environments in terms of deep learning network structure; and further discuss the field’s future development directions.

Research on Authentic Signature Identification Method Integrating Dynamic and Static Features

In many fields of social life, such as justice, finance, communication and so on, signatures are used for identity recognition. The increasingly convenient and extensive application of technology increases the opportunity for forged signatures. How to effectively identify a forged signature is still a challenge to be tackled by research. Offline static handwriting has a unique structure and strong interpretability, while online handwriting contains dynamic information, such as timing and pressure. Therefore, this paper proposes an authentic signature identification method, integrating dynamic and static features. The dynamic data and structural style of the signature are extracted by dot matrix pen technology, the global and local features, time and space features are fused and clearer and understandable features are applied to signature identification. At the same time, the classification of a forged signature is more detailed according to the characteristics of signature and a variety of machine learning models and a deep learning network structure are used for classification and recognition. When the number of classifications is 5, it is better to identify simple forgery signatures. When the classification number is 15, the accuracy rate is mostly about 96.7% and the highest accuracy reaches 100% on CNN. This paper focuses on feature extraction, incorporates the advantages of dynamic and static features and improves the classification accuracy of signature identification.

Dynamic scheduling for semiconductor manufacturing systems with uncertainties using convolutional neural networks and reinforcement learning

The dynamic scheduling problem of semiconductor manufacturing systems (SMSs) is becoming more complicated and challenging due to internal uncertainties and external demand changes. To this end, this paper addresses integrated release control and production scheduling problems with uncertain processing times and urgent orders and proposes a convolutional neural network and asynchronous advanced actor critic-based method (CNN-A3C) that involves a training phase and a deployment phase. In the training phase, actor–critic networks are trained to predict the evaluation of scheduling decisions and to output the optimal scheduling decision. In the deployment phase, the most appropriate release control and scheduling decisions are periodically generated according to the current production status based on the networks. Furthermore, we improve the four key points in the deep reinforcement learning (DRL) algorithm, state space, action space, reward function, and network structure and design four mechanisms: a slide-window-based two-dimensional state perception mechanism, an adaptive reward function that considers multiple objectives and automatically adjusts to dynamic events, a continuous action space based on composite dispatching rules (CDR) and release strategies, and actor–critic networks based on convolutional neural networks (CNNs). To verify the feasibility and effectiveness of the proposed dynamic scheduling method, it is implemented on a simplified SMS. The simulation experimental results show that the proposed method outperforms the unimproved A3C-based method and the common dispatching rules under the new uncertain scenarios.

Damage Detection Method of Mining Conveyor Belt Based on Deep Learning

The belt conveyor is one of the key equipment in coal mining. Timely identification of belt damage is necessary to ensure coal mine safety production and reduce economic losses. In order to meet the needs of practical application, a new method of belt damage identification is proposed by fusing the temporal and spatial features of the image and directly using the on-site monitoring video image. Based on the deep learning network structure, an end-to-end belt damage detection model is designed. An improved attention mechanism is designed and introduced into the image spatial feature extraction model to solve the problem of complex background in belt surveillance video. In order to reduce the influence of reflection and shadow on recognition accuracy, a method of extracting temporal features of continuous multi frame video images using Temporal Convolutional Networks (TCN) is designed. The experimental results show that the accuracy of the method for belt damage recognition by fusing the temporal and spatial features of the image is more than 20% higher than that of the method for extracting the spatial features of the image alone, which shows that our proposed method is suitable for belt damage detection directly using the on-site belt monitoring video and is practical.

Identification and diagnosis of meniscus tear by magnetic resonance imaging using a deep learning model.

ObjectiveMeniscus tear is a common problem in sports trauma, and its imaging diagnosis mainly relies on MRI. To improve the diagnostic accuracy and efficiency, a deep learning model was employed in this study and the identification efficiency was evaluated.MethodsStandard knee MRI images from 924 individual patients were used to complete the training, validation and testing processes. Mask regional convolutional neural network (R–CNN) was used to build the deep learning network structure, and ResNet50 was adopted to develop the backbone network. The deep learning model was trained and validated with a dataset containing 504 and 220 patients, respectively. Internal testing was performed based on a dataset of 200 patients, and 180 patients from 8 hospitals were regarded as an external dataset for model validation. Additionally, 40 patients who were diagnosed by the arthroscopic surgery were enrolled as the final test dataset.ResultsAfter training and validation, the deep learning model effectively recognized healthy and injured menisci. Average precision for the three types of menisci (healthy, torn and degenerated menisci) ranged from 68% to 80%. Diagnostic accuracy for healthy, torn and degenerated menisci was 87.50%, 86.96%, and 84.78%, respectively. Validation results from external dataset demonstrated that the accuracy of diagnosing torn and intact meniscus tear through 3.0T MRI images was higher than 80%, while the accuracy verified by arthroscopic surgery was 87.50%.ConclusionMask R–CNN effectively identified and diagnosed meniscal injuries, especially for tears that occurred in different parts of the meniscus. The recognition ability was admirable, and the diagnostic accuracy could be further improved with increased training sample size. Therefore, this deep learning model showed great potential in diagnosing meniscus injuries.Translational potential of this articleDeep learning model exerted unique effect in terms of reducing doctors’ workload and improving diagnostic accuracy. Injured and healthy menisci could be more accurately identified and classified based on training and learning datasets. This model could also distinguish torn from degenerated menisci, making it an effective tool for MRI-assisted diagnosis of meniscus injuries in clinical practice.

Measurement of correlated color temperature from RGB images by deep regression model

Accurate measurement of correlated color temperature (CCT) which represents the color of light source is critical in photometry research. Current research shows that the value of CCT in environment has diverse and important effects on human perception and behaviour. Since we spend a large portion of our day indoors, these effects have a considerably greater impact on our daily lives than we think. For instance, the temperature perception of the environment or shape perceptions of students is related to CCT value in the environment. The value of CCT which is created by light sources in environment can be determined in a variety of ways in literature. CCT values of the environment can be measured precisely with spectroradiometers, which are special and relatively expensive measurement devices that aim to precisely measure radiance, luminance and chromaticity of light. Additionally, CCT values can be estimated with lower accuracy using various color space transformations and predefined models instead of spectroradiometer devices. In this study, an alternative approach to these two techniques for measuring CCT in the environment is proposed. CCT values of the environments were determined closer to spectroradiometer measurement results thanks to the deep regression model developed within the scope of this study using only RGB cameras. 191 different specially created RGB images and corresponding CCT values were taught to deep learning network structure for creating a regression model. The proposed approach performance was compared to alternative CCT calculation methods in literature using a variety of real-scene test images.

Classifying Gas Data Measured Under Multiple Conditions Using Deep Learning

Gas classification is a machine learning problem that is important for various applications including monitoring systems, health care, public security, etc. Since measuring the characteristic of gas molecules is greatly affected by external factors such as wind speed and the internal setting of detecting sensors, classification should be done by taking into account the combination of these individual factors, which we call a <i>condition</i> in this paper. In particular, when classifying gas data measured under multiple conditions, the data from each condition need to be integrated, which we call <i>multi-conditioned gas classification</i>. While there have been some studies on gas classification for a single condition, no previous approach deals with the multi-conditioned gas classification problem to the best of our knowledge. In this paper, we propose a novel multi-conditioned gas classification method for the first time. We present a new deep learning network structure that can efficiently extract features from the data of multiple conditions and effectively integrate them, which is referred to as a multi-conditioned gas classification network (MCGCN). We also propose a new training loss function to guarantee good performance reliably for the varying number of given conditions. Experimental results demonstrate the superiority of the proposed method, which achieves accuracies of 99.15% &#x00B1; 0.41 regardless of the number of conditions with 15 times fewer model parameters in comparison to the existing method.

Word-Level and Pinyin-Level Based Chinese Short Text Classification

Short text classification is an important branch of Natural Language Processing. Although CNN and RNN have achieved satisfactory results in the text classification tasks, they are difficult to apply to the Chinese short text classification because of the data sparsity and the homophonic typos problems of them. To solve the above problems, word-level and Pinyin-level based Chinese short text classification model is constructed. Since homophones have the same Pinyin, the addition of Pinyin-level features can solve the homophonic typos problem. In addition, due to the introduction of more features, the data sparsity problem of short text can be solved. In order to fully extract the deep hidden features of the short text, a deep learning model based on BiLSTM, Attention and CNN is constructed, and the residual network is used to solve the gradient disappearance problem with the increase of network layers. Additionally, considering that the complex deep learning network structure will increase the text classification time, the Text Center is constructed. When there is a new text input, the text classification task can be quickly realized by calculating the Manhattan distance between the embedding vector of it and the vectors stored in the Text Center. The Accuracy, Precision, Recall and F1 of the proposed model on the simplifyweibo_4_moods dataset are 0.9713, 0.9627, 0.9765 and 0.9696 respectively, and those on the online_shopping_10_cats dataset are 0.9533, 0.9416, 0.9608 and 0.9511 respectively, which are better than that of the baseline method. In addition, the classification time of the proposed model on simplifyweibo_4_moods and online_shopping_10_cats is 0.0042 and 0.0033 respectively, which is far lower than that of the baseline method.

Rolling bearing fault diagnosis based on SSA optimized self-adaptive DBN

Due to the structure of rolling bearings and the complexity of the operating environment, collected vibration signals tend to show strong non-stationary and time-varying characteristics. Extracting useful fault feature information from actual bearing vibration signals and identifying bearing faults is challenging. In this paper, an innovative optimized adaptive deep belief network (SADBN) is proposed to address the problem of rolling bearing fault identification. The DBN is pre-trained by the minimum batch stochastic gradient descent. Then, a back propagation neural network and conjugate gradient descent are used to supervise and fine-tune the entire DBN model, which effectively improve the classification accuracy of the DBN. The salp swarm algorithm, an intelligent optimization method, is used to optimize the DBN. Then, the experience of deep learning network structure is summarized. Finally, a series of simulations based on the experimental data verify the effectiveness of the proposed method.

Complexity Assessment of Chronic Pain in Elderly Knee Osteoarthritis Based on Neuroimaging Recognition Techniques.

The chronic pain of knee osteoarthritis in the elderly is investigated in detail in this paper, as well as the complexity of chronic pain utilising neuroimaging recognition techniques. Chronic pain in knee osteoarthritis (KOA) has a major effect on patients' quality of life and functional activities; therefore, understanding the causes of KOA pain and the analgesic advantages of different therapies is important. In recent years, neuroimaging techniques have become increasingly important in basic and clinical pain research. Thanks to the application and development of neuroimaging techniques in the study of chronic pain in KOA, researchers have found that chronic pain in KOA contains both injury-receptive and neuropathic pain components. The neuropathic pain mechanism that causes KOA pain is complicated, and it may be produced by peripheral or central sensitization, but it has not gotten enough attention in clinical practice, and there is no agreement on how to treat combination neuropathic pain KOA. As a result, using neuroimaging techniques such as magnetic resonance imaging (MRI), electroencephalography (EEG), magnetoencephalography (MEG), and near-infrared spectroscopy (NIRS), this review examines the changes in brain pathophysiology-related regions caused by KOA pain, compares the latest results in pain assessment and prediction, and clarifies the central brain analgesic mechanistic. The capsule network model is introduced in this paper from the perspective of deep learning network structure to construct an information-complete and reversible image low-level feature bridge using isotropic representation, predict the corresponding capsule features from MRI voxel responses, and then, complete the accurate reconstruction of simple images using inverse transformation. The proposed model improves the structural similarity index by about 10%, improves the reconstruction performance of low-level feature content in simple images by about 10%, and achieves feature interpretation and analysis of low-level visual cortical fMRI voxels by visualising capsule features, according to the experimental results.

Malware Classification Based on Multilayer Perception and Word2Vec for IoT Security

With the construction of smart cities, the number of Internet of Things (IoT) devices is growing rapidly, leading to an explosive growth of malware designed for IoT devices. These malware pose a serious threat to the security of IoT devices. The traditional malware classification methods mainly rely on feature engineering. To improve accuracy, a large number of different types of features will be extracted from malware files in these methods. That brings a high complexity to the classification. To solve these issues, a malware classification method based on Word2Vec and Multilayer Perception (MLP) is proposed in this article. First, for one malware sample, Word2Vec is used to calculate a word vector for all bytes of the binary file and all instructions in the assembly file. Second, we combine these vectors into a 256x256x2-dimensional matrix. Finally, we designed a deep learning network structure based on MLP to train the model. Then the model is used to classify the testing samples. The experimental results prove that the method has a high accuracy of 99.54%.