Application Of Deep Learning Models Research Articles

Engineering archive management model based on big data analysis and deep learning model

Abstract In the background of the era of big data, the information management system of engineering archives has become more comprehensive and perfect because of the application of information technology. The application of deep learning model makes the management of engineering archives more systematic, scientific and standardized, which greatly improves the quality and efficiency of engineering archives management. The progress of society and the development of the times have put forward higher requirements for digital storage technology. This paper combines the characteristics of the new technological era, analyses the characteristics of traditional information management in the context of data processing, artificial intelligence, deep learning and other data, proposes a method for developing and managing web archives based on Bootstrapping technology, introduces an information meta-evaluation mechanism to improve the quality of mining, and uses a long and short-term memory model to extract multi-type fine-grained archival information elements in the corpus. Finally, the Alex network was established to manage the archives in a categorised manner. The experimental results show that the query results of the proposed method for the target archives are 100% accurate, and the query time for individual archives is basically within 5s, which has good management effect.

Analysis of the survey of voice synthesis technology

The original purpose of speech synthesis was to complete the task of converting from text to speech (TTS). With the application of deep learning models in this field of speech synthesis, the results of speech synthesis have gradually reached the level of human voice, which makes it widely used for voice assistant, navigation, reading, intelligent customer service, and many other aspects. In order to help readers expand more research ideas and understand the development process of speech synthesis technology and the ethical choices faced, this article introduces as much as possible in the development process of speech synthesis technology, several frameworks for optimization, as well as interpreting their respective advantages and disadvantages, summarize their approximate results obtained in the MOS test method and analyze the ethical issues that may be brought about by voice cloning technology.

Application of deep learning in damage classification of reinforced concrete bridges

Inspecting Reinforced Concrete (RC) Bridges is crucial to ensure their safety and perform essential maintenance. The current research introduces the knowledge base for applying deep learning to classify and detect RC bridges' five most common defects (cracks, corrosion, efflorescence, spalling, and exposed steel reinforcement). Theimage classification process was carried out using Xception & Vanilla models based on convolutional neural networks (CNN). A comparative study between the two models is presented for multi-class, multi-target image classification.The concrete defect bridge image (CODEBRIM) dataset was used to train and test the models. The outcomes showed the potential application of deep learning models (Xception & Vanilla) for defect classification of concrete bridges and the superiority of the Xception model in defect classification with an accuracy of 94.95%, compared to 85.71% accuracy for the Vanilla model.

Generating Music with Data: Application of Deep Learning Models for Symbolic Music Composition

Language models based on deep learning showed promising results for artistic generation purposes, including musical generation. However, the evaluation of symbolic musical generation models is mostly based on low-level mathematical metrics (e.g., the result of the loss function) due to the inherent difficulty in measuring the musical quality of a given performance due to the subjective nature of music. This work sought to measure and evaluate musical excerpts generated by deep learning models from a human perspective, limited to the scope of classical piano music generation. In this assessment, a population of 117 people performed blind tests with musical excerpts of human composition and musical excerpts generated using artificial intelligence models, including the models PerformanceRNN, Music Transformer, MuseNet and a custom model based on GRUs. The experiments demonstrated that musical excerpts generated using models based on the Transformer neural network architecture obtained the greatest receptivity within the tested population, surpassing the results of human compositions. In addition, the experiments also demonstrated that people with greater musical sensitivity and musical experience were more able to identify the compositional origin of the excerpts heard.

Deep learning-enabled segmentation of ambiguous bioimages with deepflash2

Bioimages frequently exhibit low signal-to-noise ratios due to experimental conditions, specimen characteristics, and imaging trade-offs. Reliable segmentation of such ambiguous images is difficult and laborious. Here we introduce deepflash2, a deep learning-enabled segmentation tool for bioimage analysis. The tool addresses typical challenges that may arise during the training, evaluation, and application of deep learning models on ambiguous data. The tool’s training and evaluation pipeline uses multiple expert annotations and deep model ensembles to achieve accurate results. The application pipeline supports various use-cases for expert annotations and includes a quality assurance mechanism in the form of uncertainty measures. Benchmarked against other tools, deepflash2 offers both high predictive accuracy and efficient computational resource usage. The tool is built upon established deep learning libraries and enables sharing of trained model ensembles with the research community. deepflash2 aims to simplify the integration of deep learning into bioimage analysis projects while improving accuracy and reliability.

Domain Adaptation with Contrastive Simultaneous Multi-Loss Training for Hand Gesture Recognition

Hand gesture recognition from images is a critical task with various real-world applications, particularly in the field of human-robot interaction. Industrial environments, where non-verbal communication is preferred, are significant areas of application for gesture recognition. However, these environments are often unstructured and noisy, with complex and dynamic backgrounds, making accurate hand segmentation a challenging task. Currently, most solutions employ heavy preprocessing to segment the hand, followed by the application of deep learning models to classify the gestures. To address this challenge and develop a more robust and generalizable classification model, we propose a new form of domain adaptation using multi-loss training and contrastive learning. Our approach is particularly relevant in industrial collaborative scenarios, where hand segmentation is difficult and context-dependent. In this paper, we present an innovative solution that further challenges the existing approach by testing the model on an entirely unrelated dataset with different users. We use a dataset for training and validation and demonstrate that contrastive learning techniques in simultaneous multi-loss functions provide superior performance in hand gesture recognition compared to conventional approaches in similar conditions.

Stock Return Prediction Based on CNN Model and Genetic Evolutionary Algorithm For A-share Medical Sector

Since the outbreak of the epidemic in December 2019, the real economy has been seriously affected, where the stock market has experienced a short-term crash. In this context, the financial market has become more turbulent and unpredictable. Previously, the traditional prediction method only captures the linear relationship between both the dependent and the independent variable. With the growing advancement of technology and science, the application of deep learning models to learn the nonlinear relationship between high latitude indicators and predictors has become the mainstream trend of stock prediction. Therefore, a convolutional neural network (CNN) model with fusion genetic algorithm is proposed in this paper to predict and analyze the stock return rate of China’s A-share medical sector from January 1,2020 to February 2020. In order to facilitate the operation, this paper chooses the GPU with better performance to process the large-scale calculation in the neural network and builds the experimental environment under the Huawei cloud model arts notebook system. This paper selects the Sequential model belonging to the tensor flow library Keras package. Subsequently, this paper optimizes the parameters in the model by using genetic algorithm. According to the analysis, the mean square error of the optimal model is 0.0017, and the prediction results of the model in the test set are validated with well performances, which is of great significance to investors' decision-making. These results shed light on guiding further exploration of stock trend prediction.

Uncertainty quantification-based robust deep learning for building energy systems considering distribution shift problem

Artificial intelligence becomes one of the key technologies in building energy conservation, where deep learning algorithms provide feasible solution for the modeling problem of complex building energy systems. However, existing deep learning models are lack of robustness under distribution shift scenarios, which greatly limits their real-world application. To tackle this problem, first, we enrich the concept of distribution shift problem in building energy domain. And a cluster-based dataset splitting method is proposed to simulate the distribution shift scenarios. Second, we adopt uncertainty quantification methods to improve the overall robustness of deep learning model. Comprehensive data experiments are conducted based on the reference modeling problem of chiller, and five widely-used uncertainty quantification methods are compared under distribution shift scenarios. Both the model precision performance and robustness performance are investigated. The experimental result demonstrates that the deep ensemble (DE) model is a simple but efficient method to solve the distribution shift problem, where its precision performance and robustness performance outperformed the other methods, followed by Bayesian neural network (BNN). In addition, the benefit of robust deep learning model is also studied. It is shown that the deep learning model with higher robustness performance can better filter out the unreliable predictions, which is essential to the online active learning strategy. The proposed research framework might provide a solid foundation for the real-world application of deep learning model.

Fog-cloud based intrusion detection system using Recurrent Neural Networks and feature selection for IoT networks

Deep learning (DL) techniques are being widely researched for their effectiveness in detecting cyber intrusions against the Internet of Things (IoT). Time sensitive Critical Infrastructures (CIs) that rely on IoT require rapid detection of cyber intrusions close to the constrained devices in order to prevent service delays. Deep learning techniques perform better in detecting attacks compared to shallow machine learning algorithms and can be used for intrusion detection. However, communication overheads due to large volume of IoT data and computation requirements for deep learning models prevents effective application of deep learning models closer to the constrained devices. Existing IDS techniques are either based on shallow learning algorithms or not trained on relevant IoT datasets and furthermore not designed for distributed fog-cloud deployment. To counter these issues, we propose a novel fog-cloud based IoT intrusion detection framework which incorporates a distributed processing by splitting the dataset according to attack class and a feature selection step on time-series IoT data. This is followed by a deep learning Recurrent Neural Network (SimpleRNN and Bi-directional Long Short-Term Memory (LSTM)) for attack detection. The effectiveness of the proposed approach was evaluated using the high-dimensional BoT-IoT dataset which contains large volumes of realistic IoT attack traffic. Results show that feature selection methods significantly reduced the dataset size by 90% under the computation requirements without compromising on the attack detection ability. The models built on reduced dataset achieved higher recall rate compared to models trained on full feature set without loosing class differentiation ability. The SimpleRNN and Bi-LSTM models also did not suffer any underfitting or overfitting with the reduced feature space. The proposed deep learning based IoT intrusion detection framework is suitable for fog-cloud based deployment and can scale well even with large volumes of IoT data.

Experimental Studies for the Application of deep learning Models in Forecasting Infection of COVID-19

This study predicts and forecasts daily positive COVID-19 counts. In the epidemic fields, statistical methods are typically used. However, recently, machine learning is being applied. The appropriate model is different in each country because of differences in measurement methods and nationality. Thus, an adequate model needs to be evaluated and selected for use in Japan. Hence, in this paper, RNNs were trained with two loss functions, MSE and AIC, and then evaluated and compared based on ARIMA. The comparison results show a 49.5% reduction of RNN from the ARIMA’s RMSE. Seq2Seq has an R2=0.92. Moreover, this paper presents the comparison results of an RNN with the others models.

Transformer Based Water Level Prediction in Poyang Lake, China

Water level is an important indicator of lake hydrology characteristics, and its fluctuation significantly affects lake ecosystems. In recent years, deep learning models have shown their superiority in the long-time range prediction of hydrology processes, while the application of deep learning models with the attention mechanism for lake water level prediction is very rare. In this paper, taking Poyang Lake as a case study, the transformer neural network model is applied to examine the model performance in lake water level prediction, to explore the effects of the Yangtze River on lake water level fluctuations, and to analyze the influence of hyper-parameters (window size and model layers) and lead time on the model accuracy. The result indicated that the transformer model performs well in simulating the lake water level variations and can reflect the temporal water level variation characteristics in Poyang Lake. In the testing stage, the RMSE values were recorded in the range of 0.26–0.70 m, and the NSE values are higher than 0.94. Moreover, the Yangtze River inflow has a great influence on the lake water level fluctuation of Poyang Lake, especially in flood and receding periods. The contribution rate of the Yangtze River in RMSE and NSE is higher than 80% and 270%, respectively. Additionally, hyper-parameters, such as window size and model layers, significantly influence the transformer model simulation accuracy. In this study, a window size of 90 d and a model layer of 6 are the most suitable hyper-parameters for water level prediction in Poyang Lake. Additionally, lead time may affect the model accuracy in lake water level prediction. With the lead time varied from one to seven days, the model accuracy was high and RMSE values were in the range of 0.46–0.73 m, while the RMSE value increased to 1.37 m and 1.82 m with the lead time of 15 and 30 days, respectively. The transformer neural network model constructed in this paper was the first to be applied to lake water forecasting and showed high efficiency in Poyang Lake. However, few studies have tried to use transformer model coupling with the attention mechanism for forecasting hydrological processes. It is suggested that the model can be used for long sequence time-series forecasting in hydrological processes in other lakes to test its performance, providing further scientific evidence for the control of lake floods and management of lake resources.

Machine Learning Methods for Cancer Classification Using Gene Expression Data: A Review

Cancer is a term that denotes a group of diseases caused by the abnormal growth of cells that can spread in different parts of the body. According to the World Health Organization (WHO), cancer is the second major cause of death after cardiovascular diseases. Gene expression can play a fundamental role in the early detection of cancer, as it is indicative of the biochemical processes in tissue and cells, as well as the genetic characteristics of an organism. Deoxyribonucleic acid (DNA) microarrays and ribonucleic acid (RNA)-sequencing methods for gene expression data allow quantifying the expression levels of genes and produce valuable data for computational analysis. This study reviews recent progress in gene expression analysis for cancer classification using machine learning methods. Both conventional and deep learning-based approaches are reviewed, with an emphasis on the application of deep learning models due to their comparative advantages for identifying gene patterns that are distinctive for various types of cancers. Relevant works that employ the most commonly used deep neural network architectures are covered, including multi-layer perceptrons, as well as convolutional, recurrent, graph, and transformer networks. This survey also presents an overview of the data collection methods for gene expression analysis and lists important datasets that are commonly used for supervised machine learning for this task. Furthermore, we review pertinent techniques for feature engineering and data preprocessing that are typically used to handle the high dimensionality of gene expression data, caused by a large number of genes present in data samples. The paper concludes with a discussion of future research directions for machine learning-based gene expression analysis for cancer classification.

Diffusion Tensor Imaging in Amyotrophic Lateral Sclerosis: Machine Learning for Biomarker Development.

Diffusion tensor imaging (DTI) allows the in vivo imaging of pathological white matter alterations, either with unbiased voxel-wise or hypothesis-guided tract-based analysis. Alterations of diffusion metrics are indicative of the cerebral status of patients with amyotrophic lateral sclerosis (ALS) at the individual level. Using machine learning (ML) models to analyze complex and high-dimensional neuroimaging data sets, new opportunities for DTI-based biomarkers in ALS arise. This review aims to summarize how different ML models based on DTI parameters can be used for supervised diagnostic classifications and to provide individualized patient stratification with unsupervised approaches in ALS. To capture the whole spectrum of neuropathological signatures, DTI might be combined with additional modalities, such as structural T1w 3-D MRI in ML models. To further improve the power of ML in ALS and enable the application of deep learning models, standardized DTI protocols and multi-center collaborations are needed to validate multimodal DTI biomarkers. The application of ML models to multiparametric MRI/multimodal DTI-based data sets will enable a detailed assessment of neuropathological signatures in patients with ALS and the development of novel neuroimaging biomarkers that could be used in the clinical workup.

Improved Soil Quality Prediction Model Using Deep Learning for Smart Agriculture Systems

Soil is the major source of infinite lives on Earth and the quality of soil plays significant role on Agriculture practices all around. Hence, the evaluation of soil quality is very important for determining the amount of nutrients that the soil require for proper yield. In present decade, the application of deep learning models in many fields of research has created greater impact. The increasing soil data availability of soil data there is a greater demand for the remotely avail open source model, leads to the incorporation of deep learning method to predict the soil quality. With that concern, this paper proposes a novel model called Improved Soil Quality Prediction Model using Deep Learning (ISQP-DL). The work considers the chemical, physical and biological factors of soil in particular area to estimate the soil quality. Firstly, pH rating of soil samples has been collected from the soil testing laboratory from which the acidic range has been categorized through soil test and the same data has been taken as input to the Deep Neural Network Regression (DNNR) model. Secondly, soil nutrient data has been given as second input to the DNNR model. By utilizing this data set, the DNNR method is used to evaluate the fertility rate by which the soil quality has been estimated. For training and testing, the model uses Deep Neural Network Regression (DNNR), by utilizing the dataset. The results show that the proposed model is effective for SQP (Soil Quality Prediction Model) with efficient good fitting and generality is enhanced with input features with higher rate of classification accuracy. The results show that the proposed model achieves 96.7% of accuracy rate compared with existing models.

The Construction of the Value System of English Language and Literature Courses Based on the Deep Learning Model

With the rapid development of artificial intelligence, big data and other modern technologies, the intelligence of education has become the trend of modern education reform, and the deep learning model, as a representative of the application of artificial intelligence in the field of education, has a natural connection with the construction of the value system of English language and literature courses. This paper analyzes the feasibility and significance of the application of deep learning model in English language and literature from the characteristics of deep learning model, and constructs the value system of English language and literature based on it, aiming to provide more choices for English majors and improve their interest and enthusiasm in learning English language and literature.

Application of Deep Learning Model of Small Image Data Based on Active Transfer Learning

Deep learning models are known for their ability to learn complex patterns and make accurate predictions from data, but their performance is heavily dependent on the quantity and quality of the data used to train them. In many cases, a large amount of labeled data is required to achieve high performance, but collecting and labeling data can be time-consuming and expensive. To solve these problems, this paper proposes a method to utilize active transfer learning-based image small-scale data. This allows us to learn only 74% less data on Fashion-MNIST data and derive 84.9% accuracy.

Deep Learning for Detecting Cyber Threats in Indian Government Networks

The increasing digitization of government services in India has led to a surge in cyber threats, necessitating advanced solutions for cybersecurity. This paper examines the application of deep learning models, including Convolutional Neural Networks (CNN) and Long Short-Term Memory (LSTM) networks, for detecting and mitigating cyber threats in Indian government networks. The study evaluates the performance of these models in detecting malware, ransomware, and phishing attacks by analyzing large datasets of network traffic. A key focus is on the scalability of these models to handle vast amounts of real-time data while maintaining high detection accuracy. The paper also highlights the challenges of implementing deep learning-based cybersecurity solutions in government networks, such as data privacy concerns, the complexity of infrastructure, and the need for policy frameworks. By presenting case studies of successful implementations in Indian government agencies, the paper showcases the potential of deep learning to revolutionize cybersecurity in the public sector.

A Light Weight Multisensory Fusion Model for Induction Motor Fault Diagnosis

Fault diagnosis keeps an essential tool to ensure the safety and reliability of a motor system. Based on deep learning, fault diagnosis models constructed by mining historical fault data of equipment have received extensive attention. However, the high computational cost constrains the application of deep learning models for fault diagnosis, especially when coping with multisource data. Inspired by the model reduction and neural network structure automatic search method, this article proposed a light weight multisensory fusion model for induction motor data fusion and diagnosis. Inverted residual block and network architecture search technology are introduced to accelerate the training speed and prediction speed of the diagnostic model, so that the diagnostic accuracy is maintained at an acceptable level. The effectiveness of the proposed model is demonstrated through motor fault diagnosis experiments. Compared with other popular neural networks, the proposed method is capable of judging fault patterns accurately with shorter prediction time.

E-prop on SpiNNaker 2: Exploring online learning in spiking RNNs on neuromorphic hardware.

In recent years, the application of deep learning models at the edge has gained attention. Typically, artificial neural networks (ANNs) are trained on graphics processing units (GPUs) and optimized for efficient execution on edge devices. Training ANNs directly at the edge is the next step with many applications such as the adaptation of models to specific situations like changes in environmental settings or optimization for individuals, e.g., optimization for speakers for speech processing. Also, local training can preserve privacy. Over the last few years, many algorithms have been developed to reduce memory footprint and computation. A specific challenge to train recurrent neural networks (RNNs) for processing sequential data is the need for the Back Propagation Through Time (BPTT) algorithm to store the network state of all time steps. This limitation is resolved by the biologically-inspired E-prop approach for training Spiking Recurrent Neural Networks (SRNNs). We implement the E-prop algorithm on a prototype of the SpiNNaker 2 neuromorphic system. A parallelization strategy is developed to split and train networks on the ARM cores of SpiNNaker 2 to make efficient use of both memory and compute resources. We trained an SRNN from scratch on SpiNNaker 2 in real-time on the Google Speech Command dataset for keyword spotting. We achieved an accuracy of 91.12% while requiring only 680 KB of memory for training the network with 25 K weights. Compared to other spiking neural networks with equal or better accuracy, our work is significantly more memory-efficient. In addition, we performed a memory and time profiling of the E-prop algorithm. This is used on the one hand to discuss whether E-prop or BPTT is better suited for training a model at the edge and on the other hand to explore architecture modifications to SpiNNaker 2 to speed up online learning. Finally, energy estimations predict that the SRNN can be trained on SpiNNaker2 with 12 times less energy than using a NVIDIA V100 GPU.

Application of deep learning models for detection of subdural hematoma: a systematic review and meta-analysis

BackgroundThis study aimed to investigate the application of deep learning (DL) models for the detection of subdural hematoma (SDH).MethodsWe conducted a comprehensive search using relevant keywords. Articles extracted were original...