Hybrid Deep Neural Network Model Research Articles

A new hybrid deep neural network for multiple sites PM2.5 forecasting

Many studies have confirmed that fine particulate matter (PM2.5) poses significant hazards to both human health and the ecological environment. Predicting future trends in PM2.5 concentrations is crucial for preventing air pollution hazards to human health. However, current machine learning-based forecasting models often focus on specific regions or sites, resulting in poor spatial generalization performance. To address this issue, this study utilizes ERA5 meteorological reanalysis data, MERRA2 assimilated aerosol optical depth (AOD) data, and ground pollutant monitoring site data to construct a three-dimensional dataset. A novel hybrid deep neural network model (D_GAT) is proposed to simultaneously forecast the future 72h PM2.5 concentration trends for all sites in China. The D_GAT model combines the advantages of a graph attention network and a long short-term memory neural network, which can effectively aggregate spatial information between sites and simulate the spatial transport process and time series change characteristics of pollutant particles. Moreover, the proposed calculation method for attention scores quantified based on the actual pollutant propagation characteristics, combined with the upper atmosphere pollution transport process represented by AOD, can better express the pollution contributions of different neighboring sites to the predicted central site. Experimental results show that, compared to four other baseline models, the novel model achieves an average R2 of 0.732 for all sites in 72-h prediction, which is the highest among the five models, and the prediction accuracy of 72h is also in the forefront compared with the current similar research. The RMSE and MAE are 14.63 μg/m3 and 9.49 μg/m3, respectively, which are 2–4 μg/m3 lower than the other four baseline models, indicating that it has higher reliability and can achieve the goal of timely prediction and early warning. Overall, the new model is an effective tool for multiple sites generalization forecasting.

A Study on the Accuracy of Micro Expression Based Deception Detection with Hybrid Deep Neural Network Models

This article details a study on enhancing deception detection accuracy by using Hybrid Deep Neural Network (HDNN) models. The research, focusing on fear-related micro-expressions, utilizes a diverse dataset of responses to high-stakes questions. It analyzes facial action units (AUs) and pupil size variations through data preprocessing and feature extraction. The HDNN model outperforms the traditional Convolutional Neural Network (CNN) with a 91% accuracy rate. The findings’ implications for security, law enforcement, psychology, and behavioral treatments are discussed. Ethical considerations of deception detection technology deployment and future research directions, including cross-cultural studies, real-world assessments, ethical guidelines, studies on emotional expression dynamics, “explainable AI” development, and multimodal data integration, are also explored. The study contributes to deception detection knowledge and highlights the potential of machine learning techniques, especially HDNN, in improving decision-making and security in high-stakes situations.

Accelerating regional-scale groundwater flow simulations with a hybrid deep neural network model incorporating mixed input types: A case study of the northeast Qatar aquifer

ABSTRACT This study presents the ‘Dual Path CNN-MLP’, a novel hybrid deep neural network (DNN) architecture that merges the strengths of convolutional neural networks (CNNs) and multilayer perceptrons (MLPs) for regional groundwater flow simulations. This model stands out from previous DNN approaches by managing mixed input types, including both imagery and numerical vectors. Such flexibility allows the diverse nature of groundwater data to be efficiently utilized without the need to convert it into a uniform format, which often leads to oversimplification or unnecessary expansion of the dataset. When applied to the northeast Qatar aquifer, the model demonstrates high accuracy in simulating transient groundwater flow fields, benchmarked against the well-established MODFLOW model. The model's efficacy is confirmed through k-fold cross-validation, showing an error margin of less than 12% across all examined locations. The study also examines the model's ability to perform uncertainty analysis using Monte Carlo simulations, finding that it achieves around 1% average absolute percentage error in estimating the mean hydraulic head. Errors are mostly found in areas with significant variations in the hydraulic head. Switching to this ML model from the conventional MODFLOW simulator boosts computational efficiency by about 99%, showcasing its advantage for tasks like uncertainty analysis in repetitive groundwater simulations.

A transformer acoustic signal analysis method based on matrix pencil and hybrid deep neural network

Acoustic signal analysis is an important component of transformer online monitoring. Currently, traditional methods have problems such as low spectral resolution, imbalanced sample distribution, and unsatisfactory classification performance. This article first introduces the matrix pencil algorithm for time-frequency spectrum analysis of acoustic signals, and then uses the SMOTE algorithm to expand the imbalanced samples. Then, an ACmix hybrid deep neural network model is constructed to classify 11 types of transformer operation and environmental acoustic signals. Finally, detailed experiments were conducted on the method proposed in this paper, and the experimental results showed that the matrix pencil algorithm has high time-frequency resolution and good noise resistance performance. The SMOTE sample expansion method can significantly improve the recognition accuracy by more than 2%. Overall accuracy of the proposed method in acoustic signal classification tasks reaches 91.81%.

A Battery Capacity Estimation Framework Combining Hybrid Deep Neural Network and Regional Capacity Calculation Based on Real-World Operating Data

Efficient battery capacity estimation is of utmost importance for safe and reliable operations of electric vehicles (EVs). This paper proposes a battery capacity estimation framework based on real-world EV operating data collected from forty electric buses of the same model operating in two cities. First, a reference capacity calculation method is presented by combining the Coulomb counting method with the incremental capacity analysis method. Then the impacts of temperature, current, and State-of-Charge on battery degradation are quantitatively analyzed. Using the historical probability distributions as battery health features, a hybrid deep neural network model that combines a convolutional neural network with a fully-connected neural network is proposed for battery capacity estimation. The validation results show that the proposed model outperforms the state-of-the-art methods and reaches a mean absolute percentage error of 2.79 <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$\%$</tex-math></inline-formula> , while maintaining low computational cost.

Application of hybrid improved temporal convolution network model in time series prediction of river water quality

Time series prediction of river water quality is an important method to grasp the changes of river water quality and protect the river water environment. However, due to the time series data of river water quality have strong periodicity, seasonality and nonlinearity, which seriously affects the accuracy of river water quality prediction. In this paper, a new hybrid deep neural network model is proposed for river water quality prediction, which is integrated with Savitaky-Golay (SG) filter, STL time series decomposition method, Self-attention mechanism, and Temporal Convolutional Network (TCN). The SG filter can effectively remove the noise in the time series data of river water quality, and the STL technology can decompose the time series data into trend, seasonal and residual series. The decomposed trend series and residual series are input into the model combining the Self-attention mechanism and TCN respectively for training and prediction. In order to verify the proposed model, this study uses opensource water quality data and private water quality data to conduct experiments, and compares with other water quality prediction models. The experimental results show that our method achieves the best prediction results in the water quality data of two different rivers.

Free nitrous acid prediction in ANAMMOX process using hybrid deep neural network model

Free nitrous acid (FNA) is a critical metric for stabilization of ANAMMOX but can not be directly and immediately measured by sensors or chemical measurement method, which hinders the effective management and operation for ANAMMOX. This study focuses on FNA prediction using hybrid model based on temporal convolutional network (TCN) combined with attention mechanism (AM) optimized by multiobjective tree-structured parzen estimator (MOTPE), called MOTPE-TCNA. A case study in an ANAMMOX reactor is carried out. Results show that nitrogen removal rate (NRR) is highly correlated with FNA concentration, indicating that it can forecast the operational status by predicting FNA. Then, MOTPE successfully optimizes the hyperparameters of TCN, helping TCN achieve a high prediction accuracy, and AM furtherly improves model accuracy. MOTPE-TCNA obtains the highest prediction accuracy, whose R2 value gets 0.992, increasing 1.71–11.80% compared to other models. As a deep neural network model, MOTPE-TCNA has more advantages than traditional machine learning methods in FNA prediction, which is beneficial to maintain the stable operation and easy control for ANAMMOX process.

Deep learning-based prediction of steady surface settlement due to shield tunnelling

Predicting ground movement produced by shield tunnelling in densely built urban areas is of practical significance. This study introduces an artificial intelligence method to predict the steady surface settlement caused by shield tunnelling. The proposed method comprises three models: a hybrid deep neural network model (HDNN), an error correction model (ECM), and an auxiliary model. Based on the settlement mechanism associated with tunnelling, the factors related to the ground movement were classified into five categories. The first four types of factors were used as the input parameters of the HDNN to predict steady surface settlement. The ECM considers unmeasurable factors. Five datasets from different subway lines were used to assess the performance and generalisation capacity of the proposed method. The results show that the proposed method outperforms classical deep neural networks as it considers the settlement mechanism and spatial and temporal characteristics of the input parameters.

Identification of thrombopoiesis inducer based on a hybrid deep neural network model

Thrombocytopenia is a common haematological problem worldwide. Currently, there are no relatively safe and effective agents for the treatment of thrombocytopenia. To address this challenge, we propose a computational method that enables the discovery of novel drug candidates with haematopoietic activities. Based on different types of molecular representations, three deep learning (DL) algorithms, namely recurrent neural networks (RNNs), deep neural networks (DNNs), and hybrid neural networks (RNNs+DNNs), were used to develop classification models to distinguish between active and inactive compounds. The evaluation results illustrated that the hybrid DL model exhibited the best prediction performance, with an accuracy of 97.8 % and Matthews correlation coefficient of 0.958 on the test dataset. Subsequently, we performed drug discovery screening based on the hybrid DL model and identified a compound from the FDA-approved drug library that was structurally divergent from conventional drugs and showed a potential therapeutic action against thrombocytopenia. The novel drug candidate wedelolactone significantly promoted megakaryocyte differentiation in vitro and increased platelet levels and megakaryocyte differentiation in irradiated mice with no systemic toxicity. Overall, our work demonstrates how artificial intelligence can be used to discover novel drugs against thrombocytopenia.

DeepFormer: a hybrid network based on convolutional neural network and flow-attention mechanism for identifying the function of DNA sequences.

Identifying the function of DNA sequences accurately is an essential and challenging task in the genomic field. Until now, deep learning has been widely used in the functional analysis of DNA sequences, including DeepSEA, DanQ, DeepATT and TBiNet. However, these methods have the problems of high computational complexity and not fully considering the distant interactions among chromatin features, thus affecting the prediction accuracy. In this work, we propose a hybrid deep neural network model, called DeepFormer, based on convolutional neural network (CNN) and flow-attention mechanism for DNA sequence function prediction. In DeepFormer, the CNN is used to capture the local features of DNA sequences as well as important motifs. Based on the conservation law of flow network, the flow-attention mechanism can capture more distal interactions among sequence features with linear time complexity. We compare DeepFormer with the above four kinds of classical methods using the commonly used dataset of 919 chromatin features of nearly 4.9 million noncoding DNA sequences. Experimental results show that DeepFormer significantly outperforms four kinds of methods, with an average recall rate at least 7.058% higher than other methods. Furthermore, we confirmed the effectiveness of DeepFormer in capturing functional variation using Alzheimer's disease, pathogenic mutations in alpha-thalassemia and modification in CCCTC-binding factor (CTCF) activity. We further predicted the maize chromatin accessibility of five tissues and validated the generalization of DeepFormer. The average recall rate of DeepFormer exceeds the classical methods by at least 1.54%, demonstrating strong robustness.

CNN-LSTM: A Novel Hybrid Deep Neural Network Model for Brain Tumor Classification

Current revelations in medical imaging have seen a slew of computer-aided diagnostic (CAD) tools for radiologists developed. Brain tumor classification is essential for radiologists to fully support and better interpret magnetic resonance imaging (MRI). In this work, we reported on new observations based on binary brain tumor categorization using HYBRID CNN-LSTM. Initially, the collected image is pre-processed and augmented using the following steps such as rotation, cropping, zooming, CLAHE (Contrast Limited Adaptive Histogram Equalization), and Random Rotation with panoramic stitching (RRPS). Then, a method called particle swarm optimization (PSO) is used to segment tumor regions in an MR image. After that, a hybrid CNN-LSTM classifier is applied to classify an image as a tumor or normal. In this proposed hybrid model, the CNN classifier is used for generating the feature map and the LSTM classifier is used for the classification process. The effectiveness of the proposed approach is analyzed based on the different metrics and outcomes compared to different methods.

Automated Hybrid Deep Neural Network Model for Fake News Identification and Classification in Social Networks

Automated Hybrid Deep Neural Network Model for Fake News Identification and Classification in Social Networks

Remaining Useful Life Prediction Based on CNN-BGRU-SA

Remaining Useful Life (RUL) prediction has very high importance for improving the safety and reliability of equipment, and its accurate estimation can provide technical support for fault Prognostics and Health Management (PHM). A hybrid deep neural network model has been proposed to improve the prognosis accuracy of equipment RUL. Convolution Neural Network (CNN) is used to extract the local features with Bidirectional Gated Recurrent Unit (BGRU) capturing the anterior and backward long-term dependence and then Self Attention (SA) assigning weights. Finally, Fully-Connection Network (FCN) is stacked to output the RUL prediction value, and its superiority is verified on C-MAPSS dataset. Experimental results show that CNN-BGRU-SA acquires better prognosis performance compared with a single network and the approach based on the BLCNN model, with high accuracy and generalization.

A hybrid deep neural network model for NOx emission prediction of heavy oil-fired boiler flames

Accurate NOx emission monitoring is essential for an in-depth understanding of the combustion state. However, establishing an accurate emission prediction model based on the traditional data-driven method is difficult, limited by low robustness and insufficient labeled data. To mitigate these limitations, this study proposes a hybrid deep neural network model for NOx emission prediction. In this hybrid model, an adversarial denoising autoencoder (ADAE) is established to extract flame deep features, and then the least support vector regression (LSSVR) is utilized to analyze the extracted features for predicting NOx emission. A novel training strategy that includes adversarial mechanisms, denoising coding and dropout is introduced to enhance the feature learning ability. The feasibility of the hybrid model ADAE-LSSVR is verified by the 4.2 MW heavy oil-fired boiler flame images, and the model hyper-parameters are optimized for higher prediction performance. Experimental results demonstrated that the ADAE can automatically extract robust features from raw flame images without manual intervention. Moreover, the proposed ADAE-LSSVR model provides satisfactory prediction accuracy with a correlation coefficient (R2) of 0.97, outperforming other state-of-art models.

The evaluation of automobile interior acceleration sound fused with physiological signal using a hybrid deep neural network

The evaluation of automobile sound quality is not only related to the inherent properties of the sound, but also to the psychological and physiological state of the evaluator, which makes the evaluation of sound quality become an interdisciplinary research. However, there are some deficiencies regarding the existing studies as follows: (1) it is imprecise enough to visualize the preference of evaluators for sound on the scale; (2) the evaluators without acoustic experience are prone to deviations in their evaluations, resulting in the universally and unapplicable results; (3) the automobile sound quality cannot be fully reflected only by physical and psychological parameters, and most of the existing evaluation models of sound quality are only characterized by shallow architectures. To alleviate the above flaws, a hybrid deep neural network (HDNN) is constructed to achieve the evaluation of automobile interior acceleration sound fused with physiological signals in this paper. An EEG test paradigm under the stimulation of automobile sound is meanwhile designed for a feasibility confirmation of the proposed method. In addition, the acquired EEG datum are respectively analyzed from two perspectives: the manual extraction of typical EEG features and the adaptive extraction of EEG features based on HDNN model. Furthermore, the performance of the proposed HDNN is also validated by comparing with conventional convolutional neural network (CNN) and long short time memory (LSTM). The results indicate that the adaptively extracted EEG features are superior to the manually extracted EEG features, and the proposed HDNN outperforms the CNN and LSTM in terms of accuracy, sensitivity as well as precision, and the optimal accuracy of HDNN is up to 88.1%. Thus, the constructed HDNN contributes to achieving accurate evaluation of automobile sound quality with EEG signal.

A hybrid deep learning approach with genetic and coral reefs metaheuristics for enhanced defect detection in software

Early detection and correction of software defects is essential in the software development process. In the production stage, software with defects negatively impacts on operational costs and ultimately affects customer satisfaction. Although different approaches exist to predict software defects, two essential factors are timely and accurate detection. This paper presents a hybrid Deep Neural Network model for enhanced prediction of software bugs . Different Nature-Inspired Algorithms have been applied to improve the exploration of the hyperparameter solution space to optimize the Deep Neural Network architecture. Experimental investigations have been conducted using NASA dataset to predict software defects and evaluation measures like accuracy, computational time and F1 score have been used for performance comparison. The approach based on the combination of Genetic and Coral Reef metaheuristics outperformed all other models, achieving highest accuracy of around 96% and average F1-score of 0.92.

Multi-objective optimization of CO2 emission and thermal efficiency for on-site steam methane reforming hydrogen production process using machine learning

Currently, hydrogen is produced primarily through steam methane reforming , a gray hydrogen production process that generates CO 2 as a by-product. Thus, it is crucial to optimize the process thermal efficiency with minimizing CO 2 generation in a hydrogen production process. This study focuses on the multi-objective optimization of low-carbon hydrogen production process, considering both process thermal efficiency maximization and CO 2 emission minimization. To this end, a hybrid deep neural network model is developed to increase the robustness of the multi-objective optimization. The developed hybrid deep neural network model is incorporated into a proposed multi-objective particle swarm optimization algorithm that performs Pareto dominance-based multi-objective optimization. In experiments conducted, Pareto-optimal solutions with thermal efficiency distribution between 77.5 and 87.0% and CO 2 emissions between 577.9 and 597.6 t/y were obtained. Furthermore, the Pareto-optimal front was analyzed to provide various representative solutions to assist decision-makers. The findings of this study can enable efficient and flexible process operations according to various requirements. • Low-carbon hydrogen production from an on-site steam methane reforming process. • Multi-objective optimization of thermal efficiency and CO 2 emission simultaneously. • Multi-objective particle swarm optimization for Pareto dominance-based optimization.

Volatility forecasting using deep neural network with time-series feature embedding

Volatility is usually a proxy indicator for market variation or tendency, containing essential information for investors and policy-makers. This paper proposes a novel hybrid deep neural network model (HDNN) with temporal embedding for volatility forecasting. The main idea of our HDNN is that it encodes one-dimensional time-series data as two-dimensional GAF images, which enables the follow-up convolution neural network (CNN) to learn volatility-related feature mappings automatically. Specifically, HDNN adopts an elegant end-to-end learning paradigm for volatility forecasting, which consists of feature embedding and regression components. The feature embedding component explores the volatility-related temporal information from GAF images via the elaborate CNN in an underlying temporal embedding space. Then, the regression component takes these embedding vectors as input for volatility forecasting tasks. Finally, we examine the feasibility of HDNN on four synthetic GBM datasets and five real-world Stock Index datasets in terms of five regression metrics. The results demonstrate that HDNN has better performance in most cases than the baseline forecasting models of GARCH, EGACH, SVR, and NN. It confirms that the volatility-related temporal features extracted by HDNN indeed improve the forecasting ability. Furthermore, the Friedman test verifies that HDNN is statistically superior to the compared forecasting models.

DeeProBot: a hybrid deep neural network model for social bot detection based on user profile data

Use of online social networks (OSNs) undoubtedly brings the world closer. OSNs like Twitter provide a space for expressing one’s opinions in a public platform. This great potential is misused by the creation of bot accounts, which spread fake news and manipulate opinions. Hence, distinguishing genuine human accounts from bot accounts has become a pressing issue for researchers. In this paper, we propose a framework based on deep learning to classify Twitter accounts as either ‘human’ or ‘bot.’ We use the information from user profile metadata of the Twitter account like description, follower count and tweet count. We name the framework ‘DeeProBot,’ which stands for Deep Profile-based Bot detection framework. The raw text from the description field of the Twitter account is also considered a feature for training the model by embedding the raw text using pre-trained Global Vectors (GLoVe) for word representation. Using only the user profile-based features considerably reduces the feature engineering overhead compared with that of user timeline-based features like user tweets and retweets. DeeProBot handles mixed types of features including numerical, binary, and text data, making the model hybrid. The network is designed with long short-term memory (LSTM) units and dense layers to accept and process the mixed input types. The proposed model is evaluated on a collection of publicly available labeled datasets. We have designed the model to make it generalizable across different datasets. The model is evaluated using two ways: testing on a hold-out set of the same dataset; and training with one dataset and testing with a different dataset. With these experiments, the proposed model achieved AUC as high as 0.97 with a selected set of features.

A Hybrid Deep Neural Network Model for Time Series Forecasting

Deep neural networks have proven to perform optimal forecasts even with the presence of noisyand non-linear nature of time series data. In thispaper, a hybrid deep neural network consisting of Convolutional Neural Networks (CNN) and Long Short Term Memory (LSTM) architecture have been proposed. The model combines the convolutional layer’s capability of feature extraction along with the LSTM’s feature of learning long term sequential dependencies. The experiments were performed on two datasets and compared with four other approaches: Recurrent Neural Network (RNN), LSTM, Gated Recurrent Unit (GRU) and Bidirectional LSTM. All five models are evaluated and compared with one step ahead forecasting. The proposed hybrid CNN-LSTM outperformed other modelsfor both datasets showing robustness against error.