Deep Neural Network Research Articles

Electric vehicle energy consumption prediction for unknown route types using deep neural networks by combining static and dynamic data

Accurate energy consumption prediction of electric vehicles (EVs) is crucial for drivers considering long trips. All the data should be provided beforehand to determine the energy consumption at the beginning of the trip. Although dynamic vehicle data (vehicle speed, state-of-charge, acceleration, etc.) cannot be known before the trip, factors related to the specified route (route type, elevation, average speed, weather, driving time, etc.) can be used to predict the consumed energy. These factors can be categorized as static and dynamic features, and thus, the question of how to effectively use static and dynamic data arises. This paper investigates the problem of predicting the energy consumption of an EV for a predetermined trip using a deep neural network (DNN) model that effectively uses static features along with dynamic segment features. Furthermore, we address the problem where the route types are unknown in advance. To include more information in the prediction model, we clustered the speed profiles using shape-based clustering with dynamic time warping (DTW) to predict the route type and used the cluster labels as static inputs. Real driving data collected from various drivers of a specific EV were used to train the DNN. The proposed DNN model was compared with the average energy consumption (AEC) model and five machine learning models. The results show that labels obtained from shape-based clustering improved the prediction more than feature-based cluster labels. The prediction errors were minimized with the proposed DNN model, where static features are introduced to the first and second layers twice.

Deep Learning-Assisted Compound Bioactivity Estimation Framework

Drug Discovery is a highly complicated process. On average, it takes six to twelve years to manufacture a new drug and have the product released in the market. It is of utmost importance to find methods that would accelerate the manufacturing process. This significant challenge in drug development can be addressed using deep learning techniques. The aim of this paper is to propose a deep learning-based framework that can help chemists examine compound biological activity in a more accurate manner. The proposed framework employs autoencoder for data representation of the compounds data, which is then classified using deep neural network followed by building a customized deep regression model to estimate an accurate value of the compound bioactivity. The proposed framework achieved an accuracy of 89% in autoencoder reconstruction error, 79.01% in classification, and MAE of 2.4 while predicting compound bioactivity using deep regression model.

PPDNN-CRP: privacy-preserving deep neural network processing for credit risk prediction in cloud: a homomorphic encryption-based approach

This study proposes a Privacy-Preserving Deep Neural Network for Credit Risk Prediction (PPDNN-CRP) framework that leverages homomorphic encryption (HE) to ensure data privacy throughout the credit risk prediction process. The PPDNN-CRP framework employs the Paillier homomorphic encryption scheme to secure sensitive loan application data during both the training and inference phases. Implemented using TensorFlow for deep neural network operations and TenSEAL for HE, the framework uses the Kaggle loan dataset to evaluate its performance. The results show that PPDNN-CRP achieved an accuracy of 80.48%, demonstrating competitive performance compared to Privacy-Preserving Logistic Regression (PPLR) at 77.23% and a slight decrease from the non-private DNN-CRP model at 86.18%. The model exhibited strong metrics with a precision of 0.84, recall of 0.91, F1-score of 0.87, and an AUC of 0.74. Security analysis confirms that PPDNN-CRP effectively defends against various privacy attacks, including poisoning, evasion, membership inference, model inversion, and model extraction, through robust encryption techniques and privacy measures. This framework offers a promising approach for achieving high-quality credit risk prediction while maintaining data privacy and complying with legal and ethical standards.

Deep neural networks in low energy algorithms for wireless sensor networks

Deep neural networks in low energy algorithms for wireless sensor networks

Enhancing local-scale groundwater quality predictions using advanced machine learning approaches

Assessing groundwater quality typically involves labor-intensive, time-consuming, and costly laboratory tests, making real-time monitoring impractical, especially at the local level. Groundwater quality projections at the local scale using broad spatial datasets have been inaccurate due to variations in hydrogeology, human activities, industrial operations, groundwater extraction, and waste disposal. This study aims to identify the most dependable and resilient machine learning algorithms for forecasting groundwater quality at nearby monitoring locations by utilizing simple water quality metrics that can be quickly assessed without extensive sampling and laboratory testing. The Entropy-weighted Water Quality Index (EWQI) was calculated using a large spatial and temporal dataset (2014–2021) of 977 wells with parameters including pH, total hardness (TH), calcium (Ca2⁺), magnesium (Mg2⁺), sodium (Na⁺), potassium (K⁺), sulfate (SO₄2⁻), chloride (Cl⁻), nitrate (NO₃⁻), total dissolved solids (TDS), and fluoride (F⁻). Further, similar parameters were also observed in 33 open wells at the three local monitoring sites from December 2022 to March 2023. The EWQI was predicted using a Random Forest (RF), eXtreme Gradient Boosting (XGB), and Deep Neural Network (DNN). The pH, TH, and TDS were used as input variables for EWQI predictions, as they can be easily measured using handheld probes or multi-parameters. The model performance was evaluated using R2, MAE, and RMSE. During the training stage, all three models predicted the EWQI with an R2 greater than 90%, with minimal errors when pH, TH, and TDS were input variables. To validate the models at a local scale, the EWQI was predicted at the village level (e.g., Antoli, Balapura, and Lapodiaya) using pH, TH, and TDS as input variables. The machine learning models were able to predict the EWQI very closely to the actual EWQI, with an R2 greater than 90%. It is also evident that the models could predict the EWQI using basic parameters that are easily measured, providing an overall idea of the water quality for a small area. Hence, these machine learning models could be useful for accurately representing groundwater quality, thereby avoiding the use of time-consuming and costly laboratory techniques.

Classification of Rice Grains by Image Processing

General Background: The assessment of the surface quality of pre-treated rice grains is crucial for determining their market acceptability, storage stability, processing quality, and overall consumer satisfaction. Traditional evaluation methods are often time-consuming and yield subjective classifications. Specific Background: The lack of a comprehensive tagged image dataset hinders the application of advanced convolutional neural networks (CNN) for detailed damage classification of healthy rice grains. Knowledge Gap: Existing datasets and methods limit the effective exploration of sophisticated CNN models for categorizing rice types, particularly in identifying subtle damage characteristics. Aims: The study aims to create a robust rice grain classification system using image processing techniques, primarily deep learning algorithms, to improve the classification of rice varieties. Results: Utilizing a dataset of 75,000 images across five widely cultivated rice varieties in Turkey, we achieved classification accuracies of 100% for CNN, 99.95% for deep neural networks (DNN), and 99.87% for artificial neural networks (ANN). Novelty: The proposed approach represents a significant advancement in rice classification technology, employing a combination of image acquisition, feature extraction, and machine learning to streamline the process, effectively addressing the challenges faced in traditional methods. Implications: The findings underscore the potential for improved sorting and grading efficiency in the rice industry, facilitating better market outcomes and consumer satisfaction through enhanced quality control Highlights: Innovative: Uses CNN for accurate rice variety classification. Data-Driven: Analyzes 75,000 images for enhanced quality evaluation. Impactful: Increases efficiency in sorting and grading processes. Keywords: Rice Classification, Image Processing, CNN, Machine Learning, Agricultural Technology

Navigating Technological Risks: An Uncertainty Analysis of Powertrain Technology in Hybrid-Electric Commuter Aircraft

Abstract This study addresses the uncertainties in hybrid-electric powertrain technology for a 19-passenger commuter aircraft, focusing on two future Entry-Into-Service timeframes: 2030 and 2040. The methodology is split into a preliminary optimization of aircraft design based on nominal technology scenarios followed by Monte Carlo simulations to investigate the impact of diverse technology projections and distribution types. Advanced surrogate modeling techniques, leveraging deep neural networks (DNN) trained on a dataset from an aircraft design framework, are employed. Key outcomes from this work reveal a marked increase in computational efficiency, with a speed-up factor of approximately 500 times when utilizing surrogate models. The results indicate that the 2040 entry-into-service (EIS) scenario could achieve larger reductions in fuel and total energy consumption—20.4% and 15.8% respectively—relative to the 2030 scenario, but with higher uncertainty. Across all scenarios examined, the hybrid-electric model showcased superior performance compared to its conventional counterpart. The battery-specific energy density is proved to be a critical parameter of the aircraft’s performance across both timeframes. The findings emphasize the importance of continuous innovation in battery and motor technologies to target toward greater system-level efficiency and reduced environmental impact.

An Efficient Ensemble Approach for Brain Tumors Classification Using Magnetic Resonance Imaging

Tumors in the brain can be life-threatening, making early and precise detection crucial for effective treatment and improved patient outcomes. Deep learning (DL) techniques have shown significant potential in automating the early diagnosis of brain tumors by analyzing magnetic resonance imaging (MRI), offering a more efficient and accurate approach to classification. Deep convolutional neural networks (DCNNs), which are a sub-field of DL, have the potential to analyze rapidly and accurately MRI data and, as such, assist human radiologists, facilitating quicker diagnoses and earlier treatment initiation. This study presents an ensemble of three high-performing DCNN models, i.e., DenseNet169, EfficientNetB0, and ResNet50, for accurate classification of brain tumors and non-tumor MRI samples. Our proposed ensemble model demonstrates significant improvements over various evaluation parameters compared to individual state-of-the-art (SOTA) DCNN models. We implemented ten SOTA DCNN models, i.e., EfficientNetB0, ResNet50, DenseNet169, DenseNet121, SqueezeNet, ResNet34, ResNet18, VGG16, VGG19, and LeNet5, and provided a detailed performance comparison. We evaluated these models using two learning rates (LRs) of 0.001 and 0.0001 and two batch sizes (BSs) of 64 and 128 and identified the optimal hyperparameters for each model. Our findings indicate that the ensemble approach outperforms individual models, having 92% accuracy, 90% precision, 92% recall, and an F1 score of 91% at a 64 BS and 0.0001 LR. This study not only highlights the superior performance of the ensemble technique but also offers a comprehensive comparison with the latest research.

HEFS-MLDR: A novel hybrid ensemble feature selection framework for improved deep neural network architecture in the diagnosis of Parkinson’s disease

HEFS-MLDR: A novel hybrid ensemble feature selection framework for improved deep neural network architecture in the diagnosis of Parkinson’s disease

How the technologies behind self-driving cars, social networks, ChatGPT, and DALL-E2 are changing structural biology.

The performance of deep Neural Networks (NNs) in the text (ChatGPT) and image (DALL-E2) domains has attracted worldwide attention. Convolutional NNs (CNNs), Large Language Models (LLMs), Denoising Diffusion Probabilistic Models (DDPMs)/Noise Conditional Score Networks (NCSNs), and Graph NNs (GNNs) have impacted computer vision, language editing and translation, automated conversation, image generation, and social network management. Proteins can be viewed as texts written with the alphabet of amino acids, as images, or as graphs of interacting residues. Each of these perspectives suggests the use of tools from a different area of deep learning for protein structural biology. Here, I review how CNNs, LLMs, DDPMs/NCSNs, and GNNs have led to major advances in protein structure prediction, inverse folding, protein design, and small molecule design. This review is primarily intended as a deep learning primer for practicing experimental structural biologists. However, extensive references to the deep learning literature should also make it relevant to readers who have a background in machine learning, physics or statistics, and an interest in protein structural biology.

Distilling Tiny and Ultrafast Deep Neural Networks for Autonomous Navigation on Nano-UAVs

Distilling Tiny and Ultrafast Deep Neural Networks for Autonomous Navigation on Nano-UAVs

Real-time, dense UAV mapping by leveraging monocular depth prediction with monocular-inertial SLAM

We present a dense and metric 3D mapping pipeline designed for embedded operation on-board UAVs, by loosely coupling deep neural networks trained to infer dense depth single images with a SLAM system that restores metric scale from sparse depth. In contrast to computationally restrictive approaches that leverage multiple views, we propose a highly efficient, single-view approach without sacrificing 3D mapping performance. This enables real-time construction of a global 3D voxel map by iterative fusion of the rescaled dense depth maps obtained via raycasting from the estimated camera poses. Quantitative and qualitative experimentations of our framework in challenging environmental conditions show comparable or superior performance with respect to state-of-the-art approaches via a better effectiveness-efficiency trade-off.

A novel effective key synchronization approach based on optimized deep neural networks for IoT-based low-power wide area networks

A novel effective key synchronization approach based on optimized deep neural networks for IoT-based low-power wide area networks

Enhancing Adversarial Robustness through Stable Adversarial Training

Deep neural network models are vulnerable to attacks from adversarial methods, such as gradient attacks. Evening small perturbations can cause significant differences in their predictions. Adversarial training (AT) aims to improve the model’s adversarial robustness against gradient attacks by generating adversarial samples and optimizing the adversarial training objective function of the model. Existing methods mainly focus on improving robust accuracy, balancing natural and robust accuracy and suppressing robust overfitting. They rarely consider the AT problem from the characteristics of deep neural networks themselves, such as the stability properties under certain conditions. From a mathematical perspective, deep neural networks with stable training processes may have a better ability to suppress overfitting, as their training process is smoother and avoids sudden drops in performance. We provide a proof of the existence of Ulam stability for deep neural networks. Ulam stability not only determines the existence of the solution for an operator inequality, but it also provides an error bound between the exact and approximate solutions. The feature subspace of a deep neural network with Ulam stability can be accurately characterized and constrained by a function with special properties and a controlled error boundary constant. This restricted feature subspace leads to a more stable training process. Based on these properties, we propose an adversarial training framework called Ulam stability adversarial training (US-AT). This framework can incorporate different Ulam stability conditions and benchmark AT models, optimize the construction of the optimal feature subspace, and consistently improve the model’s robustness and training stability. US-AT is simple and easy to use, and it can be easily integrated with existing multi-class AT models, such as GradAlign and TRADES. Experimental results show that US-AT methods can consistently improve the robust accuracy and training stability of benchmark models.

How can geostatistics help us understand deep learning? An exploratory study in SAR-based aircraft detection

Deep Neural Networks (DNNs) have garnered significant attention across various research domains due to their impressive performance, particularly Convolutional Neural Networks (CNNs), known for their exceptional accuracy in image processing tasks. However, the opaque nature of DNNs has raised concerns about their trustworthiness, as users often cannot understand how the model arrives at its predictions or decisions. This lack of transparency is particularly problematic in critical fields such as healthcare, finance, and law, where the stakes are high. Consequently, there has been a surge in the development of explanation methods for DNNs. Typically, the effectiveness of these methods is assessed subjectively via human observation on the heatmaps or attribution maps generated by eXplanation AI (XAI) methods. In this paper, a novel GeoStatistics Explainable Artificial Intelligence (GSEAI) framework is proposed, which integrates spatial pattern analysis from Geostatistics with XAI algorithms to assess and compare XAI understandability. Global and local Moran’s I indices, commonly used to assess the spatial autocorrelation of geographic data, assist in comprehending the spatial distribution patterns of attribution maps produced by the XAI method, through measuring the levels of aggregation or dispersion. Interpreting and analyzing attribution maps by Moran’s I scattergram and LISA clustering maps provide an accurate global objective quantitative assessment of the spatial distribution of feature attribution and achieves a more understandable local interpretation. In this paper, we conduct experiments on aircraft detection in SAR images based on the widely used YOLOv5 network, and evaluate four mainstream XAI methods quantitatively and qualitatively. By using GSEAI to perform explanation analysis of the given DNN, we could gain more insights about the behavior of the network, to enhance the trustworthiness of DNN applications. To the best of our knowledge, this is the first time XAI has been integrated with geostatistical algorithms in SAR domain knowledge, which expands the analytical approaches of XAI and also promotes the development of XAI within SAR image analytics.

Deep neural network-based molecular dynamics simulations for AlxGa1-xN alloys and their thermal properties

Efficient heat dissipation is crucial for the performance and lifetime of high electron mobility transistors (HEMTs). The thermal conductivity of materials and interfacial thermal conductance (ITC) play significant roles in their heat dissipation. To predict the thermal properties of AlxGa1-xN and the ITC of GaN/AlxGa1-xN in HEMTs, a dataset with first-principles accuracy was constructed using concurrent learning method and trained to obtain an interatomic potential employing deep neural networks (DNN) method. Using obtained DNN interatomic potential, equilibrium molecular dynamics simulations were employed to calculate the thermal conductivity of AlxGa1-xN, which showed excellent consistent with experimental results. Additionally, the phonon density of states of AlxGa1-xN and the ITC of GaN/AlxGa1-xN were calculated. Our study revealed a decrease in the ITC of GaN/AlxGa1-xN with increasing x, and the insertion of 1nm-thick AlN at the interface significantly reduced the ITC. This work provided a high-fidelity DNN potential for molecular dynamics simulations of AlxGa1-xN, offering valuable guidance for exploring the thermal transport of complex alloy and heterostructure.&#xD.

A deep autoencoder for electric double layer capacitance prediction in electrochemical sensors

This study explores the application of a deep autoencoder neural network to accurately predict the electric double layer capacitance from real-world parameters in binary, asymmetric electrolytes under low concentration conditions. By utilizing a modest simulation-based dataset of just 250 samples, the deep autoencoder neural network model developed in this study effectively predicted the capacitance by learning the critical features and relationships of the electric double layer model and encoding this learned representation into a low-dimensional latent space. From the latent variables, the decoder block of the neural network learned to effectively recreate the high-dimensional input. To enhance the model's robustness, prevent overfitting, and better simulate real-world conditions, noise was incorporated into the training and test data. The model demonstrated strong performance across various conditions, such as ionic size, ionic charge, and surface potential, yielding satisfactory results on both clean and noisy test datasets. A key feature of this approach was the mapping of real-world electric double layer parameters to the latent variables of the model, allowing for direct input of physical parameters to predict the electric double layer capacitance. This research highlights the potential of machine learning techniques to expedite the design and analysis of complex multi-physics systems such as electrochemical sensors by reducing the dependence on extensive domain expertise throughout the design process.

Multiple Criteria-Based Intelligent Techniques for Efficient Handover in Next-Generation Networks

Background: Several wireless technologies are assumed to be operating in cooperation for next-generation networks. These networks offer various services to mobile users; however, efficient handover between different networks is the most challenging task in high mobility scenarios. The traditional signal strength-based handover algorithms are not able to cope with mobile users' high-quality requirements. Methods: In this paper, multiple criteria-based intelligent techniques are proposed to deal with inefficiencies related to handover. This technique makes use of artificial neural networks that take multiple parameters as inputs in order to predict the degradation of parameters. These predictions are further used to design rules for initiating handover procedures prior to service quality degradation. Results: Based on the prediction results obtained by deep neural networks, the handover decisions are recommended according to the type of application: conversational or streaming. Conclusion: The simulation results demonstrate the efficacy of the proposed method as there is an improvement of up to 40% and 25% in terms of handover rate and service disruption time, respectively, with an acceptable prediction error of 0.05.

Enhancing Heart Disease Prediction with Federated Learning and Blockchain Integration

Federated learning offers a framework for developing local models across institutions while safeguarding sensitive data. This paper introduces a novel approach for heart disease prediction using the TabNet model, which combines the strengths of tree-based models and deep neural networks. Our study utilizes the Comprehensive Heart Disease and UCI Heart Disease datasets, leveraging TabNet’s architecture to enhance data handling in federated environments. Horizontal federated learning was implemented using the federated averaging algorithm to securely aggregate model updates across participants. Blockchain technology was integrated to enhance transparency and accountability, with smart contracts automating governance. The experimental results demonstrate that TabNet achieved the highest balanced metrics score of 1.594 after 50 epochs, with an accuracy of 0.822 and an epsilon value of 6.855, effectively balancing privacy and performance. The model also demonstrated strong accuracy with only 10 iterations on aggregated data, highlighting the benefits of multi-source data integration. This work presents a scalable, privacy-preserving solution for heart disease prediction, combining TabNet and blockchain to address key healthcare challenges while ensuring data integrity.

Current Status and Challenges and Future Trends of Deep Learning-Based Intrusion Detection Models.

With the advancement of deep learning (DL) technology, DL-based intrusion detection models have emerged as a focal point of research within the domain of cybersecurity. This paper provides an overview of the datasets frequently utilized in the research. This article presents an overview of the widely utilized datasets in the research, establishing a basis for future investigation and analysis. The text subsequently summarizes the prevalent data preprocessing methods and feature engineering techniques utilized in intrusion detection. Following this, it provides a review of seven deep learning-based intrusion detection models, namely, deep autoencoders, deep belief networks, deep neural networks, convolutional neural networks, recurrent neural networks, generative adversarial networks, and transformers. Each model is examined from various dimensions, highlighting their unique architectures and applications within the context of cybersecurity. Furthermore, this paper broadens its scope to include intrusion detection techniques facilitated by the following two large-scale predictive models: the BERT series and the GPT series. These models, leveraging the power of transformers and attention mechanisms, have demonstrated remarkable capabilities in understanding and processing sequential data. In light of these findings, this paper concludes with a prospective outlook on future research directions. Four key areas have been identified for further research. By addressing these issues and advancing research in the aforementioned areas, this paper envisions a future in which DL-based intrusion detection systems are not only more accurate and efficient but also better aligned with the dynamic and evolving landscape of cybersecurity threats.