Proposed Deep Neural Network Model Research Articles

Electric vehicle energy consumption prediction for unknown route types using deep neural networks by combining static and dynamic data

Accurate energy consumption prediction of electric vehicles (EVs) is crucial for drivers considering long trips. All the data should be provided beforehand to determine the energy consumption at the beginning of the trip. Although dynamic vehicle data (vehicle speed, state-of-charge, acceleration, etc.) cannot be known before the trip, factors related to the specified route (route type, elevation, average speed, weather, driving time, etc.) can be used to predict the consumed energy. These factors can be categorized as static and dynamic features, and thus, the question of how to effectively use static and dynamic data arises. This paper investigates the problem of predicting the energy consumption of an EV for a predetermined trip using a deep neural network (DNN) model that effectively uses static features along with dynamic segment features. Furthermore, we address the problem where the route types are unknown in advance. To include more information in the prediction model, we clustered the speed profiles using shape-based clustering with dynamic time warping (DTW) to predict the route type and used the cluster labels as static inputs. Real driving data collected from various drivers of a specific EV were used to train the DNN. The proposed DNN model was compared with the average energy consumption (AEC) model and five machine learning models. The results show that labels obtained from shape-based clustering improved the prediction more than feature-based cluster labels. The prediction errors were minimized with the proposed DNN model, where static features are introduced to the first and second layers twice.

A Neural Network Approach for Software Reliability Prediction

The increasing reliance on computer software has raised significant concerns regarding software reliability evaluation. Over the past four decades, various software reliability models have been developed, encompassing both parametric and nonparametric approaches. However, no single model has demonstrated effectiveness in handling all types of datasets. In response to this challenge, the deep neural network (DNN), a powerful deep learning model, has emerged as a promising solution. By leveraging the flexibility and adaptability of artificial neural networks (ANN), the DNN model exhibits remarkable prediction performance by capturing training variables and exploring deeper layers. This study presents an approach that utilizes a DNN model based on an ANN architecture with multiple activation functions. This approach aims to estimate software reliability and predict the frequency of software flaws. Through extensive experimental analysis and validation, the proposed DNN model surpasses the predictive accuracy achieved by existing parametric and nonparametric models. These results highlight the potential of the DNN model in effectively addressing the complexities involved in software reliability evaluation. By combining the power of deep learning with the adaptability of ANN, the proposed model offers a promising and accurate solution for software reliability prediction.

Development of a Machine Learning (ML)-Based Computational Model to Estimate the Engineering Properties of Portland Cement Concrete (PCC)

Portland cement concrete (PCC) is the construction material most used worldwide. Hence, its proper characterization is fundamental for the daily-basis engineering practice. Nonetheless, the experimental measurements of the PCC’s engineering properties (i.e., Poisson’s Ratio -v-, Elastic Modulus -E-, Compressive Strength -ComS-, and Tensile Strength -TenS-) consume considerable amounts of time and financial resources. Therefore, the development of high-precision indirect methods is fundamental. Accordingly, this research proposes a computational model based on deep neural networks (DNNs) to simultaneously predict the v, E, ComS, and TenS. For this purpose, the Long-Term Pavement Performance database was employed as the data source. In this regard, the mix design parameters of the PCC are adopted as input variables. The performance of the DNN model was evaluated with 1:1 lines, goodness-of-fit parameters, Shapley additive explanations assessments, and running time analysis. The results demonstrated that the proposed DNN model exhibited an exactitude higher than 99.8%, with forecasting errors close to zero (0). Consequently, the machine learning-based computational model designed in this investigation is a helpful tool for estimating the PCC’s engineering properties when laboratory tests are not attainable. Thus, the main novelty of this study is creating a robust model to determine the v, E, ComS, and TenS by solely considering the mix design parameters. Likewise, the central contribution to the state-of-the-art achieved by the present research effort is the public launch of the developed computational tool through an open-access GitHub repository, which can be utilized by engineers, designers, agencies, and other stakeholders.

Edge drop prediction of hot-rolled silicon steel based on a mechanism and data model

In this article, a high-precision edge drop prediction model for hot-rolled silicon steel based on a rolling mechanism and deep neural network (DNN) is proposed. Considering the initial roll shape, roll wear and roll thermal expansion, a mechanism model of the roll system is established according to the hot rolling 4-high finishing mill. In order to reduce the data dimension and effectively improve the prediction accuracy, the random forest algorithm is employed to analyse the feature parameters that affect the edge drop of hot-rolled silicon steel. Finally, the DNN is introduced to establish an accurate silicon steel edge drop prediction model based on mechanism data and rolling data. The experimental results demonstrated that the proposed DNN model has strong generalisation and reliability with a squared correlation coefficient ( R2) of 0.9656, and it is suitable for application in hot-rolled silicon steel lines for predicting edge drop. Furthermore, the proposed model has a certain reference value for the hot rolling process.

LM-DNN: pre-trained DNN with LSTM and cross Fold validation for detecting viral pneumonia from chest CT

ABSTRACT Some of the viruses may cause lung parenchyma and airway involvement. Usually, viral pneumonia causes ground-glass opacities, bilateral peripheral distribution, consolidation, vascular thickening, and reticular opacity. These features are common in COVID-19 rather than Non-Covid-19 viral pneumonia. However, in advanced cases, COVID-19 viral pneumonia may cause organising pneumonia and fibrosis of the lung. Atypical findings of Non-Covid-19 pneumonia have included central peripheral distribution, pleural effusion, lymphadenopathy, nodules, tree-in-bud opacities, and pneumothorax. Therefore, differentiating Non-Covid-19 pneumonia from COVID-19 pneumonia at chest computed tomography (CT) is necessary. In that case, CT scans of the thorax are one of the essential tools for early identification and future prognosis of viral pneumonia. We have proposed a Computer-Aided Diagnostic (CAD) system that can detect features of chest CT using a Deep Neural Network (DNN) with Long Short-Term Memory (LSTM). Transfer learning using pre-trained DNN models (ResNet50, VGG19, InceptionV3, Xception, DenseNet121, and VGG16) is applied to retain both high-level and low-level features effectively. The deep features are passed to the LSTM layer. The LSTM is utilised as a classifier and detects long short-term dependencies. The proposed method employs a hybrid DNN-LSTM network for automatic detection to take advantage of the uniqueness of the two models. The proposed models are trained with common and different features present in the chest CT of COVID-19 and Non-Covid-19 viral pneumonia. The 5-fold cross-validation (CV) method validated and tested the proposed model. The proposed DNN model’s performance is quite improved with LSTM and CV. As a result, the proposed LM-DNN (VGG16+LSTM+CV) model has achieved the classification test accuracy of 91.58% and specificity of 93.86%, which offers superior performance with state-of-the-art. Also, the DenseNet121+LSTM+CV model has reached the classification test accuracy of 90.1% and sensitivity of 92%.

A deep neural network model for paternity testing based on 15-loci STR for Iraqi families

Abstract Paternity testing using a deoxyribose nucleic acid (DNA) profile is an essential branch of forensic science, and DNA short tandem repeat (STR) is usually used for this purpose. Nowadays, in third-world countries, conventional kinship analysis techniques used in forensic investigations result in inadequate accuracy measurements, especially when dealing with large human STR datasets; they compare human profiles manually so that the number of samples is limited due to the required human efforts and time consumption. By utilizing automation made possible by AI, forensic investigations are conducted more efficiently, saving both time conception and cost. In this article, we propose a new algorithm for predicting paternity based on the 15-loci STR-DNA datasets using a deep neural network (DNN), where comparisons among many human profiles are held regardless of the limitation of the number of samples. For the purpose of paternity testing, familial data are artificially created based on the real data of individual Iraqi people from Al-Najaf province. Such action helps to overcome the shortage of Iraqi data due to restricted policies and the secrecy of familial datasets. About 53,530 datasets are used in the proposed DNN model for the purpose of training and testing. The Keras library based on Python is used to implement and test the proposed system, as well as the confusion matrix and receiver operating characteristic curve for system evaluation. The system shows excellent accuracy of 99.6% in paternity tests, which is the highest accuracy compared to the existing works. This system shows a good attempt at testing paternity based on a technique of artificial intelligence.

Achromatic and Resolution Enhancement Light Field Deep Neural Network for ZnO Nematic Liquid Crystal Microlens Array

Nematic liquid‐crystal microlens arrays (LC‐MLAs) often exhibit chromatic aberration and low resolution, severely compromising their optical imaging quality. This study proposes an achromatic and resolution enhancement light field (ARELF) deep neural network (DNN) to address these issues. The training set is constructed by incorporating LC‐MLA characteristics’ degradation, retrofitting the vimeo90k dataset. The network's hidden layer is trained to learn about chromatic aberration and low resolution of LC‐MLA while extracting imaging features and fusing the information of complementary features of a light field under varying voltages. The loss function includes both chromatic aberration and overall resolution. The light field images of ZnO LC‐MLA under seven consecutive voltages are used as input to test the proposed DNN model. After experimental verification, the proposed model effectively eliminates chromatic aberration while enhancing the spatial and temporal resolution of LC‐MLA. This novel network can be utilized to optimize the design process of LC‐MLA and significantly improve its imaging performance.

Towards a robust, effective and resource efficient machine learning technique for IoT security monitoring

The application of Deep Neural Networks (DNNs) for monitoring cyberattacks in Internet of Things (IoT) systems has gained significant attention in recent years. However, achieving optimal detection performance through DNN training has posed challenges due to computational intensity and vulnerability to adversarial samples. To address these issues, this paper introduces an optimization method that combines regularization and simulated micro-batching. This approach enables the training of DNNs in a robust, efficient, and resource-friendly manner for IoT security monitoring. Experimental results demonstrate that the proposed DNN model, including its performance in Federated Learning (FL) settings, exhibits improved attack detection and resistance to adversarial perturbations compared to benchmark baseline models and conventional Machine Learning (ML) methods typically employed in IoT security monitoring. Notably, the proposed method achieves significant reductions of 79.54% and 21.91% in memory and time usage, respectively, when compared to the benchmark baseline in simulated virtual worker environments. Moreover, in realistic testbed scenarios, the proposed method reduces memory footprint by 6.05% and execution time by 15.84%, while maintaining accuracy levels that are superior or comparable to state-of-the-art methods. These findings validate the feasibility and effectiveness of the proposed optimization method for enhancing the efficiency and robustness of DNN-based IoT security monitoring.

Utilizing deep neural network for web-based blood glucose level prediction system

Machine learning algorithms can be used to forecast future blood glucose (BG) levels for diabetes patients, according to recent studies. In this study, dataset from continuous glucose monitoring (CGM) system was used as the sole input for the machine learning models. To forecast blood glucose levels 15, 30, and 45 minutes in the future, we suggested deep neural network (DNN) and tested it on 7 patients with type 1 diabetes (T1D). The suggested prediction model was evaluated against a variety of machine learning models, such as k-nearest neighbor (KNN), support vector regression (SVR), decision tree (DT), adaptive boosting (AdaBoost), random forest (RF), and eXtreme gradient boosting (XGBoost). The experimental findings demonstrated that the proposed DNN model outperformed all other models, with average root mean square errors (RMSEs) of 17.295, 25.940, and 35.146 mg/dL over prediction horizons (PHs) of 15, 30, and 45 minutes, respectively. Additionally, we have included the suggested prediction model in web-based blood glucose level prediction tools. By using this web-based system, patients may readily acquire their future blood glucose levels, allowing for the generation of preventative alarms prior to crucial hypoglycemia or hyperglycemic situations.

EESD special issue: AI and data‐driven methods in earthquake engineering – (Part 1)

EESD special issue: AI and data‐driven methods in earthquake engineering – (Part 1)

ADU-Net: An Attention Dense U-Net based deep supervised DNN for automated lesion segmentation of COVID-19 from chest CT images

An automatic method for qualitative and quantitative evaluation of chest Computed Tomography (CT) images is essential for diagnosing COVID-19 patients. We aim to develop an automated COVID-19 prediction framework using deep learning.We put forth a novel Deep Neural Network (DNN) composed of an attention-based dense U-Net with deep supervision for COVID-19 lung lesion segmentation from chest CT images. We incorporate dense U-Net where convolution kernel size 5×5 is used instead of 3×3. The dense and transition blocks are introduced to implement a densely connected network on each encoder level. Also, the attention mechanism is applied between the encoder, skip connection, and decoder. These are used to keep both the high and low-level features efficiently. The deep supervision mechanism creates secondary segmentation maps from the features. Deep supervision combines secondary supervision maps from various resolution levels and produces a better final segmentation map. The trained artificial DNN model takes the test data at its input and generates a prediction output for COVID-19 lesion segmentation. The proposed model has been applied to the MedSeg COVID-19 chest CT segmentation dataset. Data pre-processing methods help the training process and improve performance.We compare the performance of the proposed DNN model with state-of-the-art models by computing the well-known metrics: dice coefficient, Jaccard coefficient, accuracy, specificity, sensitivity, and precision. As a result, the proposed model outperforms the state-of-the-art models.This new model may be considered an efficient automated screening system for COVID-19 diagnosis and can potentially improve patient health care and management system.

A New Framework for Regional Traffic Volumes Estimation with Large-Scale Connected Vehicle Data and Deep Learning Method

Connected vehicle (CV) data in this paper refer to the in-vehicle telematic data, including trajectories and driving events (e.g., hard braking) collected by vehicle manufacturers when vehicles are moving. Recently manufactured vehicles are equipped with cellular modems and Internet of Things (IoT) devices to collect vehicle data. Such data, after removing personal information, are being redistributed to third-party organizations. Compared to other probe vehicle data, the CV data has a higher penetration rate, ubiquitous coverage, and almost lane-level positioning accuracy. These features pave the road for novel transportation applications in transportation planning and traffic operations. In this paper, we represent a novel framework to estimate the regional link volumes based on the CV data and a deep neural network (DNN) model. The training data are generated according to the link volumes (targeted model output) and the corresponding CV counts (input features) at the same locations. The DNN model’s performance was compared with other estimation methods like linear regression and random forest and showed superior performance. The trained DNN model takes ubiquitous CV counts from other locations to estimate the corresponding link volumes. As a case study, the proposed DNN model was trained with a large training data set derived from CV data and time-dependent link counts collected at over 1,200 locations on freeways in the Dallas Fort Worth, Texas, area. The results reveal good accuracy and robustness.

Energy-optimal DNN model placement in UAV-enabled edge computing networks

Unmanned aerial vehicle (UAV)-enabled edge computing is emerging as a potential enabler for Artificial Intelligence of Things (AIoT) in the forthcoming sixth-generation (6G) communication networks. With the use of flexible UAVs, massive sensing data is gathered and processed promptly without considering geographical locations. Deep neural networks (DNNs) are becoming a driving force to extract valuable information from sensing data. However, the lightweight servers installed on UAVs are not able to meet the extremely high requirements of inference tasks due to the limited battery capacities of UAVs. In this work, we investigate a DNN model placement problem for AIoT applications, where the trained DNN models are selected and placed on UAVs to execute inference tasks locally. It is impractical to obtain future DNN model request profiles and system operation states in UAV-enabled edge computing. The Lyapunov optimization technique is leveraged for the proposed DNN model placement problem. Based on the observed system overview, an advanced online placement (AOP) algorithm is developed to solve the transformed problem in each time slot, which can reduce DNN model transmission delay and disk I/O energy cost simultaneously while keeping the input data queues stable. Finally, extensive simulations are provided to depict the effectiveness of the AOP algorithm. The numerical results demonstrate that the AOP algorithm can reduce 18.14% of the model placement cost and 29.89% of the input data queue backlog on average by comparing it with benchmark algorithms.

High Accuracy and Cost-Effective Fiber Optic Liquid Level Sensing System Based on Deep Neural Network.

In this paper, a novel liquid level sensing system is proposed to enhance the capacity of the sensing system, as well as reduce the cost and increase the sensing accuracy. The proposed sensing system can monitor the liquid level of several points at the same time in the sensing unit. Additionally, for cost efficiency, the proposed system employs only one sensor at each spot and all the sensors are multiplexed. In multiplexed systems, when changing the liquid level inside the container, the float position is changed and leads to an overlap or cross-talk between two sensors. To solve this overlap problem and to accurately predict the liquid level of each container, we proposed a deep neural network (DNN) approach to properly identify the water level. The performance of the proposed DNN model is evaluated via two different scenarios and the result proves that the proposed DNN model can accurately predict the liquid level of each point. Furthermore, when comparing the DNN model with the conventional machine learning schemes, including random forest (RF) and support vector machines (SVM), the DNN model exhibits the best performance.

Developing a Deep Neural Network model for COVID-19 diagnosis based on CT scan images.

COVID-19 is most commonly diagnosed using a testing kit but chest X-rays and computed tomography (CT) scan images have a potential role in COVID-19 diagnosis. Currently, CT diagnosis systems based on Artificial intelligence (AI) models have been used in some countries. Previous research studies used complex neural networks, which led to difficulty in network training and high computation rates. Hence, in this study, we developed the 6-layer Deep Neural Network (DNN) model for COVID-19 diagnosis based on CT scan images. The proposed DNN model is generated to improve accurate diagnostics for classifying sick and healthy persons. Also, other classification models, such as decision trees, random forests and standard neural networks, have been investigated. One of the main contributions of this study is the use of the global feature extractor operator for feature extraction from the images. Furthermore, the 10-fold cross-validation technique is utilized for partitioning the data into training, testing and validation. During the DNN training, the model is generated without dropping out of neurons in the layers. The experimental results of the lightweight DNN model demonstrated that this model has the best accuracy of 96.71% compared to the previous classification models for COVID-19 diagnosis.

Dynamic Super-Pixel Normalization for Robust Hyperspectral Image Classification

Deep neural networks (DNNs) have underpinned most of recent progress of hyperspectral image (HSI) classification. One premise of their success lies on the high image quality without noise corruption. However, due to the limitation of imaging sensor and imaging conditions, HSIs captured in practice inevitably suffer from random noise, which will degrade the generalization performance and robustness of most existing DNN based methods. In this study, we propose a dynamic super-pixel normalization based DNN for HSI classification, which can adaptively relieve the negative effect caused by various types of noise corruption and improve the generalization performance. To achieve this goal, we propose a dynamic super-pixel normalization module, for a given super-pixel which normalizes the inner pixel features using parameters dynamically generated based on themselves. By doing this, such a module enables adaptively restoring the similarity among pixels within the super-pixel corrupted by random noise through aligning their feature distribution, thus enhancing the generalization performance on noisy HSI. Moreover, it can be directly plugged into any other existing DNN architectures. To appropriately train the proposed DNN model, we further present a semi-supervised learning framework, which integrates the cross entropy loss and Kullback-Leibler divergence loss on labeled samples with the infomation entropy loss on the unlabeled samples for joint learning to well sidestep over-fitting. Experiments on three benchmark HSI classification datasets demonstrate the advantages of the proposed method over several state-of-the-art competitors in handling HSIs under different types of noise corruption.

An Improved Deep Structure for Accurately Brain Tumor Recognition

Brain neoplasms are recognized with a biopsy, which is not commonly done before decisive brain surgery. By using Convolutional Neural Networks (CNNs) and textural features, the process of diagnosing brain tumors by radiologists would be a noninvasive procedure. This paper proposes a features fusion model that can distinguish between no tumor and brain tumor types via a novel deep learning structure. The proposed model extracts Gray Level Co-occurrence Matrix (GLCM) textural features from MRI brain tumor images. Moreover, a deep neural network (DNN) model has been proposed to select the most salient features from the GLCM. Moreover, it manipulates the extraction of the additional high levels of salient features from a proposed CNN model. Finally, a fusion process has been utilized between these two types of features to form the input layer of additional proposed DNN model which is responsible for the recognition process. Two common datasets have been applied and tested, Br35H and FigShare datasets. The first dataset contains binary labels, while the second one splits the brain tumor into four classes; glioma, meningioma, pituitary, and no cancer. Moreover, several performance metrics have been evaluated from both datasets, including, accuracy, sensitivity, specificity, F-score, and training time. Experimental results show that the proposed methodology has achieved superior performance compared with the current state of art studies. The proposed system has achieved about 98.22% accuracy value in the case of the Br35H dataset however, an accuracy of 98.01% has been achieved in the case of the FigShare dataset.

Towards a deep learning-based outlier detection approach in the context of streaming data

Uncommon observations that significantly vary from the norm are referred to as outliers. Outlier detection, which aims to detect unexpected behavior, is a critical topic that has attracted significant attention in a wide range of research areas and application domains, including video surveillance, network intrusion detection, disease outbreak detection, and others. Deep learning-based techniques for outlier detection have currently outperformed machine learning and shallow approaches on streaming data, which are big and complicated datasets. Despite the fact that deep learning has been successfully applied in a variety of application domains, developing an effective and appropriate model is a difficult task due to the dynamic nature and variations of real-world applications and data. Hence, this research proposes a novel deep learning model based on a deep neural network (DNN) to handle the outlier detection problem in the context of streaming data. The proposed DNN model is developed with multiple hidden layers to improve feature abstraction and capabilities. Extensive experiments performed on four real-world outlier benchmark datasets, available at the UCI repository, and comparisons to state-of-the-art approaches are used to evaluate the proposed model's performance. Experiment results demonstrate that it outperforms both machine learning algorithms and deep learning competitors, resulting in significant performance gains. Particularly, when compared to other algorithms, the evaluation results clearly demonstrated the efficacy of the proposed approach, with much higher accuracy, recall and f1-score rates of 99.63%, 99.014% and 99.437%, respectively.

Cognitive level classification on information communication technology skills for blog

<span lang="EN-US">Learners can study and update their knowledge continually due to the rapid growth of online content. The Medium blog is a well-known open platform that encourages authors who want to share their experiences to publish content on various topics in multiple languages. Meanwhile, readers can query interesting content by searching for a related topic. However, finding suitable content is still challenging for learners, especially information communication technology (ICT) content in Thai, and needs to be classified into beginner, intermediate, and advanced cognitive levels. Moreover, ICT blog content is usually a mix of Thai language and technical terms in English. To overcome the challenge of content classification, a deep neural network (DNN) classification model was constructed to classify the ICT content from the Medium blog into three levels based on cognition. We examined and compared the classification results with strong baseline models, including logistic regression, multinomial naïve bayes, support vector machine (SVM), and multilayer perceptron (MLP). The experimental results indicate that the proposed DNN model attained the highest accuracy (0.878), precision (0.882), recall (0.878), and F1-score (0.875). </span>

MR-self Noise2Noise: self-supervised deep learning-based image quality improvement of submillimeter resolution 3D MR images.

The study aimed to develop a deep neural network (DNN)-based noise reduction and image quality improvement by only using routine clinical scans and evaluate its performance in 3D high-resolution MRI. This retrospective study included T1-weighted magnetization-prepared rapid gradient-echo (MP-RAGE) images from 185 clinical scans: 135 for DNN training, 11 for DNN validation, 20 for qualitative evaluation, and 19 for quantitative evaluation. Additionally, 18 vessel wall imaging (VWI) data were included to evaluate generalization. In each scan of the DNN training set, two noise-independent images were generated from the k-space data, resulting in an input-label pair. 2.5D U-net architecture was utilized for the DNN model. Qualitative evaluation between conventional MP-RAGE and DNN-based MP-RAGE was performed by two radiologists in image quality, fine structure delineation, and lesion conspicuity. Quantitative evaluation was performed with full sampled data as a reference by measuring quantitative error metrics and volumetry at 7 different simulated noise levels. DNN application on VWI was evaluated by two radiologists in image quality. Our DNN-based MP-RAGE outperformed conventional MP-RAGE in all image quality parameters (average scores = 3.7 vs. 4.9, p < 0.001). In the quantitative evaluation, DNN showed better error metrics (p < 0.001) and comparable (p > 0.09) or better (p < 0.02) volumetry results than conventional MP-RAGE. DNN application to VWI also revealed improved image quality (3.5 vs. 4.6, p < 0.001). The proposed DNN model successfully denoises 3D MR image and improves its image quality by using routine clinical scans only. • Our deep learning framework successfully improved conventional 3D high-resolution MRI in all image quality parameters, fine structure delineation, and lesion conspicuity. • Compared to conventional MRI, the proposed deep neural network-based MRI revealed better quantitative error metrics and comparable or better volumetry results. • Deep neural network application to 3D MRI whose pulse sequences and parameters were different from the training data showed improvement in image quality, revealing the potential to generalize on various clinical MRI.