DNN Model Research Articles

Determination of heat transfer characteristics of tube bundles over heating plate by machine learning

In this study machine learning is used with the aid of a deep neural network algorithm to predict convective heat transfer characteristics for inline and staggered tube bundles, and correlation equations for Nusselt number and friction factor are derived. Machine learning algorithm's data were obtained from experimental work for various transverse pitch of the tube bundles, longitudinal pitch of the tube bundles and Reynolds number. 276 experimental data points were taken for both inline and staggered tube bundles. However, considering that the data obtained from the experimental study may be insufficient for training, a two-step data augmentation method and retraining with cross-validation was used to prevent data deficiency in the deep neural network structure. Thus, the unseen data in the experimental work were also predicted. The coefficient of determination for the DNN model predictions was obtained greater than 0.96. One correlation equation for Nusselt Number and three correlation equations for friction factor were proposed from the augmented data with machine learning. The R2 values of the correlation equations varied between 89 % and 99 %. As a result, machine learning methods successfully applied to predict the Nusselt number and friction factor of tube bundles consistent with the experimental data.

Detection and classification on MRI images of brain tumor using YOLO NAS deep learning model

If a brain tumor is not properly diagnosed, it might result in fatal consequences and major health issues. As a result, a key component of diagnosis is the early identification of brain tumors and the precise categorization of brain tumor types. This research work focusses on detection and classification such as pituitary, meningioma, glioma, and non-tumorous tumors from brain tumor MRI image using YOLO NAS model. Advances in deep learning have led to an increasing awareness of computer-aided diagnosis technology. To improve classification performance, the input data set is acquired from the REMBRANDT repository using the Digital Database of Brain Tumor Magnetic Resonance Images. The primary phases of this task include segmentation, classification, and pre-processing. The researcher preprocessed the RGB image to exclude any undesired spots before locating the ROI. For each brain tumor image, we used the hybrid anisotropic diffusion filtering (HADF) technique to remove noise. The Encoder-Decoder Network (En–DeNet), which uses a deep neural network based on U-Net as the encoder and a pre-trained EfficientNet as the decoder, is then used to segment the MRI images. A proposed model was developed using a deep learning-based YOLO NAS technique to identify brain cancers from MRI images such as meningioma tumors, non-tumor, glioma tumors, and pituitary tumors. The suggested model's classification performance is weighed based on parameters such as precision, F1-score, sensitivity, accuracy, and specificity. The proposed fusion method YOLO NAS has improved classification results of accuracy (AC = 0.997%), specificity (SP = 0.985%), precision (PR = 0.982%), F1-score (F1 = 0.992%), and sensitivity (SE = 0.985%) using 2570 MRI data for training and 630 data for testing and validation of brain tumor cases. The results show that the brain tumor type classification system based on the proposed YOLO NAS technique works better than the DNN, PDCNN, DenseNet-161, and DCNN-SGD model classifiers.

A combination learning framework to uncover cyber attacks in IoT networks

The Internet of Things (IoT) is rapidly expanding, connecting an increasing number of devices daily. Having diverse and extensive networking and resource-constrained devices creates vulnerabilities to various cyber-attacks. The IoT with the supervision of Software Defined Network (SDN) enhances the network performance through its flexibility and adaptability. Different methods have been employed for detecting security attacks; however, they are often computationally efficient and unsuitable for such resource-constraint environments. Consequently, there is a significant requirement to develop efficient security measures against a range of attacks. Recent advancements in deep learning (DL) models have paved the way for designing effective attack detection methods. In this study, we leverage Genetic Algorithm (GA) with a correlation coefficient as a fitness function for feature selection. Additionally, mutual information (MI) is applied for feature ranking to measure their dependency on the target variable. The selected optimal features were used to train a hybrid DNN model to uncover attacks in IoT networks. The hybrid DNN integrates Convolutional Neural Network, Bi-Gated Recurrent Units (Bi-GRU), and Bidirectional Long Short-Term Memory (Bi-LSTM) for training the input data. The performance of our proposed model is evaluated against several other baseline DL models, and an ablation study is provided. Three key datasets InSDN, UNSW-NB15, and CICIoT 2023 datasets, containing various types of attacks, were used to assess the performance of the model. The proposed model demonstrates an impressive accuracy and detection time over the existing model with lower resource consumption.

Prediction and minimization of blasting flyrock distance, using deep neural networks and gravitational search algorithm, JAYA, and multi-verse optimization algorithms

Flyrock represents a significant and fundamental challenge in surface mine blasting, carrying inherent risks to humans and the environment. Consequently, accurate prediction, minimization, and identification of the factors influencing flyrock distance are imperative for effective control and mitigation of its destructive consequences. Machine learning and artificial intelligence methodologies have emerged as viable means to predict and simulate in different scientific fields. This study employs Deep Neural Network in conjunction with three optimization algorithms including the JAYA Algorithm, Multi-Verse Optimization Algorithm, and Gravitational Search Algorithm to predict blasting flyrock distance. The developed model consists of a combination of seven input parameters, encompassing both blasting design parameters and rock geomechanical properties. The output of the Deep Neural Networks model is the flyrock distance. For the training and testing of the model, a dataset comprising of 245 blasting records, collected from Songun copper mine, Iran, was utilized. The DNN model yielded an R2 value of 0.96 and an MSE value of 34.11. These results demonstrate the high accuracy and predictive capability of the model. Furthermore, the application of three optimization algorithms resulted in similar optimized parameter values, which minimized flyrock distances.

LSSMSD: defending against black-box DNN model stealing based on localized stochastic sensitivity

LSSMSD: defending against black-box DNN model stealing based on localized stochastic sensitivity

Testing the Possible Origins of Category Selectivity in the Brain with DNN Models

Testing the Possible Origins of Category Selectivity in the Brain with DNN Models

A novel algorithm combined X-ray fluorescence and Neural Network (XRF-NN) for coal ash content prediction: Algorithm design and performance evaluation

This paper investigated a precise algorithm combining X-ray Fluorescence and Neural Network (XRF-NN) for predicting ash content. The 261 sets of XRF tests show that the 34 elements in the chosen Guqiao coal can be categorized as major, secondary and tiny elements, whose cumulative ratio were ~95%, 4-5% and <1%, respectively. Referring to the machine learning theory, the construction strategy of Element-Ash dataset was determined viz. value determination → standardization → division → optimization of generalization ability. Then, the hyperparameters optimization displays that XRF-NN model with 34 inputs and 1 output was suitable to predict coal ash content, where the activation function, loss function, and optimizer were ReLU, MAE, and Momentum, respectively. After iterative training, the new XRF-NN model provides the precise prediction of coal ash content with the absolute errors between -2.0% and 2%. Moreover, the prediction accuracy rose from 57.69% to 100%, as the expected relative error increased from 1% to 5%. Furthermore, the comparisons between different prediction methods reveal that the minimum MRE of 1.22% can be obtained by XRF-NN with total elements, which was only half of those given by the conventional Multiple Linear Regression and Partial Least Squares Regression. Besides, XRF-NN model presents the root mean squared error 0.797%, the mean absolute error 0.625% and the coefficient of determination 0.999, which were significantly superior to those calculated by Dual-Energy Gamma-ray Transmission, Ploy, RFR, XGBoost and DNN model. The results of this study suggest the excellent performance of the new XRF-NN model in predicting ash content.

Static Scheduling of Weight Programming for DNN Acceleration with Resource Constrained PIM

Most existing architectural studies on ReRAM-based processing-in-memory (PIM) DNN accelerators assume that all weights of the DNN can be mapped to the crossbar at once. However, these studies are over-idealized. ReRAM crossbar resources for calculation are limited because of technological limitations, so multiple weight mapping procedures are required during the inference process. In this article, we propose a static scheduling framework which generates the mapping between DNN weights and ReRAM cells with minimum runtime weight programming cost. We first build a ReRAM crossbar programming latency model by simultaneously considering the DNN weight patterns, ReRAM programming operations, and PIM architecture characteristics. Then, the model is used in the searching process to obtain an optimized weight-to-OU mapping table with minimum online programming latency. Finally, an OU scheduler is used to coordinate the activation sequences of OUs in the crossbars to perform the inference computation correctly. Evaluation results show the proposed framework significantly reduces the weight programming overhead and the overall inference latency for various DNN models with different input datasets.

CHRONIC KIDNEY DISEASE STAGE IDENTIFICATION IN HIV INFECTED PATIENTS USING MACHINE LEARNING

One of the leading causes of illness and mortality in the world's medical communities is chronic kidney disease (CKD). Patients often misdiagnose CKD since there are no symptoms in the early stages of the illness. Individuals living with HIV are more likely to develop critical care kidney disease (CKD). Early diagnosis of CKD prevents the illness from worsening and enables patients to get treatment more quickly. The application of machine learning algorithms for illness categorization and prediction in healthcare has increased due to the availability of pathology data. The categorization of CKD using machine learning models is presented in this research. For individuals with CKD, the CKD stages are also determined based on the glomerular filtration rate. The DNN model performs better, diagnosing CKD patients with HIV with 99% accuracy.

A Hybrid Sparse-dense Defensive DNN Accelerator Architecture against Adversarial Example Attacks

Understanding how to defend against adversarial attacks is crucial for ensuring the safety and reliability of these systems in real-world applications. Various adversarial defense methods are proposed, which aim at improving the robustness of neural networks against adversarial attacks by changing the model structure, adding detection networks, and adversarial purification network. However, deploying adversarial defense methods in existing DNN accelerators or defensive accelerators leads to many key issues. To address these challenges, this article proposes sDNNGuard , an elastic heterogeneous DNN accelerator architecture that can efficiently orchestrate the simultaneous execution of original ( target ) DNN networks and the detect algorithm or network. It not only supports for dense DNN detect algorithms, but also allows for sparse DNN defense methods and other mixed dense-sparse (e.g., dense-dense and sparse-dense) workloads to fully exploit the benefits of sparsity. sDNNGuard with a CPU core also supports the non-DNN computing and allows the special layer of the neural network, and used for the conversion for sparse storage format for weights and activation values. To reduce off-chip traffic and improve resources utilization, a new hardware abstraction with elastic on-chip buffer/computing resource management is proposed to achieve dynamical resource scheduling mechanism. We propose an extended AI instruction set for neural networks synchronization, task scheduling and efficient data interaction. Experiment results show that sDNNGuard can effectively validate the legitimacy of the input samples in parallel with the target DNN model, achieving an average 1.42× speedup compared with the state-of-the-art accelerators.

Sea Horse Optimization-Deep Neural Network: A Medication Adherence Monitoring System Based on Hand Gesture Recognition.

Medication adherence is an essential aspect of healthcare for patients and is important for achieving medical objectives. However, the lack of standard techniques for measuring adherence is a global concern, making it challenging to accurately monitor and measure patient medication regimens. The use of sensor technology for medication adherence monitoring has received much attention lately since it makes it possible to continuously observe patients' medication adherence behavior. Sensor devices or smart wearables utilize state-of-the-art machine learning (ML) methods to analyze intricate data patterns and provide predictions accurately. The key aim of this work is to develop a sensor-based hand gesture recognition model to predict medication activities. In this research, a smart sensor device-based hand gesture prediction model is developed to recognize medication intake activities. The device includes a tri-axial gyroscope, geometric, and accelerometer sensors to sense and gather data from hand gestures. A smartphone application gathers hand gesture data from the sensor device, which is then stored in the cloud database in a .csv format. These data are collected, processed, and classified to recognize the medication intake activity using the proposed novel neural network model called Sea Horse Optimization-Deep Neural Network (SHO-DNN). The SHO technique is implemented to update the biases and weights and the number of hidden layers in the DNN model. By updating these parameters, the DNN model is improved in classifying the samples of hand gestures to identify the medication activities. The research model demonstrates impressive performance, with an accuracy of 98.59%, sensitivity of 97.82%, precision of 98.69%, and an F1 score of 98.48%. Hence, the proposed model outperformed the most available models in all the aforementioned aspects. The results indicate that this model is a promising approach for medication adherence monitoring in healthcare applications, instilling confidence in its effectiveness.

Black Box Watermarking for DNN Model Integrity Detection Using Label Loss

Black Box Watermarking for DNN Model Integrity Detection Using Label Loss

Matryoshka: Exploiting the Over-Parametrization of Deep Learning Models for Covert Data Transmission.

High-quality private machine learning (ML) data stored in local data centers becomes a key competitive factor for AI corporations. In this paper, we present a novel insider attack called Matryoshka to reveal the possibility of breaking the privacy of ML data even with no exposed interface. Our attack employs a scheduled-to-publish DNN model as a carrier model for covert transmission of secret models which memorize the information of private ML data that otherwise has no interface to the outsider. At the core of our attack, we present a novel parameter sharing approach which exploits the learning capacity of the carrier model for information hiding. Our approach simultaneously achieves: (i) High Capacity - With almost no utility loss of the carrier model, Matryoshka can transmit over 10,000 real-world data samples within a carrier model which has 220× less parameters than the total size of the stolen data, and simultaneously transmit multiple heterogeneous datasets or models within a single carrier model under a trivial distortion rate, neither of which can be done with existing steganography techniques; (ii) Decoding Efficiency - once downloading the published carrier model, an outside colluder can exclusively decode the hidden models from the carrier model with only several integer secrets and the knowledge of the hidden model architecture; (iii) Effectiveness - Moreover, almost all the recovered models either have similar performance as if it is trained independently on the private data, or can be further used to extract memorized raw training data with low error; (iv) Robustness - Information redundancy is naturally implemented to achieve resilience against common post-processing techniques on the carrier before its publishing; (v) Covertness - A model inspector with different levels of prior knowledge could hardly differentiate a carrier model from a normal model.

Factorized visual representations in the primate visual system and deep neural networks.

Object classification has been proposed as a principal objective of the primate ventral visual stream and has been used as an optimization target for deep neural network models (DNNs) of the visual system. However, visual brain areas represent many different types of information, and optimizing for classification of object identity alone does not constrain how other information may be encoded in visual representations. Information about different scene parameters may be discarded altogether ('invariance'), represented in non-interfering subspaces of population activity ('factorization') or encoded in an entangled fashion. In this work, we provide evidence that factorization is a normative principle of biological visual representations. In the monkey ventral visual hierarchy, we found that factorization of object pose and background information from object identity increased in higher-level regions and strongly contributed to improving object identity decoding performance. We then conducted a large-scale analysis of factorization of individual scene parameters - lighting, background, camera viewpoint, and object pose - in a diverse library of DNN models of the visual system. Models which best matched neural, fMRI, and behavioral data from both monkeys and humans across 12 datasets tended to be those which factorized scene parameters most strongly. Notably, invariance to these parameters was not as consistently associated with matches to neural and behavioral data, suggesting that maintaining non-class information in factorized activity subspaces is often preferred to dropping it altogether. Thus, we propose that factorization of visual scene information is a widely used strategy in brains and DNN models thereof.

RegionFilter: Region-aware video filtering mechanism on resource-constrained edge nodes

The video analytics pipeline consumes substantial network and computing resources, and in many scenarios videos are sent from cameras to servers. Due to limited camera-side resources, most existing systems are frame-level filtering approaches that filter out frames that are unchanged or irrelevant to applications before transmitting data. However, video applications usually care about the regions of interest (RoIs) of frames rather than background regions, and frame-level filtering fails to consider the spatial redundancy of intra-frame regions. We argue that video analytics should adopt a region-level filtering approach. Unfortunately, how to perform region-aware filtering on cameras without GPU is a significant challenge. To overcome this, we built RegionFilter, which achieves region awareness on resource-constrained edge nodes through feedback results from server-side DNN models. RegionFilter splits each video segment into multiple sub-segments in spatial distribution through region awareness. Then different configuration parameters (e.g., resolution, quantization parameter (QP)) are selected according to the video application, and the sub-segments are encoded into sub-streamings with different qualities that are uploaded to servers for further processing. Experiments on various video tasks and datasets show that RegionFilter achieves considerable filtering benefits (bandwidth savings of 11.7–41.2% when transmitting HD video) while always meeting the desired accuracy.

Network Intrusion Detection System using Federated Machine Learning Approach

In the quickly changing digital world of today, protecting oneself from cyberattacks is crucial. This study presents a novel method that uses TensorFlow Federated (TFF) Learning to merge BiLSTM and DNN architectures, improving the precision and effectiveness of intrusion detection systems (IDS). TFF offers a major paradigm change in model training by enabling decentralized learning on several servers or devices. TFF provides IDS with collective intelligence by fostering collaborative learning on remote data sources while protecting data privacy. This improves detection accuracy and strengthens defenses against adversarial attacks. By utilizing TensorFlow Federated methods, IDS may run DNN and BiLSTM models concurrently, maximizing processing speed and resource efficiency. The system's capacity to manage high-throughput data streams is ensured by this concurrent execution, which speeds up threat detection and response. Moreover, information sharing and smooth integration between concurrent processes are made possible via synchronization and communication protocols. The cooperative synergy between the various models improves IDS's dependability and efficacy in thwarting emerging cyberthreats.

Time-series simulation of alpine grassland cover using transferable stacking deep learning and multisource remote sensing data in the Google Earth Engine

The growth of vegetation on the Qinghai Tibet Plateau (QTP) is experiencing significant changes due to climate change. There is still a lack of high-precision simulation methods for alpine grassland cover (AGC), and the climate feedback mechanisms of AGC remain unclear, which poses challenges for the production of high-precision AGC products and the formulation of ecological conservation policies. In this study, a transferable stacking deep learning (Stacking-DL) model is proposed based on a CNN, a DNN, and a GRU for AGC time series simulation. The applicability of deep learning models for AGC simulation is evaluated based on long time series of measured data, MODIS data, and environmental factors. Finally, the AGC spatiotemporal changes and controlling environmental factors in the alpine region were analyzed based on Sen’s slope and structural equation modeling (SEM). The results showed that feature selection and parameter optimization improved the applicability of the deep learning models in AGC simulations, and the DNN (R2 = 0.899, RMSE = 0.078) model performed best among the base deep learning models. The Stacking-DL model combines the advantages of multiple models and achieves high transfer accuracy. In the YRSR, the AGC increase area (20.34 %) is greater than the AGC decrease area (3.34 %), the increase area is mainly located in the northeast, and the decrease area is mainly located in the southwest. AGC changes in the YRSR are mainly controlled by permafrost and climate. This study provides a high-precision and transferable vegetation monitoring model for alpine mountain regions based on advanced deep learning models and clarifies the response mechanism of AGC under climate change.

Deep Learning Techniques for Chronic Kidney Disease Risk Prediction

Abstract: Chronic kidney disease (CKD) and itssymptoms are mild and gradual, often go unnoticedfor years only to be realized lately. Bade, a Local Government of Yobe state in Nigeria has been a center of attention by medical practitioners due to the prevalence of CKD. Unfortunately, a technical approach In culminating the disease is yet to be attained. We obtained a record of 400 patients with 10 attributes as our dataset from Bade General Hospital. We used DNN model to predict the absence or presence of CKD in the patients. The model produced an accuracy of 98%. Furthermore, we identified and highlighted the Featuresimportance to provide the ranking of the features used in the prediction of the CKD. The outcome revealed that two attributes; Creatinine and Bicarbonate have the highest influence on the CKDprediction.

Prediction analysis of TBI 24-h survival outcome based on machine learning

BackgroundTraumatic brain injury (TBI) is the major reason for the death of young people and is well known for its high mortality and morbidity. This paper aim to predict the 24h survival of patients with TBI. MethodsA total of 1224 samples were involved in this analysis, and the clinical indicators involved included age, gender, blood pressure, MGAP and other fields, among which the target variable was “outcome”, which was a binary variable. The methods mainly involved in this paper include data visualization analysis, single factor analysis, feature engineering analysis, random forest model (RF), K-Nearst Neighbors (KNN) model, and so on. Logistic regression model (LR) and deep neural network model (DNN). We will oversample the training set using the SMOTE method because of the very unbalanced labeling of the sample itself. ResultsAlthough the accuracy of all models is very high, the recall rate is relatively low. The DNN model with the best performance only reaches 0.17, and the corresponding AUC is 0.80. After resampling, we find that the recall rate of positive samples of all models has increased a lot, but the AUC of some models has decreased. Finally, the optimal model is LR, whose positive sample recall rate is 0.67 and AUC is 0.82. ConclusionThrough resampling, we obtained that the best model is the RF model, whose recall rate and AUC are the best, and the AUC level is about 0.87, indicating that the accuracy performance of the model is still good.

A new piracy-resistant DNN watermarking method based on secret key and block-wise image transformations

This paper proposes a watermarking method that can be used for the copyright protection of DNN models, utilizing learnable block-wise image transformation techniques and a secret key to embed a watermark. A black-box watermarking approach is used, which does not require a specific predefined training or trigger set, allowing for the remote verification of model ownership. As a result, this method can achieve copyright protection using authentication methods for DNN models. Results of experiments on established datasets [1, 2] indicate that the original watermark is not easily overwritten by pirated watermarks. Moreover, its performance in pruning attack experiments is similar to that observed in the studies cited above. However, our approach demonstrates stronger robustness against fine-tuning attacks, while also achieving higher image classification accuracy.