Trained Deep Neural Network Model Research Articles

Research on Risk Assessment of Spices Based on Gray Correlation Analysis – Deep Neural Networks

Food safety is directly related to the health of the public, and the safety of condiments is also of great significance. In this study, a risk assessment model for condiments based on gray correlation analysis was established by using publicly available sampling data of soy sauce and vinegar in China. Risk indicator screening and data preprocessing were performed first, and the weight of each indicator was calculated by gray correlation analysis to formulate a comprehensive risk value label. Then, three machine learning models, Deep Neural Network (DNN), Random Forest (RF), and eXtreme Gradient Boosting (XGBoost), were used to predict the comprehensive risk values. Finally, the fuzzy synthesis analysis was utilized to classify the risk level of the composite risk value. In this study, based on the analysis of 282 sets of soy sauce and 704 sets of vinegar samples, the trained DNN model has the optimal prediction performance, which can basically predict the comprehensive risk value and risk level of a sample by inputting the detection indexes of that sample. This method can provide a more reasonable basis for relevant departments to formulate risk control strategies.

Split tensile strength of fiber-reinforced coral aggregate concrete: Deep learning model and experimental validation

Machine learning methods are increasingly becoming an important research field for solving the complex mechanical behavior of concrete materials. A reliable deep learning model was developed based on a dataset consisting of 319 experimental data points to predict the split tensile strength of fiber-reinforced coral aggregate concrete (FRCAC). The feature selection was carried out using correlation analysis, and the neural network structure was optimized by adjusting hyperparameters based on the 10-fold cross-validation (CV). Subsequently, six statistical indicators were employed to evaluate the accuracy and generalization ability of the model. The deep neural network (DNN) model demonstrates excellent performance, with R2, MAE, and MAPE of 0.98, 0.181 MPa, and 0.037 for the training dataset and 0.95, 0.286 MPa, and 0.074 for the test dataset, respectively. Additionally, 24 sets of concrete proportions are designed using the trained DNN model, and the results indicate that the absolute error and relative error between predicted and experimental values are less than 0.4 MPa and 6.8 %, respectively. The feature analysis based on SHapley Additive exPlanations (SHAP) reveals that volume fraction, curing age, water-binder ratio, and the tensile strength of fiber have the highest contribution in estimating the split tensile strength of FRCAC. This research can provide a reference for optimizing the design of FRCAC.

Activated biochar production from young coconut waste (Cocos nucifera) as bioadsorbent: a pathway through Artificial Neural Network (ANN) optimization.

This pioneering work explores the immense potential of young coconut waste, a continuously marginalized residue of the food and beverage industry, to serve as an indispensable feedstock in the production of biochar. Through an examination of the key carbonization factors that include time, temperature, and concentrations of the activating agent, KOH, the outcomes offer relevant insights that could be leveraged to maximize biochar production for tailored applications. This study stands out for its innovative use of Artificial Neural Network (ANN) approaches for predictive modeling. Fifty datasets, supplemented with secondary data obtained from the literature and experiments, were utilized for the purposes of training, testing, and validating the neural network model. Here, the datasets were processed utilizing the Deep Neural Network (DNN) framework, which was designed and implemented with the minimal loss function framework feasible. The architectural configuration comprises the following; an input layer, four hidden layers (128-neuron dense layer, batch normalization, and 64-neuron dense layer, batch normalization), a dropout layer, and an output layer. With an R2 of 0.8238 for biochar yield and 0.7324 for iodine number, the trained DNN model showed a relatively high degree of accuracy in making predictions.

Deep-neural-network model for predicting ground motion parameters using earthquake horizontal-to-vertical spectral ratios

This study proposed a deep-neural-network (DNN) model for seismic ground motion prediction by utilizing a unified strong motion database by the National Research Institute for Earth Science and Disaster Resilience, and earthquake horizontal-to-vertical spectral ratio (EHVR) database in Japan. The model aims to enhance the accuracy of predictions by incorporating the EHVRs for complementing site effects, and utilizing existing ground motion prediction equations (GMPE) as the base model for source and propagation path effects. The hybrid approach enables the prediction of peak ground accelerations (PGAs), peak ground velocities (PGVs), and 5% damped absolute acceleration response spectra (SAs). After classifying the training and test sets from the database, the trained DNN models were applied on the test set to evaluate the performance of the predicted results. The accuracy assessment by the residuals, R-squared ( R2), and root mean square error (RMSE) between the predicted and observed values in the test set revealed the superior performance of the proposed model compared with the traditional GMPE with proxy-based site effects such as V S30s especially in predicting both the spectral amplitude and shape of SAs.

Improved Hybrid Bagging Resampling Framework for Deep Learning-Based Side-Channel Analysis

As cryptographic implementations leak secret information through side-channel emissions, the Hamming weight (HW) leakage model is widely used in deep learning profiling side-channel analysis (SCA) attacks to expose the leaked model. However, imbalanced datasets often arise from the HW leakage model, increasing the attack complexity and limiting the performance of deep learning-based SCA attacks. Effective management of class imbalance is vital for training deep neural network models to achieve optimized and improved performance results. Recent works focus on either improved deep-learning methodologies or data augmentation techniques. In this work, we propose the hybrid bagging resampling framework, a two-pronged strategy for tackling class imbalance in side-channel datasets, consisting of data augmentation and ensemble learning. We show that adopting this framework can boost attack performance results in a practical setup. From our experimental results, the SMOTEENN ensemble achieved the best performance in the ASCAD dataset, and the basic ensemble performed the best in the CHES dataset, with both contributing over 70% practical improvements in performance compared to the original imbalanced dataset, and accelerating practical attack space in comparison to the classical setup of the attack.

Neural network architecture search model for thermal radiation in dense particulate systems

In the CFD-DEM simulation of the many dense particulate systems, particle-scale thermal radiation is an important heat transfer mode under high temperatures. In this work, neural network architecture search model with different layer connection matrix is applied to the view factor regression of the thermal radiation in dense particulate systems. Deep neural networks with 3346 feasible architectures are evaluated by the HyperBand algorithm to find the local optimal solution. Neural architecture search model trained by the big data of the view factor gives a good prediction of the macroscopic radiative properties and it is in general agreement with the empirical correlations and experimental data. The particle–wall radiation decreases strongly with the distance and the maximum interaction depth is about 2.0 times the sphere diameter. The trained deep neural network model provides an efficient data-driven closure to discuss the thermal radiation of the particle–particle and particle–wall interactions in particle bed.

Shear-wave velocity prediction of tight reservoirs based on poroelasticity theory: A comparative study of deep neural network and rock physics model

The absence of shear-wave (S-wave) velocity in well log data limits the effective seismic characterization of lithology, petrophysical properties, and fluid distribution of tight reservoirs in the Chang 7 shale oil formations of Ordos Basin, west China. However, conventional rock physics models (RPMs), e.g., the Xu-White model, struggle to accurately obtain the key parameters such as pore aspect ratio in the process of S-wave velocity prediction. This work performs a comparative study of S-wave velocity prediction based on deep neural network (DNN) and RPM for such reservoirs. By employing the poroelasticity equations which are capable of describing elastic wave propagation in the complex reservoirs saturated with viscous fluid, the relation between elastic and petrophysical properties is established with a DNN-based method. This approach quantifies the connection between P-/S-wave velocities and rock properties. On the other hand, we reformulate the traditional Xu-White model by incorporating the decoupled Kuster-Toksöz (K-T) model and the poroelasticity equations with viscoelastic fluid in the modeling process. The variation of pore aspect ratio with depth is also considered to characterize the complex pore structures, which is appropriate for improving the accuracy of prediction. Log data from the three wells are considered for verification, of which results show that the S-wave velocity prediction methods help improve the final result. Aided by the DNN-based method, the prediction exhibits a better performance if the training data is sufficient. In particular, provided a small amount of measured S-wave data (even ten data points), the trained DNN model can also provide reasonable predictions.

Robustness analysis and experimental validation of a deep neural network for acoustic source imaging

Deep Neural Network (DNN) models offer an attractive alternative to existing acoustic source imaging techniques, such as acoustic beamforming, due to their ever-growing potential with increasing computational power. Source resolution of acoustic beamforming methods is limited at lower frequencies and their source maps may possess sidelobes at higher frequencies. However, acoustic beamforming methods are typically robust over a wide range of simulation and experimental conditions, such as (i) the number of sources present, (ii) source frequency and (ii) extraneous noise sources. The performance of DNN models, when these conditions are varied from their specific design criteria, is yet to be investigated and much work is needed in this area before DNN models can be utilized in experiments, such as wind tunnel tests. Furthermore, few studies have been conducted on experimental validation of DNN models, predominately due to the difficulty of large sets of experimentally obtained data needed for DNN model training and the sensitivity of DNN model performance when any of the aforementioned experimental conditions are varied. In this paper, a series of studies on the robustness of DNN models based on numerical data and experimental data are presented. Numerical DNN (NDNN) models are trained using in-phase and random-phase pressure data generated from six sources over design frequencies from 500 Hz to 20,000 Hz. The robustness of the NDNN models is tested via (1) inclusion of extraneous Gaussian white noise, (2) inclusion of extraneous tonal noise near the design frequency, (3) using source frequencies that slightly differ from the design frequencies and (4) using a number of sources that differs from the design source number. DNN model performance metrics are introduced that present a promising framework for future DNN model studies and bridging the gap between NDNN and experimentally trained DNN models. A preliminary experimental validation was conducted using a single speaker that was systematically placed over a speaker grid to generate training data via acoustic superposition, from which an experimentally trained DNN (EDNN) model was produced. The EDNN model yields exceptional noise source localization capability of the DNN model, revealing a promising start for a more sophisticated EDNN model.

Spectrally tunable light source based on deep neural network model

A spectrally tunable light source based on deep neural network (DNN) model is proposed in this work, which can reproduce arbitrary spectra accurately and rapidly. After calculating the scale factors using the trained DNN model, the target spectrum can be reproduced by regulating the combined monochromatic LEDs based on the pulse width modulation (PWM) signal with corresponding duty cycle. The standard solar spectrum and measured natural spectra at different times are reproduced using the proposed spectrally tunable light source. It is demonstrated that there is a good agreement between the target spectra and the corresponding reproduced spectra, with a fitted correlation index of above 0.9. Furthermore, the proposed spectrally tunable light source is able to transform spectrum at a frequency higher than 25 Hz and occupies only 750 KB memory space. As a result, the proposed spectrally tunable light source system can offer a high accuracy, fast speed and low cost approach to the development of specially light sources.

Artificial Intelligence in Intellectual Property Protection: Application of Deep Learning Model

To create and train a deep learning model costs a lot in comparison to ascertain a trained model. So, a trained model is considered as the intellectual property (IP) of the person who creates such model. However, there is every chance of illegal copying, redistributing and abusing of any of these high-performance models by the malicious users. To protect against such menaces, a few numbers of deep neural networks (DNN) IP security techniques have been developed recently. The present study aims at examining the existing DNN IP security activities. In the first instance, there is a proposal of taxonomy in favor of DNN IP protection techniques from the perspective of six aspects such as scenario, method, size, category, function, and target models. Afterwards, this paper focuses on the challenges faced by these methods and their capability of resisting the malicious attacks at different levels by providing proactive protection. An analysis is also made regarding the potential threats to DNN IP security techniques from various perspectives like modification of models, evasion and active attacks.
 Apart from that this paper look into the methodical assessment. The study explores the future research possibilities on DNN IP security by considering different challenges it would confront in the process of its operations.
 Result Statement: A high-performance deep neural Networks (DNN) model is costlier than the trained DNN model. It is considered as an intellectual property (IP) of the person who is responsible for creating DNN model. The infringement of the Intellectual Property of DNN model is a grave concern in recent years. This article summarizes current DNN IP security works by focusing on the limitations/ challenges they confront. It also considers the model in question's capacity for protection and resistance against various stages of attacks.

An Efficient Checkpoint Strategy for Federated Learning on Heterogeneous Fault-Prone Nodes

Federated learning (FL) is a distributed machine learning method in which client nodes train deep neural network models locally using their own training data and then send that trained model to a server, which then aggregates all of the trained models into a globally trained model. This protects personal information while enabling machine learning with vast amounts of data through parallel learning. Nodes that train local models are typically mobile or edge devices from which data can be easily obtained. These devices typically run on batteries and use wireless communication, which limits their power, making their computing performance and reliability significantly lower than that of high-performance computing servers. Therefore, training takes a long time, and if something goes wrong, the client may have to start training again from the beginning. If this happens frequently, the training of the global model may slow down and the final performance may deteriorate. In a general computing system, a checkpointing method can be used to solve this problem, but applying an existing checkpointing method to FL may result in excessive overheads. This paper proposes a new FL method for situations with many fault-prone nodes that efficiently utilizes checkpoints.

The Implementation of a Deep Neural Network (DNN) Approach in a Case Study Predicting the Distribution of Carbon Dioxide (CO2) Gas Saturation

Predicting the distribution of CO2 gas saturation is one example of how multiphase flow might be evaluated in Carbon Capture and Storage (CCS). The TOUGH2 simulator is one of the numerical simulations commonly used for multiphase flow simulation. Ordinary numerical simulations have several issues, including high grid spatial resolution and high processing costs. One of the most effective deep learning approaches to predicting the distribution of CO2 gas saturation is the deep neural network (DNN). A deep neural network is a network with three interconnected layers, there are input, hidden, and output layers. DNN learns about the previously constructed architecture from the input data. DNN requires a large quantity of data as input. Thus, in this study, we use 700 data points for each of the train_a and train_b variables. The distribution of CO2 gas saturation will be predicted automatically by the trained DNN model. This technique can handle complex data patterns, such as gas saturation in multiphase flow problems. The reconstruction loss findings show that the loss value decreases as the number of epochs increases. Furthermore, we used 3 and 4 epochs to determine the difference in results between the two. As a result, the model with 4 epochs and 10−3 regularization weight obtained the lowest error value of 0.4305. In summary, this model is capable of predicting CO2 gas saturation distribution, but more research is needed to produce more optimal results. This research hopes to help monitor multiphase flow in CCS systems in the future by forecasting the distribution of CO2 gas saturation.

Federated Learning: Centralized and P2P for a Siamese Deep Learning Model for Diabetes Foot Ulcer Classification

It is a known fact that AI models need massive amounts of data for training. In the medical field, the data are not necessarily available at a single site but are distributed over several sites. In the field of medical data sharing, particularly among healthcare institutions, the need to maintain the confidentiality of sensitive information often restricts the comprehensive utilization of real-world data in machine learning. To address this challenge, our study experiments with an innovative approach using federated learning to enable collaborative model training without compromising data confidentiality and privacy. We present an adaptation of the federated averaging algorithm, a predominant centralized learning algorithm, to a peer-to-peer federated learning environment. This adaptation led to the development of two extended algorithms: Federated Averaging Peer-to-Peer and Federated Stochastic Gradient Descent Peer-to-Peer. These algorithms were applied to train deep neural network models for the detection and monitoring of diabetic foot ulcers, a critical health condition among diabetic patients. This study compares the performance of Federated Averaging Peer-to-Peer and Federated Stochastic Gradient Descent Peer-to-Peer with their centralized counterparts in terms of model convergence and communication costs. Additionally, we explore enhancements to these algorithms using targeted heuristics based on client identities and f1-scores for each class. The results indicate that models utilizing peer-to-peer federated averaging achieve a level of convergence that is comparable to that of models trained via conventional centralized federated learning approaches. This represents a notable progression in the field of ensuring the confidentiality and privacy of medical data for training machine learning models.

Deep neural network-based phase calibration in integrated optical phased arrays

Calibrating the phase in integrated optical phased arrays (OPAs) is a crucial procedure for addressing phase errors and achieving the desired beamforming results. In this paper, we introduce a novel phase calibration methodology based on a deep neural network (DNN) architecture to enhance beamforming in integrated OPAs. Our methodology focuses on precise phase control, individually tailored to each of the 64 OPA channels, incorporating electro-optic phase shifters. To effectively handle the inherent complexity arising from the numerous voltage set combinations required for phase control across the 64 channels, we employ a tandem network architecture, further optimizing it through selective data sorting and hyperparameter tuning. To validate the effectiveness of the trained DNN model, we compared its performance with 20 reference beams obtained through the hill climbing algorithm. Despite an average intensity reduction of 0.84 dB in the peak values of the beams compared to the reference beams, our experimental results demonstrate substantial agreements between the DNN-predicted beams and the reference beams, accompanied by a slight decrease of 0.06 dB in the side-mode-suppression-ratio. These results underscore the practical effectiveness of the DNN model in OPA beamforming, highlighting its potential in scenarios that necessitate the intelligent and time-efficient calibration of multiple beams.

Using a Deep Neural Network with Small Datasets to Predict the Initial Production of Tight Oil Horizontal Wells

Due to its abundant reserves, tight oil has emerged as a significant substitute for conventional petroleum resources. It has become one of the focal points of exploration and research, and a new hot spot in global unconventional oil and gas exploration and development. This has led to a significant increase in the demand for forecasting the production capacity of tight oil horizontal wells. The deep neural network (DNN), as a mature model, has demonstrated significant advantages in many fields. However, due to the confidentiality and uniqueness of oilfield data, acquiring large datasets has become a challenge. Traditional methods using small datasets for training DNN models result in low accuracy and overfitting issues, which hinders the development of neural networks in the petroleum industry. This study aims to predict the initial production capacity of tight oil horizontal wells by using a small dataset of 650 data points through a DNN model. The research results indicate that pre-trained and fine-tuned DNNs outperform shallow neural networks, supporting vector machines, and DNN trained with traditional methods in terms of better generalization performance. Their accuracy reached 91.3%, demonstrating that it is reasonable to use a small dataset with pre-trained and fine-tuned DNN models.

Attacks on Recent DNN IP Protection Techniques and Their Mitigation

With the rapid increase in the development of Deep Learning methodologies, Deep Neural Networks (DNNs) are now being commonly deployed in smart systems (e.g. autonomous vehicles) and high-end security applications (e.g. face recognition, biometric authentication, etc.). The training of such DNN models often requires exclusive valuable training datasets, enormous computational resources, and expert fine-tuning skills. Hence, a trained DNN model can be regarded as valuable proprietary Intellectual Property (IP). Piracy of such DNN IPs has emerged as a major concern, with increasing trends of illegal copying and redistribution. A number of mitigation approaches targeting DNN IP protection have been proposed in recent years. In this work, we target two recently proposed DNN IP protection schemes: (a) Chaotic Map theory based encryption of the weight parameters, and (b) traditional block cipher based encryption of the weights. We demonstrate attacks on two recent DNN IP protection techniques, with one technique each belonging to the above-mentioned schemes, under a pragmatic attack model. We also propose a novel DNN IP protection technique based on selective encryption of the weight parameters, termed LEWIP to mitigate the exposed weaknesses, while having low implementation and performance overheads. Finally, we demonstrate the effectiveness of the LEWIP technique against state-of-the-art DNN implementations.

Optimizing process parameters for hot forging of Ti-6242 alloy: A machine learning and FEM simulation approach

In this study, we investigated the hot deformation behavior of Ti–6Al–2Sn–4Zr–2Mo (Ti-6242) alloy and propose a method to derive optimal hot process parameters for grain refinement and avoidance of flow instability. Microstructural Risk Index (MRI) was introduced as a microstructural evaluation index consisting of grain size, standard deviation of grain size, and flow instability. The initial temperature of the material and the stroke speed of the die were selected as design variables. Finite element analysis (FEA) was used to calculate grain size and flow instability and determine MRI of the forgings. The grain size model coefficient and flow instability were calculated based on the flow stress curve and verified with optical microscope (OM) and electron backscatter diffraction (EBSD) analysis results. The Deep Neural Network (DNN) model was used to determine the optimal process parameters for the forging process. The MRI prediction accuracy of the trained DNN models showed excellent performance at 97.01 %. The MRI of the optimized process variables was improved by 7.95 % compared to the minimum MRI of the training data set. Optimized process parameters can improve the quality of forgings through grain refinement and avoidance of flow instability regions.

Generating synthetic gait patterns based on benchmark datasets for controlling prosthetic legs

BackgroundProsthetic legs help individuals with an amputation regain locomotion. Recently, deep neural network (DNN)-based control methods, which take advantage of the end-to-end learning capability of the network, have been proposed. One prominent challenge for these learning-based approaches is obtaining data for the training, particularly for the training of a mid-level controller. In this study, we propose a method for generating synthetic gait patterns (vertical load and lower limb joint angles) using a generative adversarial network (GAN). This approach enables a mid-level controller to execute ambulation modes that are not included in the training datasets.MethodsThe conditional GAN is trained on benchmark datasets that contain the gait data of individuals without amputation; synthetic gait patterns are generated from the user input. Further, a DNN-based controller for the generation of impedance parameters is trained using the synthetic gait pattern and the corresponding synthetic stiffness and damping coefficients.ResultsThe trained GAN generated synthetic gait patterns with a coefficient of determination of 0.97 and a structural similarity index of 0.94 relative to benchmark data that were not included in the training datasets. We trained a DNN-based controller using the GAN-generated synthetic gait patterns for level-ground walking, standing-to-sitting motion, and sitting-to-standing motion. Four individuals without amputation participated in bypass testing and demonstrated the ambulation modes. The model successfully generated control parameters for the knee and ankle based on thigh angle and vertical load.ConclusionsThis study demonstrates that synthetic gait patterns can be used to train DNN models for impedance control. We believe a conditional GAN trained on benchmark datasets can provide reliable gait data for ambulation modes that are not included in its training datasets. Thus, designing gait data using a conditional GAN could facilitate the efficient and effective training of controllers for prosthetic legs.

Overfit deep neural network for predicting drug-target interactions

Drug-target interactions (DTIs) prediction is an important step in drug discovery. As traditional biological experiments or high-throughput screening are high cost and time-consuming, many deep learning models have been developed. Overfitting must be avoided when training deep learning models. We propose a simple framework, called OverfitDTI, for DTI prediction. In OverfitDTI, a deep neural network (DNN) model is overfit to sufficiently learn the features of the chemical space of drugs and the biological space of targets. The weights of trained DNN model form an implicit representation of the nonlinear relationship between drugs and targets. Performance of OverfitDTI on three public datasets showed that the overfit DNN models fit the nonlinear relationship with high accuracy. We identified fifteen compounds that interacted with TEK, a receptor tyrosine kinase contributing to vascular homeostasis, and the predicted AT9283 and dorsomorphin were experimentally demonstrated as inhibitors of TEK in human umbilical vein endothelial cells (HUVECs).

Prediction of particle-laden pipe flows using deep neural network models

An accurate and fast prediction of particle-laden flow fields is of particular relevance for a wide variety of industrial applications. The motivation for this research is to evaluate the applicability of deep learning methods for providing statistical properties of the carrier and dispersed phases in a particle-laden vertical pipe flow. Deep neural network (DNN) models are trained for different dependent variables using 756 high-fidelity datasets acquired from point-particle large-eddy simulations for different values of Stokes number, St, bulk particle volume fraction, Φ¯v, and wall roughness, Δγ, for the range St=10−500, Φ¯v=5×10−5−10−3, and Δγ=1°−6°. The considered parameter space corresponds to the inertia-dominated regime and covers a large extent of the typical conditions in powder-based laser metal deposition. We find that the DNN models capture the nonlinear dynamics of the system and recreate the statistical properties of the particle-laden pipe flow. However, DNN predictions of the particle statistics are of higher accuracy compared to the fluid statistics, which is attributed to the highly non-monotonic dependence of the fluid statistics on the control parameters. Owing to significantly decreased time-to-solution, the trained DNN models are promising as surrogate models to expedite model development and design process of various industrial applications.