Deep Neural Network Regression Research Articles

Origami single-end capacitive sensing for continuous shape estimation of morphing structures

In this work, we propose a novel single-end morphing capacitive sensing method for shape tracking, FxC, by combining Folding origami structures and Capacitive sensing to detect the morphing structural motions using state-of-the-art sensing circuits and deep learning. It was observed through embedding areas of origami structures with conductive materials as single-end capacitive sensing patches, that the sensor signals change coherently with the motion of the structure. Different from other origami capacitors where the origami structures are used in adjusting the thickness of the dielectric layer of double-plate capacitors, FxC uses only a single conductive plate per channel, and the origami structure directly changes the geometry of the conductive plate. We examined the operation principle of morphing single-end capacitors through 3D geometry simulation combined with physics theoretical deduction, which deduced similar behavior as observed in experimentation. Then a software pipeline was developed to use the sensor signals to reconstruct the dynamic structural geometry through data-driven deep neural network regression of geometric primitives extracted from vision tracking. We created multiple folding patterns to validate our approach, based on folding patterns including Accordion, Chevron, Sunray and V-Fold patterns with different layouts of capacitive sensors using paper-based and textile-based materials. Experimentation results show that the geometry primitives predicted from the capacitive signals have a strong correlation with the visual ground truth with R-squared value of up to 95% and tracking error of 6.5 mm for patches. The simulation and machine learning constitute two-way information exchange between the sensing signals and structural geometry. By embedding part of the origami surface with morphing single-end capacitive sensors, FxC presents a unique solution that leverages both the mechanical properties of origami and sensing properties of capacitive sensing.

Improved deep neural network for predicting structural response of stiffened cylindrical shells to far-field underwater explosion

This paper is focused on the application of the machine learning for predicting structural response of ring stiffened cylindrical shells subjected to far-field underwater explosion. Three deep neural network (DNN) models are combined to predict the progressive plastic deformation behaviour of stiffened cylindrical shells subjected to the shock wave produced by a TNT charge. In the DNN regression models, the input variables include the charge mass, stand-off distance, detonation depth, cylindrical shell thickness, stiffener web and flange. These three DNN models composed of multi hidden layers are trained based on the data collection originated from a series of simulations performed by the ABAQUS finite element solver. Grid search is introduced to the hyperparameter tuning for deep learning, improving the prediction performance of machine learning methods. The DNN models are compared to the models trained using another three methods: multiple linear regression, decision tree and k-nearest neighbour. An improved Adam optimisation algorithm is used to increase the accuracy of prediction. The challenges encountered during the predictions are discussed to provide a more comprehensive insight into the advantage of the proposed DNN models. The DNN regression models can well predict the permanent plastic deformation and strain of stiffened cylindrical shells.

Ultra-short-term wind power probabilistic forecasting based on an evolutionary non-crossing multi-output quantile regression deep neural network

Ultra-short-term wind power probabilistic forecasting is of significance for stable power grid operation; however, it is still challenging due to the inherent nonlinearity and uncertainty. Most state-of-the-art methods have focused on achieving quantile prediction using a combination of linear quantile regression and nonlinear complex deep neural networks. Yet, these methods struggle with several dilemmas. Quantile regression deep neural networks require a complete training once for each quantile. The multi-training mode and complex structure of quantile regression deep neural network can lead to extremely high computational complexity. Most of the training of quantile regression deep neural networks are guided by the loss of each quantile, and the weights are adjusted by gradient descent in which the gradient explosion and quantile crossover may be encountered. Therefore, this paper proposes a non-crossing multi-output quantile regression deep neural network optimized by chaotic particle swarm optimization. It designs a multi-output deep neural network to output all quantile estimations simultaneously through one training, effectively solving the structural complexity problem of traditional quantile regression deep neural networks. Since quantile regression produces a non-differentiable loss function which significantly hinders model training, the proposed neural network is trained by chaotic particle swarm optimization. It not only achieves the effect of optimizing all quantile losses simultaneously, but also can significantly alleviate the dilemma of training in traditional neural network weight optimization. In addition, several non-crossing constraints are designed for avoiding quantile crossover. The proposed model is trained and tested on two real-world wind power case studies. The experiment results show that the proposed model shows superiority in performance criteria, training speed, and avoiding quantile crossover.

Learning Rates of Deep Nets for Geometrically Strongly Mixing Sequence.

The great success of deep learning poses an urgent challenge to establish the theoretical basis for its working mechanism. Recently, research on the convergence of deep neural networks (DNNs) has made great progress. However, the existing studies are based on the assumption that the samples are independent, which is too strong to be applied to many real-world scenarios. In this brief, we establish a fast learning rate for the empirical risk minimization (ERM) on DNN regression with dependent samples, and the dependence is expressed in terms of geometrically strongly mixing sequence. To the best of our knowledge, this is the first convergence result of DNN methods based on mixing sequences. This result is a natural generalization of the independent sample case.

A Perceptual Shape Loss for Monocular 3D Face Reconstruction

AbstractMonocular 3D face reconstruction is a wide‐spread topic, and existing approaches tackle the problem either through fast neural network inference or offline iterative reconstruction of face geometry. In either case carefully‐designed energy functions are minimized, commonly including loss terms like a photometric loss, a landmark reprojection loss, and others. In this work we propose a new loss function for monocular face capture, inspired by how humans would perceive the quality of a 3D face reconstruction given a particular image. It is widely known that shading provides a strong indicator for 3D shape in the human visual system. As such, our new ‘perceptual’ shape loss aims to judge the quality of a 3D face estimate using only shading cues. Our loss is implemented as a discriminator‐style neural network that takes an input face image and a shaded render of the geometry estimate, and then predicts a score that perceptually evaluates how well the shaded render matches the given image. This ‘critic’ network operates on the RGB image and geometry render alone, without requiring an estimate of the albedo or illumination in the scene. Furthermore, our loss operates entirely in image space and is thus agnostic to mesh topology. We show how our new perceptual shape loss can be combined with traditional energy terms for monocular 3D face optimization and deep neural network regression, improving upon current state‐of‐the‐art results.

Deep Neural Network Regression to Assist Non-Invasive Diagnosis of Portal Hypertension.

Portal hypertension is a complex medical condition characterized by elevated blood pressure in the portal venous system. The conventional diagnosis of such disease often involves invasive procedures such as liver biopsy, endoscopy, or imaging techniques with contrast agents, which can be uncomfortable for patients and carry inherent risks. This study presents a deep neural network method in support of the non-invasive diagnosis of portal hypertension in patients with chronic liver diseases. The proposed method utilizes readily available clinical data, thus eliminating the need for invasive procedures. A dataset composed of standard laboratory parameters is used to train and validate the deep neural network regressor. The experimental results exhibit reasonable performance in distinguishing patients with portal hypertension from healthy individuals. Such performances may be improved by using larger datasets of high quality. These findings suggest that deep neural networks can serve as useful auxiliary diagnostic tools, aiding healthcare professionals in making timely and accurate decisions for patients suspected of having portal hypertension.

Cogging process design of M50 bearing steel for billet quality

This study proposes a method to determine cogging process parameters aiming at homogeneous effective strain distribution and grain size uniformity and refinement during the manufacturing M50-bearing steel billets. The design parameters were selected as feed, number of rotation, depth schedule, and forging ratio. The Billet Quality Index (BQI) is introduced to comprehensively evaluate the effective strain distribution and grain size number (ASTM E112) of the final billet and used as the objective function for the process design. The results of effective strain distribution and grain size number are obtained through finite element simulation, and these values are used to investigate the BQI of the final billet. For the finite element analysis to calculate the temperature field and grain size, the heat transfer coefficients determined by reverse engineering and the grain growth model coefficients were used compared with the experimental measurements. The Taguchi method investigates the influence of parameters on BQI, excluding the forging ratio. Optimal process parameters, including forging ratio, are determined using a Deep Neural Network (DNN) regression model, targeting BQI minimization. The BQI prediction accuracy of the DNN regression model is 98.83%, emphasizing the effectiveness of DNN despite the limited dataset size. The optimized cogging process reduces the total process time by 2.394 h and improves the BQI by 6.91% compared to the minimum BQI in the dataset. The optimized cogging process promotes homogeneous effective strain distribution, grain refinement, and uniformity, enhancing billet manufacturing efficiency.

Multilevel‐in‐width training for deep neural network regression

AbstractA common challenge in regression is that for many problems, the degrees of freedom required for a high‐quality solution also allows for overfitting. Regularization is a class of strategies that seek to restrict the range of possible solutions so as to discourage overfitting while still enabling good solutions, and different regularization strategies impose different types of restrictions. In this paper, we present a multilevel regularization strategy that constructs and trains a hierarchy of neural networks, each of which has layers that are wider versions of the previous network's layers. We draw intuition and techniques from the field of Algebraic Multigrid (AMG), traditionally used for solving linear and nonlinear systems of equations, and specifically adapt the Full Approximation Scheme (FAS) for nonlinear systems of equations to the problem of deep learning. Training through V‐cycles then encourage the neural networks to build a hierarchical understanding of the problem. We refer to this approach as multilevel‐in‐width to distinguish from prior multilevel works which hierarchically alter the depth of neural networks. The resulting approach is a highly flexible framework that can be applied to a variety of layer types, which we demonstrate with both fully connected and convolutional layers. We experimentally show with PDE regression problems that our multilevel training approach is an effective regularizer, improving the generalize performance of the neural networks studied.

Decoupling thermal effects in GaN photodetectors for accurate measurement of ultraviolet intensity using deep neural network

Precise and reliable monitoring systems under various harsh conditions are important to realize an overall future automation system for diverse industries. In this study, a facile fabrication method for a gallium nitride (GaN)-based ultraviolet (UV) photodetector was developed as a system-on-chip decoupling system, and a novel decoupling system was proposed to separate the thermal effects from the GaN-based sensor using a deep neural network (DNN)-based regression model. To predict the true values of UV intensity, the currents of both UV- and thermal-responsive GaN film and thermal-responsive copper film were used as input features without additional measurement devices. In addition, the performance of the DNN regression model was evaluated using the mean absolute percentage error. Consequently, the proposed model showed a prediction accuracy of ∼97.86% with two input features. Furthermore, the thermal decoupling method using the proposed sensor was successfully demonstrated, even when the thermal conditions changed significantly. These results support a novel decoupling method for reliable real-time monitoring systems for various harsh environmental industries, such as aerospace, ocean, subterranean, power plants, and combustion engines.

Soil-Surface-Image-Feature-Based Rapid Prediction of Soil Water Content and Bulk Density Using a Deep Neural Network

This study aimed to develop a deep neural network model for predicting the soil water content and bulk density of soil based on features extracted from in situ soil surface images. Soil surface images were acquired using a Canon EOS 100d camera. The camera was installed in the vertical direction above the soil surface layer. To maintain uniform illumination conditions, a dark room and LED lighting were utilized. Following the acquisition of soil surface images, soil samples were collected using a metal cylinder to obtain measurements of soil water content and bulk density. Various features were extracted from the images, including color, texture, and shape features, and used as inputs for both a multiple regression analysis and a deep neural network model. The results show that the deep neural network regression model can predict soil water content and bulk density with root mean squared error of 1.52% and 0.78 kN/m3. The deep neural network model outperformed the multiple regression analysis, achieving a high accuracy for predicting both soil water content and bulk density. These findings suggest that in situ soil surface images, combined with deep learning techniques, can provide a fast and reliable method for predicting important soil properties.

Blood pressure estimation and its recalibration assessment using wrist cuff blood pressure monitor.

The rapid evolution of wearable technology in healthcare sectors has created the opportunity for people to measure their blood pressure (BP) using a smartwatch at any time during their daily activities. Several commercially-available wearable devices have recently been equipped with a BP monitoring feature. However, concerns about recalibration remain. Pulse transit time (PTT)-based estimation is required for initial calibration, followed by periodic recalibration. Recalibration using arm-cuff BP monitors is not practical during everyday activities. In this study, we investigated recalibration using PTT-based BP monitoring aided by a deep neural network (DNN) and validated the performance achieved with more practical wrist-cuff BP monitors. The PTT-based prediction produced a mean absolute error (MAE) of 4.746 ± 1.529 mmHg for systolic blood pressure (SBP) and 3.448 ± 0.608 mmHg for diastolic blood pressure (DBP) when tested with an arm-cuff monitor employing recalibration. Recalibration clearly improved the performance of both DNN and conventional linear regression approaches. We established that the periodic recalibration performed by a wrist-worn BP monitor could be as accurate as that obtained with an arm-worn monitor, confirming the suitability of wrist-worn devices for everyday use. This is the first study to establish the potential of wrist-cuff BP monitors as a means to calibrate BP monitoring devices that can reliably substitute for arm-cuff BP monitors. With the use of wrist-cuff BP monitoring devices, continuous BP estimation, as well as frequent calibrations to ensure accurate BP monitoring, are now feasible.

Deep quantile and deep composite triplet regression

A main difficulty in actuarial claim size modeling is that covariates may have different effects on the body of the conditional distribution and on its tail. To cope with this problem, we introduce a deep composite regression model whose splicing point is given in terms of a quantile of the conditional claim size distribution (rather than a constant). This allows us to simultaneously fit different regression models in the two different parts of the conditional distribution function. To facilitate M-estimation for such models, we introduce and characterize the class of strictly consistent scoring functions for the triplet consisting of a quantile, as well as the lower and upper expected shortfall beyond that quantile. In a second step, this elicitability result is applied to fit deep neural network regression models. We demonstrate the applicability of our approach and its superiority over classical approaches on a real data set from accident insurance.

Evaluation of a Coating Process for SiO<sub>2</sub>/TiO<sub>2</sub> Composite Particles by Machine Learning Techniques

In this study, in order to optimize a fabrication process for SiO2/TiO2 composite particles and control their coating ratio (CTi), regression models for the coating process were constructed using various machine learning techniques. The composite particles with a core (SiO2)/shell (TiO2) structure were synthesized by mechanical stress under various fabrication conditions with respect to the supply volume of raw materials (V), addition ratio of TiO2 (rTi), operation time (t), rotor rotation speed (S), and temperature (T). Regression models were constructed by the least squares method (LSM), principal component regression (PCR), support vector regression (SVR), and the deep neural network (DNN) method. The accuracy of the constructed regression models was evaluated using the determination coefficients (R2) and the predictive performance was evaluated by comparing the prediction coefficients (Q2). From the perspective of the R2 and Q2 values, the DNN regression model was found to be the most suitable model for the present coating process. Moreover, the effects of the fabrication parameters on CTi were analyzed using the constructed DNN model. The results suggested that the t value was the dominant factor determining CTi of the composite particles, with the plot of CTi versus t displaying a clear maximum.

Improved Soil Quality Prediction Model Using Deep Learning for Smart Agriculture Systems

Soil is the major source of infinite lives on Earth and the quality of soil plays significant role on Agriculture practices all around. Hence, the evaluation of soil quality is very important for determining the amount of nutrients that the soil require for proper yield. In present decade, the application of deep learning models in many fields of research has created greater impact. The increasing soil data availability of soil data there is a greater demand for the remotely avail open source model, leads to the incorporation of deep learning method to predict the soil quality. With that concern, this paper proposes a novel model called Improved Soil Quality Prediction Model using Deep Learning (ISQP-DL). The work considers the chemical, physical and biological factors of soil in particular area to estimate the soil quality. Firstly, pH rating of soil samples has been collected from the soil testing laboratory from which the acidic range has been categorized through soil test and the same data has been taken as input to the Deep Neural Network Regression (DNNR) model. Secondly, soil nutrient data has been given as second input to the DNNR model. By utilizing this data set, the DNNR method is used to evaluate the fertility rate by which the soil quality has been estimated. For training and testing, the model uses Deep Neural Network Regression (DNNR), by utilizing the dataset. The results show that the proposed model is effective for SQP (Soil Quality Prediction Model) with efficient good fitting and generality is enhanced with input features with higher rate of classification accuracy. The results show that the proposed model achieves 96.7% of accuracy rate compared with existing models.

Deep Neural Network Regression for Normalized Digital Surface Model Generation With Sentinel-2 Imagery

Deep Neural Network Regression for Normalized Digital Surface Model Generation With Sentinel-2 Imagery

Compressive Strength Estimation of Rice Husk Ash-Blended Concrete Using Deep Neural Network Regression with an Asymmetric Loss Function

Compressive Strength Estimation of Rice Husk Ash-Blended Concrete Using Deep Neural Network Regression with an Asymmetric Loss Function

MODIS Evapotranspiration Downscaling Using a Deep Neural Network Trained Using Landsat 8 Reflectance and Temperature Data

The MODIS 8-day composite evapotranspiration (ET) product (MOD16A2) is widely used to study large-scale hydrological cycle and energy budgets. However, the MOD16A2 spatial resolution (500 m) is too coarse for local and regional water resource management in agricultural applications. In this study, we propose a Deep Neural Network (DNN)-based MOD16A2 downscaling approach to generate 30 m ET using Landsat 8 surface reflectance and temperature and AgERA5 meteorological variables. The model was trained at a 500 m resolution using the MOD16A2 ET as reference and applied to the Landsat 8 30 m resolution. The approach was tested on 15 Landsat 8 images over three agricultural study sites in the United States and compared with the classical random forest regression model that has been often used for ET downscaling. All evaluation sample sets applied to the DNN regression model had higher R2 and lower root-mean-square deviations (RMSD) and relative RMSD (rRMSD) (the average values: 0.67, 2.63 mm/8d and 14.25%, respectively) than the random forest model (0.64, 2.76 mm/8d and 14.92%, respectively). Spatial improvement was visually evident both in the DNN and the random forest downscaled 30 m ET maps compared with the 500 m MOD16A2, while the DNN-downscaled ET appeared more consistent with land surface cover variations. Comparison with the in situ ET measurements (AmeriFlux) showed that the DNN-downscaled ET had better accuracy, with R2 of 0.73, RMSD of 5.99 mm/8d and rRMSD of 48.65%, than the MOD16A2 ET (0.65, 7.18 and 50.42%, respectively).

A double-step grid-free method for sound source identification using deep learning

Deep learning is a machine learning method based on the deep neural network (DNN) which is widely used in various application fields. Compared with the model-based beamforming method, the sound source identification method using deep learning is very promising. The grid-free method is one of these deep learning methods in high accuray. However, it needs to predefine a certain number of sources in advance of localization and quantization. To solve the limitation of existing grid-free methods, a double-step grid-free (DSGF) method to identify unknown number of sound sources is proposed. The classification DNN to identify sources number is trained as the first-step and 6 regression DNNs to localize and quantify sound sources are trained as the second-step. In this work, the residual neural network (ResNet) is utilized as the prediction model. Conventional beamforming (CB) map is used as input, and the output is changed with different tasks. The classification accuracy on two kinds of datasets are analyzed, then the performance of localization and quantization with correct or wrong sources number are evaluated. The results show that the DNN models trained of different tasks are reliable. Besides, the comparison beween DSGF and CLEAN-SC methods reveals that the proposed method performs better in the close-range condition. The simulation and experimental results verify the feasibility of the DSGF method.

Unraveling the complexities of urban fluvial flood hydraulics through AI

As urbanization increases across the globe, urban flooding is an ever-pressing concern. Urban fluvial systems are highly complex, depending on a myriad of interacting variables. Numerous hydraulic models are available for analyzing urban flooding; however, meeting the demand of high spatial extension and finer discretization and solving the physics-based numerical equations are computationally expensive. Computational efforts increase drastically with an increase in model dimension and resolution, preventing current solutions from fully realizing the data revolution. In this research, we demonstrate the effectiveness of artificial intelligence (AI), in particular, machine learning (ML) methods including the emerging deep learning (DL) to quantify urban flooding considering the lower part of Darby Creek, PA, USA. Training datasets comprise multiple geographic and urban hydraulic features (e.g., coordinates, elevation, water depth, flooded locations, discharge, average slope, and the impervious area within the contributing region, downstream distance from stormwater outfalls and dams). ML Classifiers such as logistic regression (LR), decision tree (DT), support vector machine (SVM), and K-nearest neighbors (KNN) are used to identify the flooded locations. A Deep neural network (DNN)-based regression model is used to quantify the water depth. The values of the evaluation matrices indicate satisfactory performance both for the classifiers and DNN model (F-1 scores- 0.975, 0.991, 0.892, and 0.855 for binary classifiers; root mean squared error- 0.027 for DNN regression). In addition, the blocked K-folds Cross Validation (CV) of ML classifiers in detecting flooded locations showed satisfactory performance with the average accuracy of 0.899, which validates the models to generalize to the unseen area. This approach is a significant step towards resolving the complexities of urban fluvial flooding with a large multi-dimensional dataset in a highly computationally efficient manner.

Deep Neural Network Regression of MANO from a Single Image

In order to improve the precision and efficiency of estimating the pose and shape of the 3D human hand from a monocular RGB image and accelerate 3D reconstruction, combining the efficiency of deep neural networks and accuracy of traditional optimization, DeepMANO is proposed. First, a DNN model is used to extract sparse features from image, then using these features to regress MANO model parameters. The regressed parameters are then used to initialize the iterative optimization routine which fits MANO to 3D joints. Finally, the parameters obtained by the routine are employed to supervise the network. Two large hand motion datasets, FreiHand and ObMan, are utilized to train the network. The approach assumes a weak perspective model. Experimental results show that the hand pose error is reduced by 52% and mesh error is reduced by 59% compared with state-of-the-art (SOTA) methods. The running time is about three times faster than SOTA methods.