Deep Regression Research Articles

Exploring the potential of deep regression model for next-location prediction

Exploring the potential of deep regression model for next-location prediction

Progressive Frequency-Guided Depth Model with Adaptive Preprocessing for Casting Defect Detection

This article proposes a progressive frequency domain-guided depth model with adaptive preprocessing to solve the problem of defect detection with weak features based on X-ray images. In distinct intuitive surface defect detection tasks, non-destructive testing of castings using X-rays presents more complex and weak defect features, leading to lower accuracy and insufficient robustness on the part of current casting defect detection methods. To address these challenges, the proposed method establishes four specialized mechanisms to improve model accuracy. First, an adaptive image contrast enhancement method is proposed to enhance the features of defects in casting images to promote subsequent feature extraction and prediction. Second, a subtle clue mining module based on frequency domain attention is proposed to fully extract the discriminative features of casting defects. Third, a feature refinement module based on progressive learning is proposed to achieve a balance between feature resolution and semantic information. Finally, a refined deep regression supervision mechanism is designed to improve defect detection accuracy under strict intersection-to-union ratio standards. We established extensive ablation studies using casting defect images in GDXray, conducted detailed comparative experiments with other methods, and performed experiments to analyze the robustness of the resulting models. Compared with other X-ray defect detection methods, our framework achieves an average +4.6 AP. Compared to the baseline, our proposed refined deep regression supervision mechanism results in an improvement of 5.3 AP.

Deep energy-pressure regression for a thermodynamically consistent EOS model

In this paper, we aim to explore novel machine learning (ML) techniques to facilitate and accelerate the construction of universal equation-Of-State (EOS) models with a high accuracy while ensuring important thermodynamic consistency. When applying ML to fit a universal EOS model, there are two key requirements: (1) a high prediction accuracy to ensure precise estimation of relevant physics properties and (2) physical interpretability to support important physics-related downstream applications. We first identify a set of fundamental challenges from the accuracy perspective, including an extremely wide range of input/output space and highly sparse training data. We demonstrate that while a neural network (NN) model may fit the EOS data well, the black-box nature makes it difficult to provide physically interpretable results, leading to weak accountability of prediction results outside the training range and lack of guarantee to meet important thermodynamic consistency constraints. To this end, we propose a principled deep regression model that can be trained following a meta-learning style to predict the desired quantities with a high accuracy using scarce training data. We further introduce a uniquely designed kernel-based regularizer for accurate uncertainty quantification. An ensemble technique is leveraged to battle model overfitting with improved prediction stability. Auto-differentiation is conducted to verify that necessary thermodynamic consistency conditions are maintained. Our evaluation results show an excellent fit of the EOS table and the predicted values are ready to use for important physics-related tasks.

Simultaneously improving accuracy and computational cost under parametric constraints in materials property prediction tasks

Modern data mining techniques using machine learning (ML) and deep learning (DL) algorithms have been shown to excel in the regression-based task of materials property prediction using various materials representations. In an attempt to improve the predictive performance of the deep neural network model, researchers have tried to add more layers as well as develop new architectural components to create sophisticated and deep neural network models that can aid in the training process and improve the predictive ability of the final model. However, usually, these modifications require a lot of computational resources, thereby further increasing the already large model training time, which is often not feasible, thereby limiting usage for most researchers. In this paper, we study and propose a deep neural network framework for regression-based problems comprising of fully connected layers that can work with any numerical vector-based materials representations as model input. We present a novel deep regression neural network, iBRNet, with branched skip connections and multiple schedulers, which can reduce the number of parameters used to construct the model, improve the accuracy, and decrease the training time of the predictive model. We perform the model training using composition-based numerical vectors representing the elemental fractions of the respective materials and compare their performance against other traditional ML and several known DL architectures. Using multiple datasets with varying data sizes for training and testing, We show that the proposed iBRNet models outperform the state-of-the-art ML and DL models for all data sizes. We also show that the branched structure and usage of multiple schedulers lead to fewer parameters and faster model training time with better convergence than other neural networks. Scientific contribution: The combination of multiple callback functions in deep neural networks minimizes training time and maximizes accuracy in a controlled computational environment with parametric constraints for the task of materials property prediction.

Spectral encoder to extract the efficient features of Raman spectra for reliable and precise quantitative analysis

Raman spectroscopy has become a powerful analytical tool highly demanded in many applications such as microorganism sample analysis, food quality control, environmental science, and pharmaceutical analysis, owing to its non-invasiveness, simplicity, rapidity and ease of use. Among them, quantitative research using Raman spectroscopy is a crucial application field of spectral analysis. However, the entire process of quantitative modeling largely relies on the extraction of effective spectral features, particularly for measurements on complex samples or in environments with poor spectral signal quality. In this paper, we propose a method of utilizing a spectral encoder to extract effective spectral features, which can significantly enhance the reliability and precision of quantitative analysis. We built a latent encoded feature regression model; in the process of utilizing the autoencoder for reconstructing the spectrometer output, the latent feature obtained from the intermediate bottleneck layer is extracted. Then, these latent features are fed into a deep regression model for component concentration prediction. Through detailed ablation and comparative experiments, our proposed model demonstrates superior performance to common methods on single-component and multi-component mixture datasets, remarkably improving regression precision while without needing user-selected parameters and eliminating the interference of irrelevant and redundant information. Furthermore, in-depth analysis reveals that latent encoded feature possesses strong nonlinear feature representation capabilities, low computational costs, wide adaptability, and robustness against noise interference. This highlights its effectiveness in spectral regression tasks and indicates its potential in other application fields. Sufficient experimental results show that our proposed method provides a novel and effective feature extraction approach for spectral analysis, which is simple, suitable for various methods, and can meet the measurement needs of different real-world scenarios.

Alzheimer image registration using hybrid random forest and deep regression network algorithm

<span>Image registration involves superimposing images (two or more) of similar background obtained at various periods of time, at different angles, and/or with various detectors. Geometrical alignment of two scans, reference image as well as capture image. The current dissimilarity between images is because of distinct image conditions. Image registration is difficult step in image analysis works on change detection, image fusion as well as <br /> multi-channel images recovery to obtain concluded data from integration of different sources. In this analysis image registration using hybrid random forest (RF) and deep regression network algorithm for magnetic resonance imaging (MRI) applications is implemented. The Alzheimer’s disease neuroimaging initiative (ADNI) database provided by the dataset utilised in this implementation. From results it can observe that compared with individual random of forest, Hybrid RF and deep regression network algorithm improves the accuracy, precision and F1-score in effective way.</span>

Predicting Soil Quality Index with a Deep Regression Approach

ABSTRACT The detection, preservation and enhancement of soil quality is essential for ecosystem managers to offer the appropriate support after discovering factors that are linked to soil’s functional capability. In this study, soil quality indicators were combined to establish a soil quality index (SQI) by conducting an efficiency analysis for using Convolutional Neural Network (CNN), Gradient Boosting Methods (GBM) and Decision Tree (DT) approaches. For this purpose, soil’s intrinsic parameters, namely indicators, are utilized in case of fitting a deeper regression model. The experiments were carried out with analysis results of 545 soil samples obtained from previously studies conducted on different areas such as watershed, delta, pasture and farmland. The performance of each model is validated by the reported data. Compared to gradient boosting models, the proposed CNN model with mean absolute loss function achieves a lower error rate of 2.71 in Mean Absolute Percentage Error (MAPE) and best average correlation coefficient 0.90 in R2, which proves generalizability of CNN based regression. The findings show that CNNs may be utilized as an effective tool as giving a better prediction accuracy level and reliability in terms the establishing correlations and providing a conceptual framework for modeling of soil quality.

AI-driven improvement of monthly average rainfall forecasting in Mecca using grid search optimization for LSTM networks

ABSTRACT Predicting the average monthly rainfall in Mecca is crucial for sustainable development, resource management, and infrastructure protection in the region. This study aims to enhance the accuracy of long short-term memory (LSTM) deep regression models used for rainfall forecasting using an advanced grid-search-based hyperparameter optimization technique. The proposed model was trained and validated on a historical dataset of Mecca's monthly average rainfall. The model's performance improved by 5.0% post-optimization, reducing the root-mean-squared error (RMSE) from 0.1201 to 0.114. The results signify the value of grid search optimization in improving the LSTM model's accuracy, demonstrating its superiority over other common hyperparameter optimization techniques. The insights derived from this research provide valuable input for decision-makers in effectively managing water resources, mitigating environmental risks, and fostering regional development.

Lake level dynamics in the centre of the East European Plain during the Holocene

Based on the lithostratigraphic and geomorphological approach, the study of lake level fluctuations in Nero and Seliger over the past 15,000 years was conducted. Common features of level changes were revealed: deep regression at the end of the late glacial period and early Holocene, intensive rise in level in the early-middle Holocene, stabilization (with a slight upward trend) in the late Holocene. Similar features in level changes were revealed in the history of a number of lakes in Germany and Scandinavia. A connection was found between lake level fluctuations and the intensity of fluvial processes in the center of the East European Plain.

Integral Bone Age Regression

The paper takes a practical approach to estimating the bone age of children and young adults based on hand radiography, by constructing an end-to-end pipeline that involves bone segmentation, deep regression algorithms and model explainability. Unlike previous approaches reported in the literature for bone age regression, we employ a methodology coined as integral regression that, in conjunction with the popular pretrained ResNeSt-50 model, exploiting also features extracted in the segmentation step, establishes new State-Of-The-Art results on the RSNA data set. Our experiments demonstrate clear advantages in favour of the integral regression layer over the much more common linear regression layer in the deep learning framework, while the explainability module showcases the robustness of our approach.

Application of Interactive Multimedia System in Early Childhood Dance and Artistic Creativity Cultivation

Abstract This paper utilizes MPEG4 technology to design an interactive multimedia system and applies the system to instruct young children’s dance movements and techniques in their dance learning. The implementation process of the system is to use a multi-layer neural network to establish a coarse estimation method of dance posture, and then through the root node deep regression network to deeply estimate and analyze the 2D posture of young children in the process of dance. Finally, the multi-feature fusion module is used to fuse the skeletal points and key point features of the body to analyze the 3D posture of young children during the dance. After testing the recognition effect of the system and analyzing the application effect of the interactive multimedia system, the overall movement completion time of the spiral dance in the experimental group was significantly reduced, and the time required for the rotational movement in the dance was also reduced from the initial 1.22 s to 0.899 s. There were significant differences in the levels of artistic creativity in all dimensions of artistic creativity before and after the application of the interactive multimedia system (P<0.05). This paper provides a reference method for improving the ability of young children to dance and cultivate artistic creativity. Also, it lays the foundation for the educational application of interactive multimedia systems.

Leveraging Unsupervised Data and Domain Adaptation for Deep Regression in Low-Cost Sensor Calibration.

Air quality monitoring is becoming an essential task with rising awareness about air quality. Low-cost air quality sensors are easy to deploy but are not as reliable as the costly and bulky reference monitors. The low-quality sensors can be calibrated against the reference monitors with the help of deep learning. In this article, we translate the task of sensor calibration into a semi-supervised domain adaptation problem and propose a novel solution for the same. The problem is challenging, because it is a regression problem with a covariate shift and label gap. We use histogram loss instead of mean-squared or mean absolute error (MAE), which is commonly used for regression, and find it useful against covariate shift. To handle the label gap, we propose the weighting of samples for adversarial entropy optimization. In experimental evaluations, the proposed scheme outperforms many competitive baselines, which are based on semi-supervised and supervised domain adaptation, in terms of R2 score and MAE. Ablation studies show the relevance of each proposed component in the entire scheme.

Corrections to “ERDeR: The Combination of Statistical Shrinkage Methods and Ensemble Approaches to Improve the Performance of Deep Regression”

Corrections to “ERDeR: The Combination of Statistical Shrinkage Methods and Ensemble Approaches to Improve the Performance of Deep Regression”

ERDeR: The Combination of Statistical Shrinkage Methods and Ensemble Approaches to Improve the Performance of Deep Regression

ERDeR: The Combination of Statistical Shrinkage Methods and Ensemble Approaches to Improve the Performance of Deep Regression

Deep Regression Modeling for Imbalanced and Incomplete Time-Series Data

Deep Regression Modeling for Imbalanced and Incomplete Time-Series Data

Spatial–Spectral–Temporal Deep Regression Model With Convolutional Long Short-Term Memory and Transformer for the Large-Area Mapping of Mangrove Canopy Height by Using Sentinel-1 and Sentinel-2 Data

Spatial–Spectral–Temporal Deep Regression Model With Convolutional Long Short-Term Memory and Transformer for the Large-Area Mapping of Mangrove Canopy Height by Using Sentinel-1 and Sentinel-2 Data

Limited agricultural spectral dataset expansion based on generative adversarial networks

With the rise of deep learning, the combination of spectroscopy analysis techniques and deep learning methods has been extensively utilized in the field of agriculture, such as the detection of crop diseases, soil analysis, and crop quality assessment. Unfortunately, it is often difficult to obtain enough spectral samples that can be used for deep learning because of environmental constraints, equipment limitations, and labour costs. To address this issue, we propose a spectral sample augmentation technique based on the K-condition boundary equilibrium generative adversarial networks (KC-BEGAN). First, a stable boundary equilibrium generative adversarial network (BEGAN) model is constructed, and the KC-BEGAN model is built by incorporating chemical property labels, multiscale gradient information, and the k-nearest neighbour algorithm. The goal is to enrich complete spectral samples with chemical properties. Second, we compare the differences between generated samples and real samples using methods such as t-distributed stochastic neighbour embedding, Mahalanobis distance, F test, and maximum mean discrepancy. Following data augmentation by the KC-BEGAN model, the R2 on the test set for traditional regression models (PLSR, SVR, RR, PCR) and deep regression models (Inception-ResNet, Inception, 1D-CNN) improved by 4.9%, 0.4%, 2.5%, 4.4%, 3.6%, 4.4%, and 6.8% respectively. Furthermore, this study replaces the GAN module in the KC-BEGAN model with Diffusion-GAN and conducts experiments following the same procedure to evaluate the feasibility of other GAN models in augmenting labelled near-infrared spectral samples. The research results indicate that the spectral samples generated by the KC-BEGAN model are reliable and can meet the expansion requirements of small-scale spectral sample sets. Simultaneously, replacing the GAN module in the KC-BEGAN model with other GAN models is also feasible, suggesting that the continuous label marking method proposed in this study is indeed effective.

Deep Regression Network with Prediction Confidence in Time Series Application for Asset Health Estimation

Many works have been focused in developing detection, monitoring and prediction routines for asset health estimation system. Classic machine learning based models benefit from quality of physics-informed features available from domain knowledge. This, however, can be labor intensive and is limited by quality of features developed through available knowledge. Deep learning based approach, if successful, can alleviate this laborious step. On the other hand, users often need to decide whether or not to trust an algorithmic prediction while the true error in the prediction is unknown. In this work, we propose a deep learning based regression network that output both prediction value and confidence score for asset health estimation in short intermittent transients time series application. In the experimental study, we show that our model has low prediction error given short intermittent transients multivariate time series as input. Furthermore, our model also provides a confidence score for each prediction that is highly negatively correlated with true prediction error. Experiments show that by setting an acceptance threshold on confidence score, our model can reach an averaged improvement of 20% on the prediction quality with 90% coverage.

Deep Regression Network for Single-Image Super-Resolution Based on Down- and Upsampling with RCA Blocks

Deep Regression Network for Single-Image Super-Resolution Based on Down- and Upsampling with RCA Blocks

A secure routing protocol using hybrid deep regression based trust evaluation and clustering for mobile ad-hoc network

A secure routing protocol using hybrid deep regression based trust evaluation and clustering for mobile ad-hoc network