Deep Learning Regression Model Research Articles

Predicting the stacking fault energy in FCC high-entropy alloys based on data-driven machine learning

The properties of high-entropy alloys (HEAs) depend primarily on the composition and content of elements. However, getting the optimal composition of alloying elements through the traditional "trial and error" method is challenging, especially for non-equiatomic HEAs with a wide range of composition space. In this study, based on the knowledge that stacking fault energy (SFE) is the most crucial intrinsic property to determine the deformation mechanism and to optimize the mechanical properties of FCC HEAs, classical machine learning classification models including support vector classification (SVC) and random forest (RF), and deep learning regression model (Back Propagation Neural Network) were established to predict the stacking fault energy of Co–Cr–Fe–Mn–Ni–V–Al high-entropy alloys. These models can obtain the SFE data of any atomic ratio composition of the FCC structured Co–Cr–Fe–Mn–Ni–V–Al high-entropy alloy quickly and accurately. The high accuracy of these models indicates that using the compositions as features to predict stacking fault energy is feasible. Meanwhile, the monotonic relationship between alloying elements and SFE makes it possible to change the SFE of high-entropy alloy by fine-tuning the composition to realize the control of material deformation mechanism and mechanical properties. Component-based machine learning models provide a new method for rapidly discovering high-entropy alloys with exceptional strength and flexibility.

Air temperature estimation over winter wheat fields by integrating machine learning and remote sensing techniques

Air temperature (Ta) is significant to numerous Earth's surface and agricultural processes. For agricultural fields (e.g., winter wheat), Ta plays an important role in crop growth monitoring, crop yield estimation, and agricultural risk assessment. In this study, a revised Ta estimation model is proposed by integrating the temperature-vegetation index (TVX) method with multiple regression models. Daily Ta from 2013 to 2021 over winter wheat fields in Henan province (the Top 1 winter wheat-producing province in China) was estimated. The revised models utilized the TVX method over regions and periods featuring extensive vegetation coverage, especially during winter wheat growing seasons, then employed three regression models including multivariate linear regression (MLR) model, random forest (RF) machine learning regression model, and deep belief network (DBN) deep learning regression model. The satellite remote sensing observations, meteorological in-situ Ta measurements, and model assimilations were integrated into the revised models to retrieve Ta. Results obtained through methods proposed in this study were calibrated and validated by in-situ Ta measurements from weather networks. The statistical results reveal that the revised methods can produce Ta estimation over winter wheat-planting areas with higher accuracy compared with traditional regression methods, with R2, MAE, and RMSE ranges from 0.915 to 0.971, 1.204 to 2.097 °C, and 1.623 to 2.911 °C, respectively. Integrating the TVX with RF has the best performance with the highest R2 (0.971), lowest MAE (1.204 °C), and lowest RMSE (1.623 °C) over the winter wheat planting area.

Deep Learning for Automated Measurement of Patellofemoral Anatomic Landmarks.

Patellofemoral anatomy has not been well characterized. Applying deep learning to automatically measure knee anatomy can provide a better understanding of anatomy, which can be a key factor in improving outcomes. 483 total patients with knee CT imaging (April 2017-May 2022) from 6 centers were selected from a cohort scheduled for knee arthroplasty and a cohort with healthy knee anatomy. A total of 7 patellofemoral landmarks were annotated on 14,652 images and approved by a senior musculoskeletal radiologist. A two-stage deep learning model was trained to predict landmark coordinates using a modified ResNet50 architecture initialized with self-supervised learning pretrained weights on RadImageNet. Landmark predictions were evaluated with mean absolute error, and derived patellofemoral measurements were analyzed with Bland-Altman plots. Statistical significance of measurements was assessed by paired t-tests. Mean absolute error between predicted and ground truth landmark coordinates was 0.20/0.26 cm in the healthy/arthroplasty cohort. Four knee parameters were calculated, including transepicondylar axis length, transepicondylar-posterior femur axis angle, trochlear medial asymmetry, and sulcus angle. There were no statistically significant parameter differences (p > 0.05) between predicted and ground truth measurements in both cohorts, except for the healthy cohort sulcus angle. Our model accurately identifies key trochlear landmarks with ~0.20-0.26 cm accuracy and produces human-comparable measurements on both healthy and pathological knees. This work represents the first deep learning regression model for automated patellofemoral annotation trained on both physiologic and pathologic CT imaging at this scale. This novel model can enhance our ability to analyze the anatomy of the patellofemoral compartment at scale.

A Deep Learning Regression Model for Photonic Crystal Fiber Sensor with XAI Feature Selection and Analysis.

A Deep Learning Multi-output regression model is employed to correctly model the relationships between optical design parameters of an asymmetric Twin Elliptical Core Photonic Crystal Fiber (TEC-PCF) and its sensing performances. TEC-PCF acts as a biosensor to detect the blood glucose level taking hemoglobin components into account. Since asymmetric TEC-PCF uses a dual elliptical core, four super modes have to be evaluated to analyze the sensing performance in terms of effective index difference, transmission spectrum, coupling length, and sensor sensitivity. The dataset used in this work is of the optical design parameters of the sensor and Finite Element Method (FEM) results with effective indices of four super modes obtained from the COMSOL Multiphysics by varying hemoglobin concentration to 120 g/L, 140 g/L, and 160 g/L. Gretel.ai's free open-source synthetic data library is used to augment the dataset to make the training more efficient. Explainable AI (XAI) feature analysis using Shapley Additive Explanations (SHAP) framework is used for two purposes: feature selection and to know the feature's effect on prediction. The former led to the development of an optimal model with much fewer computational demands and the latter made the model interpretable. The proposed model can predict the accurate super modes when given input specifications with wavelength ranging from 1.27-1.5 μm and for various glucose concentrations under the influence of hemoglobin much faster compared to the other numerical simulations which are computationally expensive. Computation time taken by the proposed Artificial Neural Network (ANN) model, the proposed model with XAI and FEM is also being compared.

FundusQ-Net: A regression quality assessment deep learning algorithm for fundus images quality grading

Objective: Ophthalmological pathologies such as glaucoma, diabetic retinopathy and age-related macular degeneration are major causes of blindness and vision impairment. There is a need for novel decision support tools that can simplify and speed up the diagnosis of these pathologies. A key step in this process is to automatically estimate the quality of the fundus images to make sure these are interpretable by a human operator or a machine learning model. We present a novel fundus image quality scale and deep learning (DL) model that can estimate fundus image quality relative to this new scale.Methods: A total of 1245 images were graded for quality by two ophthalmologists within the range 1–10, with a resolution of 0.5. A DL regression model was trained for fundus image quality assessment. The architecture used was Inception-V3. The model was developed using a total of 89,947 images from 6 databases, of which 1245 were labeled by the specialists and the remaining 88,702 images were used for pre-training and semi-supervised learning. The final DL model was evaluated on an internal test set (n=209) as well as an external test set (n=194).Results: The final DL model, denoted FundusQ-Net, achieved a mean absolute error of 0.61 (0.54–0.68) on the internal test set. When evaluated as a binary classification model on the public DRIMDB database as an external test set the model obtained an accuracy of 99%.Significance: the proposed algorithm provides a new robust tool for automated quality grading of fundus images.

Deep image compression using scene text quality assessment

Image compression is a fundamental technology for Internet communication engineering. However, a high compression rate with general methods may degrade images, resulting in unreadable texts. In this paper, we propose an image compression method for maintaining text quality. We developed a scene text image quality assessment model to assess text quality in compressed images. The assessment model iteratively searches for the best-compressed image holding high-quality text. Objective and subjective results showed that the proposed method was superior to existing methods. Furthermore, the proposed assessment model outperformed other deep-learning regression models.

Forecasting ENSO using convolutional LSTM network with improved attention mechanism and models recombined by genetic algorithm in CMIP5/6

El Niño-Southern Oscillation (ENSO) has a profound impact on global climate, and the ability to forecast it effectively over the long term is essential. In recent years, deep learning methods have demonstrated superior prediction outcomes compared to conventional numerical models. However, due to limited observational data, most of these deep learning methods learn information from simulation data derived from physical models, and ensuring the quality of such simulation data can prove challenging. As a result, the models in CMIP5/6 were recombined using genetic algorithms (GAs) to create our training dataset. A deep learning model was then used to learn features from the output of these combined CMIP models so that these physical numerical models can complement each other. To address the issue of inadequate spatiotemporal feature extraction present in many deep learning methods, we devised an ENSO deep learning regression model. An improved self-attention mechanism was introduced into the convolutional LSTM (long short-term memory) network to enable our model to better extract local and global spatiotemporal features. With our method and model, we were able to obtain less erroneous forecast results, and our method surpassed other state-of-the-art deep learning methods in terms of its capability for long-term forecasting.

Robust water level measurement method based on computer vision

Compared with traditional methods based on non-visual devices, the water level measurement method based on computer vision has the advantages of low equipment installation and maintenance cost, convenient visualization, and so on. However, the complex condition of the monitoring site, such as different weather, lighting, water quality, staff gauge appearance, camera imaging, etc., pose significant challenges to the existing computer vision method, which can cope with specific situations but often cannot be universal. We propose a robust water level measurement method to overcome this problem. Firstly, we introduce a mapping method, which preprocesses the original image's extended region of interest (EROI) to obtain the transformed image. The region of interest (ROI) here refers to the staff gauge region, and the “extended” means taking advantage of the contextual information. Secondly, we construct a high-resolution representation deep learning regression model for water line positioning in the transformed image. Then, we trained the model with a robust data augmentation method consisting of two submethods: appearance-based data augmentation (ADA) and random extension in the direction (RED). Finally, we did a series of experiments and analyses on the widely distributed test dataset we built with two groups of criteria to evaluate the results, which are normalized since the data set contains samples of different scales. The result of this study is significantly improved compared with other computer vision methods. It shows that this study has robustness in various environments and has vast promotion value and application potential.

A machine learning supported sinogram interpolation method for X-ray computed tomography

The time of an X-ray Computed Tomography (XCT) measurement is directly affected by the number of acquired X-ray projections. In general, a large number of X-ray projections, hence a large acquisition time is required to obtain a qualitative reconstruction and successive feature analysis. To expand the applicability of XCT towards low-end parts we propose a novel sinogram interpolation method that incorporates the object rotation. The method combines a forward projection model of the XCT system and a deep learning regression model to generate intermediate X-ray projections for XCT scans with a reduced number of projections. Thereby, the accompanying reconstruction artefacts can be reduced while preserving a lower acquisition time. The method is validated on simulated projections of the 3D Shepp-Logan phantom and simulated projections of real 3D printed components using the ASTRA-toolbox. Within the reconstructions, less streaking artefacts are observed and an increased contrast between different features is obtained without introducing rotational artefacts.

Does inclusion of GARCH variance in deep learning models improve financial contagion prediction?

Financial contagions have traditionally been studied using GARCH models to explain the volatility, whereas regression models have been used to predict the stock index values based on different underlying assets. Past research has shown that deep learning regression models have demonstrated superiority over traditional statistical models. This paper creates a hybrid prediction model by using the outputs of GARCH models as an input variable in deep learning regression models. It has been observed that hybrid models provide better results as compared to deep learning models. Such improved prediction shall provide portfolio managers an additional tool to determine the investment strategy.

Grid Distribution Fault Occurrence and Remedial Measures Prediction/Forecasting through Different Deep Learning Neural Networks by Using Real Time Data from Tabuk City Power Grid

Modern societies need a constant and stable electrical supply. After relying primarily on formal mathematical modeling from operations research, control theory, and numerical analysis, power systems analysis has changed its attention toward AI prediction/forecasting tools. AI techniques have helped fix power system issues in generation, transmission, distribution, scheduling and forecasting, etc. These strategies may assist today’s large power systems which have added more interconnections to meet growing load demands. They make it simple for them to do difficult duties. Identification of problems and problem management have always necessitated the use of labor. These operations are made more sophisticated and data-intensive due to the variety and growth of the networks involved. In light of all of this, the automation of network administration is absolutely necessary. AI has the potential to improve the problem-solving and deductive reasoning approaches used in fault management. This study implements a variety of artificial intelligence and deep learning approaches in order to foresee and predict the corrective measures that will be conducted in response to faults that occur inside the power distribution network of the Grid station in Tabuk city with regard to users. The Tabuk grid station is the source of the data that was gathered for this purpose; it includes a list of defects categorization, actions and remedies that were implemented to overcome these faults, as well as the number of regular and VIP users from 2017 to 2022. Deep learning, the most advanced method of learning used by artificial intelligence, is continuing to make significant strides in a variety of domain areas, including prediction. This study found that the main predictors of remedial measures against the fault occurring in the power systems are the number of customers affected and the actual cause of the fault. Consequently, the deep learning regression model, i.e., Gated Recurrent Unit (GRU), achieved the best performance among the three, which yielded an accuracy of 92.13%, mean absolute error (MAE) loss of 0.37%, and root mean square error (RMSE) loss of 0.39% while the simple RNN model’s performance is not up to the mark with an accuracy of 89.21%, mean absolute error (MAE) loss of 0.45% and root mean square error (RMSE) loss of 0.34%. Significance of the research is to provide the maximum benefit to the customers and the company by using different AI techniques.

Sustainable Smart University: A Short-Term Deep Learning Framework for Energy Consumption Forecast

A smart city should ideally be environmentally friendly and sustainable, and energy management is one technique to monitor sustainable use. Similarly, this notion might be applied in an urban form, such as the sort of city in which a university would be located. This research analyzes the possibility for a university to enhance energy management by permitting the adoption of a variety of intelligent technologies that increase the energy sustainability of a city's infrastructure and the effectiveness of its operations. In the first module of the proposed system, we place significant emphasis on the data capabilities necessary to create energy statistics for each of its various buildings. In the second phase of the technique, we employ the collected data to conduct a data analysis of the energy behavior inside micro-cities, from which we derive characteristics. In the third module, we develop baseline regressors to assess the varying degrees of efficacy of the proposed model. Last, we describe a way for developing an energy prediction model using a deep learning regression model to solve the problem of short-term energy consumption forecasting. The performance metric results show that the suggested deep learning model increases performance prediction compared to other traditional regression techniques. The proposed model has superior RMSE, MAE and R squared results compared to alternative regression models.

Machine Learning for Short-Term Load Forecasting in Smart Grids

A smart grid is the future vision of power systems that will be enabled by artificial intelligence (AI), big data, and the Internet of things (IoT), where digitalization is at the core of the energy sector transformation. However, smart grids require that energy managers become more concerned about the reliability and security of power systems. Therefore, energy planners use various methods and technologies to support the sustainable expansion of power systems, such as electricity demand forecasting models, stochastic optimization, robust optimization, and simulation. Electricity forecasting plays a vital role in supporting the reliable transitioning of power systems. This paper deals with short-term load forecasting (STLF), which has become an active area of research over the last few years, with a handful of studies. STLF deals with predicting demand one hour to 24 h in advance. We extensively experimented with several methodologies from machine learning and a complex case study in Panama. Deep learning is a more advanced learning paradigm in the machine learning field that continues to have significant breakthroughs in domain areas such as electricity forecasting, object detection, speech recognition, etc. We identified that the main predictors of electricity demand in the short term: the previous week’s load, the previous day’s load, and temperature. We found that the deep learning regression model achieved the best performance, which yielded an R squared (R2) of 0.93 and a mean absolute percentage error (MAPE) of 2.9%, while the AdaBoost model obtained the worst performance with an R2 of 0.75 and MAPE of 5.70%.

A document analysis deep learning regression model for initial coin offerings success prediction

Initial coin offerings (ICOs) provide an early-stage financing method for blockchain-based ventures. During the ICO process, whitepapers are important not only as promotional material through which ventures can demonstrate the technical and financial project details but also as references for investors. Persuasion theory and the related literature suggest that the presentation and order of information have a significant impact on the attitude of the audience. Therefore, in addition to projects’ metadata features, we construct a document analysis deep regression model (DADRM) to innovatively extract deep text and layout features from whitepapers. Based on a real-life dataset, we conduct a comparative study to assess the effectiveness of the proposed framework in predicting ICO success in terms of the funding amount. The empirical results show that our model that both extracts text content and retains the original 2D structure of the document can significantly reduce prediction error. Based on our proposed model, both ICO platforms and investors can prejudge the funding amount of cryptocurrency projects and mitigate information asymmetry. Additionally, this study demonstrates that both what is written in the business document and how the document is presented affect investor decisions.

Portable Deep Learning-Driven Ion-Sensitive Field-Effect Transistor Scheme for Measurement of Carbaryl Pesticide.

This research proposes a multiple-input deep learning-driven ion-sensitive field-effect transistor (ISFET) scheme to predict the concentrations of carbaryl pesticide. In the study, the carbaryl concentrations are varied between 1 × 10−7–1 × 10−3 M, and the temperatures of solutions between 20–35 °C. To validate the multiple-input deep learning regression model, the proposed ISFET scheme is deployed onsite (a field test) to measure pesticide concentrations in the carbaryl-spiked vegetable extract. The advantage of this research lies in the use of a deep learning algorithm with an ISFET sensor to effectively predict the pesticide concentrations, in addition to improving the prediction accuracy. The results demonstrate the very high predictive ability of the proposed ISFET scheme, given an MSE, MAE, and R2 of 0.007%, 0.016%, and 0.992, respectively. The proposed multiple-input deep learning regression model with signal compensation is applicable to a wide range of solution temperatures which is convenient for onsite measurement. Essentially, the proposed multiple-input deep learning regression model could be adopted as an effective alternative to the conventional statistics-based regression to predict pesticide concentrations.

Deep Learning and Hyperspectral Images Based Tomato Soluble Solids Content and Firmness Estimation

Cherry tomato (Solanum lycopersicum) is popular with consumers over the world due to its special flavor. Soluble solids content (SSC) and firmness are two key metrics for evaluating the product qualities. In this work, we develop non-destructive testing techniques for SSC and fruit firmness based on hyperspectral images and the corresponding deep learning regression model. Hyperspectral reflectance images of over 200 tomato fruits are derived with the spectrum ranging from 400 to 1,000 nm. The acquired hyperspectral images are corrected and the spectral information are extracted. A novel one-dimensional (1D) convolutional ResNet (Con1dResNet) based regression model is proposed and compared with the state of art techniques. Experimental results show that, with a relatively large number of samples our technique is 26.4% better than state of art technique for SSC and 33.7% for firmness. The results of this study indicate the application potential of hyperspectral imaging technique in the SSC and firmness detection, which provides a new option for non-destructive testing of cherry tomato fruit quality in the future.

Automated estimation of total lung volume using chest radiographs and deep learning.

BackgroundTotal lung volume is an important quantitative biomarker and is used for the assessment of restrictive lung diseases.PurposeIn this study, we investigate the performance of several deep‐learning approaches for automated measurement of total lung volume from chest radiographs.MethodsAbout 7621 posteroanterior and lateral view chest radiographs (CXR) were collected from patients with chest CT available. Similarly, 928 CXR studies were chosen from patients with pulmonary function test (PFT) results. The reference total lung volume was calculated from lung segmentation on CT or PFT data, respectively. This dataset was used to train deep‐learning architectures to predict total lung volume from chest radiographs. The experiments were constructed in a stepwise fashion with increasing complexity to demonstrate the effect of training with CT‐derived labels only and the sources of error. The optimal models were tested on 291 CXR studies with reference lung volume obtained from PFT. Mean absolute error (MAE), mean absolute percentage error (MAPE), and Pearson correlation coefficient (Pearson's r) were computed.ResultsThe optimal deep‐learning regression model showed an MAE of 408 ml and an MAPE of 8.1% using both frontal and lateral chest radiographs as input. The predictions were highly correlated with the reference standard (Pearson's r = 0.92). CT‐derived labels were useful for pretraining but the optimal performance was obtained by fine‐tuning the network with PFT‐derived labels.ConclusionWe demonstrate, for the first time, that state‐of‐the‐art deep‐learning solutions can accurately measure total lung volume from plain chest radiographs. The proposed model is made publicly available and can be used to obtain total lung volume from routinely acquired chest radiographs at no additional cost. This deep‐learning system can be a useful tool to identify trends over time in patients referred regularly for chest X‐ray.

Deep learning based optical curvature sensor through specklegram detection of multimode fiber

An optical fiber curvature sensor based on the detection of specklegrams from the facet of multimode fiber (MMF) is realized by using a deep learning regression model. Since the specklegrams result from mode interference in the MMF, they can be used to characterize the status of the MMF. In the experiment, light from a semiconductor laser source was injected into a section of 10-cm-long step-indexed MMF with a core diameter of 50 μm. A large number of specklegrams at the facet of the MMF were automatically detected when different curvatures were introduced for the MMF by a controllable moving translation stage. The output specklegrams of the MMF under different curvatures were then fed into a convolutional neural network (CNN) for training, validation and testing. Experimental results demonstrate that after the CNN was well-trained by the specklegrams with specified curvatures, the CNN can effectively predict the curvature from any of the specklegrams obtained from the MMF with a curvature in the trained range. In the experiment, the CNN trained by specklegrams with 20 curvatures successfully predicted the curvatures corresponding to the specklegrams from the MMF under 57 curvatures in the range of 1.55–6.93 m−1. The prediction error for 94.7% of the specklegrams is within an error range of ±0.3m-1, confirming the feasibility of curvature sensing based on the analysis of specklegrams by CNN.

PointLoc: Deep Pose Regressor for LiDAR Point Cloud Localization

In this paper, we present a novel end-to-end learning-based LiDAR sensor relocalization framework, termed PointLoc, which infers 6-DoF poses directly using only a single point cloud as input. Compared to visual sensor-based relocalization, LiDAR sensors can provide rich and robust geometric information about a scene. However, point clouds of LiDAR sensors are unordered and unstructured making it difficult to apply traditional deep learning regression models for this task. We address this issue by proposing a novel PointNet-style architecture with self-attention to efficiently estimate 6-DoF poses from 360° LiDAR sensor frames. Extensive experiments on recently released challenging Oxford Radar RobotCar dataset and real-world robot experiments demonstrate that the proposed method can achieve accurate relocalization performance.

A data-driven deep learning model of radiative heat transfer in dense granular systems

Radiative heat transfer has attracted increasing research interest in various dense granular systems, including large-scale nuclear pebble-bed reactors, fusion breeder blankets, packed beds, metal powder beds and granular lunar regolith simulants. The optimised ray tracing method (ORTM) with a Sobol quasi-random sequence is proposed to calculate the obstructed view factor between spheres, and it converges much faster than that with the traditional pseudorandom number sampling method. Based on discrete particle packing of nuclear pebble bed HTR-PM and random particle generation, a large obstructed view factor dataset of thermal radiation is generated by parallel CUDA computing for training a deep learning regression model to discuss the packing-property linkage for radiative heat transfer in the large-scale particle bed. Compared with other approaches, the deep residual neural network (ResNet) with the swish activation function applied in this paper is found to be the best neural network architecture for view factor prediction without vanishing gradients or model degradation problems during training. The testing error of the trained deep learning model is <1.0 × 10-4 for over 99.96% view-factor cases in nuclear pebble beds, and it has proven to be 7.3 × 105 times faster than the traditional numerical method in view-factor calculations. This makes it feasible to explore the local and macroscopic thermal radiation behaviours in large-scale pebble beds filled with up to 1.0 × 107 spheres.