Machine Learning Regression Models Research Articles

Machine learning to predict the relationship between Vibrio spp. concentrations in seawater and oysters and prevalent environmental conditions

Vibrio parahaemolyticus and Vibrio vulnificus are bacteria with a significant public health impact. Identifying factors impacting their presence and concentrations in food sources could enable the identification of significant risk factors and prevent incidences of foodborne illness. In recent years, machine learning has shown promise in modeling microbial presence based on prevalent external and internal variables, such as environmental variables and gene presence/absence, respectively, particularly with the generation and availability of large amounts and diverse sources of data. Such analyses can prove useful in predicting microbial behavior in food systems, particularly under the influence of the constant changes in environmental variables. In this study, we tested the efficacy of six machine learning regression models (random forest, support vector machine, elastic net, neural network, k-nearest neighbors, and extreme gradient boosting) in predicting the relationship between environmental variables and total and pathogenic V. parahaemolyticus and V. vulnificus concentrations in seawater and oysters. In general, environmental variables were found to be reliable predictors of total and pathogenic V. parahaemolyticus and V. vulnificus concentrations in seawater, and pathogenic V. parahaemolyticus in oysters (Acceptable Prediction Zone >70 %) when analyzed using our machine learning models. SHapley Additive exPlanations, which was used to identify variables influencing Vibrio concentrations, identified chlorophyll a content, seawater salinity, seawater temperature, and turbidity as influential variables. It is important to note that different strains were differentially impacted by the same environmental variable, indicating the need for further research to study the causes and potential mechanisms of these variations. In conclusion, environmental variables could be important predictors of Vibrio growth and behavior in seafood. Moreover, the models developed in this study could prove invaluable in assessing and managing the risks associated with V. parahaemolyticus and V. vulnificus, particularly in the face of a changing environment.

Analytical results for uncertainty propagation through trained machine learning regression models

Machine learning (ML) models are increasingly being used in metrology applications. However, for ML models to be credible in a metrology context they should be accompanied by principled uncertainty quantification. This paper addresses the challenge of uncertainty propagation through trained/fixed ML regression models. Analytical expressions for the mean and variance of the model output are obtained/presented for certain input data distributions and for a variety of ML models. Our results cover several popular ML models including linear regression, penalised linear regression, kernel ridge regression, Gaussian Processes (GPs), support vector machines (SVMs) and relevance vector machines (RVMs). We present numerical experiments in which we validate our methods and compare them with a Monte Carlo approach from a computational efficiency point of view. We also illustrate our methods in the context of a metrology application, namely modelling the state-of-health of lithium-ion cells based upon Electrical Impedance Spectroscopy (EIS) data.

Comparison of methods to aggregate climate data to predict crop yield: an application to soybean

High-dimensional climate data collected on a daily, monthly, or seasonal time step are now commonly used to predict crop yields worldwide with standard statistical models or machine learning models. Since the use of all available individual climate variables generally leads to calculation problems, over-fitting, and over-parameterization, it is necessary to aggregate the climate data used as predictors. However, there is no consensus on the best way to perform this task, and little is known about the impacts of the type of aggregation method used and of the temporal resolution of weather data on model performances. Based on historical data from 1981 to 2016 of soybean yield and climate on 3447 sites worldwide, this study compares different temporal resolutions (daily, monthly, or seasonal) and dimension reduction techniques (principal component analysis (PCA), partial least square regression, and their functional counterparts) to aggregate climate data used as inputs of machine learning and linear regression (LR) models predicting yields. Results showed that random forest models outperformed and were less sensitive to climate aggregation methods than LRs when predicting soybean yields. With our models, the use of daily climate data did not improve predictive performance compared to monthly data. Models based on PCA or averages of monthly data showed better predictive performance compared to those relying on more sophisticated dimension reduction techniques. By highlighting the high sensitivity of projected impact of climate on crop yields to the temporal resolution and aggregation of climate input data, this study reveals that model performances can be improved by choosing the most appropriate time resolution and aggregation techniques. Practical recommendations are formulated in this article based on our results.

Machine learning optimization of operating parameters to achieve high power density and efficiency of polymer electrolyte membrane fuel cell

The performance of PEMFCs is significantly influenced by operating conditions, including temperature, pressure, and flow rates. To optimize the complicated nonlinearity of operational conditions such as anode pressure (AP), cathode pressure (CP), cell temperature (Temp), anode flow rate (HFL), cathode stoichiometry (CS), and current density (CD) in experiments, this study employs five machine learning (ML) regression models: Artificial Neural Network, Decision Tree, Random Forest, Gradient Boosting, and Extreme Gradient Boosting to predict key performance indicators: voltage, power density, and efficiency, thereby optimizing the fuel cell’s operational parameters. Among these, the Gradient Boosting regressor demonstrated superior performance when validated with experimental data and was subsequently employed to fine-tune the operating parameters. Leveraging the prediction model, the performance of PEMFC in terms of power density and efficiency was further assessed by varying AP, CP and CS. The optimal conditions were identified with AP, CP at 2 bar, and CS at 2.50–2.75.

Workforce forecasting for state transportation agencies: A machine learning approach

A decline in the number of construction engineers and inspectors at State Transportation Agencies (STAs) to manage the ever-increasing lane miles has emphasized the importance of workforce planning in these agencies. Forecasting workforce requirements is crucial for effective planning in any industry or agency. This study developed machine learning models to estimate the person-hour requirements of STAs at the project level. The Arkansas Department of Transportation (ARDOT) was used as a case study, using its employee and project details data between 2012 and 2021. Machine learning regression models ranging from linear, tree ensembles, kernel-based, and neural network-based models were developed. These models were compared based on the accuracy of their predictions, the time taken for training the models and their prediction time. Predictions were tested based on the K-fold cross validation technique. The results indicated a high performance from the random forest regression model, a tree ensemble with bagging, which recorded a mean R-squared value of 0.91. Other machine learning models such as an ensemble neural networks model and the linear models also proved to be fit for the problem, attaining R squared value as high as 0.80 and 0.78 respectively. These findings underscore the capability of machine learning models to provide more accurate workforce demand forecasts for STAs and the construction industry. This enhanced accuracy in workforce planning will contribute to improved resource allocation and management.

Slurry Erosion Resistance, Morphology, and Machine Learning Modeling of Plasma-Sprayed Si3N4+TiC+VC and CrNi based Ceramic Coatings

This study presents a comprehensive analysis of the morphology, mechanical properties, erosion resistance, and machine learning modeling of Si3N4+TiC+VC and CrNi based ceramic coatings deposited via plasma spraying. The cross-sectional morphology analysis revealed good mechanical interlocking between the coatings and the substrate, with pores predominantly observed in the Si3N4+TiC+VC coating. The coating thicknesses were measured as follows: 260 μm for Si3N4+TiC+VC, 243 μm for Si3N4+TiC+VC blended with CrNi, and 232 μm for pure CrNi coating. Mechanical characterization showed that microhardness values ranged from 1933 HV for Si3N4+TiC+VC coating to 499 HV for pure CrNi coatings. Slurry erosion tests demonstrated that pure Si3N4+TiC+VC coating exhibited superior erosion resistance at 90º impingement angle compared to other coatings. Additionally, a rigorous evaluation of over 20 machine learning regression models revealed that Gaussian Process Regressors and Artificial Neural Network (ANN) with a layer size of 20, displayed satisfactory performance in modeling mass loss prediction. The erosion mechanisms observed in the coatings provided valuable insights into their response to slurry erosion, highlighting the influence of coating composition and impingement angles. The findings underscore the potential of Si3N4+TiC+VC ceramic coatings to enhance the erosion resistance of stainless steel, offering promising applications in various industries.

Using Machine Learning to Predict Axial Pile Capacity

Accurate estimation of the ultimate axial load bearing capacity of piles is necessary to ensure the safety of the supported structures and to prevent cost overruns. Traditional mechanics-based design methods do not always predict pile capacity accurately, or precisely, leaving room for improvement. This study focuses on the potential of machine learning (ML) in estimating pile capacity. A dataset of 546 load tests was compiled from three databases. The baseline performance of traditional design methods was first established by comparing the capacities computed using four traditional approaches, against the capacities interpreted from load tests using Davisson’s criterion. Sixteen different ML techniques were explored. First, the optimal feature selection technique for model training was investigated. Second, hyperparameters of each technique were optimized. The process involved the training of 32,000 different models and tuning their hyperparameters. Next the dataset was randomly split into training (70%) and testing (30%) for comparing the 16 different ML regression models. Each of the optimized models was then trained using six feature sets. The performance of each of the 16 ML models with the best performing feature subset was compared with the baseline performance from traditional methods. Evaluation criteria included measured versus predicted capacities, influence of soil type on accuracy, as well as the absence of pile diameter, or length effects on accuracy and precision. In general, the ML methods performed significantly better than the best traditional method. The current research demonstrated that ML may offer advantages in geotechnical design when large datasets are available.

Enhancing hydrogen production prediction from biomass gasification via data augmentation and explainable AI: A comparative analysis

Hydrogen production for clean energy is gaining a foothold, notably through the gasification of biomass. Machine learning aids in its accurate production predictions, yet its opaque nature limits explanation. This study looked into machine learning models' transparency and evaluated the impact of data augmentation on hydrogen production prediction accuracy using a publicly available numerical dataset. Fifteen (15) machine learning regression models including linear and tree-based were evaluated leading to the introduction of a hybrid model. The proposed hybrid model is based on ensemble and averaging techniques in mathematics. The deployed model's prediction was evaluated using the Mean Absolute Error (MAE), the Mean Square Error (MSE), the Root Mean Squared Error (RMSE), the Root Mean Squared Log Error (RMSLE), and R-squared (R2) based on K-Fold cross-validation. Furthermore, the effect of the grid search, random search, and Bayesian optimization techniques for model hyperparameter optimization were carried out. The evaluated models demonstrate superior predictions and explore data augmentation's potential in refining machine learning accuracy with the proposed hybrid model recording an MAE of 2.187, MSE of 9.788, RMSE of 3.129, R2 of 0.250, MSLE of 0.005, and RMSLE of 0.070 against the original data result of MAE of 2.665, MSE of 11.877, RMSE of 3.446, R2 of 0.112, MSLE of 0.006 and RMSLE of 0.076. However, when the proposed model hyperparameter training setting was optimized, the yielded result improved further with an MAE of 0.044, MSE of 0.040, RMSE of 0.215, R2 of 0.996, MSLE of 0.000 and RMSLE of 0.005. By unraveling the inner workings of black box models using the SHAP, LIME, and ELI-5 interpretable AI models, this study advances hydrogen production prediction using biomass gasification. Insights gained in this study foster the development of more transparent and effective models, aiming for the widespread adoption of sustainable hydrogen energy.

Construction of a Cutting-Tool Wear Prediction Model through Ensemble Learning

This study begins by conducting side milling experiments on SKD11 using tungsten carbide TiAlN-coated end mills to compare the surface roughness performance between two combinations of milling process parameters (feed rate and radial depth of cut), along with three ultrasonic-assisted methods (rotary, dual-axis, and rotary combined with dual-axis). The results suggest that the rotary (z-axis oscillation) ultrasonic-assisted method may provide better performance. Subsequently, this superior ultrasonic-assisted method was applied both with and without laser locally preheating assistance, respectively. Using a Taguchi orthogonal array, milling process parameters (spindle speed, feed rate, and radial depth of cut) were planned for experiments with the same cutting tool and the workpiece just mentioned above. The surface roughness serves as the objective function while being constrained by cutting-tool life. The characteristics of the smaller-the-better in the Taguchi method were applied to determine the optimal combination of process parameters. Based on the optimal milling process parameters obtained and the superior hybrid-assisted method adopted, milling experiments were repeatedly performed to collect the data on cutting force and cutting-tool wear. Feature engineering was performed on the cutting force signals, and different domain characteristics from both the time and frequency domains were extracted. Hereafter, feature selection by random forest and data standardization were further applied to feature extractions, and the data processing was thus completed. For the processed data, a cutting-tool wear prediction model was constructed by ensemble learning. This method leverages various machine learning regression models, including decision tree, random forest, extremely randomized tree, light gradient boosting machine, extreme gradient boosting, AdaBoost, stochastic gradient descent, support vector regression, linear support vector regression, and multilayer perceptron. After hyper-parameter tuning, the ensemble voting regression prediction was performed based on these ten mentioned models. The experimental results demonstrate that the ensemble voting regression model surpasses the performance of each individual machine learning regression model. In addition, this regression model achieves a coefficient of determination (R2) of 0.94576, a root mean square error (RMSE) of 0.24348, a mean squared error (MSE) of 0.05928, and a mean absolute error (MAE) of 0.18182. Therefore, the ensemble learning approach has been proven to be a feasible and effective method for monitoring cutting-tool wear.

Weather-Based Nonlinear Regressions for Digital TV Received Signal Strength Prediction

In this research, the impact of various weather conditions on digital television signals is investigated. Machine learning and nonlinear regression models were used to estimate the strength of the received signal. The received signal strength might vary significantly depending on the weather condition, especially in higher frequency ranges or millimetre wavelengths. Predictive analysis was performed for the radio-relay link Aval Tower-Vršac Hill, which is used for the distribution of television and radio programmes by the public company Broadcasting Technology and Connections in Serbia. The prediction was made using temperature, temperature index, relative humidity, and received signal strength data for the months of June, July, and August in 2022. The best results were obtained using the RandomForest model. Extreme variations in the strength of the received signal can be predicted by using the model mentioned above. More effective management of the broadcasting infrastructure can be done with the ability to predict sudden falls and fluctuations in received signal strength.

Tribo-informatics approach to predict wear and friction coefficient of Mg/Si3N4 composites using machine learning techniques

The demand for lightweight, high-performance materials has driven significant advancements in magnesium-based materials. However, their practical implementation faces challenges, primarily due to their low wear resistance, especially in industries like automotive, where weight reduction is vital for fuel efficiency. Furthermore, the process of fabricating and assessing wear behavior incurs time and cost constraints. In light of this, machine learning techniques have emerged as a crucial tool for predicting the mechanical properties, wear characteristics, and tribological performance of diverse materials, including magnesium and its composites. Accordingly, the present study aims to integrate experimental results with machine learning techniques. The objective is to predict the wear rate and friction coefficient of Mg/Si3N4 nanocomposites, thus optimizing material design and manufacturing for superior wear performance. Nanocomposites are fabricated through ultrasonic-assisted stir casting, and a dataset of 120 data points is collected using a pin-on-disc tribometer under dry sliding conditions. Five supervised machine learning regression models are employed to predict wear rate and coefficient of friction, with hyperparameter tuning for a fair comparison. Results are evaluated using various statistical metrics, identifying the most effective model for accurate wear behavior prediction. The study also demonstrates improved wear resistance and lower friction coefficients in nanocomposites compared to pure magnesium. This is attributed to the even distribution of Si3N4 nanoparticles and strong interfacial bonding with the matrix. The presence of a mechanically mixed layer further enhances wear resistance under high loads and speeds. Five wear modes are identified, including abrasion, oxidation, adhesion, delamination, and plastic deformation, providing valuable insights into the wear mechanisms. A comprehensive wear map facilitates a deeper understanding of wear behavior.

Predicting Stellar Masses of the First Galaxies Using Graph Neural Networks

Theoretical models of galaxy formation and evolution are primarily investigated through cosmological simulations and semi-analytical models. The former method consumes O(106) core-hours explicitly modeling the dynamics of the galaxies, whereas the latter method only requires O(103) core-hours foregoing directly simulating internal structure for computational efficiency. In this work, we present a proof-of-concept machine learning regression model, using a graph neural network architecture, to predict the stellar mass of high-redshift galaxies solely from their dark matter merger trees, trained from a radiation hydrodynamics cosmological simulation of the first galaxies.

Automated group constant parameterization for low sample sizes using different Machine learning approaches

This paper deals with group constant parameterization, a necessary step to utilize the results of assembly-level neutronics calculations at the full-core level. The focus is on low sample size problems when the commonly used linear interpolation approach is inadequate, a typical situation of using Monte Carlo codes for group constant generation. This work presents a newly developed code package for automated group constant parameterization. It implements several machine learning regression models − including a novel polynomial regression algorithm − performs hyperparameter optimization and selects the best model based on a detailed evaluation. The applicability of the new code package is demonstrated in a case study for a VVER-1200 fuel assembly covering both normal operation and transient conditions. In this example, the novel polynomial regression model provides a 73 pcm average error in kinf that leads to reactivity coefficients well within the desired precision.

UAV remote sensing phenotyping of wheat collection for response to water stress and yield prediction using machine learning

Water stress is a significant challenge for global food production. Rainfall pattern is becoming unpredictable due to climate change that causes unprecedent water stress conditions in cereals production including wheat which is one of the important staple food crops. To sustain wheat production under water limiting conditions, there is an urgent need to develop drought-tolerant wheat varieties. For this, screening large numbers of wheat genotype for traits related to growth and yield under water stressed conditions is crucial. In this study, we deployed high-throughput phenotyping approaches, including uncrewed aerial vehicle (UAV)-based multispectral imaging, advanced machine and deep learning regression models. Two separate field experiments, irrigated and rainfed, were conducted comprising 553 wheat genotypes, and collected dataset for traits such as plant height, phenology, grain yield, and timeseries multispectral imaging. UAV-multispectral imagery derived plant height measurements showed a high correlation (R2=0.75) with manual measurements. Vegetation indices derived from multispectral data differentiated growth pattern of genotypes under rainfed and irrigated conditions and were used in yield prediction modeling. Wheat genotypes were effectively ranked, and their response differentiated for water stress tolerance based on yield index, stress susceptibility index, and yield loss%. Importantly, yield prediction in genotypes was computed using four machine learning regression algorithms i.e., linear regression, support vector machine, random forest, and deep learning H2O-3, where H2O-3 was the most accurate model with R2=0.80. Results show that multispectral-driven traits combined with machine learning models effectively phenotyped large wheat population and such approaches can be integrated in crop breeding program to develop varieties tolerant to water stress.

Machine learning-based multi-objective optimization and thermal assessment of supercritical CO2 Rankine cycles for gas turbine waste heat recovery

Technologies for utilizing waste heat for power generation have attracted significant attention in recent years due to their potential to enhance energy efficiency and reduce greenhouse gas emissions. This research focuses on the comparative and optimization analysis of three supercritical carbon dioxide (sCO2) Rankine cycles (simple, cascade, and split) for gas turbine waste heat recuperation. The study begins with parametric analysis, investigating the significant effects of key variables, including turbine inlet temperature, condenser inlet temperature, and pinch point temperature, on the thermal performance of advanced sCO2 power cycles. To identify the most efficient cycle configuration, a multi-objective optimization approach is employed. This approach combines a Genetic Algorithm with machine learning regression models (Random Forest, XGBoost, Artificial Neural Network, Ridge Regression, and K-Nearest Neighbors) to predict cycle performance using a dataset extracted from cycle simulations. The decision-making process for determining the optimal cycle configuration is facilitated by the TOPSIS (technique for order of preference by similarity to the ideal solution) method. The study's major findings reveal that the split cycle outperforms the simple and cascade configurations in terms of power generation across various operating conditions. The optimized split cycle not only demonstrates superior power output but also exhibits enhanced net power output, heat recovery, system and exergy efficiency of 7.99 MW, 76.17 %, 26.86 % and 57.96 %, respectively, making it a promising choice for waste heat recovery applications. This research has the potential to contribute to the advancement and widespread adoption of waste heat recovery in energy technologies boosting system efficiency and economic feasibility. It provides a new perspective for future research, contributing to the improvement of energy generation infrastructure.

Machine-learning-assisted design of high strength steel I-section columns

High strength steel has been attracting attention in the building industry due to its superior mechanical properties. The accurate design of high strength steel structures is crucial to boost its wide application. In this paper, an accurate and unified design approach for high strength steel I-section columns with different material grades, boundary conditions, geometric dimensions (including cross-section sizes and member lengths) and failure modes is proposed based on machine learning. Firstly, 871 experimental and numerical data were collected from the literature to establish a database. Then, seven machine learning algorithms, including Decision Tree, Random Forest, Support Vector Machine, K-Nearest Neighbour, Adaptive Boosting, Extreme Gradient Boosting and Categorical Boosting, were applied to establish machine learning regression models to predict buckling resistances of high strength steel I-section columns. The model performance was then evaluated through statistic indices, with the evaluation results indicating that the Categorical Boosting trained model yields the highest level of accuracy. Based on the data in the collected database, the regression model trained by Categorical Boosting and existing codified design provisions, as given in the European code and American specification, were assessed and compared. The European code and American specification were found to yield scattered and inaccurate failure load predictions, while the Categorical Boosting trained model led to substantially more accurate and consistent failure load predictions for high strength steel I-section columns with different material grades, boundary conditions, geometric dimensions and failure modes.

Streamflow prediction using support vector regression machine learning model for Tehri Dam

Accurate and reliable streamflow prediction is critical for optimising water resource management, reservoir flood operations, watershed management, and urban water management. Many researchers have published on streamflow prediction using techniques like Rainfall-Runoff modelling, Time series Models, Data-driven models, Artificial intelligence, etc. Still, there needs to be generalised method practise in the real world. The resolution of this issue lies in selecting different methods for a particular study area. This paper uses the Support vector regression machine learning model to predict the streamflow for the Tehri Dam, Uttarakhand, India, at the Daily and Ten Daily time steps. Two cases are considered in predicting daily and ten daily time steps. The first case includes four input variables: Discharge, Rainfall, Temperature, and Snow cover area. The second case comprises only three input variables: Rainfall, Temperature, and Snow cover area. Radial Kernel is used to overcome the space complexity in the datasets. The K-fold cross-validation is suitable for prediction as it averages the prediction error rate after evaluating the SVR model’s performance on various subsets of the training data. The streamflow data for daily and ten daily time steps have been collected from 2006 to 2020. The calibration period is from 2006 to 2016, and the validation period is from 2017 to 2020. Nash Sutcliffe Efficiency (NSE) and Coefficient of determination (R2) are used as the accuracy indicator in this manuscript. The lag has been observed in the daily prediction time series when three input variables are considered. For other scenarios, the respective model shows excellent results at both the temporal scale and the parametres, which play a vital role in prediction. The study also enhances the effect on the potential use of input parametres in the machine learning model.

Predicting Cryptocurrency Returns Using Classification and Regression Machine Learning Model

People are starting to see the cryptocurrency market as a viable source of income and investment, similar to the stock market, as the concept of cryptocurrencies continues to gain popularity. Predicting Bitcoin returns is related to financial machine learning, which uses time series to forecast price variance. This study starts with the daily close price of Bitcoin for its initial dataset. The price is transformed into percentages and binary classes, which categorize into “Up” and “Down”, after which a time series is applied to produce two datasets: a categorical dataset for classification and a numerical dataset for regression. For classification that represents a Binary classification in asset-price forecasting, k-fold cross-validation is applied to ensure that the best classifiers are selected for testing and analysis. Most of the regression analysis was based on visualization, which displayed the predicted prices by each regressor in front of the original values and helped analyse the models’ results more accurately. The outcomes of this study were achieved by anticipating bitcoin returns using classification and regression machine learning models, despite the approaches’ low accuracy and significant precision rate to the “Up” class. At this stage, with a significant limitation regarding the dataset and a lack of other indicators, a model capable of predicting future variations is considered a beneficial addition for many trading tools or even for crypto market analysts.

Assessing Column Stability: A Comparative Study of Machine Learning Regression Models for Shear Strength Prediction

This research presents a comprehensive investigation into the accurate estimation of shear strength in rectangular reinforced concrete columns through advanced machine learning (ML) models. The study addresses the intricate challenge posed by shear strength complexity, which is crucial for evaluating column stability and ensuring structural integrity. Building upon a substantial dataset comprising 545 experimental observations sourced from diverse literature, this research establishes a robust foundation for predictive modeling. Four distinct ML regression models, Random Forest, Decision Tree, XGBoost, and LightGBM, are meticulously evaluated for their performance. The evaluation employs established metrics, including R2, RMSE, MAE, and MAPE to quantify their predictive capabilities. The outcomes highlight the models' robustness in capturing nuanced variations in shear strength, with impressive R2 values ranging from 93.6% to 93.9%, showcasing their exceptional ability to elucidate intricate shear behaviors. Furthermore, comparative analysis indicates the slightly superior performance of the Random Forest over the Decision Tree, highlighting the efficacy of ensemble methods in this context. Extending the exploration to include XGBoost and LightGBM, the study showcases their potential as accurate shear strength predictors. The performance of the models is validated through scatter plots and error distribution plots, confirming accurate shear strength predictions across various scenarios. This research contributes significantly to the advancement of structural engineering methodologies by highlighting the potential of ML to improve the accuracy of shear strength estimation. The findings not only underscore the exceptional performance of ML models but also provide valuable insights into their comparative effectiveness, paving the way for enhanced structural assessments in columns.

An improved approach to determine aerosol properties from all-sky camera imagery: Sensitivity to the partially cloud scenes

We present a new approach to determine aerosol properties from radiometrically calibrated images provided by an all-sky camera. It is designed to be used regardless of the sky conditions. However, we especially focus on partially cloudy scenes, which is the main novelty of this work. Our methodology is based on using a small sector of the image that contains the principal plane of the Sun. The RGB principal plane radiances are associated to the aerosol optical depth (AOD) and Angstrom exponent (AE) AERONET observations through a Gaussian Process Regression (GPR) machine learning (ML) model. We identify the cloudy points within our working sector and the principal plane signal for the RGB radiances is averaged and smoothed. Then, we use the Pérez model to synthesize the principal plane signal in the cloudy spots. Finally, 2-year dataset has been used to test the method considering different atmospheric conditions related to the presence of clouds and aerosols, according to their amount and type. In addition, we have developed a method to evaluate the quality of predictions based on the standard deviation of the GPR. This quality assurance method may be fine-tuned according to the desired accuracy based on the application for which it is intended. Our AOD and AE predictions show an excellent overall agreement with AERONET measurements that substantially improves when our quality assurance method is applied. In that case, we obtain a high degree of correlation (R2 ¿ 0.97) and an overall MAE lower than the nominal uncertainty of AERONET measurements (0.006 and 0.05 for AOD and AE, respectively). Moreover, more than 83% and 77% of the predictions fall within the nominal uncertainty associated with AERONET measurements for AOD and AE, respectively. A comprehensive sensitivity analysis of the factors affecting the performance of the proposed methodology confirms that our method is stable and not very sensitive to external and methodological factors, especially when we apply quality assurance criteria. All this supports that our methodology is a reliable alternative to retrieve the optical properties of aerosols independently of the cloud conditions. Our results may contribute to the operational use of all-sky cameras, which may be an interesting complement regarding the study of aerosol-cloud interactions in partially cloud scenarios.