Multi-output Regression Research Articles

Machine learning framework for extracting micro-viscoelastic and micro-structural properties of compressed oral solid dosage forms

A compressed pharmaceutical oral solid dosage (OSD) form is a strongly micro-viscoelastic material composite arranged as a network of agglomerated particles due to its constituent powders and their bonding and fractural mechanical properties. An OSD product’s Critical Quality Attributes, such as disintegration, drug release (dissolution) profile, and structural strength (“hardness”), are influenced by its micro-scale properties. Ultrasonic evaluation is direct, non-destructive, rapid, and cost-effective. However, for practical process control applications, the simultaneous extraction of the micro-viscoelastic and scattering properties from a tablet’s ultrasonic response requires a unique solution to a challenging inverse mathematical wave propagation problem. While the spatial progression of a pulse traveling in a composite medium with known micro-scale properties is a straightforward computational task when its dispersion relation is known, extracting such properties from the experimentally acquired waveforms is often non-trivial. In this work, a novel Machine Learning (ML)-based micro-property extraction technique directly from waveforms, based on Multi-Output Regression models and Neural Networks, is introduced and demonstrated. Synthetic waveforms with a given set of micro-properties of virtual tablets are computationally generated to train, validate, and test the developed ML models for their effectiveness in the inverse problem of recovering specified micro-scale properties. The effectiveness of these ML models is then tested and demonstrated for a set of physical OSD tablets. The micro-viscoelastic and micro-structural properties of physical tablets with known properties have been extracted through experimentally acquired waveforms to exhibit their consistency with the generated ML-based attenuation results.

Robust low-rank learning multi-output regression for incipient sediment motion in sewer pipes

The existing incipient sediment motion models typically apply conventional regression methods considering either velocity or shear stress. In the current study, incipient sediment motion is analyzed through a simultaneous and joint analysis of velocity and shear stress using the robust low-rank learning (RLRL) multi-output regression technique. Moreover, the experimental data compiled from five different channels are utilized to develop a generic incipient sediment motion model valid for a channel of any cross-sectional shape. The efficiency of the developed method is examined and compared against the available conventional regression models. The experimental results indicate that the RLRL model yields better results than its counterparts. In particular, while cross-section specific models fail to provide accurate estimates for shear stress or velocity for other cross sections, the proposed model provides satisfactory results for all channel shapes. The better performance of the recommended approach can be attributed to the joint modeling of the shear stress and the velocity which is realized by capturing the correlation between these parameters in terms of a low rank output mixing matrix which enhances the prediction performance of the approach.

MLA-TCN: Multioutput Prediction of Dam Displacement Based on Temporal Convolutional Network with Attention Mechanism

The displacement of concrete dams effectively reflects their structural integrity and operational status. Therefore, establishing a model for predicting the displacement of concrete dams and studying the evolution mechanism of dam displacement is essential for monitoring the structural safety of dams. Current data-driven models utilize artificial data that cannot reflect the actual status of dams for network training. They also have difficulty extracting the temporal patterns from long-term dependencies and obtaining the interactions between the targets and variables. To address such problems, we propose a novel model for predicting the displacement of dams based on the temporal convolutional network (TCN) with the attention mechanism and multioutput regression branches, named MLA-TCN (where MLA is multioutput model with attention mechanism). The attention mechanism implements information screening and weight distribution based on the importance of the input variables. The TCN extracts long-term temporal information using the dilated causal convolutional network and residual connection, and the multioutput regression branch achieves simultaneous multitarget prediction by establishing multiple regression tasks. Finally, the applicability of the proposed model is demonstrated using data on a concrete gravity dam within 14 years, and its accuracy is validated by comparing it with seven state-of-the-art benchmarks. The results show that the MLA-TCN model, with a mean absolute error (MAE) of 0.05 mm, a root-mean-square error (RMSE) of 0.07 mm, and a coefficient of determination (R2) of 0.99, has a comparably high predictive capability and outperforms the benchmarks, providing an accurate and effective method to estimate the displacement of dams.

Prediction of strip section shape for hot-rolled based on mechanism fusion data model

With the rise and vigorous development of artificial intelligence (AI) and data mining methodologies, machine learning (ML) has been successfully and widely used in industrial fields. For hot-rolled strips, both the rolling mechanism analyzing and multi-indicators integrated optimization are difficulties in hot rolling process. To overcome the limitations of multi-variable, nonlinear, and strong coupling in the process of hot rolling, and to improve the prediction accuracy of multi-output regression model, a novel model of rolling mechanism guided ML was developed. Mechanism data were calculated as added input features to participate in training process by way of dimensionality processing (DP). Simultaneously, genetic algorithm (GA) was adopted to realize multi-objective optimization of multi-output support vector regression (M-SVR) to further enhance the performance of model ultimately. Especially, the experimental results of root mean square error (RMSE) and correlation coefficient (R) of strip crown and thickness are 2.5218, 0.9891 and 2.1302, 0.9902, respectively, which fully demonstrates that this method does not limit the prediction accuracy but greatly improved the generalization ability.

Improved Copula-based conformal prediction for uncertainty quantification of multi-output regression

Though machine learning-based prediction techniques have been widely applied in industrial production, how to quantify their uncertainty remains a major challenge. Conformal prediction is a framework for uncertainty quantification that can be combined with arbitrary machine learning methods. Based on the existing methods, an improved conformal prediction multi-output method is proposed in this paper in order to obtain more effective prediction region. Firstly, a multi-output regression model is used as the underlying model and the Copula function is estimated by the calibration set; Secondly, given the global confidence level, the volume of the prediction region obtained from the calibration set is used as the optimization objective function, and the Differential Evolution (DE) algorithm is used to calculate different individual confidence levels for each output variable; Then, the uncertainty of the test set is calculated based on the individual confidence level obtained from the optimization solution. Finally, the experiments and analysis of the data from the iron production process show that the proposed method improves the efficiency of prediction region under the guarantee of coverage.

Machine Learning Based b-Shaped Monopole Antenna for RF Energy Harvesting Applications

In this paper, a machine learning (ML) based b-shaped multiband monopole antenna for RF energy harvesting system is presented. ML algorithms are used to optimize the antenna in order to reduce the simulation time and speed up the design process. Geometric parameters on the antenna have been determined as input in the ML model. There are two different output data: the complex and decibel magnitudes of the reflection coefficient (S11) parameters, which have been determined as output parameters. The dataset consists of 1890 samples in total. Initially, attempts were made to predict the decibel magnitude of reflection coefficient parameters using different ML algorithms, and these ML algorithms were compared with each other. Then, for the complex part of the reflection coefficientparameters, a Multi-Output regression model was applied to the real and imaginary values. The best results were obtained with the K-Nearest Neighbors algorithm for the decibel magnitude of reflection coefficient parameters, achieving a 0.14% RMSE value and a 99% R2value. The Multi-Output Regression algorithm was applied to complex (Real, Imaginary) values, achieving an R2value of 92.51% and a RMSE value of 0.10.

Investigation of S1046 profile bladed vertical axis wind turbine and artificial intelligence-based performance evaluation

ABSTRACT It is very important to determine the parameters affecting the performance of the Darrieus-type wind turbine and its effects. In particular, it should be specified at which TSR value the peak power coefficient is obtained. In this study, standard and modified S1046 airfoils and aspect ratios (H/D), angle of attack (AoA), turbulent/non-turbulent flow (WT), number of blades (N), and chord length (C) were tested. Then, four different machines learning-based multi-output regression models (Decision Tree, Linear Regression, K-Nearest Neighbors, and Random Forest) were trained to make performance predictions with the data obtained from the evaluated test setup. Thirdly, feature selection based on the Random Forest algorithm, which is the best performing multi-output regression model, was performed using data due to changing parameter values on the established system. The importance of the parameters was determined. The operating range of the system was at relatively low TSR values. When analyzing the blade profile, the modified blade version performed better in certain combinations compared to the standard profile. Maximum power coefficient (Cp) was obtained from the modified turbine structure with 5 degrees of attack angle, H/D = 1.85, and C = 60 mm. The present study aims to increase the turbine’s power coefficient and aims to predict results as power coefficient without doing many different experiments.

A Deep Learning Regression Model for Photonic Crystal Fiber Sensor with XAI Feature Selection and Analysis.

A Deep Learning Multi-output regression model is employed to correctly model the relationships between optical design parameters of an asymmetric Twin Elliptical Core Photonic Crystal Fiber (TEC-PCF) and its sensing performances. TEC-PCF acts as a biosensor to detect the blood glucose level taking hemoglobin components into account. Since asymmetric TEC-PCF uses a dual elliptical core, four super modes have to be evaluated to analyze the sensing performance in terms of effective index difference, transmission spectrum, coupling length, and sensor sensitivity. The dataset used in this work is of the optical design parameters of the sensor and Finite Element Method (FEM) results with effective indices of four super modes obtained from the COMSOL Multiphysics by varying hemoglobin concentration to 120 g/L, 140 g/L, and 160 g/L. Gretel.ai's free open-source synthetic data library is used to augment the dataset to make the training more efficient. Explainable AI (XAI) feature analysis using Shapley Additive Explanations (SHAP) framework is used for two purposes: feature selection and to know the feature's effect on prediction. The former led to the development of an optimal model with much fewer computational demands and the latter made the model interpretable. The proposed model can predict the accurate super modes when given input specifications with wavelength ranging from 1.27-1.5 μm and for various glucose concentrations under the influence of hemoglobin much faster compared to the other numerical simulations which are computationally expensive. Computation time taken by the proposed Artificial Neural Network (ANN) model, the proposed model with XAI and FEM is also being compared.

A novel adaptive dynamic GA combined with AM to optimize ANN for multi-output prediction: Small samples enhanced in industrial processing

Multi-output prediction based on small samples (SSs) produced in industrial processing plays an important role in improving processing efficiency and reducing production costs. However, insufficient sample capacity will result in inaccurate and unstable multi-output regression prediction. To solve this problem, this paper jointly optimizes an artificial neural network (ANN) by employing an adaptive dynamic genetic algorithm and adaptive moment estimation (ADGA-AM-ANN) based on enhanced SSs. The data are enhanced by adding noise to the output data to prevent overfitting due to insufficient sample capacity. The adaptive dynamic genetic algorithm is employed to grope for the optimal parameters of the ANN in the global range. This prevents the ANN from falling into a local minimum in the training process, so as to improve the overall accuracy of the model in simultaneously predicting multiple variables, and to promote its internal prediction stability. Adaptive moment estimation is employed to optimize the gradient of optimal parameters to accelerate convergence and deepen the external prediction stability of the model. The cases of cutting and laser processes are used to verify the accuracy and stability of the proposed method.

Predicting the effect of voids generated during RTM on the low-velocity impact behaviour by machine learning-based surrogate models

The main objective of the present paper is to demonstrate the feasibility of machine-learning-based surrogate models for predicting low-velocity impact behaviour considering void content and location generated during the resin transfer moulding process. Generating reliable experimental datasets for training those models is almost impossible, therefore an adapted finite element model was implemented providing reliable results to generate the synthetic datasets. The optimum hyperparameter combination for training the Random Forest model was found based on the grid search technique. The accuracy of the classification, single-output regression and multi-output regression models was sufficient. It was concluded that the multi-output regression model, which predicts the force-time, displacement-time, and energy-time curves, provides the best information, and is sufficiently accurate (R2 > 0.995) and fast (5 s per sample) as an online structural performance monitoring tool.

Simultaneous prediction the strain and energy absorption capacity of ultra-high performance fiber reinforced concretes by using multi-output regression model

Although a few machine learning (ML) models have successfully been developed for Ultra-high performance fiber reinforced concrete (UHPFRC), they only limited to one output per model and do not explain the effects of its ingredients. This paper proposes an novel approach to tackle the multiple outputs problems by using multi-output regression model (MORM) and providing some insights into the relations between dosages and their outputs via partial dependence plots. A UHPFRC database with 980 mixtures designs with 34 features and two outputs, namely the strain at peak tensile stress and energy absorption capacity is used to verify the propose approach. The MORM with three estimators, namely XGBoost Regressor, Decision Tree Regressor and Gradient Boosting Regressor is trained and tested. Three quantitative measures (R2, MAE and RMSE) are employed to evaluate the accuracy and the obtained results are superior to previous study. Among the results found, the greater feasibility of deformed high-strength steel macrofibers to achieve their exceptional values can be highlighted. On the one hand, the energy absorption capacity directly depends on the cementitious matrix’s quality. On the other hand, the strain at peak tensile stress decreases with the improvement of the matrix’s compressive strength to around 200 MPa. From this resistance, the trend is reversed due to an exponential improvement caused by the improvement in the bond strength between fiber and matrix. The proposed method could be properly used for the optimization of new UHPFRC dosages.

A novel non-profiled side channel attack based on multi-output regression neural network

A novel non-profiled side channel attack based on multi-output regression neural network

Distributionally robust learning-to-rank under the Wasserstein metric.

Despite their satisfactory performance, most existing listwise Learning-To-Rank (LTR) models do not consider the crucial issue of robustness. A data set can be contaminated in various ways, including human error in labeling or annotation, distributional data shift, and malicious adversaries who wish to degrade the algorithm's performance. It has been shown that Distributionally Robust Optimization (DRO) is resilient against various types of noise and perturbations. To fill this gap, we introduce a new listwise LTR model called Distributionally Robust Multi-output Regression Ranking (DRMRR). Different from existing methods, the scoring function of DRMRR was designed as a multivariate mapping from a feature vector to a vector of deviation scores, which captures local context information and cross-document interactions. In this way, we are able to incorporate the LTR metrics into our model. DRMRR uses a Wasserstein DRO framework to minimize a multi-output loss function under the most adverse distributions in the neighborhood of the empirical data distribution defined by a Wasserstein ball. We present a compact and computationally solvable reformulation of the min-max formulation of DRMRR. Our experiments were conducted on two real-world applications: medical document retrieval and drug response prediction, showing that DRMRR notably outperforms state-of-the-art LTR models. We also conducted an extensive analysis to examine the resilience of DRMRR against various types of noise: Gaussian noise, adversarial perturbations, and label poisoning. Accordingly, DRMRR is not only able to achieve significantly better performance than other baselines, but it can maintain a relatively stable performance as more noise is added to the data.

Machine Learning Models for Ecofriendly Optimum Design of Reinforced Concrete Columns

CO2 emission is one of the biggest environmental problems and contributes to global warming. The climatic changes due to the damage to nature is triggering a climate crisis globally. To prevent a possible climate crisis, this research proposes an engineering design solution to reduce CO2 emissions. This research proposes an optimization-machine learning pipeline and a set of models trained for the prediction of the design variables of an ecofriendly concrete column. In this research, the harmony search algorithm was used as the optimization algorithm, and different regression models were used as predictive models. Multioutput regression is applied to predict the design variables such as section width, height, and reinforcement area. The results indicated that the random forest algorithm performed better than all other machine learning algorithms that have also achieved high accuracy.

Use of a machine learning-based framework to approximate the input features of an intrinsic cohesive zone model of recycled asphalt mixes tested at low temperatures

Although the cohesive zone model (CZM) provides numerous advancements in simulating the crack initiation and evolution in asphalt mixes, its efficiency and applicability are still challenging. This is because asphalt mixes are principally assumed to be homogeneous in CZM modeling despite intrinsic heterogeneity. Therefore, it is essential to calibrate the CZM model, by adjusting the input factors, aiming at alleviating this inconsistency between the model and reality. Accordingly, this research was aimed at presenting a model to approximate the calibrated input features as a function of a wide variety in testing conditions and mix criteria. To this end, an experimental dataset was collected by investigating the fracture mechanic responses of various recycled warm asphalt mixtures (prepared using three categories using organic and chemical materials, and using foam bitumen technology). Mixes were consisted of virgin aggregates and those containing up to 70% recycled asphalt pavement particles (i.e. 0, 30, 50, and 70%). Mixes were subjected to freeze and thaw cycles (i.e. 0, 1, and 3 cycles) and semi-circular bending test was conducted at subzero temperatures (i.e. 0, −10, and −20℃). Experimental results were analyzed using machine learning (ML) algorithms including random forest, gradient boosting regression, and multi-output regression methods. The results proved the efficiency of the random forest regression model in predicting the required input factors of CZM model. The ML model was also validated by predicting the input factors (of CZM model) for another fracture dataset, demonstrating that this ML model can be promisingly used as an approximation tool.

A Kaizen Programming algorithm for multi-output regression based on a heterogeneous island model

A Kaizen Programming algorithm for multi-output regression based on a heterogeneous island model

Large scale multi-output multi-class classification using Gaussian processes

Multi-output Gaussian processes (MOGPs) can help to improve predictive performance for some output variables, by leveraging the correlation with other output variables. In this paper, our main motivation is to use multiple-output Gaussian processes to exploit correlations between outputs where each output is a multi-class classification problem. MOGPs have been mostly used for multi-output regression. There are some existing works that use MOGPs for other types of outputs, e.g., multi-output binary classification. However, MOGPs for multi-class classification has been less studied. The reason is twofold: 1) when using a softmax function, it is not clear how to scale it beyond the case of a few outputs; 2) most common type of data in multi-class classification problems consists of image data, and MOGPs are not specifically designed to image data. We thus propose a new MOGPs model called Multi-output Gaussian Processes with Augment & Reduce (MOGPs-AR) that can deal with large scale classification and downsized image input data. Large scale classification is achieved by subsampling both training data sets and classes in each output whereas downsized image input data is handled by incorporating a convolutional kernel into the new model. We show empirically that our proposed model outperforms single-output Gaussian processes in terms of different performance metrics and multi-output Gaussian processes in terms of scalability, both in synthetic and in real classification problems. We include an example with the Ommiglot dataset where we showcase the properties of our model.

Multi-Class Classification and Multi-Output Regression of Three-Dimensional Objects Using Artificial Intelligence Applied to Digital Holographic Information.

Digital holographically sensed 3D data processing, which is useful for AI-based vision, is demonstrated. Three prominent methods of learning from datasets such as sensed holograms, computationally retrieved intensity and phase from holograms forming concatenated intensity-phase (whole information) images, and phase-only images (depth information) were utilized for the proposed multi-class classification and multi-output regression tasks of the chosen 3D objects in supervised learning. Each dataset comprised 2268 images obtained from the chosen eighteen 3D objects. The efficacy of our approaches was validated on experimentally generated digital holographic data then further quantified and compared using specific evaluation matrices. The machine learning classifiers had better AUC values for different classes on the holograms and whole information datasets compared to the CNN, whereas the CNN had a better performance on the phase-only image dataset compared to these classifiers. The MLP regressor was found to have a stable prediction in the test and validation sets with a fixed EV regression score of 0.00 compared to the CNN, the other regressors for holograms, and the phase-only image datasets, whereas the RF regressor showed a better performance in the validation set for the whole information dataset with a fixed EV regression score of 0.01 compared to the CNN and other regressors.

Machine learning toward high-performance electrochemical sensors.

The so-coined fourth paradigm in science has reached the sensing area, with the use of machine learning (ML) toward data-driven improvements in sensitivity, reproducibility, and accuracy, along with the determination of multiple targets from a single measurement using multi-output regression models. Particularly, the use of supervised ML models trained on large data sets produced by electrical and electrochemical bio/sensors has emerged as an impacting trend in the literature by allowing accurate analyses even in the presence of usual issues such as electrode fouling, poor signal-to-noise ratio, chemical interferences, and matrix effects. In this trend article, apart from an outlook for the coming years, we present examples from the literature that demonstrate how helpful ML algorithms can be for dispensing the adoption of experimental methods to address the aforesaid interfering issues, ultimately contributing to translate testing technologies into on-site, practical, and daily applications.

Predicting volumetric error compensation for five-axis machine tool using machine learning

ABSTRACT This work proposes a rapid and robust machine-learning model to predict the volumetric error of a five-axis machine tool. For this purpose, several machine learning models – which are MultiOutput regression, Chained MultiOutput regression, Linear regression, SVM regression, XGB regression and ANN regression – have been selected, and their performances were compared to other models in the literature and to one another to find the best model for the problem at hand. The robustness of each model is investigated using three statistical metrics: namely Root Mean Square Error (RMSE), Mean Absolute Error (MAE) and Coefficient of Determination (R2). The results show that the models proposed in this paper are more effective than those in the literature, using the same data. Amongst the proposed models, the SVR Regression model has proven to be the best, considering all statistical metrics. Compared to the polynomial method in the literature with varying order levels, the method proposed here improves accuracy and predictive performance by 27%, with an RMSE of 0.03246 mm for the SVR model. Finally, based on the best model, an explicit equation has been deduced for practical applications. That prediction equation has been implemented in Excel and included in this paper.