Multi-target Regression Model Research Articles

Machine learning-based modeling of variable compression ratio engine performance and emissions with JME-ZnO nanoemulsion

ABSTRACT Diesel engines fueled with nanoemulsion of biodiesel and diesel have shown promising results in reducing emissions without significant engine problems. However, the evaluation of the impact of biodiesel-nano emulsion on engine performance and exhaust emissions requires time-consuming and costly experimental testing. Modelling simulations have gained attention as a useful approach to overcome these challenges. Artificial intelligence (AI) has an extensive variety of applications in various domains, including energy systems. By training computers using machine learning (ML), they can make better decisions and outperform humans. AI-based models offer promising prospects in energy generation predictions. However, these prediction models can be computationally demanding. In this paper, the authors propose a Boosting-based Multi-Target Regression model using ML techniques to predict the pursuance and exhaust of a diesel engine. The study aims to advance the understanding of enhanced engine performance through the application of ML models. The MTXGB model demonstrates high average accuracy (high R2-score of 0.9989) and low error rates of 0.0123 for all six targets.

Filter method-based feature selection process for unattributed-identity multi-target regression problem

Unattributed-identity multi-target regression (UIMTR) is defined as a multi-target regression problem in which the identity of the target and predictor variables is not predefined. It is a problem that can be found in several real-world applications. For example, when historical data is available from a set of devices, but real-time data can only be requested from a subset of them (so called sentinels). For estimating real-time status of non-sentinels, it will be necessary to generate multi-target regression models. Therefore, attributing the identity of the real-time communicators (sentinels), i.e., the predictor variables, is a critical aspect. Moreover, unlike classical feature selection problems, the set of target variables is determined after applying the selection methods and not before, thus, some adaptations are necessary. We introduce three novel methods to solve the UIMTR and, after extensive evaluation, we demonstrate: (i) the feasibility of the methods, (ii) the usefulness of the approach, and (iii) the improvement over other classical techniques. The results have been evaluated from three perspectives: (i) the quality of the predictions, (ii) the stability of the methods and (iii) the execution time.

Predicting Student Performance: Analyzing Socio-economic and Personal Factors

The purpose of this study is to predict student performance in Mathematics, Reading, and Writing exams by analyzing a comprehensive dataset containing student profiles and exam scores. Python's data processing and machine learning libraries are utilized for data analysis and modeling. The dataset is explored, followed by data preprocessing that includes feature encoding and standardization. Support Vector Regression (SVR) is employed as a multi-target regression model to train and evaluate the model for predicting student exam scores. Feature importance analysis and visualization of prediction errors are performed to gain deeper insights into student performance. The results demonstrate significant correlations between factors such as gender, lunch type, ethnic group, and parent education level with exam scores. The prediction model performs well on the test dataset and effectively predicts student exam scores in different subjects. The findings of this study provide valuable insights for educational decision-making and promote personalized and effective education.

Multitarget regression models for predicting compressive strength and chloride resistance of concrete

Multitarget regression models for predicting compressive strength and chloride resistance of concrete

A Federated Learning Framework for Fingerprinting-Based Indoor Localization in Multibuilding and Multifloor Environments

The participatory nature of federated learning (FL) makes it attractive for fingerprinting-based indoor localization in multibuilding and multifloor environments. A group of sensing clients can collaboratively leverage their private, local fingerprint data to help their edge server update a location prediction model. However, it is challenging to jointly handle the two involved issues, i.e., building-floor classification (BFC) and latitude–longitude regression (LLR), in a wide 3-D space through enabling FL on decentralized yet heterogeneous data and over an imperfect wireless network. In this article, we confront these challenges and propose an FL framework, FedLoc3D, for both BFC and LLR. Specifically, the former issue is addressed by an FedDSC-BFC approach, which generates a multilabel classification model based on a convolutional neural network with depthwise separable convolutions. The latter issue is addressed by an FedADA-LLR approach, which develops a multitarget regression model based on a deep neural network with autoencoder and data augmentation. Extensive experiments on a real-world data set of WiFi fingerprints are carried out, and our approaches with enhanced capabilities of feature extraction, generalization, and convergence are validated to improve both localization accuracy and learning efficiency under data heterogeneity and network instability.

Building Chemical Property Models for Energetic Materials from Small Datasets Using a Transfer Learning Approach.

For many experimentally measured chemical properties that cannot be directly computed from first-principles, the existing physics-based models do not extrapolate well to out-of-sample molecules, and experimental datasets themselves are too small for traditional machine learning (ML) approaches. To overcome these limitations, we apply a transfer learning approach, whereby we simultaneously train a multi-target regression model on a small number of molecules with experimentally measured values and a large number of molecules with related computed properties. We demonstrate this methodology on predicting the experimentally measured impact sensitivity of energetic crystals, finding that both characteristics of the computed dataset and model architecture are important to prediction accuracy of the small experimental dataset. Our directed-message passing neural network (D-MPNN) ML model using transfer learning outperforms direct-ML and physics-based models on a diverse test set, and the new methods described here are widely applicable to modeling many other structure-property relationships.

Improved prediction of gene expression through integrating cell signalling models with machine learning

BackgroundA key problem in bioinformatics is that of predicting gene expression levels. There are two broad approaches: use of mechanistic models that aim to directly simulate the underlying biology, and use of machine learning (ML) to empirically predict expression levels from descriptors of the experiments. There are advantages and disadvantages to both approaches: mechanistic models more directly reflect the underlying biological causation, but do not directly utilize the available empirical data; while ML methods do not fully utilize existing biological knowledge.ResultsHere, we investigate overcoming these disadvantages by integrating mechanistic cell signalling models with ML. Our approach to integration is to augment ML with similarity features (attributes) computed from cell signalling models. Seven sets of different similarity feature were generated using graph theory. Each set of features was in turn used to learn multi-target regression models. All the features have significantly improved accuracy over the baseline model - without the similarity features. Finally, the seven multi-target regression models were stacked together to form an overall prediction model that was significantly better than the baseline on 95% of genes on an independent test set. The similarity features enable this stacking model to provide interpretable knowledge about cancer, e.g. the role of ERBB3 in the MCF7 breast cancer cell line.ConclusionIntegrating mechanistic models as graphs helps to both improve the predictive results of machine learning models, and to provide biological knowledge about genes that can help in building state-of-the-art mechanistic models.

Precise prediction of multiple anticancer drug efficacy using multi target regression and support vector regression analysis

Background and objectivesThe prediction of multiple drug efficacies using machine learning prediction techniques based on clinical and molecular attributes of tumors is a new approach in the field of precision medicine of oncology. The selection of suitable and effective therapeutic drugs among the potential drugs is performed computationally considering the tumor features. In this study, we developed and validated machine learning models to predict the efficacy of five anti-cancer drugs according to the clinical and molecular attributes of 30 oral squamous cell carcinoma (OSCC) cohorts. This sounds a bit odd – consider: Ranking of the drugs was achieved using their apoptotic priming. MethodsWe developed multiple drug efficacy prediction models based on three types of tumor characteristics by applying machine learning methods, including multi-target regression (MTR) and support vector regression (SVR). The prediction accuracy of existing machine learning methods was enhanced by introducing novel pre-processing techniques to develop Enhanced MTR (E_MTR), Enhanced Log-based MTR (EL_MTR), Enhanced Multi-target SVR (EM_SVR), and Enhanced Log-based Multi-target SVR (ELM_SVR). As a unique capability, ELM_SVR and EL_MTR rank the drugs based on their predicted efficacy. All the drug efficacy prediction models were built using OSCC real samples and theoretical samples. The best model was selected was based on dataset size and evaluation metrics, such as error terms, residuals and parameter tuning, and cross-validated (CV) using 30 real samples and 340 theoretical samples. ResultsWhen 30 real tumor samples were used for the train-test and CV methods, MTR models predicted the efficacy with less error than SVR models. Comparatively, using 340 theoretical samples for the train-test and CV methods, though MTR improved the performance, SVR predicted the efficacy with zero error. We found that, for small samples, the proposed MTR provided a 0.01 difference between actual apoptotic priming and predicted priming of five drugs. For large samples, the predicted values by the proposed SVR had a difference of 0.00001. The error terms (Actual vs. Predicted) also reveal that the enhanced log model is suitable when MTR is applied. Meanwhile, the enhanced model is suitable for SVR learning for multiple drug efficacy prediction. It was found that the predicted ranks of the drugs based on the multi-targeted efficacy prediction exactly match the actual rankings. ConclusionWe developed efficient statistical and machine learning models using MTR and SVR analysis for anticancer drug efficacy, which will be useful in the field of precision medicine to choose the most suitable drugs in personalized manner. The performance results of the proposed enhanced ranking techniques are described as follows: i) EL_MTR is the best to predict multiple anticancer drug efficacies and improve the accuracy of ranking drugs, irrespective of sample size; and ii) ELM_SVR performs better than other MTR models with a large sample size and precise ranking process.

Transformer-based Multi-target Regression on Electronic Health Records for Primordial Prevention of Cardiovascular Disease.

Machine learning algorithms have been widely used to capture the static and temporal patterns within electronic health records (EHRs). While many studies focus on the (primary) prevention of diseases, primordial prevention (preventing the factors that are known to increase the risk of a disease occurring) is still widely under-investigated. In this study, we propose a multi-target regression model leveraging transformers to learn the bidirectional representations of EHR data and predict the future values of 11 major modifiable risk factors of cardiovascular disease (CVD). Inspired by the proven results of pre-training in natural language processing studies, we apply the same principles on EHR data, dividing the training of our model into two phases: pre-training and fine-tuning. We use the fine-tuned transformer model in a "multi-target regression" theme. Following this theme, we combine the 11 disjoint prediction tasks by adding shared and target-specific layers to the model and jointly train the entire model. We evaluate the performance of our proposed method on a large publicly available EHR dataset. Through various experiments, we demonstrate that the proposed method obtains a significant improvement (12.6% MAE on average across all 11 different outputs) over the baselines.

Monitoring of the refractory lining in a shielded electric arc furnace: An online multitarget regression trees approach

Being able to predict future temperatures on the wall lining is key when controlling and scheduling maintenance for large industrial smelting furnaces. In this paper, we propose and test a machine learning approach for predicting lining temperatures in a ferronickel smelting furnace. This approach was deployed and evaluated in a real-world scenario, i.e., in one of Cerro Matoso S.A.'s (CMSA) industrial plant furnaces. Different techniques were tested, and finally, a multitarget regression (MTR) model showed the best performance. Previous state of the art focused on predicting only one target sensor; in contrast, our model is capable of predicting up to 12 targets. Two MTR models were tested: the incremental structured output prediction tree (iSOUP-Tree) and the stacked single-target Hoeffding tree regressor (SST-HT). The SST-HT method had the best behavior in terms of the average mean absolute error (AMAE) and average root mean square error (ARMSE). The results indicate that the developed MTR models can accurately predict the measured temperature on multiple point sensors. Results of this work are expected to help the process of structural control and health monitoring of the furnace linings located at CMSA's plant.

Multi-level information fusion for learning a blood pressure predictive model using sensor data

The availability of commercial wearable bio-sensors provides an opportunity for developing smart phone applications for real-time diagnosis that can be used to improve the health of the user. We propose a multi-level information fusion approach for learning a predictive model for blood pressure (BP) using electrocardiogram (ECG) sensor data. The approach fuses the information on five different levels: i) data collection, where data from multiple ECG sensors is collected; ii) feature extraction, where features are extracted from the collected data by different preprocessing methods; iii) information fusion, fusing the evaluation information from different classifiers; iv) information fusion using the information from multi-target regression models for each BP class; and v) information fusion using the information from multi-target regression models from all configurations as a single model. This is used for predicting the blood pressure values (systolic BP (SBP), diastolic BP (DBP), and mean arterial pressure (MAP)). Evaluating the methodology by using a separate test set indicates that the multi-level information fusion provides promising results, which are acceptable and comparable to the state-of-the-art results obtained for blood pressure prediction.

Fast learning of fiber orientation distribution function for MR tractography using convolutional neural network.

In diffusion-weighted magnetic resonance imaging (DW-MRI), the fiber orientation distribution function (fODF) is of great importance for solving complex fiber configurations to achieve reliable tractography throughout the brain, which ultimately facilitates the understanding of brain connectivity and exploration of neurological dysfunction. Recently, multi-shell multi-tissue constrained spherical deconvolution (MSMT-CSD) method has been explored for reconstructing full fODFs. To achieve a reliable fitting, similar to other model-based approaches, a large number of diffusion measurements is typically required for MSMT-CSD method. The prolonged acquisition is, however, not feasible in practical clinical routine and is prone to motion artifacts. To accelerate the acquisition, we proposed a method to reconstruct the fODF from downsampled diffusion-weighted images (DWIs) by leveraging the strong inference ability of the deep convolutional neural network (CNN). The method treats spherical harmonics (SH)-represented DWI signals and fODF coefficients as inputs and outputs, respectively. To compensate for the reduced gradient directions with reduced number of DWIs in acquisition in each voxel, its surrounding voxels are incorporated by the network for exploiting their spatial continuity. The resulting fODF coefficients are fitted with applying the CNN in a multi-target regression model. The network is composed of two convolutional layers and three fully connected layers. To obtain an initial evaluation of the method, we quantitatively measured its performance on a simulated dataset. Then, for invivo tests, we employed data from 24 subjects from the Human Connectome Project (HCP) as training set and six subjects as test set. The performance of the proposed method was primarily compared to the super-resolved MSMT-CSD with the decreasing number of DWIs. The fODFs reconstructed by MSMT-CSD from all available 288 DWIs were used as training labels and the reference standard. The performance was quantitatively measured by the angular correlation coefficient (ACC) and the mean angular error (MAE). For the simulated dataset, the proposed method exhibited the potential advantage over the model reconstruction. For the invivo dataset, it achieved superior results over the MSMT-CSD in all the investigated cases, with its advantage more obvious when a limited number of DWIs were used. As the number of DWIs was reduced from 95 to 25, the median ACC ranged from 0.96 to 0.91 for the CNN, but 0.93 to 0.77 for the MSMT-CSD (with perfect score of 1). The angular error in the typical regions of interest (ROIs) was also much lower, especially in multi-fiber regions. The average MAE for the CNN method in regions containing one, two, three fibers was, respectively, 1.09°, 2.75°, and 8.35° smaller than the MSMT-CSD method. The visual inception of the fODF further confirmed this superiority. Moreover, the tractography results validated the effectiveness of the learned fODF, in preserving known major branching fibers with only 25 DWIs. Experiments on HCP datasets demonstrated the feasibility of the proposed method in recovering fODFs from up to 11-fold reduced number of DWIs. The proposed method offers a new streamlined reconstruction procedure and exhibits promising potential in acquisition acceleration for the reconstruction of fODFs with good accuracy.

Data-Driven Structuring of the Output Space Improves the Performance of Multi-Target Regressors

The task of multi-target regression (MTR) is concerned with learning predictive models capable of predicting multiple target variables simultaneously. MTR has attracted an increasing attention within research community in recent years, yielding a variety of methods. The methods can be divided into two main groups: problem transformation and problem adaptation. The former transform a MTR problem into simpler (typically single target) problems and apply known approaches, while the latter adapt the learning methods to directly handle the multiple target variables and learn better models which simultaneously predict all of the targets. Studies have identified the latter group of methods as having competitive advantage over the former, probably due to the fact that it exploits the interrelations of the multiple targets. In the related task of multi-label classification, it has been recently shown that organizing the multiple labels into a hierarchical structure can improve predictive performance. In this paper, we investigate whether organizing the targets into a hierarchical structure can improve the performance for MTR problems. More precisely, we propose to structure the multiple target variables into a hierarchy of variables, thus translating the task of MTR into a task of hierarchical multi-target regression (HMTR). We use four data-driven methods for devising the hierarchical structure that cluster the real values of the targets or the feature importance scores with respect to the targets. The evaluation of the proposed methodology on 16 benchmark MTR datasets reveals that structuring the multiple target variables into a hierarchy improves the predictive performance of the corresponding MTR models. The results also show that data-driven methods produce hierarchies that can improve the predictive performance even more than expert constructed hierarchies. Finally, the improvement in predictive performance is more pronounced for the datasets with very large numbers (more than hundred) of targets.

Cognitive Assessment Prediction in Alzheimer's Disease by Multi-Layer Multi-Target Regression.

Accurate and automatic prediction of cognitive assessment from multiple neuroimaging biomarkers is crucial for early detection of Alzheimer's disease. The major challenges arise from the nonlinear relationship between biomarkers and assessment scores and the inter-correlation among them, which have not yet been well addressed. In this paper, we propose multi-layer multi-target regression (MMR) which enables simultaneously modeling intrinsic inter-target correlations and nonlinear input-output relationships in a general compositional framework. Specifically, by kernelized dictionary learning, the MMR can effectively handle highly nonlinear relationship between biomarkers and assessment scores; by robust low-rank linear learning via matrix elastic nets, the MMR can explicitly encode inter-correlations among multiple assessment scores; moreover, the MMR is flexibly and allows to work with non-smooth ℓ2,1-norm loss function, which enables calibration of multiple targets with disparate noise levels for more robust parameter estimation. The MMR can be efficiently solved by an alternating optimization algorithm via gradient descent with guaranteed convergence. The MMR has been evaluated by extensive experiments on the ADNI database with MRI data, and produced high accuracy surpassing previous regression models, which demonstrates its great effectiveness as a new multi-target regression model for clinical multivariate prediction.

A spatial-constrained multi-target regression model for human brain activity prediction

Analyzing functional magnetic resonance imaging (fMRI) data from the encoding perspective provides a powerful tool to explore human vision. Using voxel-wise encoding models, previous studies predicted the brain activity evoked by external stimuli successfully. However, these models constructed a regularized regression model for each single voxel separately, which overlooked the intrinsic spatial property of fMRI data. In this work, we proposed a multi-target regression model that predicts the activities of adjacent voxels simultaneously. Different from the previous models, the spatial constraint is considered in our model. The effectiveness of the proposed model is demonstrated by comparing it with two state-of-the-art voxel-wise models on a publicly available dataset. Results indicate that the proposed method can predict voxel responses more accurately than the competing methods.

Fingerprint orientation field regularisation via multi‐target regression

Orientation field estimation is a key step in fingerprint feature extraction and recognition. A complete orientation field estimation algorithm usually consists of two steps, i.e. initial orientation field estimation and post regularisation. In this Letter, a multi-target regression model to regularise the initial orientation field is proposed. A large number of orientation patches with simulated noises, together with their regression targets are fed to a deep neural networks to train a multi-target regression model. For a given initial orientation field at testing stage, a refined orientation field is obtained by applying the regression model in patch-wise and then combining all predicted patches. Experimental results on FVC2002, FVC2004 and FVC2006 databases show remarkable performance compared with state of the art algorithms. Our algorithm is also highly efficient and easy to implement.