Multi-task Lasso Research Articles

Deep Learning-Based Prediction Models for the Detection of Vitamin D Deficiency and 25-Hydroxyvitamin D Levels Using Complete Blood Count Tests

Vitamin D (VitD) is an essential nutrient that is critical for the well-being of both adults and children, and its deficiency is recognized as a precursor to several diseases. In previous studies, researchers have approached the problem of detecting vitamin D deficiency (VDD) as a single "sufficient/deficient" classification problem using machine learning or statistics-based methods. The main objective of this study is to predict a patient's VitD status (i.e., sufficiency, insufficiency, or deficiency), severity of VDD (i.e., mild, moderate, or severe), and 25-hydroxyvitamin D (25(OH)D) level in a separate deep learning (DL)-based models. An original dataset consisting of complete blood count (CBC) tests from 907 patients, including 25(OH)D concentrations, collected from a public health laboratory was used for this purpose. CNN, RNN, LSTM, GRU and Auto-encoder algorithms were used to develop DL-based models. The top 25 features in the CBC tests were carefully selected by implementing the Extra Trees Classifier and Multi-task LASSO feature selection algorithms. The performance of the models was evaluated using metrics such as accuracy, F1-score, mean absolute error, root mean square error and R-squared. Remarkably, all three models showed satisfactory results when compared to the existing literature; however, the CNN-based prediction models proved to be the most successful.

The Potential of Multi-Task Learning in CFDST Design: Load-Bearing Capacity Design with Three MTL Models.

Concrete-filled double steel tubes (CFDSTs) are a load-bearing structure of composite materials. By combining concrete and steel pipes in a nested structure, the performance of the column will be greatly improved. The performance of CFDSTs is closely related to their design. However, existing codes for CFDST design often focus on how to verify the reliability of a design, but specific design parameters cannot be directly provided. As a machine learning technique that can simultaneously learn multiple related tasks, multi-task learning (MTL) has great potential in the structural design of CFDSTs. Based on 227 uniaxial compression cases of CFDSTs collected from the literature, this paper utilized three multi-task models (multi-task Lasso, VSTG, and MLS-SVR) separately to provide multiple parameters for CFDST design. To evaluate the accuracy of models, four statistical indicators were adopted (R2, RMSE, RRMSE, and ρ). The experimental results indicated that there was a non-linear relationship among the parameters of CFDSTs. Nevertheless, MLS-SVR was still able to provide an accurate set of design parameters. The coefficient matrices of two linear models, multi-task Lasso and VSTG, revealed the potential connection among CFDST parameters. The latent-task matrix V in VSTG divided the prediction tasks of inner tube diameter, thickness, strength, and concrete strength into three groups. In addition, the limitations of this study and future work are also summarized. This paper provides new ideas for the design of CFDSTs and the study of related codes.

Machine learning optimization for catalytic desulfurization of petroleum: Multi-layered perceptron, Multi Task Lasso, and Gaussian process regression models

Machine learning models were developed and optimized in this study to optimize the process of hydrodesulfurization (HDS) for fuel production. A number of input parameters were considered and optimized to find the best response parameters. The input parameters of process/model included: pressure, catalyst dosage, temperature, and feed’s sulfur content. This paper focuses on optimizing and evaluating AdaBoost ensemble models for predicting sulfur concentration, emission percentage, and HDS cost in industrial processes for fuel treatment. Employing Multilayer Perceptron (MLP), Gaussian Process Regression (GPR), and Multi-Task Lasso (MTL) as base models, and utilizing Tabu Search for tuning of hyper-parameter, we report the performance of the best models. The Bagged GPR stands out with a significant R2 of 0.967 for sulfur concentration and 0.959 for HDS cost, accompanied by RMSE values of 1.7691E+02 and 1.4191, respectively. For emission prediction, Bagged MTL demonstrates notable performance with an R2 of 0.877 and RMSE of 8.4235. These outcomes highlight the efficacy of ensemble models in enhancing predictive accuracy and underscore the potential for informed decision-making in industrial settings for the area of petroleum industry.

Machine learning-based optimization for catalytic sulfur removal: Computational modeling and analysis of fuel purification for reduction of environmental impacts

Hydrodesulfurization (HDS) process is an important process for separation of sulfur compounds from petroleum-based products due to operational and environmental problems that the sulfur compounds can cause. In this study, this process was evaluated to optimize its performance in removing sulfur compounds from petroleum to reduce its adverse effects. Multiple machine learning models were implemented for optimization of HDS process considering several inputs/outputs. Each data point has four input parameters: pressure, temperature, initial sulfur content of petroleum, and dosage of catalyst in the reactor. Sulfur concentration (ppm), SO2 emission percentage (%), and HDS cost ($) are also outputs to be optimized by the machine learning models. Multi-layered perceptron (MLP), Multi-task Lasso (MTL), and Gaussian process regression (GPR) are core models in this study developed for the first time for HDS process. These models were optimized utilizing Artificial Bee Colony (ABC) and applied on cleansed and normalized dataset. According to assessments done on final models, sulfur concentration, emission %, and HDS cost are predicted by R2-scores of 0.983, 0.999, and 0.990 respectively using models proposed in this study. Also, absence of overfitting can be guaranteed using these models according to analysis done in results section.

An Inexact Semismooth Newton-Based Augmented Lagrangian Algorithm for Multi-Task Lasso Problems

This paper is concerned with the [Formula: see text]-norm ball constrained multi-task learning problem, which has received extensive attention in many research areas such as machine learning, cognitive neuroscience, and signal processing. To address the challenges of solving large-scale multi-task Lasso problems, this paper develops an inexact semismooth Newton-based augmented Lagrangian (Ssnal) algorithm. When solving the inner problems in the Ssnal algorithm, the semismooth Newton (Ssn) algorithm with superlinear or even quadratic convergence is applied. Theoretically, this paper presents the global and asymptotically superlinear local convergence of the Ssnal algorithm under standard conditions. Computationally, we derive an efficient procedure to construct the generalized Jacobian of the projector onto [Formula: see text]-norm ball, which is an important component of the Ssnal algorithm, making the computational cost in the Ssn algorithm very cheap. Comprehensive numerical experiments on the multi-task Lasso problems demonstrate that the Ssnal algorithm is more efficient and robust than several existing state-of-the-art first-order algorithms.

Multitask AI Models for the Joint Prediction of Overall Survival, Progression-Free Survival, and Death without Progression as a Composite Endpoint for LA-NSCLC Patients Treated with Chemoradiotherapy

Prior methods model the risk of endpoints separately. Herein, we construct a composite AI model that considers multiple endpoints jointly, including overall survival (OS), progression-free survival (PFS), and death without progression (DWP). Our hypothesis is that the composite model potentially improves predictive performance for patients with locally advanced non-small cell lung cancer (LANSCLC) treated with chemoradiotherapy (CRT). A total of 335 LANSCLC patients treated with definitive CRT, including all evaluable patients accrued from Oct 2017 to Dec 2021, were randomly split into training/test subsets (n = 234/101). Cardio-pulmonary substructures (CPSs) were autocontoured, manually reviewed, and edited if necessary. A total of 1093 non-independent dosimetric parameters were extracted, including GTVp, GTVn, GTV, PTV, esophagus, lungs minus IGTV, left/right lung, 15 CPSs, and the overlapping volume of each OAR with PTV and the distance from each OAR to GTVp/GTVn. Other clinical parameters included age, consolidation immunotherapy (CI), ECOG score, Charlson comorbidity index, coronary heart disease, histology, PD-L1 expression, and clinical stage (AJCC 8). Within training, censored time-to-event data were imputed based on conditional event distributions derived from Kaplan-Meier estimators for casting survival analysis as a regression problem and training neural additive model (NAM) regressors. Features were selected by LASSO regression for a single endpoint (OS, PFS, DWP) and multi-task (MT) LASSO regression for four separate composite endpoints (OS-PFS, OS-DWP, PFS-DWP, OS-PFS-DWP). The performance of MT NAMs in the test set that jointly predicted the composite endpoints was evaluated using the C-index and compared to that of a single task (ST) NAM that predicted each endpoint separately. The best testing performance in predicting OS and DWP was attained by the MT NAM that jointly predicted all endpoints (c-index = 0.65, 95% CI 0.58-0.71 for OS; c-index = 0.78, 95% CI 0.69-0.87 for DWP). The best model to predict PFS was also MT between PFS and DWP (c-index = 0.59, 95% CI 0.52-0.65). The c-indices of all ST NAMs were less than 0.56. The best MT NAMs significantly outperformed ST NAMs in predicting OS (p = 0.001) and DWP (p = 0.01) except for PFS (p = 0.32). The best MT NAM in predicting OS and DWP included ECOG score, atria-PTV overlap volume, D75% [Gy] to the left atrium (LA), pulmonary arterial volume, histology (adenocarcinoma), D65% [Gy] to the descending aorta (DA), V10 Gy [%] of the LA and CI in order of overall importance. ECOG score consistently ranked as the most important feature for all four MT NAMs. An increase of ECOG score from 0 to 2 indicated a 6-month earlier risk of mortality and DWP. Atria-PTV overlap volume and D65% [Gy] to the DA were included in all four MT NAMs. MT AI models improved outcome prediction in patients with LANSCLC treated with CRT by jointly learning commonalities between the primary and auxiliary endpoints.

A reconstructed feasible solution-based safe feature elimination rule for expediting multi-task lasso

Multi-task lasso (MTL) is an effective algorithm to handle multi-task problems. By introducing ℓ2,1-norm, it can realize joint feature selection across a group of related tasks. But it is time-consuming when handling high-dimensional problems. Motivated by the row sparsity of the optimal solution, an improved safe feature elimination rule termed IEDPP is proposed to accelerate the training process. It could delete most of the redundant features before we solve the problem. Then the computational efficiency will be improved since only a reduced problem should be solved. Moreover, the properties of the projection operator and the reconstructed feasible solutions ensure the safety of the proposed method. That is to say, it will derive an identical optimal solution to the original problem both in theory and in practice. But our IEDPP could only be used once before solving, there are still some redundant features that are not deleted. Therefore, we further propose an integrated IIEG-ML rule by combining IEDPP with GAP. Then, more and more redundant features could be deleted as the algorithm converges. Moreover, by embedding IIEG-ML into the grid search method, the whole training process will be accelerated. Finally, an improved Nesterov's method is used to solve the reduced problems. Experimental results on different datasets confirm the effectiveness of our method.

Short-term wind speed prediction based on feature extraction with Multi-task Lasso and Multilayer Perceptron

The vigorous development of wind energy will help to promoting the adjustment of energy structure, the control of atmospheric smog, and the transformation of economic development mode. However, the randomness and volatility of wind speed make it difficult to predict with high accuracy. High-precision short-term wind speed prediction is inseparable from effective variable selection. In order to select highly relevant variables for short-term wind speed prediction, Multi-Task Lasso is applied to extract features, so the change trend of wind speed can be expressed exactly, and the Multilayer Perceptron (MLP) is applied to predict based on the extracted features. Firstly, Multi-Task Lasso is used to establish a linear regression model, the leading periods and the original input variables are determined from the perspective of prediction accuracy; Secondly, according to the optimization results of Multi-Task Lasso, the optimal features that can best express the wind speed fluctuation is extracted from different influencing factors and different leading periods based on the weight; Finally, the extracted optimal feature set is used as the input variable of MLP to establish prediction model. The results show that the effect of feature selection is obvious, the prediction accuracy is improved by more than 17%, and the short-term wind speed prediction model has good prediction accuracy and good generalization ability.

Prediction of the healthcare resource utilization using multi-output regression models

With the rapidly increasing healthcare cost and the scarcity of inpatient resources, it is of paramount importance to accurately predict the healthcare resource utilization. Previous research mainly focuses on predicting the healthcare cost using single-output models. However, the intensity of the healthcare resource utilization is reflected by multiple measures. For example, the Diagnosis Related Group (DRG) system adopted in China measures the healthcare resource utilization using both cost and length of stay (LoS), which motivates us to jointly predict these two measures. Compared to constructing several independent single-output models for each task, using multi-output models can provide unified prediction rules, reduce the training time, and improve the generalization by leveraging the correlations across tasks. We utilize four multi-output machine learning models, including the multi-task Lasso, the decision tree, the random forest, and the neural network. We evaluate their performance based on the Electronic Health Record (EHR) dataset with approximately 750,000 records. Based on extensive numerical experiments, we provide a guideline for model selection and construction. This research has the potential to improve the management of healthcare resources and provide decision support for healthcare payment system.

Regionalization of Groundwater Residence Time Using Metamodeling

AbstractGroundwater residence‐time distributions (RTDs) are critical for assessing susceptibility of water resources to degradation. A novel combination of numerical modeling and statistical methods allows estimation of regional RTDs with unprecedented speed. In this method, particle RTDs are generated in 30 type locales in the northeastern glaciated U.S. by using automated generalized finite‐difference groundwater flow and advective transport models. Targets for statistical learning were created from particle RTDs by fitting Weibull, gamma, and inverse Gaussian distributions. Whole‐basin flux‐weighted RTDs were well fit by one‐component Weibull distributions. Flux‐weighted RTDs at stressed receptors such as wells often produced more complicated RTDs that required a two‐component mixture to fit. A Multitask LASSO regression was trained on the parametric RTDs using hydrogeographic features of the modeled areas as explanatory features. In this way, RTDs are regionalized with mappable physical features such as recharge and aquifer volume. The shape, location, and scale parameters of the parametric RTDs are strongly related to the mean exponential age. The shape parameter of the distribution, which controls deviation from exponential, is additionally a function of aquifer heterogeneity and hydrologic features. Regionalized RTDs provide useful metrics with respect to groundwater lag times and solute loading to streams. The lag time between input and output contained in the RTD is critical to understanding the relation between the land surface and human and ecological receptors.

Multitask Lasso Model for Investigating Multimodule Design Factors, Operational Factors, and Covariates in Tubular Microbial Fuel Cells

Microbial fuel cells (MFCs) are a promising technology for simultaneous wastewater treatment and direct electricity generation. To build an effective large-scale MFC in field operation, it is important to identify the critical design and operational factors. However, there are limited studies that focus on developing systematic models to predict MFC performance and conduct variable selection. Herein, a multitask Lasso model (MLM) was employed to characterize the relationship among the input variables (design factors, operational factors, and covariates) and the output variables (normalized energy recovery (NER) and organic removal efficiency) of a tubular MFC with five modules. The proposed MLM can not only select the important input variables to predict the MFC performance but also simultaneously and accurately predict multiple output variables. The important design and operational factors were identified on the basis of the variable selection in the MLM. It was found that cathode moisture, especially th...

Application of Multi-task Lasso Regression in the Parametrization of Stellar Spectra

The multi-task learning approaches have attracted the increasing attention in the fields of machine learning, computer vision, and artificial intelligence. By utilizing the correlations in tasks, learning multiple related tasks simultaneously is better than learning each task independently. An efficient multi-task Lasso (Least Absolute Shrinkage Selection and Operator) regression algorithm is proposed in this paper to estimate the physical parameters of stellar spectra. It not only can obtain the information about the common features of the different physical parameters, but also can preserve effectively their own peculiar features. Experiments were done based on the ELODIE synthetic spectral data simulated with the stellar atmospheric model, and on the SDSS data released by the American large-scale survey Sloan. The estimation precision of our model is better than those of the methods in the related literature, especially for the estimates of the gravitational acceleration (lg g) and the chemical abundance ([Fe/H]). In the experiments we changed the spectral resolution, and applied the noises with different signal-to-noise ratios (SNRs) to the spectral data, so as to illustrate the stability of the model. The results show that the model is influenced by both the resolution and the noise. But the influence of the noise is larger than that of the resolution. In general, the multi-task Lasso regression algorithm is easy to operate, it has a strong stability, and can also improve the overall prediction accuracy of the model.

Collective Support Recovery for Multi-Design Multi-Response Linear Regression

The multi-design multi-response linear regression problem is investigated, in which design matrices are Gaussian with covariance matrices Σ (1:K) = (Σ (1) , ... , 1 (K) ) for K linear regression tasks. Design matrices across tasks are assumed to be independent. The support union of K p-dimensional regression vectors (collected as columns of matrix B*) is recovered using l 1 /l 2 -regularized lasso. Sufficient and necessary conditions on sample complexity are characterized as a sharp threshold to guarantee successful recovery of the support union. This model has been previously studied via l 1 /l ∞ -regularized lassoand via l 1 /l 1 + l 1 /l ∞ -regularized lasso, in which sharp threshold on sample complexity is characterized only for K = 2 and under special conditions. In this paper, using l 1 /l 2 -regularized lasso, sharp threshold on sample complexity is characterized under standard regularization conditions. Namely, if n > c p1 ψ(B*, Σ (1:K) ) log(p - s) where c p1 is a constant, and s is the size of the support set, then l 1 /l 2 -regularized lasso correctly recovers the support union; and if n <; c p2 ψ(B*, Σ (1:K) )log(p - s) where c p2 is a constant, then l 1 /l 2 -regularized lasso fails to recover the support union. In particular, the function ψ(B*, Σ (1:K) ) captures the impact of the sparsity of K regression vectors and the statistical properties of the design matrices on the threshold on sample complexity. Therefore, such threshold function also demonstrates the advantages of joint support union recovery using multitask lasso over individual support recovery using single-task lasso.

Multi-label learning under feature extraction budgets

We consider the problem of learning sparse linear models for multi-label prediction tasks under a hard constraint on the number of features. Such budget constraints are important in domains where the acquisition of the feature values is costly. We propose a greedy multi-label regularized least-squares algorithm that solves this problem by combining greedy forward selection search with a cross-validation based selection criterion in order to choose, which features to include in the model. We present a highly efficient algorithm for implementing this procedure with linear time and space complexities. This is achieved through the use of matrix update formulas for speeding up feature addition and cross-validation computations. Experimentally, we demonstrate that the approach allows finding sparse accurate predictors on a wide range of benchmark problems, typically outperforming the multi-task lasso baseline method when the budget is small.

Fast projections onto mixed-norm balls with applications

Joint sparsity offers powerful structural cues for feature selection, especially for variables that are expected to demonstrate a grouped behavior. Such behavior is commonly modeled via group-lasso, multitask lasso, and related methods where feature selection is effected via mixed-norms. Several mixed-norm based sparse models have received substantial attention, and for some cases efficient algorithms are also available. Surprisingly, several constrained sparse models seem to be lacking scalable algorithms. We address this deficiency by presenting batch and online (stochastic-gradient) optimization methods, both of which rely on efficient projections onto mixed-norm balls. We illustrate our methods by applying them to the multitask lasso. We conclude by mentioning some open problems.