Lasso Penalty Research Articles

Distributed Bootstrap Simultaneous Inference for High-Dimensional Quantile Regression

Modern massive data with enormous sample size and tremendous dimensionality are usually impossible to process with a single machine. They are typically stored and processed in a distributed manner. In this paper, we propose a distributed bootstrap simultaneous inference for a high-dimensional quantile regression model using massive data. Meanwhile, a communication-efficient (CE) distributed learning algorithm is developed via the CE surrogate likelihood framework and ADMM procedure, which can handle the non-smoothness of the quantile regression loss and the Lasso penalty. We theoretically prove the convergence of the algorithm and establish a lower bound on the number of communication rounds ιmin that warrant statistical accuracy and efficiency. The distributed bootstrap validity and efficiency are corroborated by an extensive simulation study.

Supervised Clustering of Persian Handwritten Images Using Regularization and Dimension Reduction Methods

Clustering, as a fundamental exploratory data technique, not only is used to discover patterns and structures in complex datasets but also is utilized to group variables in high-dimensional data analysis. Dimension reduction through clustering helps identify important variables and reduce data dimensions without losing significant information. High-dimensional image datasets, such as Persian handwritten images, have numerous pixels, making statistical inference difficult. Such high-dimensionality property pose challenges for analysis and processing, requiring specialized techniques like clustering to extract information. Incorporating response variable information enhances clustering analysis, transforming it into a supervised method. This article evaluates a supervised clustering approach using Ridge and Lasso penalties, comparing them in analyzing a real dataset while identifying important variables. We demonstrate that despite choosing a small number of variables as important variables, Lasso penalty performs relatively well in predicting the labels of new observations for this multi-class dataset.

Sentiment Analysis of Finnish Twitter Discussions on COVID-19 During the Pandemic

With the outbreak of the COVID-19 pandemic, researchers have studied how people reacted on social media during the pandemic. Sentiment analysis has been leveraged to gain insight. However, much of the research conducted on both sentiment analysis and social media analysis of COVID-19 often focuses on widespread languages, such as English and Chinese. This is partly due to the scarcity of resources for natural language processing and sentiment analysis for morphologically complex and less prevalent languages such as Finnish. This paper aims to analyze sentiments on Twitter in the Finnish language during the COVID-19 pandemic. We manually annotate with sentiments a random sample of 1943 tweets about COVID-19 in Finnish. We use it to build binomial and multinomial logistic regression models with Lasso penalty by exploiting ngrams and two existing sentiment lexicons. We also build two similar models using an existing (pre-COVID-19) Twitter dataset for comparison. The best-performing model for the Finnish language is then used to determine the trends of positive, negative, and neutral opinions on a collection of tweets in Finnish extracted between April 21 and June 18, 2020. The best sentiment polarity prediction model for the Finnish language attain 0.785 AUC, 0.710 balanced accuracy, and 0.723 macro-averaged F1 for predicting positive and negative polarity (binomial classification), and 0.667 AUC, 0.607 balanced accuracy, and 0.475 F1 when adding neutral tweets (multinomial classification). On the other hand, the pre-COVID-19 model trained on the same number of tweets exhibits higher accuracy for the multinomial model (0.687 balanced accuracy, and 0.588 F1). We hypothesize that this loss of performance is due to the COVID-19 context that makes sentiment analysis of neutral tweets more difficult for the machine learning algorithm to predict. Running the model on all the extracted Finnish tweets, we observe a decrease in negativity and an increase in positivity over the observed time as the Finnish government lifts restrictions. Our results show that applying an existing general-purpose sentiment analyzer on tweets that are domain-specific, such as COVID-19, provides lower accuracy. More effort in the future needs to be invested in using and developing sentiment analysis tools tailored to their application domain when conducting large-scale social media analysis of specific medical issues, such as a global pandemic.

On the Efficiency of the newly Proposed Convex Olanrewaju-Olanrewaju Lo-oλγ(|θ|) Penalized Regression-Type Estimator via GLMs Technique.

In this article, we proposed a novel convex penalized regression-type estimator, termed Olanrewaju-Olanrewaju penalized regression-type estimator, denoted by Lo-oλγ(|θ|) for ultra and high-dimensional data consideration whenever the number of covariates (p) is strictly greater than the sample size(n⁡). The newly proposed penalized estimator was formulated via the unique combination of arctangent and hyperbolic functions. It was noted that the newly proposed estimator could efficiently work under the constraint p<n formulated for, and as well as for the conventional regression methods of n>p. The proposed convex penalized regression-type estimator was formulated via three symmetric distributional noises of Gaussian, Laplace, and Normal Inverse Gaussian (NIG), considered being the three certified symmetric distributions that belong to the exponential family of distributions. The regularity axioms and oracle properties of the newly proposed penalized estimator were worked-out, as well as the parameter estimation for the penalized regression-type estimator for the three symmetric noises via canonical expressible form of Generalized Linear Models (GLMs) and first-order Karush-Kuhn-Tucker (KKT) condition. The Lo-oλγ(|θ|) regression-type estimator was subjected to simulation studies of well-generated symmetric observations. The US Longley macroeconomic dataset with the constraint p<n was fitted to Lo-oλγ(|θ|) regression-type estimator under the umbrella of three symmetric distributional noises. In conclusion, Gaussian noise for Lo-oλγ(|θ|) penalized regression-type estimator under the umbrella of p<n for the simulation and real life analyzes was considered effectively efficient because of its most reduced Mean Square Error (mse) measured error-type in comparison to Gaussian, Laplace, and NIG noises for LASSO and Atan penalties; and as well for Laplace, and NIG noises for Lo-oλγ(|θ|).

Application of fused graphical lasso to statistical inference for multiple sparse precision matrices.

In this paper, the fused graphical lasso (FGL) method is used to estimate multiple precision matrices from multiple populations simultaneously. The lasso penalty in the FGL model is a restraint on sparsity of precision matrices, and a moderate penalty on the two precision matrices from distinct groups restrains the similar structure across multiple groups. In high-dimensional settings, an oracle inequality is provided for FGL estimators, which is necessary to establish the central limit law. We not only focus on point estimation of a precision matrix, but also work on hypothesis testing for a linear combination of the entries of multiple precision matrices. We apply a de-biasing technology, which is used to obtain a new consistent estimator with known distribution for implementing the statistical inference, and extend the statistical inference problem to multiple populations. The corresponding de-biasing FGL estimator and its asymptotic theory are provided. A simulation study and an application of the diffuse large B-cell lymphoma data show that the proposed test works well in high-dimensional situation.

Heterogeneous macroeconomic factors' effects on stocks across sizes, styles, and sectors in the South Korean market.

Knowledge of the key macroeconomic variables that influence stock volatility across capital sizes, styles, and sectors can provide clues for investment strategies and policy decisions. We use the GARCH-MIDAS model with feature selection to analyze Korean Benchmark Indices from 2009 to 2022. This study maximizes memory retention through an optimal fractional differentiated price series and uses an adaptive lasso penalty for feature selection. The housing price-sales index and realized volatility were consistently influential across most indices and sectors. The GARCH-MIDAS model, paired with these variables, significantly improves long-term stock volatility forecasts. This study underscores the need to monitor housing prices in South Korea because of their substantial effects on long-term stock volatility.

Demand Forecasting and Project Development of Sports Tourism Based on User Behavior under the Perspective of Harmonious Coexistence of Human and Nature

Abstract Man and nature are interconnected and interdependent. The naturalization of man reflects the scientific nature of the landscape, the development of ecotourism is to establish the development of man and nature in harmony and unity of the development of ideas. This paper proposes measures to enhance the tourism experience based on the above concept. After a series of operations, such as similarity calculation and data cleaning, the user category, consumption level, and geographic location preference are calculated to establish a user model and portrait based on behavioral analysis. Based on LASSO penalty regression, the link between covariates is incorporated into the penalty term to complete the optimization and improvement of the LASSO regression model. Associate the user preference degree model with the tourism demand prediction model based on user preference. The model is applied in practice, and the average error rate for the number of tourists predicted by the LASSO regression coefficient is about 9.9%, which is smaller than the average error rate of the comparison model. Based on the results of the demand prediction, strategies related to the development of sports tourism projects are proposed, and the impact of sports tourism projects on the environment is evaluated. Based on the IPA analysis method, the user’s tourism experience of the subject area is assessed, and the user is more concerned about the internal transportation of the tourist place, with I and P values of 12.455 and 1.457, respectively, and the performance effect of the subject area in this point is poor, which needs to be solved in time.

Robust Variable Selection with Exponential Squared Loss for the Spatial Error Model

With the widespread application of spatial data in fields like econometrics and geographic information science, the methods to enhance the robustness of spatial econometric model estimation and variable selection have become a central focus of research. In the context of the spatial error model (SEM), this paper introduces a variable selection method based on exponential square loss and the adaptive lasso penalty. Due to the non-convex and non-differentiable nature of this proposed method, convex programming is not applicable for its solution. We develop a block coordinate descent algorithm, decompose the exponential square component into the difference of two convex functions, and utilize the CCCP algorithm in combination with parabolic interpolation for optimizing problem-solving. Numerical simulations demonstrate that neglecting the spatial effects of error terms can lead to reduced accuracy in selecting zero coefficients in SEM. The proposed method demonstrates robustness even when noise is present in the observed values and when the spatial weights matrix is inaccurate. Finally, we apply the model to the Boston housing dataset.

Variable Selection for Sparse Logistic Regression with Grouped Variables

We present a new penalized method for estimation in sparse logistic regression models with a group structure. Group sparsity implies that we should consider the Group Lasso penalty. In contrast to penalized log-likelihood estimation, our method can be viewed as a penalized weighted score function method. Under some mild conditions, we provide non-asymptotic oracle inequalities promoting the group sparsity of predictors. A modified block coordinate descent algorithm based on a weighted score function is also employed. The net advantage of our algorithm over existing Group Lasso-type procedures is that the tuning parameter can be pre-specified. The simulations show that this algorithm is considerably faster and more stable than competing methods. Finally, we illustrate our methodology with two real data sets.

Joint sparse least squares via generalized fused lasso penalty for identifying nonlinear dynamical systems

Joint sparse least squares via generalized fused lasso penalty for identifying nonlinear dynamical systems

Debiased inference for heterogeneous subpopulations in a high-dimensional logistic regression model

Due to the prevalence of complex data, data heterogeneity is often observed in contemporary scientific studies and various applications. Motivated by studies on cancer cell lines, we consider the analysis of heterogeneous subpopulations with binary responses and high-dimensional covariates. In many practical scenarios, it is common to use a single regression model for the entire data set. To do this effectively, it is critical to quantify the heterogeneity of the effect of covariates across subpopulations through appropriate statistical inference. However, the high dimensionality and discrete nature of the data can lead to challenges in inference. Therefore, we propose a novel statistical inference method for a high-dimensional logistic regression model that accounts for heterogeneous subpopulations. Our primary goal is to investigate heterogeneity across subpopulations by testing the equivalence of the effect of a covariate and the significance of the overall effects of a covariate. To achieve overall sparsity of the coefficients and their fusions across subpopulations, we employ a fused group Lasso penalization method. In addition, we develop a statistical inference method that incorporates bias correction of the proposed penalized method. To address computational issues due to the nonlinear log-likelihood and the fused Lasso penalty, we propose a computationally efficient and fast algorithm by adapting the ideas of the proximal gradient method and the alternating direction method of multipliers (ADMM) to our settings. Furthermore, we develop non-asymptotic analyses for the proposed fused group Lasso and prove that the debiased test statistics admit chi-squared approximations even in the presence of high-dimensional variables. In simulations, the proposed test outperforms existing methods. The practical effectiveness of the proposed method is demonstrated by analyzing data from the Cancer Cell Line Encyclopedia (CCLE).

Interpretation of a change point detection problem via adaptive lasso penalty and its application

Interpretation of a change point detection problem via adaptive lasso penalty and its application

Multi-block alternating direction method of multipliers for ultrahigh dimensional quantile fused regression

In this paper, we consider a quantile fused LASSO regression model that combines quantile regression loss with the fused LASSO penalty. Intuitively, this model offers robustness to outliers, thanks to the quantile regression, while also effectively recovering sparse and block coefficients through the fused LASSO penalty. To adapt our proposed method for ultrahigh dimensional datasets, we introduce an iterative algorithm based on the multi-block alternating direction method of multipliers (ADMM). Moreover, we demonstrate the global convergence of the algorithm and derive comparable convergence rates. Importantly, our ADMM algorithm can be easily applied to solve various existing fused LASSO models. In terms of theoretical analysis, we establish that the quantile fused LASSO can achieve near oracle properties with a practical penalty parameter, and additionally, it possesses a sure screening property under a wide class of error distributions. The numerical experimental results support our claims, showing that the quantile fused LASSO outperforms existing fused regression models in robustness, particularly under heavy-tailed distributions.

Evaluating the cost of simplicity in score building: An example from alcohol research.

Building a score from a questionnaire to predict a binary gold standard is a common research question in psychology and health sciences. When building this score, researchers may have to choose between statistical performance and simplicity. A practical question is to what extent it is worth sacrificing the former to improve the latter. We investigated this research question using real data, in which the aim was to predict an alcohol use disorder (AUD) diagnosis from 20 self-reported binary questions in young Swiss men (n = 233, mean age = 26). We compared the statistical performance using the area under the ROC curve (AUC) of (a) a "refined score" obtained by logistic regression and several simplified versions of it ("simple scores"): with (b) 3, (c) 2, and (d) 1 digit(s), and (e) a "sum score" that did not allow negative coefficients. We used four estimation methods: (a) maximum likelihood, (b) backward selection, (c) LASSO, and (d) ridge penalty. We also used bootstrap procedures to correct for optimism. Simple scores, especially sum scores, performed almost identically or even slightly better than the refined score (respective ranges of corrected AUCs for refined and sum scores: 0.828-0.848, 0.835-0.850), with the best performance been achieved by LASSO. Our example data demonstrated that simplifying a score to predict a binary outcome does not necessarily imply a major loss in statistical performance, while it may improve its implementation, interpretation, and acceptability. Our study thus provides further empirical evidence of the potential benefits of using sum scores in psychology and health sciences.

Geriatric assessment-derived deficit accumulation and patient-reported treatment burden in older adults with bladder cancer.

When a person's workload of healthcare exceeds their resources, they experience treatment burden. At the intersection of cancer and aging, little is known about treatment burden. We evaluated the association between a geriatric assessment-derived Deficit Accumulation Index (DAI) and patient-reported treatment burden in older adults with early-stage, non-muscle-invasive bladder cancer (NMIBC). We conducted a cross-sectional survey of older adults with NMIBC (≥65 years). We calculated DAI using the Cancer and Aging Research Group's geriatric assessment and measured urinary symptoms using the Urogenital Distress Inventory-6 (UDI-6). The primary outcome was Treatment Burden Questionnaire (TBQ) score. A negative binomial regression with LASSO penalty was used to model TBQ. We further conducted qualitative thematic content analysis of responses to an open-ended survey question ("What has been your Greatest Challenge in managing medical care for your bladder cancer") and created a joint display with illustrative quotes by DAI category. Among 119 patients, mean age was 78.9 years (SD 7) of whom 56.3% were robust, 30.3% pre-frail, and 13.4% frail. In the multivariable model, DAI and UDI-6 were significantly associated with TBQ. Individuals with DAI above the median (>0.18) had TBQ scores 1.94 times greater than those below (adjusted IRR 1.94, 95% CI 1.33-2.82). Individuals with UDI-6 greater than the median (25) had TBQ scores 1.7 times greater than those below (adjusted IRR 1.70, 95% CI 1.16-2.49). The top 5 themes in the Greatest Challenge question responses were cancer treatments (22.2%), cancer worry (19.2%), urination bother (18.2%), self-management (18.2%), and appointment time (11.1%). DAI and worsening urinary symptoms were associated with higher treatment burden in older adults with NMIBC. These data highlight the need for a holistic approach that reconciles the burden from aging-related conditions with that resulting from cancer treatment.

Kernel Cox partially linear regression: Building predictive models for cancer patients' survival.

Wide heterogeneity exists in cancer patients' survival, ranging from a few months to several decades. To accurately predict clinical outcomes, it is vital to build an accurate predictive model that relates the patients' molecular profiles with the patients' survival. With complex relationships between survival and high-dimensional molecular predictors, it is challenging to conduct nonparametric modeling and irrelevant predictors removing simultaneously. In this article, we build a kernel Cox proportional hazards semi-parametric model and propose a novel regularized garrotized kernel machine (RegGKM) method to fit the model. We use the kernel machine method to describe the complex relationship between survival and predictors, while automatically removing irrelevant parametric and nonparametric predictors through a LASSO penalty. An efficient high-dimensional algorithm is developed for the proposed method. Comparison with other competing methods in simulation shows that the proposed method always has better predictive accuracy. We apply this method to analyze a multiple myeloma dataset and predict the patients' death burden based on their gene expressions. Our results can help classify patients into groups with different death risks, facilitating treatment for better clinical outcomes.

Exploring the factors that influence academic stress among elementary school students using a LASSO penalty regression model

Exploring the factors that influence academic stress among elementary school students using a LASSO penalty regression model

Distributed debiased estimation of high-dimensional partially linear models with jumps

In this paper, we focus on the estimations of both parameter vector and nonparametric component in a high-dimensional partially linear model with jumps within the framework of divide and conquer strategy. We find that a three-stage estimation procedure works well in this setting. Applying the lasso penalty and projected spline approximation, first a profiled estimator for the linear part and a projected spline estimator for the nonparametric part are obtained on each local machine. In the second stage, an efficient jump detection algorithm is developed to obtain the new knot sequence, and then based on this, the estimate of the nonparametric function is obtained and averaged after plugging in the linear part estimate on each local machine. The aggregated estimate of the nonparametric function is then computed by pooling these local estimates. In the third stage, a debiased lasso estimator is averaged to obtain a distributed debiased estimator of the linear part after plugging in the aggregated estimate of nonparametric function. Asymptotic properties of resultant estimators are established under some mild assumptions. Some simulations are conducted to illustrate the empirical performances of our proposed method.

Integrative evaluation and experimental validation of the immune-modulating potential of dysregulated extracellular matrix genes in high-grade serous ovarian cancer prognosis

BackgroundHigh-grade serous ovarian cancer (HGSOC) is a challenging malignancy characterized by complex interactions between tumor cells and the surrounding microenvironment. Understanding the immune landscape of HGSOC, particularly the role of the extracellular matrix (ECM), is crucial for improving prognosis and guiding therapeutic interventions.Methods and resultsUsing univariate Cox regression analysis, we identified 71 ECM genes associated with prognosis in seven HGSOC populations. The ECMscore signature, consisting of 14 genes, was validated using Cox proportional hazards regression with a lasso penalty. Cox regression analyses demonstrated that ECMscore is an excellent indicator for prognostic classification in prevalent malignancies, including HGSOC. Moreover, patients with higher ECMscores exhibited more active stromal and carcinogenic activation pathways, including apical surface signaling, Notch signaling, apical junctions, Wnt signaling, epithelial-mesenchymal transition, TGF-beta signaling, and angiogenesis. In contrast, patients with relatively low ECMscores showed more active immune-related pathways, such as interferon alpha response, interferon-gamma response, and inflammatory response. The relationship between the ECMscore and genomic anomalies was further examined. Additionally, the correlation between ECMscore and immune microenvironment components and signals in HGSOC was examined in greater detail. Moreover, the expression of MGP, COL8A2, and PAPPA and its correlation with FAP were validated using qRT-PCR on samples from HGSOC. The utility of ECMscore in predicting the prospective clinical success of immunotherapy and its potential in guiding the selection of chemotherapeutic agents were also explored. Similar results were obtained from pan-cancer research.ConclusionThe comprehensive evaluation of the ECM may help identify immune activation and assist patients in HGSOC and even pan-cancer in receiving proper therapy.

Regularized parametric survival modeling to improve risk prediction models.

We propose to combine the benefits of flexible parametric survival modeling and regularization to improve risk prediction modeling in the context of time-to-event data. Thereto, we introduce ridge, lasso, elastic net, and group lasso penalties for both log hazard and log cumulative hazard models. The log (cumulative) hazard in these models is represented by a flexible function of time that may depend on the covariates (i.e., covariate effects may be time-varying). We show that the optimization problem for the proposed models can be formulated as a convex optimization problem and provide a user-friendly R implementation for model fitting and penalty parameter selection based on cross-validation. Simulation study results show the advantage of regularization in terms of increased out-of-sample prediction accuracy and improved calibration and discrimination of predicted survival probabilities, especially when sample size was relatively small with respect to model complexity. An applied example illustrates the proposed methods. In summary, our work provides both a foundation for and an easily accessible implementation of regularized parametric survival modeling and suggests that it improves out-of-sample predictionperformance.