Sparse Logistic Regression Model Research Articles

Epilepsy EEG signals classification based on sparse principal component logistic regression model

Epilepsy is a chronic brain disease caused by excessive discharge of brain neurons. Long-term recurrent seizures bring a lot of trouble to patients and their families. Prediction of different stages of epilepsy is of great significance. We extract pearson correlation coefficients (PCC) between channels in different frequency bands as features of EEG signals for epilepsy stages prediction. However, the features are of large feature dimension and serious multi-collinearity. To eliminate these adverse influence, the combination of traditional dimension reduction method principal component analysis (PCA) and logistic regression method with regularization term is proposed to avoid over-fitting and achieve the feature sparsity. The experiments are conducted on the widely used CHB-MIT dataset using different regularization terms L 1 and L 2 , respectively. The proposed method identifies various stages of epilepsy quickly and efficiently, and it presents the best average accuracy of 94.86%, average precision of 96.71%, average recall of 93.48%, average kappa value of 0.90 and average Matthews correlation coefficient (MCC) value of 0.90 for all patients.

Variable Selection for Sparse Logistic Regression with Grouped Variables

We present a new penalized method for estimation in sparse logistic regression models with a group structure. Group sparsity implies that we should consider the Group Lasso penalty. In contrast to penalized log-likelihood estimation, our method can be viewed as a penalized weighted score function method. Under some mild conditions, we provide non-asymptotic oracle inequalities promoting the group sparsity of predictors. A modified block coordinate descent algorithm based on a weighted score function is also employed. The net advantage of our algorithm over existing Group Lasso-type procedures is that the tuning parameter can be pre-specified. The simulations show that this algorithm is considerably faster and more stable than competing methods. Finally, we illustrate our methodology with two real data sets.

A fast trans-lasso algorithm with penalized weighted score function

An efficient transfer learning algorithm for high-dimensional sparse logistic regression models is proposed using penalized weighted score function based on square root Lasso, which intends to prespecify the tuning parameter. Three different choices of the tuning parameter are considered in the case of fixed design matrix. With a novel weight construction, the estimator of the regression vector is showed to be consistent when the inequality with respect to the Karush-Kuhn-Tucker (KKT) optimality conditions holds with high probability and the sparsity assumption for the regression vectors is required. There is information from source data to fit target data such that with high probability tending to 1-α, which is sharper than the corresponding probability bound without using the auxiliary samples, the KKT optimality conditions hold for the asymptotic choice. To detect which sources are transferable, an efficient data-driven method is proposed, which helps avoid negative transfer in practice. Simulation studies are carried out to demonstrate the numerical performance of the proposed procedure and their superiority over some existing methods. The procedures are also illustrated by analyzing the China Migrants Dynamic Survey dataset with binary outcomes concerning the associations among different provinces.

Relaxation Quadratic Approximation Greedy Pursuit Method Based on Sparse Learning

Abstract A high-performance sparse model is very important for processing high-dimensional data. Therefore, based on the quadratic approximate greed pursuit (QAGP) method, we can make full use of the information of the quadratic lower bound of its approximate function to get the relaxation quadratic approximate greed pursuit (RQAGP) method. The calculation process of the RQAGP method is to construct two inexact quadratic approximation functions by using the m-strongly convex and L-smooth characteristics of the objective function and then solve the approximation function iteratively by using the Iterative Hard Thresholding (IHT) method to get the solution of the problem. The convergence analysis is given, and the performance of the method in the sparse logistic regression model is verified on synthetic data and real data sets. The results show that the RQAGP method is effective.

Classification and biomarker selection in lower-grade glioma using robust sparse logistic regression applied to RNA-seq data

Effective diagnosis and treatment in cancer is a barrier for the development of personalized medicine, mostly due to tumor heterogeneity. In the particular case of gliomas, highly heterogeneous brain tumors at the histological, cellular and molecular levels, and exhibiting poor prognosis, the mechanisms behind tumor heterogeneity and progression remain poorly understood. The recent advances in biomedical high-throughput technologies have allowed the generation of large amounts of molecular information from the patients that combined with statistical and machine learning techniques can be used for the definition of glioma subtypes and targeted therapies, an invaluable contribution to disease understanding and effective management.In this work sparse and robust sparse logistic regression models with the elastic net penalty were applied to glioma RNA-seq data from The Cancer Genome Atlas (TCGA), to identify relevant transcriptomic features in the separation between lower-grade glioma (LGG) subtypes and identify putative outlying observations. In general, all classification models yielded good accuracies, selecting different sets of genes. Among the genes selected by the models, TXNDC12, TOMM20, PKIA, CARD8 and TAF12 have been reported as genes with relevant role in glioma development and progression. This highlights the suitability of the present approach to disclose relevant genes and fosters the biological validation of non-reported genes.

Multidimensional machine learning models predicting outcomes after trauma

BackgroundAn emerging body of literature supports the role of individualized prognostic tools to guide the management of patients after trauma. The aim of this study was to develop advanced modeling tools from multidimensional data sources, including immunological analytes and clinical and administrative data, to predict outcomes in trauma patients. MethodsThis was a prospective study of trauma patients at Level 1 centers from 2015 to 2019. Clinical, flow cytometry, and serum cytokine data were collected within 48 hours of admission. Sparse logistic regression models were developed, jointly selecting predictors and estimating the risk of ventilator-associated pneumonia, acute kidney injury, complicated disposition (death, rehabilitation, or nursing facility), and return to the operating room. Model parameters (regularization controlling model sparsity) and performance estimation were obtained via nested leave-one-out cross-validation. ResultsA total of 179 patients were included. The incidences of ventilator-associated pneumonia, acute kidney injury, complicated disposition, and return to the operating room were 17.7%, 28.8%, 22.5%, and 12.3%, respectively. Regarding extensive resource use, 30.7% of patients had prolonged intensive care unit stay, 73.2% had prolonged length of stay, and 23.5% had need for prolonged ventilatory support. The models were developed and cross-validated for ventilator-associated pneumonia, acute kidney injury, complicated dispositions, and return to the operating room, yielding predictive areas under the curve from 0.70 to 0.91. Each model derived its optimal predictive value by combining clinical, administrative, and immunological analyte data. ConclusionClinical, immunological, and administrative data can be combined to predict post-traumatic outcomes and resource use. Multidimensional machine learning modeling can identify trauma patients with complicated clinical trajectories and high resource needs.

Variable Selection for Spatial Logistic Autoregressive Models

When the spatial response variables are discrete, the spatial logistic autoregressive model adds an additional network structure to the ordinary logistic regression model to improve the classification accuracy. With the emergence of high-dimensional data in various fields, sparse spatial logistic regression models have attracted a great deal of interest from researchers. For the high-dimensional spatial logistic autoregressive model, in this paper, we propose a variable selection method with for the spatial logistic model. To identify important variables and make predictions, one efficient algorithm is employed to solve the penalized likelihood function. Simulations and a real example show that our methods perform well in a limited sample.

Autonomous Learning Interactive Features for Hyperspectral Remotely Sensed Data

In the field of remote sensing, most of the feature indexes are obtained based on expert knowledge or domain analysis. With the rapid development of machine learning and artificial intelligence, this method is time-consuming and lacks flexibility, and the indexes obtained cannot be applied to all areas. In order to not rely on expert knowledge and find the effective feature index with regard to a certain material automatically, this paper proposes a data-driven method to learn interactive features for hyperspectral remotely sensed data based on a sparse multiclass logistic regression model. The key point explicitly expresses the interaction relationship between original features as new features by multiplication or division operation in the logistic regression. Through the strong constraint of the L1 norm, the learned features are sparse. The coefficient value of the corresponding features after sparse represents the basis for judging the importance of the features, and the optimal interactive features among the original features. This expression is inspired by the phenomenon that usually the famous indexes we used in remote sensing, like NDVI, NDWI, are the ratio between different spectral bands, and also in statistical regression, the relationship between features is captured by feature value multiplication. Experiments were conducted on three hyperspectral data sets of Pavia Center, Washington DC Mall, and Pavia University. The results for binary classification show that the method can extract the NDVI and NDWI autonomously, and a new type of metal index is proposed in the Pavia University data set. This framework is more flexible and creative than the traditional method based on laboratory research to obtain the key feature and feature interaction index for hyperspectral remotely sensed data.

425Artificial Intelligence Approaches to Type 2 Diabetes Risk Prediction and Exploration of Predictive Factors

Abstract Background Major barriers exist in incorporating artificial intelligence into epidemiology, particularly in data interpretation. Thus, we examined the application of highly interpretable machine-learning methods— Random Forest (RF) and Sparse Logistic Regression (SLR)— to a large-scale health check-up dataset, examining the advantages of creating prediction models using these. Methods This study involved 392,791 participants who underwent healthcare checkups in Japan from 1999 to 2018. Participants who received diabetes treatment, or had an HbA1c level of 6.5% or higher, were excluded. The objective variable examined was type 2 diabetes onset over five years. Each prediction model was created using 26 health status items over three consecutive years. We examined three analytical methods to compare their predictive powers: RF, SLR, and a multivariate stepwise logistic regression (MSLR) as a conventional method. Variable Importance (VI) was calculated in the RF analysis, with Standard Regression Coefficients (SRC) being calculated in the SLR and MSLR analyses. Results Predictive accuracy is highest in the SLR model (AUC:0.955), followed by the RF model (AUC:0.949), and then the MSLR model (AUC:0.939). The RF model measures blood glucose, HbA1c, height, red blood cells, and aspartate transaminase with a higher predictive power. In the SLR model, HbA1c, blood glucose, systolic blood pressure, HDL-Cholesterol, and age have higher SRC. Conclusions Machine learning techniques enable more accurate diabetes risk predictions than existing methods and suggest new ways of identifying associated predictors. Key messages Applying machine-learning methods to health check-up data achieves a high accuracy in predicting type 2 diabetes while maintaining data interpretability.

Using topic models to identify clients' functioning levels and alliance ruptures in psychotherapy.

Computerized natural language processing techniques can analyze psychotherapy sessions as texts, thus generating information about the therapy process and outcome and supporting the scaling-up of psychotherapy research. We used topic modeling to identify topics discussed in psychotherapy sessions and explored (a) which topics best identified clients' functioning and alliance ruptures and (b) whether changes in these topics were associated with changes in outcome. Transcripts of 873 sessions from 58 clients treated by 52 therapists were analyzed. Before each session, clients self-reported functioning and symptom level. After each session, therapists reported the extent of alliance rupture. Latent Dirichlet allocation was used to extract latent topics from psychotherapy textual data. Then a sparse multinomial logistic regression model was used to predict which topics best identified clients' functioning levels and the occurrence of alliance ruptures in psychotherapy sessions. Finally, we used multilevel growth models to explore the associations between changes in topics and changes in outcome. Session-based processing yielded a list of semantic topics. The model identified the labels above chance (65% to 75% accuracy). Change trajectories in topics were associated with change trajectories in outcome. The results suggest that topic models can exploit rich linguistic data within sessions to identify psychotherapy process and outcomes. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

RETRIEVAL OF COMPLEX IMAGES USING VISUAL SALIENCY GUIDED COGNITIVE CLASSIFICATION

Data storage via multimedia technology is more preferred as the information in multimedia contain rich meanings and are concise when compared to the traditional textual information. However, efficient information retrieval is a crucial factor in such storage. This paper presents a cognitive classification based visual saliency guided model for the efficient retrieval of information from multimedia data storage. The Itti visual saliency model is described here for generation of an overall saliency map with the integration of color saliency, intensity and direction maps. Multi-feature fusion paradigms are used for providing clear description of the image pattern. The definition is based on two stages namely complexity based on cognitive load and classification of complexity at a cognitive level. The image retrieval system is finalized by integrating a group sparse logistic regression model. In complex scenarios, the baselines are overcome by the proposed system when tested on multiple databased as compared to other state-of-the-art models.

Alzheimer-type dementia prediction by sparse logistic regression using claim data

This study aimed to predict the risk of Alzheimer-type dementia for persons aged over 75 years old without receiving long-term care services using regularly collected claim data. A refined dataset including 48,123 persons was prepared from claim data of health insurance and long-term care insurance in a large city in the metropolitan area in Japan. The utilized features include the age and sex of subjects, 502 diseases based on ICD-10 diagnosis codes, and 107 prescription drugs based on therapeutic classes. The most important challenge in this work was feature selection form a large number of features. We adopted sparse logistic regression models with L0 regularization (SLR-L0) and L1 regularization (SLR-L1) as classification models based on machine learning. These regularizations enable feature selection by estimating sparse solution of non-zero coefficients in the model optimization. Predictions were performed by integrating 100 predictors trained by bootstrap samples. As a result, the area under the ROC curves (AUCs) were 0.663 for SLR-L0 and 0.660 for SLR-L1. These performances were similar, however, the average numbers of selected features were 13 out of a total of 611 for SLR-L0 and 253 for SLR-R1. The results indicate that SLR-L1 tended to include less useful features, whereas SLR-L0 narrowed down influential features. SLR-L0 might be more useful than SLR-L1 for practical use or the discussion of risk factors with medical experts.

Comparison of Different Machine Learning Approaches to Predict Small for Gestational Age Infants

Diagnosing infants who are small for gestational age (SGA) at early stages could help physicians to introduce interventions for SGA infants earlier. Machine learning (ML) is envisioned as a tool to identify SGA infants. However, ML has not been widely studied in this field. To develop effective SGA prediction models, we conducted four groups of experiments that considered basic ML methods, imbalanced data, feature selection and the time characteristics of variables, respectively. Infants with SGA data collected from 2010 to 2013 with gestational weeks between 24 and 42 were detected. Support vector machine (SVM), random forest (RF), logistic regression (LR) and Sparse LR models were trained on 10-fold cross validation. Precision and the area under the curve (AUC) of the receiver operator characteristic curve were evaluated. For each group, the performance of SVM and Sparse LR was similarly well. LR without any sparsity penalties performed worst, possibly caused by the overfitting problem. With the combination of handling imbalanced data and feature selection, the RF ensemble classifier performed best, which even obtained the highest AUC value (0.8547) with the help of expert knowledge. In other cases, RF performed worse than Sparse LR and SVM, possibly because of fully grown trees.

Prediction of tumour pathological subtype from genomic profile using sparse logistic regression with random effects

The purpose of this study is to highlight the application of sparse logistic regression models in dealing with prediction of tumour pathological subtypes based on lung cancer patients' genomic information. We consider sparse logistic regression models to deal with the high dimensionality and correlation between genomic regions. In a hierarchical likelihood (HL) method, it is assumed that the random effects follow a normal distribution and its variance is assumed to follow a gamma distribution. This formulation considers ridge and lasso penalties as special cases. We extend the HL penalty to include a ridge penalty (called ‘HLnet’) in a similar principle of the elastic net penalty, which is constructed from lasso penalty. The results indicate that the HL penalty creates more sparse estimates than lasso penalty with comparable prediction performance, while HLnet and elastic net penalties have the best prediction performance in real data. We illustrate the methods in a lung cancer study.

Shape-Adaptive Tensor Factorization Model for Dimensionality Reduction of Hyperspectral Images

Tensor-based dimensionality reduction (DR) of hyperspectral images is a promising research topic. However, patch-based tensorization usually adopts a squared neighborhood with fixed window size, which may be inaccurate in modeling the local spatial information in a hyperspectral image scene. In this work, we propose a novel shape-adaptive tensor factorization (SATF) model for dimensionality reduction and classification of hyperspectral images. Firstly, shape-adaptive patch features are extracted to build fourth-order tensors. Secondly, multilinear singular value decomposition (MLSVD) is adopted for tensor factorization and latent features are extracted via mode-i tensor-matrix product. Finally, classification is conducted by using a sparse multinomial logistic regression (SMLR) model. Experimental results, conducted with two popular hyperspectral data sets collected over the Indian Pines and the University of Pavia, respectively, indicate that the proposed method outperforms the other traditional and tensor-based DR methods.

A miRNA Host Response Signature Accurately Discriminates Acute Respiratory Infection Etiologies.

Background: Acute respiratory infections (ARIs) are the leading indication for antibacterial prescriptions despite a viral etiology in the majority of cases. The lack of available diagnostics to discriminate viral and bacterial etiologies contributes to this discordance. Recent efforts have focused on the host response as a source for novel diagnostic targets although none have explored the ability of host-derived microRNAs (miRNA) to discriminate between these etiologies.Methods: In this study, we compared host-derived miRNAs and mRNAs from human H3N2 influenza challenge subjects to those from patients with Streptococcus pneumoniae pneumonia. Sparse logistic regression models were used to generate miRNA signatures diagnostic of ARI etiologies. Generalized linear modeling of mRNAs to identify differentially expressed (DE) genes allowed analysis of potential miRNA:mRNA relationships. High likelihood miRNA:mRNA interactions were examined using binding target prediction and negative correlation to further explore potential changes in pathway regulation in response to infection.Results: The resultant miRNA signatures were highly accurate in discriminating ARI etiologies. Mean accuracy was 100% [88.8–100; 95% Confidence Interval (CI)] in discriminating the healthy state from S. pneumoniae pneumonia and 91.3% (72.0–98.9; 95% CI) in discriminating S. pneumoniae pneumonia from influenza infection. Subsequent differential mRNA gene expression analysis revealed alterations in regulatory networks consistent with known biology including immune cell activation and host response to viral infection. Negative correlation network analysis of miRNA:mRNA interactions revealed connections to pathways with known immunobiology such as interferon regulation and MAP kinase signaling.Conclusion: We have developed novel human host-response miRNA signatures for bacterial and viral ARI etiologies. miRNA host response signatures reveal accurate discrimination between S. pneumoniae pneumonia and influenza etiologies for ARI and integrated analyses of the host-pathogen interface are consistent with expected biology. These results highlight the differential miRNA host response to bacterial and viral etiologies of ARI, offering new opportunities to distinguish these entities.

A novel variational Bayesian method for variable selection in logistic regression models

With high-dimensional data emerging in various domains, sparse logistic regression models have gained much interest of researchers. Variable selection plays a key role in both improving the prediction accuracy and enhancing the interpretability of built models. Bayesian variable selection approaches enjoy many advantages such as high selection accuracy, easily incorporating many kinds of prior knowledge and so on. Because Bayesian methods generally make inference from the posterior distribution with Markov Chain Monte Carlo (MCMC) techniques, however, they become intractable in high-dimensional situations due to the large searching space. To address this issue, a novel variational Bayesian method for variable selection in high-dimensional logistic regression models is presented. The proposed method is based on the indicator model in which each covariate is equipped with a binary latent variable indicating whether it is important. The Bernoulli-type prior is adopted for the latent indicator variable. As for the specification of the hyperparameter in the Bernoulli prior, we provide two schemes to determine its optimal value so that the novel model can achieve sparsity adaptively. To identify important variables and make predictions, one efficient variational Bayesian approach is employed to make inference from the posterior distribution. The experiments conducted with both synthetic and some publicly available data show that the new method outperforms or is very competitive with some other popular counterparts.

F-score feature selection based Bayesian reconstruction of visual image from human brain activity

Abstract Decoding perceptual experience from human brain activity is a big challenge in neuroscience. Recent advances in human neuroimaging have shown that it is possible to reconstruct a person’s visual experience based on the retinotopy in the early visual cortex and the multivariate pattern analysis (MVPA) method using functional magnetic resonance imaging (fMRI). Previous researches reconstructed binary contrast-defined images using combination of multi-scale local image decoders in V1, V2 and V3, where contrast for local image bases was predicted from fMRI activity by sparse multinomial logistic regression (SMLR) and other models. However, the precision and efficiency of the visual image reconstruction remain insufficient. Proper feature selection is widely known to be as critical for prediction and reconstruction. Aiming at the shortcomings of existing reconstruction models, we proposed a new model of Bayesian reconstruction based on F-score feature selection (Bayes+F). The results indicate that the proposed Bayes+F model has better reconstruction accuracy and higher efficiency than the SMLR and other models, showing better robustness and noise resistant ability. It can improve the spatial correlation coefficient (Mean ± variance: 0.7078 ± 0.2104) and decrease the standard error (Mean ± variance: 0.2693 ± 0.0871) between the stimulus and the reconstructed image. Furthermore, the proposed model can reconstruct the images extremely rapid, 100 times faster than SMLR does.

Visual saliency guided complex image retrieval

ABSTRACT Compared with the traditional text data, multimedia data are concise and contains rich meanings, so people are more willing to use the multimedia data to store information. How to effectively retrieve information is essential. This paper proposes a novel visual saliency guided complex image retrieval model. Initially, Itti visual saliency model is presented. In this model, the overall saliency map is generated by the integration of direction, intensity and color saliency map, respectively. Then, to help describe the image pattern more clearly, we present the multi-feature fusion paradigm of images. To address the complexity of the images, we propose a two-stage definition: (1) Cognitive load based complexity; (2) Cognitive level of complexity classification. The group sparse logistic regression model is integrated to finalize the image retrieval system. The performance of the proposed system is tested on different databases compared with the other state-of-the-art models which overcome the baselines in complex scenarios.

A novel SMLR-PSO model to estimate the chlorophyll content in the crops using hyperspectral satellite images

The estimating the chlorophyll contents in the crops helps to identify the condition of crops and different classification of crops with soil characteristics in order to assist the farmer or others with agriculture growth. In this paper, a hybrid approach is introduced to estimate the Chlorophyll contents in the crops using hyperspectral image segmentation with active learning, which consists of two main steps. First, we use a sparse multinomial logistic regression (SMLR) model to learn the class posterior probability distributions with Quadratic Programming or joint probability distribution. Second, we use the information acquired in the previous step to segment the hyper spectral image using a Markov Random field segments to estimate the dependencies using spatial information and edge Information by minimum spanning forest rooted on markers. In order to reduce the cost of acquiring large training sets, PSO optimization is performed based on the SMLR posterior probabilities on the Normalized difference vegetation index (NDVI). The state-of-the-art performance of the proposed approach is illustrated using real hyper spectral data sets collected from the North Karnataka in a number of experimental comparisons with recently developed or statistical hyperspectral image analysis methods in terms of precision, recall and f—measure.