Greedy Forward Selection Research Articles

Targeted and Untargeted Amine Metabolite Quantitation in Single Cells with Isobaric Multiplexing.

We developed a single cell amine analysis approach utilizing isobarically multiplexed samples of 6 individual cells along with analyte abundant carrier. This methodology was applied for absolute quantitation of amino acids and untargeted relative quantitation of amines in a total of 108 individual cells using nanoflow LC with high-resolution mass spectrometry. Together with individually determined cell sizes, this provides quantification of intracellularconcentrations within individual cells. The targeted method was partially validated for 10 amino acids with limits of detection in low attomoles, linear calibration range covering analyte amounts typically from 30 amol to 120 fmol, and correlation coefficients (R) above 0.99. This was applied with cell sizes recorded during dispensing to determine millimolar intracellular amino acid concentrations. The untargeted approach yielded 249 features that were detected in at least 25% of the single cells, providing modest cell type separation on principal component analysis. Using Greedy forward selection with regularized least squares, a sub-selection of 100 features explaining most of the difference was determined. These features were annotated using MS2 from analyte standards and accurate mass with library search. The approach provides accessible, sensitive, and high-throughput method with the potential to be expanded also to other forms of ultrasensitive analysis.

Scalable classifier-agnostic channel selection for multivariate time series classification

Accuracy is a key focus of current work in time series classification. However, speed and data reduction are equally important in many applications, especially when the data scale and storage requirements rapidly increase. Current multivariate time series classification (MTSC) algorithms need hundreds of compute hours to complete training and prediction. This is due to the nature of multivariate time series data which grows with the number of time series, their length and the number of channels. In many applications, not all the channels are useful for the classification task, hence we require methods that can efficiently select useful channels and thus save computational resources. We propose and evaluate two methods for channel selection. Our techniques work by representing each class by a prototype time series and performing channel selection based on the prototype distance between classes. The main hypothesis is that useful channels enable better separation between classes; hence, channels with a larger distance between class prototypes are more useful. On the UEA MTSC benchmark, we show that these techniques achieve significant data reduction and classifier speedup for similar levels of classification accuracy. Channel selection is applied as a pre-processing step before training state-of-the-art MTSC algorithms and saves about 70% of computation time and data storage with preserved accuracy. Furthermore, our methods enable efficient classifiers, such as ROCKET, to achieve better accuracy than using no selection or greedy forward channel selection. To further study the impact of our techniques, we present experiments on classifying synthetic multivariate time series datasets with more than 100 channels, as well as a real-world case study on a dataset with 50 channels. In both cases, our channel selection methods result in significant data reduction with preserved or improved accuracy.

End-to-end learnable EEG channel selection for deep neural networks with Gumbel-softmax

Objective. To develop an efficient, embedded electroencephalogram (EEG) channel selection approach for deep neural networks, allowing us to match the channel selection to the target model, while avoiding the large computational burdens of wrapper approaches in conjunction with neural networks. Approach. We employ a concrete selector layer to jointly optimize the EEG channel selection and network parameters. This layer uses a Gumbel-softmax trick to build continuous relaxations of the discrete parameters involved in the selection process, allowing them be learned in an end-to-end manner with traditional backpropagation. As the selection layer was often observed to include the same channel twice in a certain selection, we propose a regularization function to mitigate this behavior. We validate this method on two different EEG tasks: motor execution and auditory attention decoding. For each task, we compare the performance of the Gumbel-softmax method with a baseline EEG channel selection approach tailored towards this specific task: mutual information and greedy forward selection with the utility metric respectively. Main results. Our experiments show that the proposed framework is generally applicable, while performing at least as well as (and often better than) these state-of-the-art, task-specific approaches. Significance. The proposed method offers an efficient, task- and model-independent approach to jointly learn the optimal EEG channels along with the neural network weights.

Cost-Constrained feature selection in binary classification: adaptations for greedy forward selection and genetic algorithms

BackgroundWith modern methods in biotechnology, the search for biomarkers has advanced to a challenging statistical task exploring high dimensional data sets. Feature selection is a widely researched preprocessing step to handle huge numbers of biomarker candidates and has special importance for the analysis of biomedical data. Such data sets often include many input features not related to the diagnostic or therapeutic target variable. A less researched, but also relevant aspect for medical applications are costs of different biomarker candidates. These costs are often financial costs, but can also refer to other aspects, for example the decision between a painful biopsy marker and a simple urine test. In this paper, we propose extensions to two feature selection methods to control the total amount of such costs: greedy forward selection and genetic algorithms. In comprehensive simulation studies of binary classification tasks, we compare the predictive performance, the run-time and the detection rate of relevant features for the new proposed methods and five baseline alternatives to handle budget constraints.ResultsIn simulations with a predefined budget constraint, our proposed methods outperform the baseline alternatives, with just minor differences between them. Only in the scenario without an actual budget constraint, our adapted greedy forward selection approach showed a clear drop in performance compared to the other methods. However, introducing a hyperparameter to adapt the benefit-cost trade-off in this method could overcome this weakness.ConclusionsIn feature cost scenarios, where a total budget has to be met, common feature selection algorithms are often not suitable to identify well performing subsets for a modelling task. Adaptations of these algorithms such as the ones proposed in this paper can help to tackle this problem.

Comparison of estimators and feature selection procedures in forest inventory based on airborne laser scanning and digital aerial imagery

ABSTRACTDigital maps of forest resources are a crucial factor in successful forestry applications. Since manual measurement of this data on large areas is infeasible, maps must be constructed using a sample field data set and a prediction model constructed from remote sensing materials, of which airborne laser scanning (ALS) data and aerial images are currently widely used in management planning inventories. ALS data is suitable for the prediction of variables related to the size and volume of trees, whereas optical imagery helps in improving distinction between tree species. We studied the prediction of forest attributes using field data from National Forest Inventory complemented with ad hoc field plots in combination with ALS and aerial imagery data in Aland province, Finland. We applied feature selection with genetic algorithm and greedy forward selection and compared multiple linear and nonlinear estimators. Maximally around 40 features from a total of 154 were required to achieve the best prediction performances. Tree height was predicted with normalized root mean squared error value of 0.1 and tree volume with a value around 0.25. Predicting the volumes of spruce and broadleaved trees was the most challenging due to small proportions of these tree species in the study area.

Load forecasting techniques for power systems with high levels of unmetered renewable generation: A comparative study

Load forecasting remains a challenging problem in power system operation due to growth in low carbon technologies and distributed small scale renewable generation. In this paper we provide a comparative evaluation of a number of linear and non-linear (machine learning) load forecasting models for day-ahead load forecasting under these new conditions. Both autoregressive and exogenous input only models are considered with regressors determined either empirically or by a greedy forward selection methodology. Using data from the Northern Ireland power system as a case study, we show that non-linear models yield significant performance improvements for exogenous input (EI) based models, but that linear models remain competitive for same day last week (SDLW) models that include a historical load term as a regressor.

Method of automatic ROI selection on lung CT images

We developed a method of automatic ROI selection on two-dimensional slices of computed tomography of the lungs for further computer diagnostics of emphysema. The method is based on a greedy forward selection using local texture features as an indicator of similarity with regions of interest from the learning sample. An experimental study of the method was performed on the set of 160 real digital images. The estimation of the error probability with automatic ROI selection was 0.03, which is less than in the case of manual selection.

Predicting Negative Emotions Based on Mobile Phone Usage Patterns: An Exploratory Study.

BackgroundPrompt recognition and intervention of negative emotions is crucial for patients with depression. Mobile phones and mobile apps are suitable technologies that can be used to recognize negative emotions and intervene if necessary.ObjectiveMobile phone usage patterns can be associated with concurrent emotional states. The objective of this study is to adapt machine-learning methods to analyze such patterns for the prediction of negative emotion.MethodsWe developed an Android-based app to capture emotional states and mobile phone usage patterns, which included call logs (and use of apps). Visual analog scales (VASs) were used to report negative emotions in dimensions of depression, anxiety, and stress. In the system-training phase, participants were requested to tag their emotions for 14 consecutive days. Five feature-selection methods were used to determine individual usage patterns and four machine-learning methods were tested. Finally, rank product scoring was used to select the best combination to construct the prediction model. In the system evaluation phase, participants were then requested to verify the predicted negative emotions for at least 5 days.ResultsOut of 40 enrolled healthy participants, we analyzed data from 28 participants, including 30% (9/28) women with a mean (SD) age of 29.2 (5.1) years with sufficient emotion tags. The combination of time slots of 2 hours, greedy forward selection, and Naïve Bayes method was chosen for the prediction model. We further validated the personalized models in 18 participants who performed at least 5 days of model evaluation. Overall, the predictive accuracy for negative emotions was 86.17%.ConclusionWe developed a system capable of predicting negative emotions based on mobile phone usage patterns. This system has potential for ecological momentary intervention (EMI) for depressive disorders by automatically recognizing negative emotions and providing people with preventive treatments before it escalates to clinical depression.

Greedy Learning of Sparse Eigenfaces for Face Recognition and Tracking

Appearance-based subspace models such as eigenfaces have been widely recognized as one of the most successful approaches to face recognition and tracking. The success of eigenfaces mainly has its origins in the benefits offered by principal component analysis (PCA), the representational power of the underlying generative process for high-dimensional noisy facial image data. The sparse extension of PCA (SPCA) has recently received significant attention in the research community. SPCA functions by imposing sparseness constraints on the eigenvectors, a technique that has been shown to yield more robust solutions in many applications. However, when SPCA is applied to facial images, the time and space complexity of PCA learning becomes a critical issue (e.g., real-time tracking). In this paper, we propose a very fast and scalable greedy forward selection algorithm for SPCA. Unlike a recent semidefinite program-relaxation method that suffers from complex optimization, our approach can process several thousands of data dimensions in reasonable time with little accuracy loss. The effectiveness of our proposed method was demonstrated on real-world face recognition and tracking datasets.

Software defect prediction using ensemble learning on selected features

ContextSeveral issues hinder software defect data including redundancy, correlation, feature irrelevance and missing samples. It is also hard to ensure balanced distribution between data pertaining to defective and non-defective software. In most experimental cases, data related to the latter software class is dominantly present in the dataset. ObjectiveThe objectives of this paper are to demonstrate the positive effects of combining feature selection and ensemble learning on the performance of defect classification. Along with efficient feature selection, a new two-variant (with and without feature selection) ensemble learning algorithm is proposed to provide robustness to both data imbalance and feature redundancy. MethodWe carefully combine selected ensemble learning models with efficient feature selection to address these issues and mitigate their effects on the defect classification performance. ResultsForward selection showed that only few features contribute to high area under the receiver-operating curve (AUC). On the tested datasets, greedy forward selection (GFS) method outperformed other feature selection techniques such as Pearson’s correlation. This suggests that features are highly unstable. However, ensemble learners like random forests and the proposed algorithm, average probability ensemble (APE), are not as affected by poor features as in the case of weighted support vector machines (W-SVMs). Moreover, the APE model combined with greedy forward selection (enhanced APE) achieved AUC values of approximately 1.0 for the NASA datasets: PC2, PC4, and MC1. ConclusionThis paper shows that features of a software dataset must be carefully selected for accurate classification of defective components. Furthermore, tackling the software data issues, mentioned above, with the proposed combined learning model resulted in remarkable classification performance paving the way for successful quality control.

Multi-label learning under feature extraction budgets

We consider the problem of learning sparse linear models for multi-label prediction tasks under a hard constraint on the number of features. Such budget constraints are important in domains where the acquisition of the feature values is costly. We propose a greedy multi-label regularized least-squares algorithm that solves this problem by combining greedy forward selection search with a cross-validation based selection criterion in order to choose, which features to include in the model. We present a highly efficient algorithm for implementing this procedure with linear time and space complexities. This is achieved through the use of matrix update formulas for speeding up feature addition and cross-validation computations. Experimentally, we demonstrate that the approach allows finding sparse accurate predictors on a wide range of benchmark problems, typically outperforming the multi-task lasso baseline method when the budget is small.

Integrated classifier hyperplane placement and feature selection

The process of placing a separating hyperplane for data classification is normally disconnected from the process of selecting the features to use. An approach for feature selection that is conceptually simple but computationally explosive is to simply apply the hyperplane placement process to all possible subsets of features, selecting the smallest set of features that provides reasonable classification accuracy. Two ways to speed this process are (i) use a faster filtering criterion instead of a complete hyperplane placement, and (ii) use a greedy forward or backwards sequential selection method. This paper introduces a new filtering criterion that is very fast: maximizing the drop in the sum of infeasibilities in a linear-programming transformation of the problem. It also shows how the linear programming transformation can be applied to reduce the number of features after a separating hyperplane has already been placed while maintaining the separation that was originally induced by the hyperplane. Finally, a new and highly effective integrated method that simultaneously selects features while placing the separating hyperplane is introduced.

Kernel Least-Squares Models Using Updates of the Pseudoinverse

Sparse nonlinear classification and regression models in reproducing kernel Hilbert spaces (RKHSs) are considered. The use of Mercer kernels and the square loss function gives rise to an overdetermined linear least-squares problem in the corresponding RKHS. When we apply a greedy forward selection scheme, the least-squares problem may be solved by an order-recursive update of the pseudoinverse in each iteration step. The computational time is linear with respect to the number of the selected training samples.