Nonparametric Feature Research Articles

Wearable EEG-Based Depth of Anesthesia Monitoring: A Nonparametric Feature Set

Wearable EEG-Based Depth of Anesthesia Monitoring: A Nonparametric Feature Set

Denoising point clouds with fewer learnable parameters

Point cloud denoising is a crucial task in the field of geometric processing. Recent years have witnessed significant advancements in deep learning-based point cloud denoising algorithms. These methods, compared to traditional techniques, demonstrate enhanced robustness against noise and produce point cloud data of higher fidelity. Despite their impressive performance, achieving a balance between denoising efficacy and computational efficiency remains a formidable challenge in learning-based methods. To solve this problem, we introduce LPCDNet, a novel lightweight point cloud denoising network. LPCDNet consists of three main components: a lightweight feature extraction module utilizing trigonometric functions for relative position encoding; a non-parametric feature aggregation module to leverage semantic similarities for global context comprehension; and a decoder module designed to realign noise points with the underlying surface. The network is designed to capture both local details and non-local structures, thereby ensuring high-quality denoising outcomes with a minimal parameter count. Extensive experimental evaluations demonstrate that LPCDNet achieves comparable or superior performance to state-of-the-art methods, while significantly reducing the number of learnable parameters and the necessary running time.

Investigating Growth-at-Risk Using a Multicountry Nonparametric Quantile Factor Model

We develop a nonparametric quantile panel regression model. Within each quantile, the quantile function is a combination of linear and nonlinear parts, which we approximate using Bayesian Additive Regression Trees (BART). Cross-sectional information is captured through a conditionally heteroscedastic latent factor. The nonparametric feature enhances flexibility, while the panel feature increases the number of observations in the tails. We develop Bayesian methods for inference and apply several versions of the model to study growth-at-risk dynamics in a panel of 11 advanced economies. Our framework usually improves upon single-country quantile models in recursive growth forecast comparisons.

Linking stability with molecular geometries of perovskites and lanthanide richness using machine learning methods

Oxide perovskite materials of type ABO3 have a wide range of technological applications, such as catalysts in solid oxide fuel cells and as light-absorbing materials in solar photovoltaics. These materials often exhibit differential structural and electrostatic properties through their varied A-sites and B-sites especially if they are derived from lanthanides. Although, experimental and/or computational verification of these differences are often difficult and expensive. In this paper, we thus take a data-driven approach. Specifically, we run three analysis using the dataset by [1] applying advanced machine learning tools to perform nonparametric regressions and also to produce data visualizations using latent factor analysis (LFA) and principal component analysis (PCA). We also implement a nonparametric feature screening step while performing our high dimensional regression analysis, ensuring robustness in our results.

Nonparametric feature impact and importance

Practitioners use feature importance to rank and eliminate weak predictors during model development in an effort to simplify models and improve generality. Unfortunately, they also routinely conflate such feature importance measures with feature impact, the isolated effect of an explanatory variable on the response variable. This can lead to real-world consequences when importance is inappropriately interpreted as impact in applications like medicine and business. The dominant approach for computing feature importance is through interrogation of a fitted model, which works well for feature selection, but gives distorted measures of feature impact. For example, the same method applied to the same data set can yield different feature importances, depending on the model, leading us to conclude that impact should be computed directly from the data. While there are nonparametric feature selection algorithms, they typically provide feature rankings, rather than direct measures of impact or importance. They also often focus on single-variable associations with the response. In this paper, we provide mathematical definitions of feature impact and importance, derived from partial dependence curves, that operate directly on the data. We develop two methods, StratImpact and StratImp, that estimate feature impact and importance from partial dependence measures using stratification of the explanatory variables. We show that features ranked by these definitions are competitive with, and often better than, existing feature selection techniques. We validate our methods through a comparison with contemporary methods using three real data sets and a testbed of simulated data.

Aboveground Biomass Retrieval in Tropical and Boreal Forests Using L-Band Airborne Polarimetric Observations

Forests play a crucial part in regulating global climate change since their aboveground biomass (AGB) relates to the carbon cycle, and its changes affect the main carbon pools. At present, the most suitable available SAR data for wall-to-wall forest AGB estimation are exploiting an L-band polarimetric SAR. However, the saturation issues were reported for AGB estimation using L-band backscatter coefficients. Saturation varies depending on forest structure. Polarimetric information has the capability to identify different aspects of forest structure and therefore shows great potential for reducing saturation issues and improving estimation accuracy. In this study, 121 polarimetric decomposition observations, 10 polarimetric backscatter coefficients and their derived observations, and six texture features were extracted and applied for forest AGB estimation in a tropical forest and a boreal forest. A parametric feature optimization inversion model (Multiple linear stepwise regression, MSLR) and a nonparametric feature optimization inversion model (fast iterative procedure integrated into a K-nearest neighbor nonparameter algorithm, KNNFIFS) were used for polarimetric features optimization and forest AGB inversion. The results demonstrated the great potential of L-band polarimetric features for forest AGB estimation. KNNFIFS performed better both in tropical (R2 = 0.80, RMSE = 22.55 Mg/ha, rRMSE = 14.59%, MA%E = 12.21%) and boreal (R2 = 0.74, RMSE = 19.82 Mg/ha, rRMSE = 20.86%, MA%E = 20.19%) forests. Non-model-based polarimetric features performed better compared to features extracted by backscatter coefficients, model-based decompositions, and texture. Polarimetric observations also revealed site-dependent performances.

A Novel Nonparametric Feature Selection Approach Based on Mutual Information Transfer Network.

The filter feature selection algorithm is habitually used as an effective way to reduce the computational cost of data analysis by selecting and implementing only a subset of original features into the study. Mutual information (MI) is a popular measurement adopted to quantify the dependence among features. MI-based greedy forward methods (MIGFMs) have been widely applied to escape from computational complexity and exhaustion of high-dimensional data. However, most MIGFMs are parametric methods that necessitate proper preset parameters and stopping criteria. Improper parameters may lead to ignorance of better results. This paper proposes a novel nonparametric feature selection method based on mutual information and mixed-integer linear programming (MILP). By forming a mutual information network, we transform the feature selection problem into a maximum flow problem, which can be solved with the Gurobi solver in a reasonable time. The proposed method attempts to prevent negligence on obtaining a superior feature subset while keeping the computational cost in an affordable range. Analytical comparison of the proposed method with six feature selection methods reveals significantly better results compared to MIGFMs, considering classification accuracy.

A general framework of nonparametric feature selection in high-dimensional data.

Nonparametric feature selection for high-dimensional data is an important and challenging problem in the fields of statistics and machine learning. Most of the existing methods for feature selection focus on parametric or additive models which may suffer from model misspecification. In this paper, we propose a new framework to perform nonparametric feature selection for both regression and classification problems. Under this framework, we learn prediction functions through empirical risk minimization over a reproducing kernel Hilbert space. The space is generated by a novel tensor product kernel, which depends on a set of parameters that determines the importance of the features. Computationally, we minimize the empirical risk with a penalty to estimate the prediction and kernel parameters simultaneously. The solution can be obtained by iteratively solving convex optimization problems. We study the theoretical property of the kernel feature space and prove the oracle selection property and Fisher consistency of our proposed method. Finally, we demonstrate the superior performance of our approach compared to existing methods via extensive simulation studies and applications to two realstudies.

Nonparametric feature selection by random forests and deep neural networks

Random forests are a widely used machine learning algorithm, but their computational efficiency is undermined when applied to large-scale datasets with numerous instances and useless features. Herein, we propose a nonparametric feature selection algorithm that incorporates random forests and deep neural networks, and its theoretical properties are also investigated under regularity conditions. Using different synthetic models and a real-world example, we demonstrate the advantage of the proposed algorithm over other alternatives in terms of identifying useful features, avoiding useless ones, and the computation efficiency. Although the algorithm is proposed using standard random forests, it can be widely adapted to other machine learning algorithms, as long as features can be sorted accordingly.

An Efficient Method for Detection and Localization of Myocardial Infarction

Electrocardiography (ECG) is a noninvasive diagnostic tool used to diagnose coronary artery disease, myocardial infarction (MI), and other heart disorders. As MI is the death of cardiac muscle tissue owing to a coronary artery blockage. It should be diagnosed at an early stage to provide timely treatment and thereby save lives. This study proposes a novel efficient technique for MI detection and localization based on variational mode decomposition (VMD) and regularized neighborhood component analysis (RNCA). Statistical and nonlinear features calculated from intrinsic mode functions decomposed by VMD create the final feature set. These features are ranked using RNCA, a nonparametric feature ranking approach, and then fed into the k-nearest neighbors (KNN) and AdaBoost classifiers with minimum features. With 12-Lead ECG System, i.e., ten electrodes on the body for detection might be costly and restricts patient movement. The Physikalisch-Technische Bundesanstalt ECG diagnostic database consisting of all 12 leads data is studied to find the minimum number of leads needed for successful MI diagnosis and localizing infarcted artery. Using 33 features from lead 8 (V2) with the KNN, the proposed approach gave the best accuracy of 99.82% for detection compared to previous related studies. The technique also distinguished the 11 types of MI with 99.75% accuracy utilizing 22 features from lead 7 (V1) with the KNN. The study reveals that an automated diagnostic tool for the portable device may be built just by chest lead. As VMD provides a robust solution against noise, this algorithm is excellent for portable health devices.

Parallel dynamic topic modeling via evolving topic adjustment and term weighting scheme

The parallel Hierarchical Dirichlet Process (pHDP) is an efficient topic model which explores the equivalence of the generation process between Hierarchical Dirichlet Process (HDP) and Gamma-Gamma-Poisson Process (G2PP), in order to achieve parallelism at the topic level. Unfortunately, pHDP loses the non-parametric feature of HDP, i.e., the number of topics in pHDP is predetermined and fixed. Furthermore, under the bootstrap structure of pHDP, the topic-indiscriminate words are of high probabilities to be assigned to different topics, resulting in poor qualities of the extracted topics. To achieve parallelism without sacrificing the non-parametric feature of HDP, in addition to improve the quality of extracted topics, we propose a parallel dynamic topic model by developing an adjustment mechanism of evolving topics and reducing the sampling probabilities of topic-indiscriminate words. Both supervised and unsupervised experiments on benchmark datasets show the competitive performance of our model.

Low-resolution face recognition in resource-constrained environments

• A LR face recognition model for resource-constrained environments is proposed. • Our model is lightweight and capable of being effectively trained on small data. • Our model adopts PixelHop++ which is designed based on the SSL principle. • Active learning is incorporated to minimize the required labeled training data. • We show the competitive performance of our model compared with state-of-the-art. Although Deep Neural Networks (DNNs) have achieved tremendous success in the face recognition task, utilizing them in resource-constrained environments with limited networking and computing is challenging. Such environments often demand a small model capable of being effectively trained on a small number of labeled training data, with low training complexity, and low-resolution input images. To address these challenges, we adopt an emerging machine learning methodology called Successive Subspace Learning (SSL) to propose LRFRHop, a high-performance data-efficient low-resolution face recognition model for resource-constrained environments. SSL offers an explainable non-parametric feature extraction submodel that flexibly trades the model size for the verification performance. Its training complexity is significantly lower than DNN-based models since it is trained in a one-pass feedforward manner without backpropagation. Furthermore, active learning can be conveniently incorporated to reduce the labeling cost. We demonstrate the effectiveness of LRFRHop by conducting experiments on the LFW and the CMU Multi-PIE datasets.

Univariate measurement error selection likelihood for variable selection of additive model

In this paper, we introduce a measurement error selection likelihood to select important variables and estimate additive components simultaneously in a high-dimensional additive model. Although the model contains multi-variates, the proposed estimation is a type of univariate nonparametric form. This format matches the feature of the additive structure in the sense that both the model and the nonparametric estimation are of univariate nonparametric feature, essentially. Consequently, the variable selection is valid even if the number of variables is large. The selection consistency is obtained and finite performances are illustrated via Monte Carlo experiments.

Nonparametric independence feature screening for ultrahigh-dimensional missing data

Missing data are common in medical and social science studies and often face a serious challenge in ultrahigh-dimensional data analysis. In this paper, a nonparametric feature screening approach based on the imputation technique is proposed for ultrahigh-dimensional data with responses missing at random, where the imputation technique is used to replacing each missing value with a set of plausible values. Our approach has many advantages. On one hand, the suggested method relies only on imputation, and its impact is considerably less than that of the feature screening procedure based on the inverse probability weighting approach in missing probabilities. On the other hand, our method does not rely on any model assumption and works generally for all kinds of situations. Simulation studies are conducted to examine the performance of our approach, and a real data example is also presented for illustration.

Copula Guided Parallel Gibbs Sampling for Nonparametric and Coherent Topic Discovery

Hierarchical Dirichlet Process (HDP) has attracted much attention in the research community of natural language processing. Given a corpus, HDP is able to determine the number of topics automatically, possessing an important feature dubbed nonparametric that overcomes the challenging issue of manually specifying a suitable topic number in parametric topic models, such as Latent Dirichlet Allocation (LDA). Nevertheless, HDP requires a much higher computational cost than LDA for parameter estimation. By taking the advantage of multi-threading, a parallel Gibbs sampling algorithm is proposed to estimate parameters for HDP based on the equivalence between HDP and Gamma-Gamma Poisson Process (G2PP) in terms of the generative process. Unfortunately, the above parallel Gibbs sampling algorithm requires to apply the finite approximation on the number of topics manually (i.e., predefine the topic number), thus can not retain the nonparametric feature of HDP. Another drawback of the above models is the lack of capturing the semantic dependencies between words, because the topic assignment of words is independent with each other. Although some works have been done in phrase-based topic modelling, these existing methods are still limited by either enforcing the entire phrase to share a common topic or requiring much complex and time-consuming phrase mining methods. In this paper, we aim to develop a copula guided parallel Gibbs sampling algorithm for HDP which can adjust the number of topics dynamically and capture the latent semantic dependencies between words that compose a coherent segment. Extensive experiments on real-world datasets indicate that our method achieves low perplexities and high topic coherence scores with a small time cost. In addition, we validate the effectiveness of our method on the modelling of word semantic dependencies by comparing the extracted topical phrases with those learned by state-of-the-art phrase-based baselines.

Sparse Nonparametric Regression With Regularized Tensor Product Kernel.

With growing interest to use black-box machine learning for complex data with many feature variables, it is critical to obtain a prediction model that only depends on a small set of features to maximize generalizability. Therefore, feature selection remains to be an important and challenging problem in modern applications. Most of existing methods for feature selection are based on either parametric or semiparametric models, so the resulting performance can severely suffer from model misspecification when high-order nonlinear interactions among the features are present. A very limited number of approaches for nonparametric feature selection were proposed, but they are computationally intensive and may not even converge. In this paper, we propose a novel and computationally efficient approach for nonparametric feature selection in regression field based on a tensor-product kernel function over the feature space. The importance of each feature is governed by a parameter in the kernel function which can be efficiently computed iteratively from a modified alternating direction method of multipliers (ADMM) algorithm. We prove the oracle selection property of the proposed method. Finally, we demonstrate the superior performance of our approach compared to existing methods via simulation studies and application to the prediction of Alzheimer's disease.

High‐dimensional spectral data classification with nonparametric feature screening

AbstractTwo nonparametric feature screening methods, namely, the Kolmogorov filter and model free, marginally measure the relationship between categorical response and predictor variables without the parametrical assumption. And they can select important variables in the high‐dimensional classification data. Random forest, as a classical nonparametric method, can solve various classification problems. In this paper, we combine the two nonparametric feature screening methods with random forest to handle with spectral data classification. And then other conventional classification methods are compared with ours on three spectral datasets. The comparison results illustrated that our methods have more desirable ability about classification performance and variable selection than other methods.

Brain-Computer Interface for Persons with Motor Disabilities - A Review

Aim /Objective: A Brain-Computer Interface (BCI) is a communication medium, which restructures brain signals into respective commands for an external device. Methodology: A BCI allows its target users like persons with motor disabilities to act on their environment using brain signals without using peripheral nerves or muscles. In this review article, we have presented a view on different BCIs for humans with motor disabilities. Results & Conclusion: From the study, it is clear that the P300 based Electroencephalography (EEG)BCIs with Steady-State Visually Evoked Potential (SSVEP) non-parametric feature extraction techniques work with high efficiency in the major parameters like Information Bit Transfer Rate (ITR), Mutual Information (MI) rate and Low Signal to Noise Ratio (SNR) and achieve a maximum classification accuracy using Self Organized Fuzzy Neural Network (SOFNN).

Multisensor Estimation Fusion with Gaussian Process for Nonlinear Dynamic Systems

The Gaussian process is gaining increasing importance in different areas such as signal processing, machine learning, robotics, control and aerospace and electronic systems, since it can represent unknown system functions by posterior probability. This paper investigates multisensor fusion in the setting of Gaussian process estimation for nonlinear dynamic systems. In order to overcome the difficulty caused by the unknown nonlinear system models, we associate the transition and measurement functions with the Gaussian process regression models, then the advantages of the non-parametric feature of the Gaussian process can be fully extracted for state estimation. Next, based on the Gaussian process filters, we propose two different fusion methods, centralized estimation fusion and distributed estimation fusion, to utilize the multisensor measurement information. Furthermore, the equivalence of the two proposed fusion methods is established by rigorous analysis. Finally, numerical examples for nonlinear target tracking systems demonstrate the equivalence and show that the multisensor estimation fusion performs better than the single sensor. Meanwhile, the proposed fusion methods outperform the convex combination method and the relaxed Chebyshev center covariance intersection fusion algorithm.

A nonparametric feature screening method for ultrahigh-dimensional missing response

This paper addresses the feature screening issue for ultrahigh-dimensional data with responses missing at random. A novel nonparametric feature screening procedure is developed to identify the important features via the conditionally imputing marginal Spearman rank correlation. The proposed nonparametric screening approach has several desirable merits. First, it is nonparametric without assuming any regression form of predictors on response variable. Second, it is robust to outliers and heavy-tailed data. Third, under some regularity conditions, it is shown that the proposed feature screening procedure has the sure screening and ranking consistency properties. Simulation studies evidence that the proposed screening procedure outperforms several existing model-free screening procedures. An example taken from the microarray diffuse large-B-cell lymphoma study is used to illustrate the proposed methodologies.