Finite Set Of Samples Research Articles

A Reproducing Kernel Hilbert Space Framework for Functional Classification

The intrinsic infinite-dimensional nature of functional data creates a bottleneck in the application of traditional classifiers to functional settings. These classifiers are generally either unable to generalize to infinite dimensions or have poor performance due to the curse of dimensionality. To address this concern, we propose building a distance-weighted discrimination (DWD) classifier on scores obtained by projecting data onto one specific direction. We choose this direction by minimizing, over a reproducing kernel Hilbert space, an empirical risk function containing the DWD classifier loss function. Our proposed classifier avoids overfitting and enjoys the appealing properties of DWD classifiers. We further extend this framework to accommodate functional data classification problems where scalar covariates are involved. In contrast to previous work, we establish a nonasymptotic estimation error bound on the relative misclassification rate. Through simulation studies and a real-world application, we demonstrate that the proposed classifier performs favorably relative to other commonly used functional classifiers in terms of prediction accuracy in finite-sample settings. Supplementary materials for this article are available online.

CoxMKF: a knockoff filter for high-dimensional mediation analysis with a survival outcome in epigenetic studies.

It is of scientific interest to identify DNA methylation CpG sites that might mediate the effect of an environmental exposure on a survival outcome in high-dimensional mediation analysis. However, there is a lack of powerful statistical methods that can provide a guarantee of false discovery rate (FDR) control in finite-sample settings. In this article, we propose a novel method called CoxMKF, which applies aggregation of multiple knockoffs to a Cox proportional hazards model for a survival outcome with high-dimensional mediators. The proposed CoxMKF can achieve FDR control even in finite-sample settings, which is particularly advantageous when the sample size is not large. Moreover, our proposed CoxMKF can overcome the randomness of the unstable model-X knockoffs. Our simulation results show that CoxMKF controls FDR well in finite samples. We further apply CoxMKF to a lung cancer dataset from The Cancer Genome Atlas (TCGA) project with 754 subjects and 365306 DNA methylation CpG sites, and identify four DNA methylation CpG sites that might mediate the effect of smoking on the overall survival among lung cancer patients. The R package CoxMKF is publicly available at https://github.com/MinhaoYaooo/CoxMKF. Supplementary data are available at Bioinformatics online.

Non-asymptotic analysis and inference for an outlyingness induced winsorized mean

Robust estimation of a mean vector, a topic regarded as obsolete in the traditional robust statistics community, has recently surged in machine learning literature in the last decade. The latest focus is on the sub-Gaussian performance and computability of the estimators in a non-asymptotic setting. Numerous traditional robust estimators are computationally intractable, which partly contributes to the renewal of the interest in the robust mean estimation. Robust centrality estimators, however, include the trimmed mean and the sample median. The latter has the best robustness but suffers a low efficiency drawback. Trimmed mean and median of means, achieving sub-Gaussian performance have been proposed and studied in the literature. This article investigates the robustness of leading sub-Gaussian estimators of mean and reveals that none of them can resist greater than \(25\%\) contamination in data and consequently introduces an outlyingness induced winsorized mean which has the best possible robustness (can resist up to \(50\%\) contamination without breakdown) meanwhile achieving high efficiency. Furthermore, it has a sub-Gaussian performance for uncontaminated samples and a bounded estimation error for contaminated samples at a given confidence level in a finite sample setting. It can be computed in linear time.

Nonintrusive model order reduction for cross-diffusion systems

In this paper, we investigate tensor based nonintrusive reduced-order models (ROMs) for parametric cross-diffusion equations. The full-order model (FOM) consists of ordinary differential equations (ODEs) in matrix or tensor form resulting from finite difference discretization of the differential operators by taking the advantage of Kronecker structure. The matrix/tensor differential equations are integrated in time with the implicit–explicit (IMEX) Euler method. The reduced bases, relying on a finite sample set of parameter values, are constructed in form of a two-level approach by applying higher-order singular value decomposition (HOSVD) to the space–time snapshots in tensor form, which leads to a large amount of computational and memory savings. The nonintrusive reduced approximation for an arbitrary parameter value is obtained through tensor product of the reduced basis by the parameter dependent core tensor that contains the reduced coefficients. The reduced coefficients for new parameter values are computed with the radial basis functions. The efficiency of the proposed method is illustrated through numerical experiments for two-dimensional Schnakenberg and three-dimensional Brusselator cross-diffusion equations. The spatiotemporal patterns are accurately predicted by the reduced-order models with speedup factors of orders two and three over the full-order models.

An Embedding of ReLU Networks and an Analysis of Their Identifiability

Neural networks with the rectified linear unit (ReLU) nonlinearity are described by a vector of parameters $$\theta $$ and realized as a piecewise linear continuous function $$\varvec{R}_{\theta }: x \in \mathbb {R}^{d} \mapsto \varvec{R}_{\theta }(x) \in \mathbb {R}^{k}$$ . Natural scalings and permutations operations on the parameters $$\theta $$ leave the realization unchanged, leading to equivalence classes of parameters that yield the same realization. These considerations in turn lead to the notion of identifiability—the ability to recover (the equivalence class of) $$\theta $$ from the sole knowledge of its realization $$\varvec{R}_{\theta }$$ . The overall objective of this paper is to introduce an embedding for ReLU neural networks of any depth, $${\varvec{\Phi }}(\theta )$$ , that is invariant to scalings and that provides a locally linear parameterization of the realization of the network. Leveraging these two key properties, we derive some conditions under which a deep ReLU network is indeed locally identifiable from the knowledge of the realization on a finite set of samples $$x_{i} \in \mathbb {R}^{d}$$ . We study the shallow case in more depth, establishing necessary and sufficient conditions for the network to be identifiable from a bounded subset $$\mathcal {X}\subseteq \mathbb {R}^{d}$$ .

Error-Based Knockoffs Inference for Controlled Feature Selection

Recently, the scheme of model-X knockoffs was proposed as a promising solution to address controlled feature selection under high-dimensional finite-sample settings. However, the procedure of model-X knockoffs depends heavily on the coefficient-based feature importance and only concerns the control of false discovery rate (FDR). To further improve its adaptivity and flexibility, in this paper, we propose an error-based knockoff inference method by integrating the knockoff features, the error-based feature importance statistics, and the stepdown procedure together. The proposed inference procedure does not require specifying a regression model and can handle feature selection with theoretical guarantees on controlling false discovery proportion (FDP), FDR, or k-familywise error rate (k-FWER). Empirical evaluations demonstrate the competitive performance of our approach on both simulated and real data.

Non-parametric estimator of a multivariate madogram for missing-data and extreme value framework

The modeling of dependence between maxima is an important subject in several applications in risk analysis. To this aim, the extreme value copula function, characterized via the madogram, can be used as a margin-free description of the dependence structure. From a practical point of view, the family of extreme value distributions is very rich and arise naturally as the limiting distribution of properly normalized component-wise maxima. In this paper, we investigate the nonparametric estimation of the madogram where data are completely missing at random. We provide the functional central limit theorem for the considered multivariate madrogram correctly normalized, towards a tight Gaussian process for which the covariance function depends on the probabilities of missing. Explicit formula for the asymptotic variance is also given. Our results are illustrated in a finite sample setting with a simulation study. Our method is also illustrated on a sparse dataset of annual maxima rainfall in Central Eastern Canada.

Quantum solvability of noisy linear problems by divide-and-conquer strategy

Noisy linear problems have been studied in various science and engineering disciplines. A class of ‘hard’ noisy linear problems can be formulated as follows: Given a matrix and a vector b constructed using a finite set of samples, a hidden vector or structure involved in b is obtained by solving a noise-corrupted linear equation , where η is a noise vector that cannot be identified. For solving such a noisy linear problem, we consider a quantum algorithm based on a divide-and-conquer strategy, wherein a large core process is divided into smaller subprocesses. The algorithm appropriately reduces both the computational complexities and size of a quantum sample. More specifically, if a quantum computer can access a particular reduced form of the quantum samples, polynomial quantum-sample and time complexities are achieved in the main computation. The size of a quantum sample and its executing system can be reduced, e.g., from exponential to sub-exponential with respect to the problem length, which is better than other results we are aware. We analyse the noise model conditions for such a quantum advantage, and show when the divide-and-conquer strategy can be beneficial for quantum noisy linear problems.

Nonintrusive Model Order Reduction for Cross-Diffusion Systems

In this paper, we investigate tensor based nonintrusive reduced-order models (ROMs) for parametric cross-diffusion equations. The full-order model (FOM) consists of ordinary differential equations (ODEs) in matrix or tensor form resulting from finite-difference discretization of the differential operators by taking the advantage of Kronecker structure. The matrix/tensor differential equations are integrated in time with the implicit-explicit (IMEX) Euler method. The reduced bases, relying on a finite sample set of parameter values, are constructed in form of a two-level approach by applying higher-order singular value decomposition (HOSVD) to the space-time snapshots in tensor form, which leads to a large amount of computational and memory savings. The nonintrusive reduced approximations for an arbitrary parameter value are obtained through tensor product of the reduced basis by the parameter dependent core tensor that contains the reduced coefficients. The reduced coefficients for new parameter values are computed using radial basis function (RBF) interpolation. The efficiency of the proposed method is illustrated through numerical experiments for two-dimensional Schnakenberg and three-dimensional Brusselator cross-diffusion equations. The spatiotemporal patterns are accurately predicted by the reduced-order models with speed-up factors of orders two and three over the full-order models.

Deterministic error bounds for kernel-based learning techniques under bounded noise

We consider the problem of reconstructing a function from a finite set of noise-corrupted samples. Two kernel algorithms are analyzed, namely kernel ridge regression and ɛ-support vector regression. By assuming the ground-truth function belongs to the reproducing kernel Hilbert space of the chosen kernel, and the measurement noise affecting the dataset is bounded, we adopt an approximation theory viewpoint to establish deterministic, finite-sample error bounds for the two models. Finally, we discuss their connection with Gaussian processes and two numerical examples are provided. In establishing our inequalities, we hope to help bring the fields of non-parametric kernel learning and system identification for robust control closer to each other.

A deep learning semiparametric regression for adjusting complex confounding structures

Deep Treatment Learning (deepTL), a robust yet efficient deep learning-based semiparametric regression approach, is proposed to adjust the complex confounding structures in comparative effectiveness analysis of observational data, for example, electronic health record (EHR) data in which complex confounding structures are often embedded. Specifically, we develop a deep learning neural network with a score-based ensembling scheme for flexible function approximation. An improved semiparametric procedure is further developed to enhance the performance of the proposed method under finite sample settings. Comprehensive numerical studies have demonstrated the superior performance of the proposed methods, as compared with existing methods, with a remarkably reduced bias and mean squared error in parameter estimates. The proposed research is motivated by a postsurgery pain study, which is also used to illustrate the practical application of deepTL. Finally, an R package, “deepTL,” is developed to implement the proposed method.

Deciphering the optimal exergy field in closed-wet cooling towers using Bi-level reduced-order models

This paper introduces a bi-level reduced-order models (ROMs) approach for quickly deciphering the optimal exergy fields in closed wet cooling towers (CWCTs) with consideration of weather variations. First, an efficient sampling method based on stochastic reduced-order model is performed for the approximation of the multivariate probability distributions by generating a finite set of samples. The uncertainty associated with input variables is propagated via multi-sample CFD simulations of the CWCT model for each of the samples. The results of the state and output variables stored in the CFD solutions are used to construct the data-driven and physics-based ROMs by combining principal component analysis and artificial neural network methods. The constructed data-driven ROM is embedded in a sampling-based stochastic optimization model that seeks the maximization of the expected exergy efficiency ratio. The physics-based ROM is used to visualize the optimal field profiles of the thermal-, mechanical-, and chemical-exergy fluxes. Finally, the results of a case study demonstrate that the main strengths of the proposed approach is to simultaneously obtain the optimal exergy efficiency ratios and the exergy field profiles of the CWCT system in a computationally efficient manner.

Leveraging the Fisher Randomization Test using Confidence Distributions: Inference, Combination and Fusion Learning

AbstractThe flexibility and wide applicability of the Fisher randomization test (FRT) make it an attractive tool for assessment of causal effects of interventions from modern-day randomized experiments that are increasing in size and complexity. This paper provides a theoretical inferential framework for FRT by establishing its connection with confidence distributions. Such a connection leads to development’s of (i) an unambiguous procedure for inversion of FRTs to generate confidence intervals with guaranteed coverage, (ii) new insights on the effect of size of the Monte Carlo sample on the estimation of a p-value curve and (iii) generic and specific methods to combine FRTs from multiple independent experiments with theoretical guarantees. Our developments pertain to finite sample settings but have direct extensions to large samples. Simulations and a case example demonstrate the benefit of these new developments.

Robust Universal Inference.

Learning and making inference from a finite set of samples are among the fundamental problems in science. In most popular applications, the paradigmatic approach is to seek a model that best explains the data. This approach has many desirable properties when the number of samples is large. However, in many practical setups, data acquisition is costly and only a limited number of samples is available. In this work, we study an alternative approach for this challenging setup. Our framework suggests that the role of the train-set is not to provide a single estimated model, which may be inaccurate due to the limited number of samples. Instead, we define a class of “reasonable” models. Then, the worst-case performance in the class is controlled by a minimax estimator with respect to it. Further, we introduce a robust estimation scheme that provides minimax guarantees, also for the case where the true model is not a member of the model class. Our results draw important connections to universal prediction, the redundancy-capacity theorem, and channel capacity theory. We demonstrate our suggested scheme in different setups, showing a significant improvement in worst-case performance over currently known alternatives.

A Bayesian nonparametric mixture model for grouping dependence structures and selecting copula functions

The demand for advanced dependence modeling arises in a variety of fields, including finance, insurance and health science. When analyzing dependent data, it is important but challenging to properly model the dependence structure in order to carry out valid and efficient inferences. Grouping the data according to the similarity in the dependence structure is necessary, especially for data of a small size. A copula-based model, indexed by copula selection indicators and dependence parameters, is introduced to delineate dependent data and group similar dependence structures. To conduct inference, a Bayesian nonparametric method with the prior distributions specified as a Dirichlet Process is proposed as a mixture of Dirichlet process mixture copula model (M-DPM-CM). Extensive simulation studies have been conducted to evaluate the performance of the proposed procedure, and the results show that the proposed M-DPM-CM can recover the true grouping structure and achieve high accuracy in copula model selection under various finite sample settings. The M-DPM-CM is applied to analyze the Vertebral Column dataset from UCI Machine Learning Repository.

A Distributed Approach to Speaker Count Problem in an Open-Set Scenario by Clustering Pitch Features

Counting the number of speakers in an audio sample can lead to innovative applications, such as a real-time ranking system. Researchers have studied advanced machine learning approaches for solving the speaker count problem. However, these solutions are not efficient in real-time environments, as it requires pre-processing of a finite set of data samples. Another approach for solving the problem is via unsupervised learning or by using audio processing techniques. The research in this category is limited and does not consider the large-scale open set environment. In this paper, we propose a distributed clustering approach to address the speaker count problem. The separability of the speaker is computed using statistical pitch parameters. The proposed solution uses multiple microphones available in smartphones in a large geographical area to capture and extract statistical pitch features from the audio samples. These features are shared between the nodes to estimate the number of speakers in the neighborhood. One of the major challenges is to reduce the error count that arises due to the proximity of the users and multiple microphones. We evaluate the algorithm’s performance using real smartphones in a multi-group arrangement by capturing parallel conversations between the users in both indoor and outdoor scenarios. The average error count distance is 1.667 in a multi-group scenario. The average error count distances in indoor environments are 16% which is better than in the outdoor environment.

Quantum secure learning with classical samples

Studies addressing the question "Can a learner complete the learning securely?" have recently been spurred from the standpoints of fundamental theory and potential applications. In the relevant context of this question, we present a classical-quantum hybrid sampling protocol and define a security condition that allows only legitimate learners to prepare a finite set of samples that guarantees the success of the learning; the security condition excludes intruders. We do this by combining our security concept with the bound of the so-called probably approximately correct (PAC) learning. We show that while the lower bound on the learning samples guarantees PAC learning, an upper bound can be derived to rule out adversarial learners. Such a secure learning condition is appealing, because it is defined only by the size of samples required for the successful learning and is independent of the algorithm employed. Notably, the security stems from the fundamental quantum no-broadcasting principle. No such condition can thus occur in any classical regime, where learning samples can be copied. Owing to the hybrid architecture, our scheme also offers a practical advantage for implementation in noisy intermediate-scale quantum devices.

Type I error control for cluster randomized trials under varying small sample structures

BackgroundLinear mixed models (LMM) are a common approach to analyzing data from cluster randomized trials (CRTs). Inference on parameters can be performed via Wald tests or likelihood ratio tests (LRT), but both approaches may give incorrect Type I error rates in common finite sample settings. The impact of different combinations of cluster size, number of clusters, intraclass correlation coefficient (ICC), and analysis approach on Type I error rates has not been well studied. Reviews of published CRTs find that small sample sizes are not uncommon, so the performance of different inferential approaches in these settings can guide data analysts to the best choices.MethodsUsing a random-intercept LMM stucture, we use simulations to study Type I error rates with the LRT and Wald test with different degrees of freedom (DF) choices across different combinations of cluster size, number of clusters, and ICC.ResultsOur simulations show that the LRT can be anti-conservative when the ICC is large and the number of clusters is small, with the effect most pronouced when the cluster size is relatively large. Wald tests with the between-within DF method or the Satterthwaite DF approximation maintain Type I error control at the stated level, though they are conservative when the number of clusters, the cluster size, and the ICC are small.ConclusionsDepending on the structure of the CRT, analysts should choose a hypothesis testing approach that will maintain the appropriate Type I error rate for their data. Wald tests with the Satterthwaite DF approximation work well in many circumstances, but in other cases the LRT may have Type I error rates closer to the nominal level.

Stable recovery of planar regions with algebraic boundaries in Bernstein form

We present a new method for the stable reconstruction of a class of binary images from a small number of measurements. The images we consider are characteristic functions of algebraic domains, that is, domains defined as zero loci of bivariate polynomials, and we assume to know only a finite set of uniform samples for each image. The solution to such a problem can be set up in terms of linear equations associated to a set of image moments. However, the sensitivity of the moments to noise makes the numerical solution highly unstable. To derive a robust image recovery algorithm, we represent algebraic polynomials and the corresponding image moments in terms of bivariate Bernstein polynomials and apply polynomial-generating, refinable sampling kernels. This approach is robust to noise, computationally fast and simple to implement. We illustrate the performance of our reconstruction algorithm from noisy samples through extensive numerical experiments. Our code is released open source and freely available.

Cause-specific quantile regression on inactivity time.

In time-to-event analysis, the traditional summary measures have been based on the hazard function, survival function, quantile event time, restricted mean event time, and residual lifetime. Under competing risks, furthermore, typical summary measures have been the cause-specific hazard function and cumulative incidence function. Recently inactivity time has recaptured attention in the literature, being interpreted as life lost. In this paper, we further interpret it as quality of life reduced and time period after transition to a drug, and propose a quantile regression model to associate the inactivity time with potential predictors under competing risks. We define the proper cumulative distribution function of the inactivity time distribution for each specific event type among those subjects who experience the same type of events during a follow-up period. A score function-type estimating equation is developed and asymptotic properties of the regression coefficient estimators are derived by assuming that competing events are censored at their occurrence times as in the cause-specific hazard analysis. The proposed approach reduces to a regular quantile regression on the inactivity time without competing risks when all types of competing events are collapsed into the same type. Due to difficulty in estimating the improper probability density function of the cause-specific inactivity distribution to evaluate the variance of the quantiles, a computationally efficient perturbation method is adopted to infer the regression coefficients. Simulation results show that our proposed method works well under the assumed finite sample settings. The proposed method is illustrated with a real dataset from a breast cancer study.