Heteroscedastic Setting Research Articles

Achieving robustness to aleatoric uncertainty with heteroscedastic Bayesian optimisation

Bayesian optimisation is a sample-efficient search methodology that holds great promise for accelerating drug and materials discovery programs. A frequently-overlooked modelling consideration in Bayesian optimisation strategies however, is the representation of heteroscedastic aleatoric uncertainty. In many practical applications it is desirable to identify inputs with low aleatoric noise, an example of which might be a material composition which displays robust properties in response to a noisy fabrication process. In this paper, we propose a heteroscedastic Bayesian optimisation scheme capable of representing and minimising aleatoric noise across the input space. Our scheme employs a heteroscedastic Gaussian process surrogate model in conjunction with two straightforward adaptations of existing acquisition functions. First, we extend the augmented expected improvement heuristic to the heteroscedastic setting and second, we introduce the aleatoric noise-penalised expected improvement (ANPEI) heuristic. Both methodologies are capable of penalising aleatoric noise in the suggestions. In particular, the ANPEI acquisition yields improved performance relative to homoscedastic Bayesian optimisation and random sampling on toy problems as well as on two real-world scientific datasets. Code is available at: https://github.com/Ryan-Rhys/Heteroscedastic-BO

Optimal classification of Gaussian processes in homo- and heteroscedastic settings

A procedure to derive optimal discrimination rules is formulated for binary functional classification problems in which the instances available for induction are characterized by random trajectories sampled from different Gaussian processes, depending on the class label. Specifically, these optimal rules are derived as the asymptotic form of the quadratic discriminant for the discretely monitored trajectories in the limit that the set of monitoring points becomes dense in the interval on which the processes are defined. The main goal of this work is to provide a detailed analysis of such optimal rules in the dense monitoring limit, with a particular focus on elucidating the mechanisms by which near-perfect classification arises. In the general case, the quadratic discriminant includes terms that are singular in this limit. If such singularities do not cancel out, one obtains near-perfect classification, which means that the error approaches zero asymptotically, for infinite sample sizes. This singular limit is a consequence of the orthogonality of the probability measures associated with the stochastic processes from which the trajectories are sampled. As a further novel result of this analysis, we formulate rules to determine whether two Gaussian processes are equivalent or mutually singular (orthogonal).

Rank-Based Procedures in Factorial Designs: Hypotheses About Non-Parametric Treatment Effects

Summary Existing tests for factorial designs in the non-parametric case are based on hypotheses formulated in terms of distribution functions. Typical null hypotheses, however, are formulated in terms of some parameters or effect measures, particularly in heteroscedastic settings. Here this idea is extended to non-parametric models by introducing a novel non-parametric analysis-of-variance type of statistic based on ranks or pseudoranks which is suitable for testing hypotheses formulated in meaningful non-parametric treatment effects in general factorial designs. This is achieved by a careful detailed study of the common distribution of rank-based estimators for the treatment effects. Since the statistic is asymptotically not a pivotal quantity we propose three different approximation techniques, discuss their theoretic properties and compare them in extensive simulations together with two additional Wald-type tests. An extension of the presented idea to general repeated measures designs is briefly outlined. The rank- and pseudorank-based procedures proposed maintain the preassigned type I error rate quite accurately, also in unbalanced and heteroscedastic models.

Partitioning estimation of local variance based on nearest neighbors under censoring

In a nonparametric and heteroscedastic setting, our primary interest is in the local variance estimation when the response variable is subject to right censoring. For the proposed partitioning local variance estimators, based on the first and second nearest neighbors, some transformations on the observed censoring times are involved, using their estimated survival functions. Proofs of consistency and rate of convergence for the presented estimators are given. Moreover, local variance estimation is demonstrated on the basis of real survival data.

Nonparametric lack-of-fit testing and consistent variable selection

Let X be a d dimensional vector of covariates and Y be the response variable. Under the nonparametric model Y = m(X) + σ(X) we develop an ANOVA-type test for the null hypothesis that a particular coordinate of X has no influence on the regression function. The asymptotic distribution of the test statistic, using residuals based on local polynomial regression, is established under the null hypothesis and local alternatives. Simulations suggest that the test outperforms existing procedures in heteroscedastic settings. Using p-values from this test, a variable selection method based on False Discovery Rate corrections is proposed, and proved to be consistent in estimating the set of indices corresponding to the significant covariates. Simulations suggest that, under a sparse model, dimension reduction techniques can help avoid the curse of dimensionality. We also propose a backward elimination version of this procedure, called BEAMS (Backward Elimination ANOVA-type Model Selection), which performs competitively against well established procedures in linear regression settings, and outperforms them in nonparametric settings. A real data set is analyzed.

Minimax rates in permutation estimation for feature matching

The problem of matching two sets of features appears in various tasks of computer vision and can be often formalized as a problem of permutation estimation. We address this problem from a statistical point of view and provide a theoretical analysis of the accuracy of several natural estimators. To this end, the minimax rate of separation is investigated and its expression is obtained as a function of the sample size, noise level and dimension of the features. We consider the cases of homoscedastic and heteroscedastic noise and establish, in each case, tight upper bounds on the separation distance of several estimators. These upper bounds are shown to be unimprovable both in the homoscedastic and heteroscedastic settings. Interestingly, these bounds demonstrate that a phase transition occurs when the dimension d of the features is of the order of the logarithm of the number of features n. For d = O(log n), the rate is dimension free and equals σ(log n)1/2, where σ is the noise level. In contrast, when d is larger than c log n for some constant c < 0, the minimax rate increases with d and is of the order of σ(d log n)1/4. We also discuss the computational aspects of the estimators and provide empirical evidence of their consistency on synthetic data. Finally, we show that our results extend to more general matching criteria.

Non asymptotic minimax rates of testing in signal detection with heterogeneous variances

The aim of this paper is to establish non-asymptotic minimax rates for goodness-of-fit hypotheses testing in an heteroscedastic setting. More precisely, we deal with sequences (Yj)j∈J of independent Gaussian random variables, having mean (θj)j∈J and variance (σj)j∈J. The set J will be either finite or countable. In particular, such a model covers the inverse problem setting where few results in test theory have been obtained. The rates of testing are obtained with respect to l2 norm, without assumption on (σj)j∈J and on several functions spaces. Our point of view is entirely non-asymptotic.

Testing for Interaction in Two-Way Random and Mixed Effects Models: The Fully Nonparametric Approach

In a recent paper, Gaugler and Akritas (unpublished manuscript) considered testing for no main effect in a two-factor mixed effects design when the traditional assumptions do not hold. Here we extend the nonparametric modeling to the random effects design and consider the problem of testing for no interaction effect. The new models for these designs allow for dependence among the random effects, heteroscedasticity in the error and interaction terms, and do not require normality. At a more systemic level, these models differ from the classical ones in that they do not consider the random interaction term as an additional, extraneous source of variability. The proposed test procedure applies to settings where the random factor in the case of the mixed model or at least one of the random factors in the case of the random effects model has many levels. The number of replications can be small and possibly unbalanced. Moreover, the model and test procedure are general enough to accommodate data missing at random (MAR), provided the missingness mechanism is the same for each level of the random effect. The limiting distribution of the test statistic is normal. Extensive simulations indicate that our test procedure, with or without missing data, maintains the nominal Type I error rate in all simulation settings. On the contrary, the standard procedures (the F-test of PROC GLM in SAS, and the ML and REML methods of PROC MIXED in SAS), as well as the exact F-test of Khuri, Mathew, and Sinha (1998 in Statistical Tests for Mixed Linear Models), are extremely liberal in heteroscedastic settings, while under homoscedasticity and normality, the proposed test procedure is comparable to them. An analysis of a dataset from the Mussel Watch Project is presented.

Testing for superiority among two regression curves

This paper discusses the problem of testing the equality of two nonparametric regression curves against one-sided alternatives in a two sample heteroscedastic setting in which design and error densities may differ between the two populations. The paper proposes a class of tests using covariate matching and derives their asymptotic power for local alternatives. Using a semiparametric approach, an upper bound on the asymptotic power of all tests against a given local alternative is obtained. For a given local alternative, a member of the proposed class of tests is shown to achieve this upper bound.

Bayesian Model Selection Approach to Analysis of Variance Under Heteroscedasticity

The classical Bayesian approach to analysis of variance assumes the homoscedastic condition and uses conventional uniform priors on the location parameters and on the logarithm of the common scale. The problem has been developed as one of estimation of location parameters. We argue that this does not lead to an appropriate Bayesian solution. A solution based on a Bayesian model selection procedure is proposed. Our development is in the general heteroscedastic setting in which a frequentist exact test does not exist. The Bayes factor involved uses intrinsic and fractional priors which are used instead of the usual default prior distributions for which the Bayes factor is not well defined. The behaviour of these Bayes factors is compared with the Bayesian information criterion of Schwarz and the frequentist asymptotic approximations of Welch and Brown and Forsythe.

E-optimal block designs under heteroscedastic model

This paper mainly studies the E-optimality of block designs under a general heteroscedastic setting. The C-matrix of a block design under a heteroscedastic setting is obtained by using generalized least squares. Some bounds for the smallest positive eigenvalue of C-matrix are obtained in some general classes of connected designs. Use of these bounds is then made to obtain certain E-optimal block designs in various classes of connected block designs.

Confidence Bands for Polynomial Regression with Fixed Intercepts

Confidence bands are considered for linear and quadratic regression when the regression equation is forced through a fixed point. In the linear case, it is noted that the construction of confidence bands reduces to the construction of a confidence interval on the slope parameter. The corresponding bands are straight lines whose widths are independent of the size of the set over which the band is constructed. The construction for the quadratic case involves more complex arguments; these are developed, and tables are presented for band use in both the homoscedastic and heteroscedastic settings. An example is included.

Confidence Bands for Polynomial Regression With Fixed Intercepts

Confidence bands are considered for linear and quadratic regression when the regression equation is forced through a fixed point. In the linear case, it is noted that the construction of confidence bands reduces to the construction of a confidence interval on the slope parameter. The corresponding bands are straight lines whose widths are independent of the size of the set over which the band is constructed. The construction for the quadratic case involves more complex arguments; these are developed, and tables are presented for band use in both the homoscedastic and heteroscedastic settings. An example is included.