Order-restricted Estimators Research Articles

Order-restricted inference for clustered ROC data with application to fingerprint matching accuracy.

Receiver operating characteristic (ROC) curve is commonly used to evaluate and compare the accuracy of classification methods or markers. Estimating ROC curves has been an important problem in various fields including biometric recognition and diagnostic medicine. In real applications, classification markers are often developed under two or more ordered conditions, such that a natural stochastic ordering exists among the observations. Incorporating such a stochastic ordering into estimation can improve statistical efficiency (Davidov and Herman, 2012). In addition, clustered and correlated data arise when multiple measurements are gleaned from the same subject, making estimation of ROC curves complicated due to within-cluster correlations. In this article, we propose to model the ROC curve using a weighted empirical process to jointly account for the order constraint and within-cluster correlation structure. The algebraic properties of resulting summary statistics of the ROC curve such as its area and partial area are also studied. The algebraic expressions reduce to the ones by Davidov and Herman (2012) for independent observations. We derive asymptotic properties of the proposed order-restricted estimators and show that they have smaller mean-squared errors than the existing estimators. Simulation studies also demonstrate better performance of the newly proposed estimators over existing methods for finite samples. The proposed method is further exemplified with the fingerprint matching data from the National Institute of Standards and Technology Special Database4.

Isotonic designs for phase I trials in partially ordered groups.

Dose-finding trials can be conducted such that patients are first stratified into multiple risk groups before doses are allocated. The risk groups are often completely ordered in that, for a fixed dose, the probability of toxicity is monotonically increasing across groups. In some trials, the groups are only partially ordered. For example, one of several groups in a trial may be known to have the least risk of toxicity for a given dose, but the ordering of the risk among the remaining groups may not be known. The aim of the article is to introduce a method for designing dose-finding trials of cytotoxic agents in completely or partially ordered groups of patients. This article presents a method for dose-finding that combines previously proposed mathematical models, augmented with results using order restricted inference. The resulting method is computationally convenient and allows for dose-finding in trials with completely or partially ordered groups. Extensive simulations are done to evaluate the performance of the method, using randomly generated dose-toxicity curves where, within each group, the risk of toxicity is an increasing function of dose. Our simulations show that the hybrid method, in which order-restricted estimation is applied to parameters of a parsimonious mathematical model, gives results that are similar to previously proposed methods for completely ordered groups. Our method generalizes to a wide range of partial orders among the groups. The problem of dose-finding in partially ordered groups has not been extensively studied in the statistical literature. The proposed method is computationally feasible, and provides a potential solution to the design of dose-finding studies in completely or partially ordered groups.

A design for phase I trials in completely or partially ordered groups.

We propose a design for dose finding for cytotoxic agents in completely or partially ordered groups of patients. By completely ordered groups, we mean that prior to the study, there is clinical information that would indicate that for a given dose, the groups can be ordered with respect to the probability of toxicity at that dose. With partially ordered groups, at a given dose, only some of the groups can be ordered with respect to the probability of toxicity at that dose. The method we propose includes elements of the parametric model used in the continual reassessment method combined with the Hwang-Peddada order-restricted estimation procedure. We evaluate the operating characteristics of these designs in a family of dose-toxicity curves representing complete and partial orders. Copyright © 2017 John Wiley & Sons, Ltd.

Ordinal Dominance Curve Based Inference for Stochastically Ordered Distributions

SummaryIn a variety of applications researchers collect data that are sampled under ordered experimental conditions. In such situations it is reasonable to assume that outcomes are ordered stochastically by the level of the treatment. For example a toxicologist may want to assess the effect of a toxin on the reproductive system by studying the number of offspring produced by an animal as a function of the dose that it received. A variety of scientific questions arise in such a context. The most basic is whether a dose–response relationship is present. If so, determining the minimal toxic dose is of great interest. The strength of the ordering between groups i<j may be assessed by the probabilities ℙXi⩽Xj or monotone functions thereof. Large probabilities indicate a strong ordering. We find that such (and other) questions are naturally addressed by studying the ordinal dominance curve and the area under the curve (AUC). We develop new, non-parametric, order-restricted estimators for the AUCs in K⩾2 samples and investigate them in detail. We show that the restricted estimators are strongly consistent and have lower mean-squared error compared with the unrestricted estimators. In the two-sample case estimators are considerably improved in those situations where the data are highly variable and/or sample sizes are small, and where the AUC is close to its boundary value. Note that these are precisely the conditions under which estimating the AUC is difficult. Simulations indicate that in the multisample case estimators are improved under a much broader set of conditions with substantial reductions in mean-squared error. These estimators provide a basis for a family of distribution-free tests for the usual stochastic order in K⩾2 samples. Limiting distributions are derived. An accurate and simple approximation to the tail area is given for K = 2. Simulation studies show that the new tests perform well compared with existing alternatives. The methodology proposed is illustrated by applying it to an aquatic toxicology study where it is used to determine the minimal toxic dose more accurately.

Order-Restricted Inference for Multivariate Binary Data With Application to Toxicology

In many applications, researchers collect multivariate binary response data under two or more naturally ordered experimental conditions. In such situations, one is often interested in using all binary outcomes simultaneously to detect an ordering among the experimental conditions. To make such comparisons, we develop a general methodology for testing for the multivariate stochastic order between K ≥ 2 multivariate binary distributions. Our proposed test uses order-restricted estimators, which, according to our simulation study, are more efficient than the unrestricted estimators in terms of their mean squared error. We compared the power of the proposed test with that of several alternative tests, including procedures that combine individual univariate tests for order, such as union-intersection tests and a Bonferroni-based test. We also compared the proposed test with an unrestricted Hotelling T2-type test. Our simulations suggest that the proposed method competes well with these alternatives. The gain in power is often substantial. The proposed methodology is illustrated by applying it to a two-year rodent cancer bioassay data obtained from the U.S. National Toxicology Program. Supplemental materials are available online.

Modifying estimators of ordered positive parameters under the Stein loss

This paper treats the problem of estimating positive parameters restricted to a polyhedral convex cone which includes typical order restrictions, such as simple order, tree order and umbrella order restrictions. In this paper, two methods are used to show the improvement of order-preserving estimators over crude non-order-preserving estimators without any assumption on underlying distributions. One is to use Fenchel’s duality theorem, and then the superiority of the isotonic regression estimator is established under the general restriction to polyhedral convex cones. The use of the Abel identity is the other method, and we can derive a class of improved estimators which includes order-statistics-based estimators in the typical order restrictions. When the underlying distributions are scale families, the unbiased estimators and their order-restricted estimators are shown to be minimax. The minimaxity of the restrictedly generalized Bayes estimator against the prior over the restricted space is also demonstrated in the two dimensional case. Finally, some examples and multivariate extensions are given.

Sample size selection in clinical trials when population means are subject to a partial order: one-sided ordered alternatives

The statistical methodology under order restriction is very mathematical and complex. Thus, we provide a brief methodological background of order-restricted likelihood ratio tests for the normal theoretical case for the basic understanding of its applications, and relegate more technical details to the appendices. For data analysis, algorithms for computing the order-restricted estimates and computation of p-values are described. A two-step procedure is presented for obtaining the sample size in clinical trials when the minimum power, say 0.80 or 0.90 is specified, and the normal means satisfy an order restriction. Using this approach will result in reduction of 14–24% in the sample size required when one-sided ordered alternatives are used, as illustrated by several examples.

Consistent estimation of the minimum normal mean under the tree-order restriction

Let ( X ij ∣ j = 1 , … , n i ( s ) , i = 0 , 1 , … , s ) be independent observations from s + 1 univariate normal populations, with X ij ∼ N ( μ i , σ 2 ) . The tree-order restriction ( μ 0 ⩽ μ i , i = 1 , … , s ) arises naturally when comparing a treatment ( μ 0 ) to several controls ( μ 1 , … , μ s ). When the sample sizes and population means and variances are equal and fixed, the maximum likelihood-based estimator (MLBE) of μ 0 is negatively biased and diverges to - ∞ a.s. as s → ∞ , leading some to assert that maximum likelihood may “fail disastrously” in order-restricted estimation. By viewing this problem as one of estimating a target parameter μ 0 in the presence of an increasing number of nuisance parameters μ 1 , … , μ s , however, this behavior is reminiscent of the classical Neyman–Scott example. This suggests an alternative formulation of the problem wherein the sample size n 0 ( s ) for the target parameter increases with s. Here the MLBE of μ 0 is either consistent or admits a bias-reducing adjustment, depending on the rate of increase of n 0 ( s ) . The consistency of an estimator due to Cohen and Sackrowitz [2002. Inference for the model of several treatments and a control. J. Statist. Plann. Inference 107, 89–101] is also discussed.

An adaptive rule for stopping a discovery process under time censorship

Consider a finite population of large but unknown size of hidden objects. Consider searching for these objects for a period of time, at a certain cost, and receiving a reward depending on the sizes of the objects found. Suppose that the size and discovery time of the objects both have unknown distributions, but the conditional distribution of time given size is exponential with an unknown non-negative and non-decreasing function of the size as failure rate. The goal is to find an optimal way to stop the discovery process. Assuming that the above parameters are known, an optimal stopping time is derived and its asymptotic properties are studied. Then, an adaptive rule based on order restricted estimates of the distributions from truncated data is presented. This adaptive rule is shown to perform nearly as well as the optimal stopping time for large population size.

The role of reversals in order‐restricted inference

AbstractA statistical model is said to be an order‐restricted statistical model when its parameter takes its values in a closed convex cone C of the Euclidean space. In recent years, order‐restricted likelihood ratio tests and maximum likelihood estimators have been criticized on the grounds that they may violate a cone order monotonicity (COM) property, and hence reverse the cone order induced by C. The authors argue here that these reversals occur only in the case that C is an obtuse cone, and that in this case COM is an inappropriate requirement for likelihood‐based estimates and tests. They conclude that these procedures thus remain perfectly reasonable procedures for order‐restricted inference.

Estimation in hazard regression models under ordered departures from proportionality

Estimation of age-varying covariate effects in hazard regression models is considered, where covariate dependence is monotone rather than proportional. It is well-known that the proportionality assumption underlying the Cox proportional hazards model does not hold in many applications. Further, the effect of a covariate is often monotone, in the sense that the lifetime (or duration) conditional on a higher value of the covariate ages faster or slower than that conditional on a lower value. These ordered/monotone alternatives to proportionality constitute a popular model for covariate effects in the two-sample setup, and have recently been extended to the case of continuous covariates. Under these models, the ratio of hazard functions (or cumulative hazard functions) increases or decreases with duration. Ordered departures of this kind are common in applications, and the models provide useful and intuitively appealing descriptions of covariate dependence in non-proportional situations. Here, methods for estimating hazard regression models under monotone departures are proposed. In particular, it is shown how the histogram sieve estimators can be smoothed and order restricted estimation performed using biased bootstrap techniques such as adaptive bandwidth kernel estimators or data tilting. The performance of the estimators is compared using simulated data, and their use is illustrated with applications from biomedicine and economic duration data.

Order restricted estimators: some bias results

Stimulated by practical concerns in the use of order restricted estimators, we study some bias issues of isotonic estimators of ordered parameters. We obtain explicit expressions for biases of order restricted estimators of ordered means of two normal populations with common variance. Based on intuitive considerations, we also propose a new estimator of the common variance of two normal populations having ordered normal means and show that it has smaller bias than the maximum likelihood estimator of the variance and has smaller mean squared error than the usual unbiased estimator of the common variance.

Estimating distributions with increasing failure rate in an imperfect repair model.

A failed system is repaired minimally if after failure, it is restored to the working condition of an identical system of the same age. We extend the nonparametric maximum likelihood estimator (MLE) of a system's lifetime distribution function to test units that are known to have an increasing failure rate. Such items comprise a significant portion of working components in industry. The order-restricted MLE is shown to be consistent. Similar results hold for the Brown-Proschan imperfect repair model, which dictates that a failed component is repaired perfectly with some unknown probability, and is otherwise repaired minimally. The estimators derived are motivated and illustrated by failure data in the nuclear industry. Failure times for groups of emergency diesel generators and motor-driven pumps are analyzed using the order-restricted methods. The order-restricted estimators are consistent and show distinct differences from the ordinary MLEs. Simulation results suggest significant improvement in reliability estimation is available in many cases when component failure data exhibit the IFR property.

An empirical comparison of inference using order-restricted and linear logit models for a binary response

In many applications with a binary response and an ordinal or quantitative predictor, it is natural to expect the response probability to change monotonically. Two possible models are a linear model with some link, such as the linear logit model, and a more general order–restricted model that assumes monotonicity alone. The order–restricted approach is more complex to apply, and we investigate whether it may be worth the extra effort. Specifically, suppose the order restriction truly holds but a simpler linear model does not. For testing the hypothesis of independence, is there the potential of a substantive power gain by performing an order-restricted test? For estimating a set of binomial parameters, how large must the sample size be before the consistency of the order-restricted estimates and inconsistency of the model–based estimates makes a substantive difference to mean square errors? We conducted a limited simulation study comparing estimators and likelihood–ratio tests for the linear logit model and for the order–restricted model. Results suggest that order–restricted inference is preferable for moderate to large sample sizes when the true probabilities take only a couple of levels, such as in a dose–response experiment when all doses provide a uniform improvement over placebo. If the true probabilities are strictly monotone but deviate somewhat from the linear logit model, the logit-based inference is usually more powerful unless the sample size is extremely large. When the true probabilities may have slight departures from monotonicity, the order-restricted estimates often perform better, particularly for moderate to large samples.

Order-Restricted Bayesian Estimation of the Age Composition of a Population of Atlantic Cod

Order-Restricted Bayesian Estimation of the Age Composition of a Population of Atlantic Cod

Analysis of Contingency Tables by Correspondence Models Subject to Order Constraints

Abstract Inferential correspondence analysis, which has gained much attention in recent years, is applied here to contingency tables with ordered categories. To reflect such order, the parameters of the underlying correspondence models are constrained to follow the order induced by the categories of the analyzed table. A reparameterization of the correspondence model in terms of a latent variable model is presented. This allows a simple and straightforward use of the EM algorithm to obtain efficient order-restricted estimates. A goodness-of-fit test is also discussed, and an example is analyzed. A small Monte Carlo example is presented.

Analysis of Contingency Tables by Correspondence Models Subject to Order Constraints

Abstract Inferential correspondence analysis, which has gained much attention in recent years, is applied here to contingency tables with ordered categories. To reflect such order, the parameters of the underlying correspondence models are constrained to follow the order induced by the categories of the analyzed table. A reparameterization of the correspondence model in terms of a latent variable model is presented. This allows a simple and straightforward use of the EM algorithm to obtain efficient order-restricted estimates. A goodness-of-fit test is also discussed, and an example is analyzed. A small Monte Carlo example is presented.

Recent Developments in Nonparametric Density Estimation

Advances in computation and the fast and cheap computational facilities now available to statisticians have had a significant impact upon statistical research, and especially the development of nonparametric data analysis procedures. In particular, theoretical and applied research on nonparametric density estimation has had a noticeable influence on related topics, such as nonparametric regression, nonparametric discrimination, and nonparametric pattern recognition. This article reviews recent developments in nonparametric density estimation and includes topics that have been omitted from review articles and books on the subject. The early density estimation methods, such as the histogram, kernel estimators, and orthogonal series estimators are still very popular, and recent research on them is described. Different types of restricted maximum likelihood density estimators, including order-restricted estimators, maximum penalized likelihood estimators, and sieve estimators, are discussed, where restrictions are imposed upon the class of densities or on the form of the likelihood function. Nonparametric density estimators that are data-adaptive and lead to locally smoothed estimators are also discussed; these include variable partition histograms, estimators based on statistically equivalent blocks, nearest-neighbor estimators, variable kernel estimators, and adaptive kernel estimators. For the multivariate case, extensions of methods of univariate density estimation are usually straightforward but can be computationally expensive. A method of multivariate density estimation that did not spring from a univariate generalization is described, namely, projection pursuit density estimation, in which both dimensionality reduction and density estimation can be pursued at the same time. Finally, some areas of related research are mentioned, such as nonparametric estimation of functionals of a density, robust parametric estimation, semiparametric models, and density estimation for censored and incomplete data, directional and spherical data, and density estimation for dependent sequences of observations.

Review Papers: Recent Developments in Nonparametric Density Estimation

Abstract Advances in computation and the fast and cheap computational facilities now available to statisticians have had a significant impact upon statistical research, and especially the development of nonparametric data analysis procedures. In particular, theoretical and applied research on nonparametric density estimation has had a noticeable influence on related topics, such as nonparametric regression, nonparametric discrimination, and nonparametric pattern recognition. This article reviews recent developments in nonparametric density estimation and includes topics that have been omitted from review articles and books on the subject. The early density estimation methods, such as the histogram, kernel estimators, and orthogonal series estimators are still very popular, and recent research on them is described. Different types of restricted maximum likelihood density estimators, including order-restricted estimators, maximum penalized likelihood estimators, and sieve estimators, are discussed, where restrictions are imposed upon the class of densities or on the form of the likelihood function. Nonparametric density estimators that are data-adaptive and lead to locally smoothed estimators are also discussed; these include variable partition histograms, estimators based on statistically equivalent blocks, nearest-neighbor estimators, variable kernel estimators, and adaptive kernel estimators. For the multivariate case, extensions of methods of univariate density estimation are usually straightforward but can be computationally expensive. A method of multivariate density estimation that did not spring from a univariate generalization is described, namely, projection pursuit density estimation, in which both dimensionality reduction and density estimation can be pursued at the same time. Finally, some areas of related research are mentioned, such as nonparametric estimation of functionals of a density, robust parametric estimation, semiparametric models, and density estimation for censored and incomplete data, directional and spherical data, and density estimation for dependent sequences of observations.

Order-restricted goodness-of-fit tests based on spacings

Goodness-of-fit tests are proposed for unimodal densities and U-shaped hazards. The tests are based on maximum-product-of-spacings estimators, and incorporate unimodality or U-shapedness using order restrictions. A slightly improved “maximum violator” algorithm is given for computing the order-restricted estimates and test statistics. Modified spacings such as “k-spacings”, which may actually increase power, ensure computational feasibility when sample sizes are large. Simulations demonstrate that for samples of size less than twenty, the use of order restrictions can increase power, even with modified spacings. The proposed methods can be used as approximations in cases of null hypotheses that are specified only up to unknown parameters that are estimated.