Abstract
Simultaneous hypothesis tests can fail to provide results that meet logical requirements. For example, if A and B are two statements such that A implies B, there exist tests that, based on the same data, reject B but not A. Such outcomes are generally inconvenient to statisticians (who want to communicate the results to practitioners in a simple fashion) and non-statisticians (confused by conflicting pieces of information). Based on this inconvenience, one might want to use tests that satisfy logical requirements. However, Izbicki and Esteves shows that the only tests that are in accordance with three logical requirements (monotonicity, invertibility and consonance) are trivial tests based on point estimation, which generally lack statistical optimality. As a possible solution to this dilemma, this paper adapts the above logical requirements to agnostic tests, in which one can accept, reject or remain agnostic with respect to a given hypothesis. Each of the logical requirements is characterized in terms of a Bayesian decision theoretic perspective. Contrary to the results obtained for regular hypothesis tests, there exist agnostic tests that satisfy all logical requirements and also perform well statistically. In particular, agnostic tests that fulfill all logical requirements are characterized as region estimator-based tests. Examples of such tests are provided.
Highlights
One of the practical shortcomings of simultaneous test procedures is that they can lack logical consistency [1,2]
Izbicki and Esteves [3] prove that the only tests that are fully coherent are trivial tests based on point estimation, which are generally void of statistical optimality
This section describes the mathematical setup for agnostic testing schemes
Summary
One of the practical shortcomings of simultaneous test procedures is that they can lack logical consistency [1,2]. Izbicki and Esteves [3] prove that the only tests that are fully coherent are trivial tests based on point estimation, which are generally void of statistical optimality. This finding suggests that alternatives to the standard “reject versus accept” tests should be explored. An agnostic test enables one to explicitly deal with the difference between “accepting a hypothesis H” and “not rejecting H (remaining agnostic)” This distinction will be made clearer, which derives agnostic tests under a Bayesian decision-theoretic standpoint by means of specific penalties for false rejection, false acceptance and excessive abstinence.
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Free) 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
