Alternative Test Procedure Research Articles

Simultaneous Non‐inferiority Test of Sensitivity and Specificity for Two Diagnostic Procedures in the Presence of a Gold Standard

AbstractSensitivity and specificity have traditionally been used to assess the performance of a diagnostic procedure. Diagnostic procedures with both high sensitivity and high specificity are desirable, but these procedures are frequently too expensive, hazardous, and/or difficult to operate. A less sophisticated procedure may be preferred, if the loss of the sensitivity or specificity is determined to be clinically acceptable. This paper addresses the problem of simultaneous testing of sensitivity and specificity for an alternative test procedure with a reference test procedure when a gold standard is present. The hypothesis is formulated as a compound hypothesis of two non‐inferiority (one‐sided equivalence) tests. We present an asymptotic test statistic based on the restricted maximum likelihood estimate in the framework of comparing two correlated proportions under the prospective and retrospective sampling designs. The sample size and power of an asymptotic test statistic are derived. The actual type I error and power are calculated by enumerating the exact probabilities in the rejection region. For applications that require high sensitivity as well as high specificity, a large number of positive subjects and a large number of negative subjects are needed. We also propose a weighted sum statistic as an alternative test by comparing a combined measure of sensitivity and specificity of the two procedures. The sample size determination is independent of the sampling plan for the two tests.

Confounding Factors in Bioassays with Freshwater and Marine Organisms

The use of bioassays in ecological risk assessments often raises questions about the causative factors, and insight into the possibility that confounding factors, such as pH or increased ammonia concentrations, might be responsible for the observed toxicity is needed. It was decided to develop a practical approach for the Dutch situation, in which a first screening is carried out based on provisional criteria. In collecting the required data, dozens of experiments were performed, while the scientific literature was searched for additional information. It is concluded that the provisional criteria specified are at present useful tools in interpreting results of bioassays. Depending on the outcome and the aim of the research, it might be necessary to further reduce uncertainties in the interpretation. This might require some additional experiments, using alternative controls or test procedures or altering the composition of the original sample.

Coating systems for infra structural works

“Rijkswaterstaat” is a government organisation responsible for the construction, operation and maintenance of a large number of structures. The organisation consist of a general engineering division and a number of regional offices responsible for the Rijkswaterstaat objects in their particular region. Rijkswaterstaat spends between 40 to 50 K€ per year on coating and coating related activities. When the steel structures of the Eastern Scheldt barrier (± 200,000 m2 of steel) were coated in the mid eighties the applied high solid epoxy system developed extensive cracking. This resulted in numerous locations with pit corrosion. Investigations were carried out and it was concluded that the cracking of this specific high-solids epoxy system was caused by restrained shrinkage. This resulted in the build up of high internal mechanical stresses in the coating system. Problems with stress build up were also encountered on other projects, although less severe and mainly concentrated at edges and welded joints. To prevent future problems, RWS has taken a number of initiatives such as the developed a procedure for the selection of coating systems. This selection procedure has proven to be useful for RWS in selecting coating systems for use on infra structural works. The basic principle of the selection procedure of the RWS selection procedure is different from most other procedures. The RWS selection procedure uses the object to be coated as a starting point instead of the coating system. This is done by first evaluating all relevant aspects of the object which may influence the selection of a system. These aspects are then translated into so called “Functional requirements”. Coating systems with properties that are in compliance with the performance requirements are gathered and through comparison of requirements and properties a selection is made. The objects that, until now, have been coated with systems selected with the selection procedure have performed adequately. Systems that have been selected with the procedure to replace or repair those that have developed cracking have until now not shown any new problems. At the present time more and more alternative tests and test procedures become available, most of them derived from ISO 12944. These will be studied closely and compared with the currently used methods on the level of functional requirements. RWS is already actively involved in the search for alternative test methods and will remain to do so in the future. In view of this the Rijkswaterstaat selection procedure will remain in function.

A permutation alternative and other test procedures for spatially correlated data in one-way anova

Research in many disciplines involves data with spatially correlated observations. Spatial dependence violates the independent errors assumption required for techniques such as the standard one-way analysis of variance for a completely randomized design. The testing methodology within the correlated errors approach has not been investigated within a spatial context. For one-way fixed effects analysis of variance, a permutation test and tests associated with the correlated errors approach are investigated through simulation. No single test was superior with respect to both power and size but the standard Wald F test and a simple adjustment to it performed well overall.

Rainbow trout primary epidermal cell proliferation as an indicator of aquatic toxicity: an in vitro/in vivo exposure comparison

Little or no work has been carried out on primary cell cultures in terms of cellular proliferation and toxicity studies. Cell proliferation represents one of the most relevant cellular functions. Anti-PCNA antibodies have aroused considerable interest recently as potential immunocytochemical markers of proliferation for use in toxicity studies. In this study, PCNA methodology, which was developed primarily for mammalian tissues, was adapted to rainbow trout ( Oncorhynchus mykiss (R.)) primary cultured epidermal cells exposed in vivo i.e. whole animal exposures and in vitro for the study of the ecotoxicological potential of the aromatic amine, 2,4-dichloroaniline (2,4-DCA), a member of a little studied and widespread class of aquatic pollutants. There are many approaches to assess the proliferative activity of cells. Immunocytochemical methods offer a high sensitivity and specificity. The immunohistochemical avidin–biotin complex (ABC) method was used for the detection and quantification of PCNA, one of the best-known endogenous proliferation markers, applying the mammalian monoclonal antibody PC-10 to formalin-fixed primary cultures of rainbow trout skin. Here we describe our experience with the immunocytochemical detection and quantification of this proliferation marker. Results indicate that the antibody cross reacts with the corresponding rainbow trout epitope and that the alterations in PCNA labelling in the in vivo and in vitro exposed cultures followed similar patterns. This paper presents data on the validation of rainbow trout primary epidermal culture as an in vitro ecotoxicity model with epidermal proliferation as an endpoint. It can be concluded that cellular proliferation could be used as an indicator of the aquatic toxicity potential of xenobiotics. Correlations between cellular proliferation responses in primary cultures derived from in vivo exposed rainbow trout and primary cultures exposed in vitro were assessed. A dose-response was evidenced in both approaches, however the in vivo exposures appeared to be approximately two orders of magnitude more sensitive than the in vitro exposures. Responses in vitro occurred between 200 and 1000 μM while in vivo responses were between 2 and 10 μM. The good qualitative correspondence between the in vitro and in vivo results indicates that studies using trout epidermal cells allow the identification of xenobiotic effects in fish skin. However, further work is required before quantitative predictions i.e. effective concentrations in vivo, can be made from in vitro studies. This study suggests that the in vitro exposed rainbow trout primary cultured cell model with proliferation as an endpoint can be used as an alternative testing procedure to the whole animal assay.

Testing That a Dependent Process Is Uncorrelated

An analysis is presented of a new testing procedure for the null hypothesis that a stochastic process is uncorrelated when the process is possibly dependent. Unlike with existing procedures, the user does not need to choose any arbitrary number to implement the proposed test. The asymptotic null distribution of the proposed test statistic is not standard, but it is tabulated by means of simulations. The test is compared with two alternative test procedures that require selection of user-chosen numbers on the basis of asymptotic local power and finite sample behavior. Although the asymptotic local power of the proposed test is lower than those corresponding to the alternative tests, in a Monte Carlo study I show that in small samples the test typically better controls the type I error and that the loss of power is not substantial.

Interlaboratory validation of an atrazine immunoassay

An immunoassay method was validated in an interlaboratory study as an alternative test procedure to US Environmental Protection Agency (USEPA) method 508.1 rev. 2.0 for monitoring atrazine in drinking water in compliance with the Safe Drinking Water Act (SDWA). The study plan was approved by the USEPA Office of Water (OW). Nineteen laboratories participated in the study. Participants included personnel from water treatment plants and private and government laboratories certified to perform SDWA compliance monitoring. Field samples and reagent water were fortified with atrazine and shipped to study participants in a blind fashion. Data were used to generate quality control criteria to provide guidance for laboratories to evaluate method performance. The results of the study have been submitted to the USEPA OW for approval and promulgation of the method. This is the first immunoassay method to be validated under the SDWA.

COMPARING EMPIRICAL DISTRIBUTIONS OF P-VALUES FROM SIMULATIONS

In the context of hypothesis testing simulation studies, this paper advocates using graphical and numerical tools to summarize the results, beyond the conventional practice of just reporting empirical levels. Some of these tools are defined by us. They are mainly based on the computation of distances between the empirical distribution function of the p-values derived from the simulation experiment and the distribution function of the U([0, 1]) random variable. Their null distribution is tabulated. A joint study of several distances reveals that important aspects of a test can pass unnoticed if only empirical significance levels are calculated. The proposed tools are applied to two practical examples, demonstrating their usefulness in the discrimination between alternative test procedures and also in the detection of data not according to the null hypothesis.

A critical note to IEEE 1057-94 standard on hysteretic ADC dynamic testing

In recent gears, the IEEE 1057-93 Standard (1994) for specifying and testing measurement devices based on analog-to-digital converters (ADCs) earned scientific interest in several topics. In particular, investigations showed the histogram test of the standard to be insensitive to ADC hysteresis. In this paper, an alternative test procedure for determining the dynamic transfer function of an ADC with hysteresis is proposed. The procedure exploits digital signal processing to reduce the large amount of data required by the histogram test. Numerical and experimental results of performance characterization, comparison with a state-of-the-art procedure, and noise-sensitivity analysis are presented and discussed.

Testing Long-Run PPP in the Presence of Sticky Prices

In this paper we show that standard tests for long-run Purchasing Power Parity (PPP) are misspecified if aggregate prices are sticky. Using Monte Carlo simulations, we show that in small samples the ADF test has low power to reject the null of no cointegration when long-run PPP is tested using current prices instead of long-run equilibrium prices. We propose an alternative two-step testing procedure, which consists of, first, estimating long-run equilibrium prices from a standard money demand function and, then, testing long-run PPP as a relationship between the current exchange rate and long-run \QTR{em}{equilibrium} price differentials. Finally, we show that in small samples our proposed test has considerably higher power than the standard ADF test for a unit root in the real exchange rate. We apply our test to the effective exchange rate of the Greek drachma in the post-Bretton Woods period and find that it is closely linked to long-run equilibrium price differentials.

Swapping space for time and unfair tests of ecological models

AbstractTesting ecological models for management is an increasingly important part of the maturation of ecology as an applied science. Consequently, we need to work at applying fair tests of models with adequate data. We demonstrate that a recent test of a discrete time, stochastic model was biased towards falsifying the predictions. If the model was a perfect description of reality, the test falsified the predictions 84% of the time. We introduce an alternative testing procedure for stochastic models, and show that it falsifies the predictions only 5% of the time when the model is a perfect description of reality. The example is used as a point of departure to discuss some of the philosophical aspects of model testing.

Swapping space for time and unfair tests of ecological models

Testing ecological models for management is an increasingly important part of the maturation of ecology as an applied science. Consequently, we need to work at applying fair tests of models with adequate data. We demonstrate that a recent test of a discrete time, stochastic model was biased towards falsifying the predictions. If the model was a perfect description of reality, the test falsified the predictions 84% of the time. We introduce an alternative testing procedure for stochastic models, and show that it falsifies the predictions only 5% of the time when the model is a perfect description of reality. The example is used as a point of departure to discuss some of the philosophical aspects of model testing.

Analysing long memory and asymmetries

The paper presents evidence on nonlinearities in Finnish financial time series. The analysis concentrates on the so-called long-memory property which is examined using, various alternative test procedures. This analysis makes use of relatively long monthly Finnish time series which cover the period 1922–1996. The results give some evidence on long memory but one cannot say that the results would overwhelmingly support the existence of long memory in Finnish time series. There are, however, considerable differences between variables and the results are quite sensitive in terms of the treatment of short memory which also applies to different ways of prefiltering the data. Clearly more work is required to obtain more affirmative results in this respect. One way, of doing that is to apply asymmetric time models to find the source of nonlinearity. When that is done with the Finnish data some weak evidence on asymmetry is obtained.

Crossover Experiments: A Comparison of ANOVA Tests and Alternative Analyses

Crossover experiments are really special types of repeated measures experiments. The correct analysis of a repeated measures experiment depends on the structure of the variance-covariance matrix of the repeated measures. Certain structures of the variance-covariance matrix assure that analysis of variance procedures lead to valid statistical tests. Other covariance structures invalidate some of the analysis of variance tests. This paper provides conditions under which the traditional ANOVA tests are valid. For those situations when the traditional tests may be invalid, alternative test procedures are proposed, and these procedures are compared to one another in a simulation study. These alternative procedures include degrees of freedom adjustment techniques, a multivariate model approach, and a mixed models (estimated generalized least squares) approach. In cases where the usual ANOVA methods are not valid, the multivariate model approach is the method that met the nominal size requirement for the hypotheses tests of equal treatment and equal carryover effects.

An alternative one sided test of normal mean

This paper provides an alternative test procedure for the problem oftesting of normal mean for one sided alternative when the variance is known in case of costly trials. The proposed test is based on a sampling scheme which we call 'continuous adaptive design'. Some exact andasymptotic results related to the test and design are studied.

A note on generality of the Cagan model for money demand

The generality of the Cagan model for money demand is questioned. Some high inflationary economies are hit by external shocks that may have a non-transitory effect on macroeconomic variables including consumer prices, high-powered money, or narrow money which are usually used in these studies. When a structural break exists, standard unit-root tests such as the augmented Dickey-Fuller test or Phillips-Perron test are biased in favour of accepting the hypothesis of non-stationarity. An alternative Perron's testing procedure, which includes dummy variables that recognize the possibility of exogenous changes in level or trend, was applied on high inflation plagued Yugoslavia and Mexico. Consumer price and money variables are in both cases stationary around a deterministic breaking trend function. Therefore, no cointegration between the variables exists and the Cagan model cannot explain the nature of inflationary processes in these countries. Alternative specifications are necessary.

Comparison of One‐Step Acidic Extraction Versus Two‐Step Basic and Acidic Extraction Procedures for Semivolatile Analysis of Wastewater

The feasibility of using the U.S. Environmental Protection Agency's (U.S. EPA's) contract laboratory protocol method for analyzing semivolatile compounds in industrial wastewater pretreatment program samples was investigated because of time and labor savings involved. Various laboratory evaluation studies were conducted to determine whether this alternate one‐step acidic extraction method would give comparable results to U.S. EPA Method 625. These studies consisted of comparisons of method detection limits, spike recoveries in reagent water and actual field samples, and the precision between Method 625 and the alternate one‐step acidic extraction. More than 60 base‐neutral and acid extractables (55 target compounds and 6 surrogates) were used in the comparative study of method detection limits and spike recoveries of the two procedures. Using reagent water, one‐step acidic extraction gave consistently better recoveries than two‐step basic and acidic extraction. Similar results were obtained with representative field sample matrices. Based on results of this investigation, the Metropolitan Water Reclamation District of Greater Chicago, Illinois, submitted an application to U.S. EPA to have one‐step acidic extraction accepted as a site‐specific alternate test procedure for analyzing industrial wastewater samples. After thorough review, U.S. EPA Region V approved the use of this alternate one‐step acidic extraction method. It is recommended that other municipal analytical laboratories apply to U.S. EPA for similar approval.

Use of Two-Stage Test Statistic in the Two-Period Crossover Trials

For two-period crossover trials where the residual carryover can only exist in the presence of treatment effect, Willan (1988, Biometrics 44, 211-218) recommended use of the maximum test statistic that chooses the analysis with the larger test statistic corresponding to the parallel analysis and the crossover analysis. We construct two alternative two-stage test procedures, based on Grizzle's approach, that maintain the actual type I error rate at the desirable level. The power, accuracy, and precision of the analysis based on the modified two-stage procedures are compared to those based on the parallel analysis, crossover analysis, and the maximum test statistic.

Defined Structural and Performance Criteria would Facilitate the Validation and Acceptance of Alternative Test Procedures

Defined Structural and Performance Criteria would Facilitate the Validation and Acceptance of Alternative Test Procedures

ALTERNATIVE ANALYSES OF CROSSOVER DESIGNS WITH MORE THAN TWO PERIODS

A crossover experiment is a special form of a repeated measures experiment. An appropriate analysis of a repeated measures experiment depends on the form of the varian-cecovariance matrix of the repeated measures. Certain forms of this matrix yield valid analysis of variance F -tests while other forms invalidate these tests. In a crossover experiment where analysis of variance tests are invalid, two alternative tests of a linear contrast of the parameters are proposed. In addition to these approximate t-tests, three alternative methods for testing for equal treatment effects and equal carryover effects are proposed. A simulation study is conducted to evaluate these proposed alternative test procedures for power. Confidence levels and confidence interval lengths are also examined for those procedures from which an estimate of the linear contrast can be made.