Observed Significance Level Research Articles

Technical note: MAMBA-Multi-pAradigM voxel-Based Analysis: A computational cookbot.

Voxel-Based (VB) analysis embraces a multifaceted ensemble of sophisticated techniques, lying at the boundary between image processing and statistical modeling, that allow for a frequentist inference of pathophysiological properties anchored to an anatomical reference. VB methods has been widely adopted in neuroimaging studies and, more recently, they are gaining momentum in radiation oncology research. However, the price for the power of VB analysis is the complexity of the underlying mathematics and algorithms. In this paper, we present the Multi-pAradigM voxel-Based Analysis (MAMBA) toolbox, which is intended for a flexible application of VB analysis in a wide variety of scenarios in medical imaging and radiation oncology. The MAMBA toolbox is implemented in Matlab. It provides open-source functions to compute VB statistical models of the input data, according to a great variety of regression schemes, and to derive VB maps of the observed significance level, performing a non-parametric permutation inference. The toolbox allows for including VB and global outcomes, as well as an arbitrary amount of VB and global Explanatory Variables (EVs). In addition, the Matlab Parallel Computing Toolbox is exploited to take advantage of the perfect parallelizability of most workloads. The use of MAMBA was demonstrated by means of several realistic examples on a synthetic dataset mimicking a radiation oncology scenario. MAMBA is an open-source toolbox, freely available for academic and non-commercial purposes. It is designed to make state-of-the-art VB analysis accessible to research scientists without the programming resources needed to build from scratch their own software solutions. At the same time, the source code is handed out for more experienced users to complement their own tools, also customizing user-defined models. MAMBA guarantees high generality and flexibility in the design of the statistical models, significantly expanding on the features of available free tools for VB analysis. The presented toolbox aims at increasing the reach of VB studies as well as the sharing of research results.

Evaluation methods for estimation of Weibull parameters used in Monte Carlo simulations for safety analysis of pressure vessels

Abstract The test data for static burst strength and load cycle fatigue strength of pressure vessels can often be well described by Gaussian normal or Weibull distribution functions. There are various approaches which can be used to determine the parameters of the Weibull distribution function; however, the performance of these methods is uncertain. In this study, six methods are evaluated by using the criterion of OSL (observed significance level) from Anderson-Darling (AD) goodness of Fit (GoF), These are: a) the norm-log based method, b) least squares regression, c) weighted least squares regression, d) a linear approach based on good linear unbiased estimators, e) maximum likelihood estimation and f) method of moments estimation. In addition, various approaches of ranking function are considered. The results show that there are no outperforming methods which can be identified clearly, primarily due to the limitation of the small sample size of the test data used for Weibull analysis. This randomness resulting from the sampling is further investigated by using Monte Carlo simulations, concluding that the sample size of the experimental data is more crucial than the exact method used to derive Weibull parameters. Finally, a recommendation is made to consider the uncertainties of the limitations due to the small size for pressure vessel testing and also for general material testing.

R-Squared-Bootstrapping for Gegenbauer-Type Long Memory

The autocorrelation of long memory processes decays much slower than that of short memory processes, e.g. autoregressive processes. Fractional integration and Gegenbauer are basic models that show long memory behavior. Many tests for long memory of the fractional integration type (at f = 0) have been developed based on test statistics (e.g. the R/S-type statistics) or parameter estimators (e.g. the GPH estimator). But, so far, no work has shown reasonable power on testing for cyclic long memory. The authors investigate a parametric bootstrap procedure with an R-squared statistic as an assessment of cyclic long memory behavior. According to simulation results, the R-squared-bootstrapping method performs excellently in detecting Gegenbauer-type processes (e.g. with long memory behavior associated with frequencies $$f\in (0,0.5]$$ ) while at the same time controlling observed significance levels. The R-squared-bootstrapping test is also applied to the Lynx data and the result suggests the presence of long range dependence.

Statistical Inference for Least Absolute Deviation Regression with Autocorrelated Errors

The method of least absolute deviation provides a robust alternative to least squares, particularly when the data follow distributions that are non-normal and subject to outliers. While inference in least squares estimation is well understood, inferential procedures in the situation of least absolute deviation estimation have not been studied as extensively, particularly in the presence of autocorrelation. In this search, we study two alternative significance test procedures in least absolute deviation regression, along with two approaches used to correct for serial correlation. The study is based on a Monte Carlo simulation, and comparisons are made based on observed significance levels.

Using Monte Carlo Normal Distributions to Evaluate Structural Models With Nonnormal Data

Statistical theories of goodness-of-fit tests in structural equation modeling are based on asymptotic distributions of test statistics. When the model includes a large number of variables or the population is not from a multivariate normal distribution, the asymptotic distributions do not approximate the distribution of the test statistics very well at small sample sizes. A variety of methods have been developed to improve the accuracy of hypothesis testing at small sample sizes. However, all these methods have their limitations, specially for nonnormal distributed data. We propose a Monte Carlo test that is able to control Type I error with more accuracy compared to existing approaches in both normal and nonnormally distributed data at small sample sizes. Extensive simulation studies show that the suggested Monte Carlo test has a more accurate observed significance level as compared to other tests with a reasonable power to reject misspecified models.

On the functional extinction of the Passenger Pigeon.

The Passenger Pigeon (Ectopistes migratorius) was a social breeder, and it has been suggested that the species experienced functional extinction, defined as a total reproductive failure, prior to its actual extinction in the early years of the 20th century. We applied a novel randomization test based on the relative times of the most recent egg- and skin-specimen sightings (i.e., recorded date of specimen collection) to test for functional extinction. For a total of 6 eggs and 27 skins, the observed significance level was 0.38, which indicated that the species did not become functionally extinct. Thus, proposals to reverse its rapid decline in the late 19th century could have been successful.

Asymmetric nonlinear mean reversion in real effective exchange rates: A Fisher-type panel unit root test applied to Sub-Saharan Africa

This study investigates the nonlinear data generating processes of real effective exchange rates in a panel of Sub-Saharan African countries; the region with the highest transportation costs, trade barriers in international arbitrage and frequent central bank intervention in the foreign exchange market, which are plausible main sources of nonlinear and asymmetric deviations from purchasing power parity. By means of Monte Carlo simulations, we use the empirical distributions of the exponential smooth transition autoregressive (ESTAR), and the asymmetric ESTAR data generating processes to test for mean reversion in monthly real effective exchange rates. We then apply Fisher's inverse chi-square test that combines the observed significance levels of independent univariate unit root tests to test for panel unit roots. The findings suggest that once nonlinearities and asymmetries are taken into account, there is more evidence in favor of the purchasing power parity hypothesis.

One Swallow Doesn't Make a Summer: New Evidence on Anchoring Effects

Some researchers have argued that anchoring in economic valuations casts doubt on the assumption of consistent and stable preferences. We present new evidence that explores the strength of certain anchoring results. We then present a theoretical framework that provides insights into why we should be cautious of initial empirical findings in general. The model importantly highlights that the rate of false positives depends not only on the observed significance level, but also on statistical power, research priors, and the number of scholars exploring the question. Importantly, a few independent replications dramatically increase the chances that the original finding is true. (JEL D12, C91)

The Availability of Skills among Workers in Commercial Banks in Aqaba, Jordan: Hashemite Kingdom of Jordan: An Exploratory Study in National Banks in the City of Aqaba-Jordan

The aim of this study is to identify the impact of computerized management information systems in the performance of workers in the commercial banks of Aqaba-Jordan, a questionnaire was developed and designed to achieve the objectives of this study in order to collect data, and was distributed on the sample of (92) employees from various commercial banks in Aqaba-Jordan, in a set of 8-10 questionnaires for each bank, (88) questionnaires have been restored in a percentage of 93%. The questionnaire was based on closed type of questionnaires. The researcher has calculated several statistical measures, namely arithmetic average, Alpha Cronbach's coefficient, t test, standard deviation, ANOVA, and Toki test. The acceptance criteria for hypotheses or question (item) were that the arithmetic mean should be greater than 3.00 and to be significant, the observed significance level of t must be less than 0.05. Also this study aimed to investigate the perceptions of the sample about the dimensions of the information systems for job performance requirements, programming requirements, human requirements and material requirements, it was found that there a number of results that contribute to the development of manpower and the functionality of the banks. The followings represent the most important results of the study: 1. There is a relationship between the characteristics of the beneficiaries and their attitudes towards management information systems. 2. The level of utilization of information systems is influenced by the attitudes of the beneficiaries towards the adoption of computer in the applications of information systems and by their attitudes toward the requirements of information systems and information systems activities, which are attributable to demographic variables (gender, scientific qualifications, experience, and job level). The study also showed that there are statistically significant differences between respondents' perceptions of job performance and attributed to demographic variables (gender, qualifications, experience, and job level).

A new Bayesian procedure for testing point null hypotheses

Testing point null hypotheses is a very common activity in various applied situations. However, the existing Bayesian testing procedure may give evidence which does not agree with the classical frequentist p-value in many point null testing situations. A typical example for this is the well known Lindley’s paradox (Lindley in Biometrika 44:187–192, 1957). In this paper we propose an alternative testing procedure in the Bayesian framework. It is shown that for many classical testing examples, the Bayesian evidence derived by our new testing procedure is not contradictory to its frequentist counterpart any more. In fact, the new Bayesian evidence under the noninformative prior is usually coincident with the frequentist observed significance level.

Design Allowable for Durability Analysis of Advanced Composites

An allowable design method is presented, along with its experimental validation, to obtain engineering-standard design allowable for fatigue life evaluation of advanced composites. A small sample test method for stress/life curves was established to overcome these issues based on the regression analysis theory. Statistically-based design values, such as the B-basis value, the Anderson-Darling test statistic and the observed significance level, were obtained depends on the characteristics of the composite material test data. For evaluation of the fatigue life of advanced composites, the S-N curve-fitting parameters are obtained using the linear heteroscedastic regression method based on experimental data. Another advantage of this method is that all the test data at different stress levels can be analyzed coinstantaneously, which greatly saved the experimental efforts and specimen for confidential statistical results. Finally, an experimental test program has been conducted on typical laminate composites to generate statistically meaningful data for durability design. The numerical results show that the proposed method can be used as a guide to obtaining design allowable for durability analysis of advanced composite structures.

Sensitivity analysis of the t-distribution under truncated normal populations

In recent literature, the truncated normal distribution has been used to model the stochastic structure for a variety of random structures. In this paper, the sensitivity of the t-random variable under a left-truncated normal population is explored. Simulation results are used to assess the errors associated when applying the student t-distribution to the case of an underlying left-truncated normal population. The maximum errors are modelled as a linear function of the magnitude of the truncation and sample size. In the case of a left-truncated normal population, adjustments to standard inferences for the mean, namely confidence intervals and observed significance levels, based on the t-random variable are introduced.

A Scale‐space Approach for Detecting Non‐stationarities in Time Series

Abstract. The presented method called Significant Non‐stationarities, represents an exploratory tool for identifying significant changes in the mean, the variance, and the first‐lag autocorrelation coefficient of a time series. The changes are detected on different time scales. The statistical inference for each scale is based on accurate approximation of the probability distribution, using test statistics being ratios of quadratic forms. No assumptions concerning the autocovariance function of the time series are made as the dependence structure is estimated non‐parametrically. The results of the analyses are summarized in significance maps showing at which time points and on which time scales significant changes in the parameters occur. The performance of the given method is thoroughly studied by simulations in terms of observed significance level and power. Several examples, including a real temperature data set, are studied. The examples illustrate that it is important to carry out the analysis on several time horizons.

Case comment: an expert's report criticizing plaintiff's failure to account for multiple comparisons is deemed admissible in EEOC v.Autozone

In its opinions in Daubert, Joiner and Kumho Tire, the U.S. Supreme Court assigned trial judges a ‘gate-keeping’ role in assuring that proposed expert testimony is sufficiently reliable that it should be admitted into evidence. A section of the Daubert decision described guidelines to assist trial judges in their evaluation of expert testimony. This comment discusses a careful district court opinion concerning a motion, under the principles of Daubert, to exclude the report of defendant's expert in the sex discrimination case, EEOC v. Autozone. The technical statistical issue dealt with the need for adjusting the observed significance levels or p-values of individual statistical tests when a large number of related tests are conducted. The Equal Employment Opportunity Commission and its expert argued that while such adjustments are used in medical applications of statistics, they are not needed in labour economics. The decision is noteworthy because the judge realized that the principles of statistical inference remain the same regardless of the origin of the data. After describing the issue and reanalysing some of the data, alternative statistical methods that stratify the data into comparable subgroups and combine the results for the separate strata are often more appropriate. They avoid making many separate tests and have higher statistical power to detect discrimination. Since 10 years of hiring and promotion data were analysed, which cover both pre- and post-charge time periods, two possible scenarios are described. The first is the one that apparently was the focus of both experts and the second focuses on the status of a particular female plaintiff for a job as guard in the early period. It is shown that overall the data do not support a claim of class-wide discrimination but might support a claim of sex discrimination in hiring for guard positions in the pre-charge time period.

Detection of the Number of Signals Using the Benjamini-Hochberg Procedure

This paper presents a novel approach to detect multiple signals embedded in noisy observations from a sensor array. We formulate the detection problem as a multiple hypothesis test. To control the global level of the multiple test, we apply the false discovery rate (FDR) criterion proposed by Benjamini and Hochberg. Compared to the classical familywise error rate (FWE) criterion, the FDR-controling procedure leads to a significant gain in power for large size problems. In addition, we apply the bootstrap technique to estimate the observed significance level required by the FDR-controling procedure. Simulations show that the FDR-controling procedure always provides higher probability of correct detection than the FWE-controling procedure. Furthermore, the reliability of the proposed test procedure is not affected by the gain in power of the test

Evidence for novel susceptibility genes for late-onset Alzheimer's disease from a genome-wide association study of putative functional variants

This study sets out to identify novel susceptibility genes for late-onset Alzheimer's disease (LOAD) in a powerful set of samples from the UK and USA (1808 LOAD cases and 2062 controls). Allele frequencies of 17 343 gene-based putative functional single nucleotide polymorphisms (SNPs) were tested for association with LOAD in a discovery case-control sample from the UK. A tiered strategy was used to follow-up significant variants from the discovery sample in four independent sample sets. Here, we report the identification of several candidate SNPs that show significant association with LOAD. Three of the identified markers are located on chromosome 19 (meta-analysis: full sample P = 6.94E - 81 to 0.0001), close to the APOE gene and exhibit linkage disequilibrium (LD) with the APOEepsilon4 and epsilon2/3 variants (0.09 < D'<1). Two of the three SNPs can be regarded as study-wide significant (expected number of false positives reaching the observed significance level less than 0.05 per study). Sixteen additional SNPs show evidence for association with LOAD [P = 0.0010-0.00006; odds ratio (OR) = 1.07-1.45], several of which map to known linkage regions, biological candidate genes and novel genes. Four SNPs not in LD with APOE show a false positive rate of less than 2 per study, one of which shows study-wide suggestive evidence taking account of 17 343 tests. This is a missense mutation in the galanin-like peptide precursor gene (P = 0.00005, OR = 1.2, false positive rate = 0.87).

肺アスペルギルス症の発症リスクコントロール要因―入院環境と抗真菌剤選択の重要性―

In patients with blood diseases, maintaining a proper ward environment as regards temperature and humidity and effective drug treatment are important in preventing the development of deep mycosis. We investigated the influences of temperature and humidity and use of antifungal agents on the incidence of pulmonary aspergillosis in such patients. The subjects were 132 patients admitted to our hematology ward between January 1999 and December 2004. Of the 132 patients, 22 developed pulmonary aspergillosis but 110 did not. There were no significant differences in patient background between these patients. The incidence of pulmonary aspergillosis was high (observed significance level of test : 0.004) during summer when the mean humidity in the ward was high. Administration of the antifungal agent micafungin for prevention or treatment reduced incidence (observed significance level of test : 0.042). Based on these findings, we conclude that proper control of ward humidity and selection of an effective antifungal agent are important factors in reducing the incidence of aspergillosis in wards where patients are being treated for blood diseases.

Testing equality of covariance matrices when data are incomplete

In the statistics literature, a number of procedures have been proposed for testing equality of several groups’ covariance matrices when data are complete, but this problem has not been considered for incomplete data in a general setting. This paper proposes statistical tests for equality of covariance matrices when data are missing. A Wald test (denoted by T 1 ), a likelihood ratio test (LRT) (denoted by R), based on the assumption of normal populations are developed. It is well-known that for the complete data case the classic LRT and the Wald test constructed under the normality assumption perform poorly in instances when data are not from multivariate normal distributions. As expected, this is also the case for the incomplete data case and therefore has led us to construct a robust Wald test (denoted by T 2 ) that performs well for both normal and non-normal data. A re-scaled LRT (denoted by R * ) is also proposed. A simulation study is carried out to assess the performance of T 1 , T 2 , R, and R * in terms of closeness of their observed significance level to the nominal significance level as well as the power of these tests. It is found that T 2 performs very well for both normal and non-normal data in both small and large samples. In addition to its usual applications, we have discussed the application of the proposed tests in testing whether a set of data are missing completely at random (MCAR).

Dogs Still Do Resemble Their Owners

It need not be, in a rational, scientific world, that we should see the case for dogs resembling their owners as stronger than would any other observer of the available data, but it traditionally falls to the original authors to defend their original conclusions (Roy & Christenfeld, 2004), and we do so here. We also present new data, based on a new method, that support our conclusions. Levine (2005) raised an important issue concerning the nonindependence of ratings in our original study, though we suggest that the implications are not what he indicated. In our study, each dog appeared twice, once as a foil and once as the correct choice. Levine is worried that if the judges operated nonindependently across ratings, refusing, for example, to pick the same dog twice, then the expected value of getting a match right by chance could be very different from the .5 we used in our statistical analyses. However, the effect that concerns him, and that he illustrated with a small example, is attenuated in our study because we had 15, not 3, dogs in each set; because we used not 1 but 14 different presentation orders of dog-owner choices; and because judges’ actual behavior turns out to have been largely independent. (It is not logically necessary, nor even strategically advisable, for judges always to avoid picking again a dog they have previously chosen, and judges show little, albeit some, inclination to avoid this.) Although calculating the effect of these factors is mathematically formidable, it is readily examined with a computer simulation. We randomly generated 1,000 orders of dogs, each with 20,000 random ‘‘judges.’’ In the simulation, the judges were created to show the same slight tendency to avoid picking the same dog twice as the actual judges in the original study. This simulation suggests that the expected probability of getting a dog right is .5, the probability we used in our study, and that the standard error of this probability is .00005. Thus, the fear that the expected value is different from the value we chose, in either the liberal or the conservative direction, is unrealized. More subtly, nonindependence can change the distribution of scores under the null hypothesis, without changing their mean. Consider the simplest example of two owners. Nonindependent judges would get both right or both wrong, whereas independent judges, given a separate foil for each match, could get one right and one wrong. Thus, the shape of the distribution, and so the observed significance level, can change from that assumed with independent judges. This effect was attenuated in our experiment because there were 15 dogs, and also because our judges made nearly independent choices. Indeed, the computer simulation indicates that there is no cause for concern: Using the observed level of nonindependence, the simulation yielded effectively the same distribution under the null hypothesis and, therefore, the same significance of the observed effects as in the original report. Perhaps the strongest evidence, however, comes from fresh data, collected with a different method that avoids the nonindependence issue. We used 24 of the original dog and owner pictures; half the dogs were purebred, and half were not. The pictures were chosen from the original set, excluding pictures with a beach background that distinguished them from the others with a grassy park background. We also excluded two mixed-breed dogs that were close to purebred, and then reduced the number of purebreds to match the dozen nonpurebreds in the sample. For each dog, six potential owners were selected. One was the real owner, and the other five were randomly selected owners of other dogs—other purebreds in the case of a purebred picture, and nonpurebreds otherwise. With this choice of foils, any ability to match dogs with their owners would have to be based on more than just having some idea of whether a particular person was likely to own a purebred. Ninety-six new, naive judges were each shown one of the purebred dog-owners sets and one of the nonpurebred dog-owner sets, in counterbalanced order. They ranked each of the six possible owners in likelihood of being the correct match by placing slips of paper with the numbers 1 through 6 on the possible-owner photographs. No subject saw any dog or owner more than once, and each contributed only one rating to the evaluation of purebreds and one to the evaluation of nonpurebreds. For the nonpurebreds, the real owner was chosen, on average, in position 3.6, no different from the random-guessing value of 3.5, t(95) 5 0.62, p 5 .54, effect size d 5 0.06. For the Address correspondence to Nicholas J.S. Christenfeld, Department of Psychology, University of California, San Diego, 9500 Gilman Dr., La Jolla, CA 92093-0109; e-mail: nchristenfeld@ucsd.edu. PSYCHOLOGICAL SCIENCE

Testing for a ‘within-subjects’ association in repeated measures data

‘Within-subjects association’ is a tendency for a subject's personal highs in a variable Y to be associated with personal highs (or lows) in a variable X. We show that the within-subject Spearman correlations, individually of little reliability, can be aggregated across subjects in a repeated measures data set to effectively detect such an association. Observed significance levels can be assigned using an approximate randomization procedure even in small samples with variable numbers of observations per subject and ties in the ranks. We investigate the power of this method under the assumptions that the ranks within subjects follow a distribution proposed by Henze. On the basis of these results, we make recommendations for sample designs.