Null Testing Research Articles

Evaluating the Impact of Supplemental Computer-Assisted Math Instruction in Elementary School: A Conceptual Replication

Despite the surge in efficacy studies, the relative effectiveness of any two math interventions is largely unknown and replications are rare. To ensure that students benefit from high-quality learning experiences, the current conceptual replication of Wang and Woodworth, a randomized control trial, evaluated the relative impacts of two computer programs used in a school district as supplements to students’ regular education math instruction, DreamBox Learning and Zearn Math. For 115 students in kindergarten and first grade, data were collected before and after an 11-week implementation period. Measures of effect size, but not null hypothesis statistical tests, suggested that participants in the DreamBox condition had higher test scores on posttest numeracy and geometry (Hedges g = .32 and .14). There was also a statistically significant group by pretest interaction, evidence of “Matthew Effects” for numeracy. Although some support for adopting DreamBox as a supplemental method for improving math competencies of students in early elementary school was provided, more research is needed as the true effect may be smaller than what the present study reports. The study’s implications are discussed.

Mediation analysis method review of high throughput data.

High-throughput technologies have made high-dimensional settings increasingly common, providing opportunities for the development of high-dimensional mediation methods. We aimed to provide useful guidance for researchers using high-dimensional mediation analysis and ideas for biostatisticians to develop it by summarizing and discussing recent advances in high-dimensional mediation analysis. The method still faces many challenges when extended single and multiple mediation analyses to high-dimensional settings. The development of high-dimensional mediation methods attempts to address these issues, such as screening true mediators, estimating mediation effects by variable selection, reducing the mediation dimension to resolve correlations between variables, and utilizing composite null hypothesis testing to test them. Although these problems regarding high-dimensional mediation have been solved to some extent, some challenges remain. First, the correlation between mediators are rarely considered when the variables are selected for mediation. Second, downscaling without incorporating prior biological knowledge makes the results difficult to interpret. In addition, a method of sensitivity analysis for the strict sequential ignorability assumption in high-dimensional mediation analysis is still lacking. An analyst needs to consider the applicability of each method when utilizing them, while a biostatistician could consider extensions and improvements in the methodology.

A new approach for uncertainty quantification in predictor-corrector quasi-static Monte Carlo transient simulation

Two different approaches are widely accepted for transient Monte Carlo (MC) simulation namely the Dynamic Monte Carlo (DMC) and the Predictor-Corrector Quasi-Static Monte Carlo (PCQS-MC). The MC transport code iMC developed at the Korea Advanced Institute of Science and Technology incorporates both approaches. In this paper, an original method for properly assessing the uncertainty of PCQS-MC is proposed and demonstrated using the iMC code. A detailed description of applying the PCQS method for transient MC calculation is presented with an emphasis on the origins of PCQS-MC uncertainties. The implementation of a quasi-static method, i.e., calculation of point-kinetics (PK) parameters, incurs an additional uncertainty for PCQS-MC calculation alongside the conventional stochastic MC uncertainty. Such quasi-static treatment-driven stochasticity cannot be recognized through the conventional MC sampling scheme, insinuating a significant underestimation of uncertainty without proper measures. To verify the findings, null-transient simulations have been performed for GODIVA and C5G7 benchmarks, where the former and latter problems require continuous and multi-group energy treatment respectively. For an improved uncertainty evaluation, a new sampling scheme referred to as the PK-sampling scheme is proposed where cycle-wise PK correction is made. Proper estimation of uncertainty can be made from the heap of cycle-wise corrected power through a unique screening process based on the null hypothesis testing. The proposed PK sampling scheme is tested for C5G7-TD transient benchmarks using iMC where the result plainly attests to the effectiveness of the new sampling approach.

Novel Ringdown Amplitude-Phase Consistency Test.

The ringdown signal emitted during a binary black hole coalescence can be modeled as a linear superposition of the characteristic damped modes of the remnant black hole that get excited during the merger phase. While checking the consistency of the measured frequencies and damping times against theKerr BH spectrum predicted by general relativity (GR) is a cornerstone of strong-field tests of gravity, the consistency of measured excitation amplitudes and phases have been largely left unexplored. For a nonprecessing, quasicircular binary black hole merger, we find that GR predicts a narrow region in the space of mode amplitude ratio and phase difference, independently of the spin of the binary components. Using this unexpected result, we develop a new null test of strong-field gravity which demands that the measured amplitudes and phases of different ringdown modes should lie within this narrow region predicted by GR. We call this the amplitude-phase consistency test and introduce a procedure for performing it using information from the ringdown signal. Lastly, we apply this test to the GW190521 event, using the multimodal ringdown parameters inferred by Capano etal. [arXiv:2105.05238]. While ringdown measurements errors for this event are large, we show that GW190521 is consistent with the amplitude-phase consistency test. Our test is particularly well suited for accommodating multiple loud ringdown detections as those expected in the near future, and can be used complementarily to standard black-hole spectroscopy as a proxy for modified gravity, compact objects other than black holes, binary precession and eccentricity.

Internal Control System and Financial Statement Fraud of Quoted Banks in Nigeria

The menace of financial statement fraud which has sent notable and reputable corporate entities to their unimaginable early grave soon after the publication of juicy profits has continuing posing catastrophic threats and bleeding the global economy into a state of comma. This dreaded phenomenon around the globe may be demystified through effective internal control system to provide managers with the means to increase accountability for their programs and to obtain reasonable assurance that the programs they direct meet established goals and objectives. Intuitively, an effective system is not only restricted to a digital one but to one that is able to deter, prevent and detect fraudulent activities frontally. The main aim of this study is to develop an internal control system that will help in reducing financial statement fraud of quoted banks in Nigeria through not only its deterrent, predictive and detective properties; but also, the recommended internal control procedures and policies which the accounting profession should adopt. Deterrent control mechanism, and preventive control mechanism were the proxies of internal control system while improper revenue recognition, overstatement of assets were the sub-constructs of the financial statements fraud. The researchers adopted correlational survey design for the study and used primary data via structured questionnaire. The null hypotheses test and other data were analyzed by Pearson Product Moment Correlation and simple linear regression tools with the aid of Statistical Packages for the Social Sciences version 25. The study found that all the internal control system proxies; deterrent control mechanism, and preventive control mechanism positively and significantly related to improper revenue recognition and overstatement of assets of quoted banks in Nigeria in the period of this study. The study concludes that internal control system effectively and significantly related with the financial statement fraud. Based on the empirical findings, the study recommends the need for banks as a matter of necessity to continue to reinvigorate and maintain the tone at the top culture with effective risk management, energizes segregation of duties and job rotation with mandatory annual leave policies for the purposes of rooting out financial statement fraud in Nigeria banks.

Beyond Statistical Significance: A Holistic View of What Makes a Research Finding "Important"

Students often believe that statistical significance is the only determinant of whether a quantitative result is “important.” In this paper, I review traditional null hypothesis statistical testing to identify what questions inferential statistics can and cannot answer, including statistical significance, effect size and direction, causality, generalizability, and changeability of the independent variable. I illustrate these issues with examples from an empirical study of the association between how much time teenagers spent playing video games and time spent reading. I describe how study design and context determine each of those aspects of “importance,” and close by summarizing how to provide a holistic view of importance when writing about a quantitative analysis. I also include exercises to guide students through applying these concepts to articles in newspapers and scholarly journals.

Ministry of Defence expenditures impact on economic growth of Ukraine

B a c k g r o u n d . With the start of the second stage of the Russian-Ukrainian war, the Government of Ukraine and its military command faced an objective examination of the results of years of efforts to develop Ukraine's national security and defence sector. At the same time as defence capability was being enhanced, economic performance was on the rise. This led us to test the hypothesis of whether the increased defence capability caused this growth. This study aims to clarify the relationship between military expenditures and economic growth in Ukraine. The object of the study is a consistent and long-term process of economic growth of the state along with an increase in the level of defence capability as a result of military expenditures in the context of financial support from the Ministry of Defence of Ukraine. Methods. Theoretical methods: analysis, synthesis, abstraction, generalisation, induction, deduction, explanation, classification. Empirical methods: hypothesis, observation, experiment, description. Results. This paper defines "military capabilities" and their relationship with military potential. It has been established that the ability to use military potential (military power/strength) is a military capability. Military expenditures are used to develop capabilities in the Ministry of Defence of Ukraine's system. Thus, allocating and redistributing expenditures for the army's needs is financial support. The armed forces' financial support drives the country's economic growth. To establish this fact, the leading indicators of military expenditures in the Ministry of Defence of Ukraine (from now on – MoD) were analysed and the indicator of economic growth – gross domestic product (from now – GDP) from 2012 to 2021. This period is crucial for Ukraine's national security, defence, and economic growth, as the basic principles of state defence policy were established during this time. This motivated us to analyse financial and defence indicators to determine their interrelationship. C o n c l u s i o n s . Two variables were defined as military expenditures and the GDP, where the first variable is independent and the second is dependent. A high correlation between military spending and economic growth in Ukraine was found by constructing a scatter plot, performing the relevant calculations, obtaining the data in an ANOVA table, setting up a regression line, and conducting null hypothesis tests.

Wildfire smoke exposure and emergency department visits for headache: A case-crossover analysis in California, 2006-2020.

To evaluate the association of short-term exposure to overall fine particulate matter of <2.5 μm (PM2.5 ) and wildfire-specific PM2.5 with emergency department (ED) visits for headache. Studies have reported associations between PM2.5 exposure and headache risk. As climate change drives longer and more intense wildfire seasons, wildfire PM2.5 may contribute to more frequent headaches. Our study included adult Californian members (aged ≥18 years) of a large de-identified commercial and Medicare Advantage claims database from 2006 to 2020. We identified ED visits for primary headache disorders (subtypes: tension-type headache, migraine headache, cluster headache, and "other" primary headache). Claims included member age, sex, and residential zip code. We linked daily overall and wildfire-specific PM2.5 to residential zip code and conducted a time-stratified case-crossover analysis considering 7-day average PM2.5 concentrations, first for primary headache disorders combined, and then by headache subtype. Among 9898 unique individuals we identified 13,623 ED encounters for primary headache disorders. Migraine was the most frequently diagnosed headache (N=5534/13,623 [47.6%]) followed by "other" primary headache (N=6489/13,623 [40.6%]). For all primary headache ED diagnoses, we observed an association of 7-day average wildfire PM2.5 (odds ratio [OR] 1.17, 95% confidence interval [CI] 0.95-1.44 per 10 μg/m3 increase) and by subtype we observed increased odds of ED visits associated with 7-day average wildfire PM2.5 for tension-type headache (OR 1.42, 95% CI 0.91-2.22), "other" primary headache (OR 1.40, 95% CI 0.96-2.05), and cluster headache (OR 1.29, 95% CI 0.71-2.35), although these findings were not statistically significant under traditional null hypothesis testing. Overall PM2.5 was associated with tension-type headache (OR 1.29, 95% CI 1.03-1.62), but not migraine, cluster, or "other" primary headaches. Although imprecise, these results suggest short-term wildfire PM2.5 exposure may be associated with ED visits for headache. Patients, healthcare providers, and systems may need to respond to increased headache-related healthcare needs in the wake of wildfires and on poor air quality days.

A systematic review and meta-analysis of the clinical performance of implant-supported overdentures retained by CAD-CAM bars.

To evaluate the performance of IODs involving CAD-CAM bars. A comprehensive search of studies published until May 2023 was conducted in many databases, including PubMed/MEDLINE, Web of Science, Cochrane Library, and SciELO, following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA). The population, intervention, comparison, outcome (PICO) question was: "How do IODs retained by bars fabricated by CAD-CAM technology perform in daily clinical practice?" The meta-analysis included clinical studies based on effect size and a two-tailed null test with a 95% confidence interval (CI). Ten studies were included in the meta-analysis. Among them, nine reported a 100% implant survival rate for all CAD-CAM milled bars. Complications were reported in two studies with CAD/CAM-milled titanium bars, and one study reported more fractures in soldered gold bars used in maxillary rehabilitation. However, no fractures were observed in IODs retained by PEEK and zirconia bars. According to six studies, biological complications, including peri-implantitis, were minimal in the BioHPP and PEEK bar groups, while no cases were reported in the titanium or zirconia bar groups. CAD-CAM-milled zirconia bars had higher plaque and bleeding indices compared with titanium bars, as evidenced by findings from five studies. All four studies that evaluated Oral Health Impact Profile (OHIP) scores showed a positive effect of IODs retained by CAD-CAM milled titanium bars on quality of life. Patient satisfaction and acceptance by prosthodontists were significantly high, according to the results of five studies. Overdentures retained with CAD-CAM milled titanium bars show great potential for use in daily clinical practice. Moreover, patient and practitioner satisfaction was very high when this method was used.

Clinical Investigation of Chemotherapeutic Resistance and miRNA Expressions in Head and Neck Cancers: A Thorough PRISMA Compliant Systematic Review and Comprehensive Meta-Analysis.

Background: Chemoresistance is a significant barrier to combating head and neck cancer, and decoding this resistance can widen the therapeutic application of such chemotherapeutic drugs. This systematic review and meta-analysis explores the influence of microRNA (miRNA) expressions on chemoresistance in head and neck cancers (HNC). The objective is to evaluate the theragnostic effects of microRNA expressions on chemoresistance in HNC patients and investigate the utility of miRNAs as biomarkers and avenues for new therapeutic targets. Methods: We performed a comprehensive bibliographic search that included the SCOPUS, PubMed, and Science Direct bibliographic databases. These searches conformed to a predefined set of search strategies. Following the PRISMA guidelines, inclusion and exclusion criteria were framed upon completing the literature search. The data items extracted were tabulated and collated in MS Excel. This spreadsheet was used to determine the effect size estimation for the theragnostic effects of miRNA expressions on chemoresistance in HNC, the hazard ratio (HR), and 95% confidence intervals (95% CI). The comprehensive meta-analysis was performed using the random effects model. Heterogeneity among the data collected was assessed using the Q test, Tau2, I2, and Z measures. Publication bias of the included studies was checked using the Egger's bias indicator test, Orwin and classic fail-safe N test, Begg and Mazumdar rank collection test, and Duval and Tweedie's trim and fill methods. Results: After collating the data from 23 studies, dysregulation of 34 miRNAs was observed in 2189 people. These data were gathered from 23 studies. Out of the 34 miRNAs considered, 22 were up-regulated, while 12 were down-regulated. The TaqMan transcription kits were the most used miRNA profiling platform, and miR-200c was seen to have a mixed dysregulation. We measured the overall pooled effect estimate of HR to be 1.516 for the various analyzed miRNA at a 95% confidence interval of 1.303-1.765, with a significant p-value. The null hypothesis test's Z value was 5.377, and the p-value was correspondingly noted to be less than 0.0001. This outcome indicates that the risk of death is determined to be higher in up-regulated groups than in down-regulated groups. Among the 34 miRNAs that were investigated, seven miRNAs were associated with an improved prognosis, especially with the overexpression of these seven miRNAs (miR15b-5p, miR-548b, miR-519d, miR-1278, miR-145, miR-200c, Hsa- miR139-3p). Discussion: The findings reveal that intricate relationships between miRNAs' expression and chemotherapeutic resistance in HNC are more likely to exist and can be potential therapeutic targets. This review suggests the involvement of specific miRNAs as predictors of chemoresistance and sensitivity in HNC. The examination of the current study results illustrates the significance of miRNA expression as a theragnostic biomarker in medical oncology.

Bayesian Performance Analysis for Algorithm Ranking Comparison

In the field of optimization and machine learning, the statistical assessment of results has played a key role in conducting algorithmic performance comparisons. Classically, null hypothesis statistical tests have been used. However, recently, alternatives based on Bayesian statistics have shown great potential in complex scenarios, especially when quantifying the uncertainty in the comparison. In this work, we delve deep into the Bayesian statistical assessment of experimental results by proposing a framework for the analysis of several algorithms on several problems/instances. To this end, experimental results are transformed to their corresponding rankings of algorithms, assuming that these rankings have been generated by a probability distribution (defined on permutation spaces). From the set of rankings, we estimate the posterior distribution of the parameters of the studied probability models, and several inferences concerning the analysis of the results are examined. Particularly, we study questions related to the probability of having one algorithm in the first position of the ranking or the probability that two algorithms are in the same relative position in the ranking. Not limited to that, the assumptions, strengths, and weaknesses of the models in each case are studied. To help other researchers to make use of this kind of analysis, we provide a Python package and source code implementation at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://zenodo.org/record/6320599</uri> .

Neural network reconstruction of H'(z) and its application in teleparallel gravity

In this work, we explore the possibility of using artificial neural networks to impose constraints on teleparallel gravity and its f(T) extensions. We use the available Hubble parameter observations from cosmic chronometers and baryon acoustic oscillations from different galaxy surveys. We discuss the procedure for training a network model to reconstruct the Hubble diagram. Further, we describe the procedure to obtain H'(z), the first order derivative of H(z), using artificial neural networks which is a novel approach to this method of reconstruction. These analyses are complemented with further studies on the impact of two priors which we put on H 0 to assess their impact on the analysis, which are the local measurements by the SH0ES team (H 0 R20 = 73.2 ± 1.3 km Mpc-1 s-1) and the updated TRGB calibration from the Carnegie Supernova Project (H 0 TRGB = 69.8 ± 1.9 km Mpc-1 s-1), respectively. Additionally, we investigate the validity of the concordance model, through some cosmological null tests with these reconstructed data sets. Finally, we reconstruct the allowed f(T) functions for different combinations of the observational Hubble data sets. Results show that the ΛCDM model lies comfortably included at the 1σ confidence level for all the examined cases.

Optical design of a null test for off-axis three-mirror system based on refractive-diffractive zoom hybrid compensator

In general, the reflectors of off-axis three-mirror system have large apertures and large differences in surface shape, which means it is difficult to use one set of null compensators to test all the aspheric mirrors in the off-axis three-mirror system because of the limitation of the dynamic range of the testing equipment. In this paper, we propose a surface shape test method that combines a refractive-zoom lens group and a Spatial Light Modulator (SLM) as a null compensator to test all the aspheric surfaces of one off-axis three-mirror system. The proposed method is verified by simulating the null test process of three aspheric mirrors of one off-axis three-mirror system, with the consideration of the fabrication and alignment errors. Simulation results show that the probability that residual wave aberration is within 0.1λ RMS is 50% for the primary mirror and 40% for the secondary mirror and 80% for the tertiary mirror.

Are P-values and Bayes factors valid measures of evidential strength?

P-values and Bayes factors are commonly used as measures of the evidential strength of the data collected in hypothesis tests. It is not clear, however, that they are valid measures of that evidential strength; that is, whether they have the properties that we intuitively expect a measure of evidential strength to have. I argue here that measures of evidential strength should be stochastically ordered by both the effect size and the sample size. I consider the case that the data are normally distributed and show that, for that case, P-values are valid measures of evidential strength while Bayes factors are not. Specifically, I show that in a sharp Null hypothesis test the Bayes factor is stochastically ordered by the sample size only if the effect size or the sample size is sufficiently large. This lack of stochastic ordering lies at the root of the Jeffreys-Lindley paradox.

Remarkably reproducible psychological (memory) phenomena in the classroom: some evidence for generality from small-N research

BackgroundMainstream psychology is experiencing a crisis of confidence. Many of the methodological solutions offered in response have focused largely on statistical alternatives to null hypothesis statistical testing, ignoring nonstatistical remedies that are readily available within psychology; namely, use of small-N designs. In fact, many classic memory studies that have passed the test of replicability used them. That methodological legacy warranted a retrospective look at nonexperimental data to explore the generality of the reported effects.MethodVarious classroom demonstrations were conducted over multiple semesters in introductory psychology courses with typical, mostly freshman students from a predominantly white private Catholic university in the US Midwest based on classic memory experiments on immediate memory span, chunking, and depth of processing.ResultsStudents tended to remember 7 ± 2 digits, remembered more digits of π following an attached meaningful story, and remembered more words after elaborative rehearsal than after maintenance rehearsal. These results amount to replications under uncontrolled classroom environments of the classic experiments originally conducted largely outside of null hypothesis statistical testing frameworks.ConclusionsIn light of the ongoing replication crisis in psychology, the results are remarkable and noteworthy, validating these historically important psychological findings. They are testament to the reliability of reproducible effects as the hallmark of empirical findings in science and suggest an alternative approach to commonly proffered solutions to the replication crisis.

Bound orbits around charged black strings

We study the geodesics of 5d Reissner–Nordstrom and nonsingular black strings, and establish a rational bound orbit taxonomy for both massive as well as null test particles. For the timelike case, test particles with high energy (that would have made them plunge into or scatter off a black hole) could still form bound orbits around the black strings. We calculate the accumulated angles of the corresponding radial periods and show that they are higher than their 4d counterparts. For the null case, we found the existence of stable null orbits outside their respective horizons, which do not exist in the four dimensions except at their extremal limit.

Implementation of a null test for freeform optics using a high-definition spatial light modulator.

We report the implementation of an interferometric null test using a high-definition spatial light modulator (SLM) as a reconfigurable alternative to a computer-generated hologram. We detail the alignment process chain, including novel techniques using the SLM to project alignment fiducials on the test part. To validate the alignment protocol, we measure a mild off-axis conic with the SLM-based system and cross-validate with conventional interferometry within 30 nm root-mean-square (RMS) surface figure. Finally, we report the null test of a 65 mm clear aperture concave freeform with 91 μm peak-valley sag departure from the base sphere. The measured surface figure of the freeform is within 40 nm RMS compared to the measurement with a commercial metrology instrument.

ORMA: A strategy to reduce Psychology's replication problems

Because of their historical reliance upon null hypothesis statistical tests (NHST), the human sciences have developed a number of potentially problematic research literatures. While aware of the file drawer effect since the 1970s, scientists have been largely unsuccessful at addressing its pernicious effects. Because significant results have a greater likelihood of being published than do nonsignificant effects, many of our research literatures might currently be constructed upon a series of Type I errors and inflated effect sizes. A method (called Original Replication of Meta-Analyses or ORMA) has recently been developed for identifying problematic research literatures and offering a method to address the problems due to publication bias. Philosophers of science have long argued that a chief reason for science's preeminence as a source of knowledge rests in its ability to self-correct. Researchers in the human sciences are now able to empirically test their research literatures to ascertain which are in need of repair. The use of ORMA serves to lessen the problems that led to the recent calls for bans on significant/nonsignificant statistics in human science research. ORMA will also improve psychology's ability to successfully replicate its research findings.

Modeling and suppressing the wavefront degeneration in a CGH interferometric null test.

The computer-generated hologram (CGH) enables the ultra-high accuracy of surface measurement but causes the wavefront degeneration in the optical system. In this article, we give a high-accuracy analytical simulation of the wavefront degeneration in null test by the elliptical Gaussian model. We propose an analytical expression of instrumental transfer function (ITF) for the CGH null test without knowing the phase distribution of CGH, which gives an efficient instruction to suppress the wavefront degeneration. The ITF of the interferometric null test for a ∅3m aspheric mirror can be optimized from 0 to 0.65 at 0.4 Nyquist frequency.

How to Evaluate Theory-Based Hypotheses in Meta-Analysis Using an AIC-Type Criterion.

Meta-analysis techniques allow researchers to aggregate effect sizes-like standardized mean difference(s), correlation(s), or odds ratio(s)-of different studies. This leads to overall effect-size estimates and their confidence intervals. Additionally, researchers can aim for theory development or theory evaluation. That is, researchers may not only be interested in these overall estimates but also in a specific ordering or size of them, which then reflects a theory. Researchers may have expectations regarding the ordering of standardized mean differences or about the (ranges of) sizes of an odds ratio or Hedges' g. Such theory-based hypotheses most probably contain inequality constraints and can be evaluated with the Akaike's information criterion type (i.e., AIC-type) confirmatory model selection criterion called generalized order-restricted information criterion (GORICA). This paper introduces and illustrates how the GORICA can be applied to meta-analyzed estimates. Additionally, it compares the use of the GORICA to that of classical null hypothesis testing and the AIC, that is, the use of theory-based hypotheses versus null hypotheses. By using the GORICA, researchers from all types of fields (e.g., psychology, sociology, political science, biomedical science, and medicine) can quantify the support for theory-based hypotheses specified a priori. This leads to increased statistical power, because of (i) the use of theory-based hypotheses (cf. one-sided vs. two-sided testing) and (ii) the use of meta-analyzed results (that are based on multiple studies which increase the combined sample size). The quantification of support and the power increase aid in, for instance, evaluating and developing theories and, therewith, developing evidence-based treatments and policy.