Abstract
A combination of confirmation bias, hindsight bias, and pressure to publish may prompt the (unconscious) exploration of various methodological options and reporting only the ones that lead to a (statistically) significant outcome. This undisclosed analytic flexibility is particularly relevant in EEG research, where a myriad of preprocessing and analysis pipelines can be used to extract information from complex multidimensional data. One solution to limit confirmation and hindsight bias by disclosing analytic choices is preregistration: researchers write a time-stamped, publicly accessible research plan with hypotheses, data collection plan, and the intended preprocessing and statistical analyses before the start of a research project. In this manuscript, we present an overview of the problems associated with undisclosed analytic flexibility, discuss why and how EEG researchers would benefit from adopting preregistration, provide guidelines and examples on how to preregister data preprocessing and analysis steps in typical ERP studies, and conclude by discussing possibilities and limitations of this open science practice.
Highlights
Over the last decade, findings from a number of research disciplines have been under careful scrutiny
We present an overview of the problems associated with undisclosed analytic flexibility, discuss why and how EEG researchers would benefit from adopting preregistration, provide guidelines and examples on how to preregister data pre processing and analysis steps in typical event-related potentials (ERPs) studies, and conclude by discussing possibilities and limitations of this open science practice
A preregistration is specific when it includes a detailed description of all phases of the research workflow, from the initial design of the study to the information reported in the final manuscript; precise when the research plan is interpretable in only one way; and exhaustive when the research plan states that only the mentioned analyses will be considered as diagnostic to confirm or falsify predictions, thereby clar ifying that other analyses have been conducted after seeing the data
Summary
Findings from a number of research disciplines have been under careful scrutiny. The rationale behind the selection of data preprocessing and analysis pipelines – e.g., the selection of cut-off values when identifying outliers or the choice of a particular statistical technique3 – is not properly described This is not necessarily due to blind following of “statistical rituals” (Gigerenzer, 2004), because there may very well be multiple reasonable processing steps that can be applied to the same dataset (Steegen et al, 2016). We describe how undisclosed analytic flexibility may influence the interpretation of results in human electrophysiology research
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Free) 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
