Excess success in articles on object-based attention

Gregory Francis,Evelina Thunell

doi:10.3758/s13414-022-02459-6

Abstract

Twenty-five years of research has explored the object-based attention effect using the two-rectangles paradigm and closely related paradigms. While reading this literature, we noticed statistical attributes that are sometimes related to questionable research practices, which can undermine the reported conclusions. To quantify these attributes, we applied the Test for Excess Success (TES) individually to 37 articles that investigate various properties of object-based attention and comprise four or more experiments. A TES analysis estimates the probability that a direct replication of the experiments in a given article with the same sample sizes would have the same success (or better) as the original article. If the probability is low, then readers should be skeptical about the conclusions that are based on those experimental results. We find that 19 of the 37 analyzed articles (51%) seem too good to be true in that they have a replication probability below 0.1. In a new large sample study, we do find evidence for the basic object-based attention effect in the two-rectangles paradigm, which this literature builds on. A power analysis using this data shows that commonly used sample sizes in studies that investigate properties of object-based attention with the two-rectangles paradigm are, in fact, much too small to reliably detect even the basic effect.

Highlights

A neutral condition typically produces a response time between those for the cued and uncued conditions. These results suggest that attentional resources are guided to aid processing at the cued location, and that it takes time to redirect attention from the cued side of the visual field to the uncued side
Visual perception is often concerned with objects rather than with a certain spatial location, and many researchers suspected that attentional processing could have an object-based component
Experiments 5, 6, and 7 explored various stimulus and task manipulations to compare the findings reported in Experiments 2–4 against previous reports in the literature

Summary

Introduction

It is well known that people can process visual information even without directly looking at a stimulus. We seem to be able to focus processing resources to a certain spatial area, with stimuli outside of this focus taking longer to detect and identify (e.g., Posner, 1980). Such spatial attention effects are well established, and various paradigms have investigated their temporal and spatial properties and limitations. Et al (1994) reported empirical evidence for such object-based attention in a two-rectangles paradigm, schematized, which has since become the most commonly used paradigm in object-based attention research (Chen, 2012). Consistent with spatial cuing effects, response times (e.g., for identifying a subsequent letter as “T” or “L”) are fastest for valid trials, i.e., when the letter appears

Methods

Results

Conclusion