Identification of unreliable subject data from remote testing

Abstract

The less controlled nature of conducting behavioral testing outside of the typical sound booth and not under the direct observation of the experimenter introduces additional factors that may lead to unreliable data. These factors arise from the environment (e.g., room acoustics and background noise), the participant (e.g., attention, engagement, and understanding of the task), and the testing apparatus (e.g., calibration and network latency). Remote-testing protocols should include features to allow for the detection of unreliable data. To aid in the detection of unreliable data in future remote-testing protocols, we retrospectively examined a large dataset (N > 1000) of speech intelligibility and tone-in-noise detection that was collected over headphones with a tablet-based system in a waiting room environment. The dataset provides the ability to determine the extent to which unreliable data can be detected based on the presence of high levels of background noise during the testing, abnormally short response times, high lapse rates, flat psychometric functions, and patterns in the response sequence. [The views expressed in this abstract are those of the authors and do not reflect the official policy of the Department of Army/Navy/Air Force, Department of Defense, or U.S. Government.]