Test-Retest Reliability of a Visual-Cognitive Technology (BlazePod™) to Measure Response Time

Abstract

A new technology (BlazePod™) that measures response time (RT) is currently on the market and has been used by strength and conditioning professionals. Nevertheless, to trust in the measurement, before the use of a new device to measure any outcome in the research or clinical setting, a reliability analysis of its measurement must be established (Koo and Li, 2016). Hence, we assessed the test-retest reliability (repeatability) of the BlazePod™ (Play Coyotta Ltd., Aviv, Israel) technology during a pre-defined activity to provide information about the level of agreement and the magnitude of errors incurred when using the technology. This information can assist practitioners and researchers in the use of BlazePod™ technology. We recruited 24 physically active young adults (age = 23.9 ± 4.0 years; height = 1.67 ± 0.09 m; body mass = 68.2 ± 13.1 kg), who were free of injuries, and any orthopedic, or cardiorespiratory diseases. Participants reported to the laboratory on two occasions, separated by one week. One week before, participants performed a familiarization session with the instrument. During the first session, the one-leg balance activity (OLBA) was performed. This activity was chosen randomly among all BlazePod™ pre-defined activities. We conducted all sessions in a physiology laboratory at the same time for each participant and under similar environmental conditions (~23° C; ~60% humidity). The OLBA consisted of a unipedal balance activity performed with four pods arranged in a square on the floor. Participants stood up in the center of the square, and the OLBA aim was to tap out as many lights as possible with the dominant foot during 30 seconds. The system lighted up in a random order not known by the participants neither the researchers. The distance between the Pods was the individual lower limb length. Three trials were performed. The best value obtained was recorded. A one-minute rest interval between all trials was given. The total number of taps and average RT of all taps in the OLBA were recorded for further analysis. Data are presented as mean ± SD or 95% confidence interval (CI). We confirmed the normal data distribution using the Shapiro-Wilk test. A paired t-test, Cohen’s d effect size (ES) and its 95% CI were calculated to assess the magnitude of the mean difference between sessions. The interpretation of the ES was: trivial (<0.20), small (0.20-0.59), moderate (0.60-1.19), large (1.2-2.0) and very large (>2.0) effect (Hopkins et al., 2009). The intraclass correlation coefficient (ICC) and its 95% CI was used to assess the reliability based on a single measurement, absolute-agreement, two-way mixed-effects model. The ICC value was interpreted as follows: poor (<0.5), moderate (0.5-0.75), good (0.75-0.9), and excellent (>0.9) reliability (Koo and Li, 2016). We also calculated the standard error of measurement (SEM), the coefficient of variation (CV), the smallest detectable change (SDC), the level of agreement between sessions by a Bland-Altman plot, the systematic bias, and its 95% limits of agreement (LoA = bias ± 1.96 SD) (Bland and Altman, 1986). We observed a small to moderate increase between sessions for the number of taps (Day 1 = 20 ± 3 taps, Day 2 = 22 ± 4 taps; t(23) = -4.121; p < 0.001; ES = 0.55, 95% CI = 0.43 to 0.67) and a trivial to small decrease for the RT (Day 1 = 1418 ± 193 ms, Day 2 = 1358 ± 248 ms; t(23) = 1.721; p = 0.099; ES = -0.27, 95% CI = -0.15 to -0.38 CI). All reliability indexes for both outcome measures are shown in Table 1. Moderate to excellent levels of reliability were found by the ICC (95% CI) values and acceptable reliability by the CV for both measures. Bland-Altman plots are depicted in Figure 1. The systematic bias that we found showed that on average in the second day, participants achieved two taps more than the first day and were 59 ms faster than the first day. The LoA showed that the number of taps measured in the first day might be 7 units below or 3 units above Day 2. Besides, the RT measured in Day 1 might be 272 ms below or 391 ms above Day 2. In conclusion, the BlazePod™ technology provides reliable information during its OLBA in physically active young adults. We considered the measurement error as acceptable for practical use since low systematic biases and errors of measurement were reported in this study, besides a moderate ICC and excellent CV. These results suggest that practitioners can use the information provided by the BlazePod™ technology to monitor performance changes during cognitive training and to evaluate the effects of a training intervention.