Evaluating Step Counting Algorithms on Subsecond Wrist-Worn Accelerometry: A Comparison Using Publicly Available Data Sets

Abstract

Background: Walking-based metrics, including step count and total time walking, are easily interpretable measures of physical activity. Algorithms can estimate steps from accelerometry, which increasingly is measured with accelerometers located on the wrist. However, many existing step counting algorithms have not been validated in free-living settings, exhibit high error rates, or cannot be used without proprietary software. We compare the performance of several existing open-source step counting algorithms on three publicly available data sets, including one with free-living data. Methods: We applied five open-source algorithms: Adaptive Empirical Pattern Transformation, Oak, Step Detection Threshold, Verisense, and stepcount, and one proprietary algorithm (ActiLife) to three publicly available data sets with ground truth step counts: Clemson Ped-Eval, Movement Analysis in Real-World Environments Using Accelerometers, and OxWalk. We evaluate F1 score, precision, recall, mean absolute percent error (MAPE), and mean bias for each algorithm and setting. Results: The machine learning-based stepcount algorithm exhibited the highest F1 score (0.89 ± 0.11) and lowest MAPE (8.6 ± 9%) across all data sets and had the best, or comparable, F1 scores and MAPE in each individual data set. All algorithms performed worse with respect to both F1 score and MAPE in free-living compared with regular walking scenarios, and stepcount and Verisense were most sensitive to sampling frequency of input data. Conclusion: Machine learning-based algorithms, including stepcount, are a promising avenue for step counting. More free-living accelerometry data sets with ground truth step counts are needed for testing, validation, and continued refinement of algorithms.