Abstract
The rapid growth of point-of-care polysomnographic alternatives has necessitated standardized evaluation and validation frameworks. The current average across participant validation methods may overestimate the agreement between wearable sleep tracker devices and polysomnography (PSG) systems because of the high base rate of sleep during the night and the interindividual difference across the sampling population. This study proposes an evaluation framework to assess the aggregating differences of the sleep architecture features and the chronologically epoch-by-epoch mismatch of the wearable sleep tracker devices and the PSG ground truth. An AASM-based sleep stage categorizing method was proposed to standardize the sleep stages scored by different types of wearable trackers. Sleep features and sleep stage architecture were extracted from the PSG and the wearable device’s hypnograms. Therefrom, a localized quantifier index was developed to characterize the local mismatch of sleep scoring. We evaluated different commonly used wearable sleep tracking devices with the data collected from 22 different subjects over 30 nights of 8-h sleeping. The proposed localization quantifiers can characterize the chronologically localized mismatches over the sleeping time. The outperformance of the proposed method over existing evaluation methods was reported. The proposed evaluation method can be utilized for the improvement of the sensor design and scoring algorithm.
Highlights
The emerging trend in transforming point-of-care sleep tracking devices to polysomnographic alternatives has necessitated methods to evaluate and validate sleep monitoring functions
We explain the results of the aggregating sleep features and the localized mismatch quantifier of each sleep tracker, and the comparison among the four sleep trackers
Our results from the proposed framework confirmed the findings from previous studies that, compared to the PSG ground truth, wearable sleep tracker devices effectively detect sleep onset and sleep period time, but are deficient in estimating N1, N2, N3, and REM stages [31]
Summary
The emerging trend in transforming point-of-care sleep tracking devices to polysomnographic alternatives has necessitated methods to evaluate and validate sleep monitoring functions. An assessment method to compare the detailed distribution of sleep stages and quantify the transition among sleep stages between wearable sleep tracking devices and PSG is critical. The first group of methods utilized Tryon’s approach to compare the sensitivity, specificity, and overall accuracy of the correct sleep staging [18] generated by the wearable devices and PSG system. The third group of methods evaluates the overall epoch-by-epoch correlations using the confusion matrix of Pearson correlations [18,19,22], Cohen’s kappa, and Fleiss’ kappa [23] Those comparative methods provide only overall sleep staging comparisons without quantifying the transition mismatch among sleep stages and the monotone correlations among sleep episodes. The validation of the proposed framework to evaluate four different wearable devices with sleep tracking functions is described in the Results Section. A discussion of the results and conclusions is presented in the last section
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Free) 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
