Abstract
Accurate and efficient detection of sleep spindles is a methodological challenge. The present study describes a method of using non-experts for manual detection of sleep spindles. We recruited five experts and 168 non-experts to manually identify spindles in stage N2 and stage N3 sleep data using a MATLAB interface. Scorers classified each spindle into definite and indefinite spindle (with weights of 1 and 0.5, respectively). First, a method of optimizing the thresholds of the expert/non-expert group consensus according to the results of experts and non-experts themselves is described. Using this method, we established expert and non-expert group standards from expert and non-expert scorers, respectively, and evaluated the performance of the non-expert group standards by compared with the expert group standard (termed EGS). The results indicated that the highest performance was the non-expert group standard when definite spindles were only considered (termed nEGS-1; F1 score = 0.78 for N2; 0.68 for N3). Second, four automatic spindle detection methods were compared with the EGS. We found that the performance of nEGS-1 versus EGS was higher than that of the four automated methods. Our results also showed positive correlation between the mean F1 score of individual expert in EGS and the F1 score of nEGS-1 versus EGS across 30 segments of stage N2 data (r = 0.61, P < 0.001). Further, we found that six and nine non-experts were needed to manually identify spindles in stages N2 and N3, respectively, while maintaining acceptable performance of nEGS-1 versus EGS (F1 score = 0.79 for N2; 0.64 for N3). In conclusion, this study establishes a detailed process for detection of sleep spindles by non-experts in a crowdsourcing scheme.
Highlights
Sleep spindles are characterized by a waxing and waning shape, within the sigma frequency range (11~16 Hz) with a minimum duration of 0.5s, and they are a key hallmark of stage 2 (N2) non-rapid eye movement (NREM) sleep [1]
We found that when definite spindles were only considered, the non-expert group standard showed acceptable agreement with the EGS
Because the nEGS-1 had been identified as the final non-expert group standard according to the above results, we only considered the performance of nEGS-1 when determining the minimum number of nonexperts for reliable spindle identification
Summary
Sleep spindles are characterized by a waxing and waning shape, within the sigma frequency range (11~16 Hz) with a minimum duration of 0.5s, and they are a key hallmark of stage 2 (N2) non-rapid eye movement (NREM) sleep [1]. The roles of sleep spindles in basic and clinical sleep research are well documented recently. Much evidence indicates that the functional significance of sleep spindles has been implicated in intelligence [2, 3] and sleep-dependent memory consolidation [4, 5], which suggests that sleep spindles may be considered both as a physiological index of intellectual abilities and a marker of the capacity for learning [6, 7]. Sleep spindle identification by non-experts design, data collection and analysis, decision to publish, or preparation of the manuscript
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Free) 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
