A comparison of probabilistic classifiers for sleep stage classification

Pedro Fonseca,Xi Long,Niek Den Teuling,Ronald M Aarts

doi:10.1088/1361-6579/aabbc2

Abstract

Objective: To compare conditional random fields (CRF), hidden Markov models (HMMs) and Bayesian linear discriminants (LDs) for cardiorespiratory sleep stage classification on a five-class sleep staging task (wake/N1/N2/N3/REM), to explore the benefits of incorporating time information in the classification and to evaluate the feasibility of sleep staging on obstructive sleep apnea (OSA) patients. Approach: The classifiers with and without time information were evaluated with 10-fold cross-validation on five-, four- (wake/N1 + N2/N3/REM) and three-class (wake/NREM/REM) classification tasks using a data set comprising 443 night-time polysomnography (PSG) recordings of 231 participants (180 healthy participants, 100 of which had a ‘regular’ sleep architecture, and 51 participants previously diagnosed with OSA). Main results: CRF with time information (CRFt) outperforms all other classifiers on all tasks, achieving a median accuracy and Cohen’s κ for all participants of 62.8% and 0.44 for five classes, 68.8% and 0.47 for four classes, and 77.6% and 0.55 for three classes. An advantage was found in training classifiers, specifically for ‘regular’ and ‘OSA’ participants, achieving an improvement in classification performance for these groups. For ‘regular’ participants, CRFt achieved a median accuracy and Cohen’s κ of 67.0% and 0.51, 70.8% and 0.53 and 81.3% and 0.62 for five-, four- and three-classes respectively, and for ‘OSA’ patients, of 59.9% and 0.40, 69.7% and 0.45, and 75.8% and 0.51 for five-, four- and three-classes respectively. Significance: The results suggest that CRFt is not only better at learning and predicting more complex and irregular sleep architectures, but that it also performs reasonably well in five-class classification—the standard for sleep scoring used in clinical PSG. Additionally, and albeit with a decrease in performance when compared with healthy participants, sleep stage classification in OSA patients using cardiorespiratory features and CRFt seems feasible with reasonable accuracy.

Highlights

Despite substantial advances over the last few decades in simpler and less obtrusive techniques for sleep monitoring, overnight polysomnography (PSG) remains the gold standard for sleep assessment and diagnosis of sleep disorders
For the three-class classification task, CRF with time information (CRFt) performed significantly better than the other classifiers, achieving an average κ of 0.59 ± 0.15 and accuracy of 80.67 ± 6.93% in the ‘regular’ data set, a κ of 0.50 ± 0.13 and accuracy of 75.40 ± 6.83%, for ‘obstructive sleep apnea (OSA)’, and a κ of 0.54 ± 0.14 and accuracy of 76.70 ± 7.43% for ‘all’
The performance of a conditional random fields (CRF) classifier was evaluated in a home sleep monitoring context with cardiorespiratory features, and compared to hidden Markov models (HMMs) and linear discriminants (LDs) classifiers

Summary

Introduction

Despite substantial advances over the last few decades in simpler and less obtrusive techniques for sleep monitoring, overnight polysomnography (PSG) remains the gold standard for sleep assessment and diagnosis of sleep disorders. The only physiological characteristics that can be conveniently and reliably monitored are body movements, for example by means of actigraphy (Ancoli-Israel et al 2003), cardiac activity, for instance with electrocardiography (ECG), photoplethysmography (PPG) (Yu et al 2006, Beattie et al 2017), or ballistocardiography (BCG) (Mack et al 2009), and surrogates of respiratory effort based on respiratory movements measured with respiratory inductance plethysmography (RIP) belts around the thorax, BCG, or even Doppler radars (de Chazal et al 2011) These modalities can be comfortably used in a home setting, and even set up by the subjects themselves, the relation between these physiological characteristics and sleep stages cannot be trivially annotated using visual rules, as is done with standard PSG. Exploiting the varying autonomic characteristics during different stages of sleep, algorithms have been described which automate this process, typically using cardiac characteristics computed from heart rate variability and respiratory features from respiratory effort to classify epochs in states of ‘wake’, ‘light sleep’ (typically combining N1 and N2), ‘deep sleep’ or N3, and ‘rapid eye movement’ (REM) (Willemen et al 2014, Fonseca et al 2015, Beattie et al 2017)

Methods

Results

Discussion

Conclusion