A Novel Sleep Staging Network Based on Data Adaptation and Multimodal Fusion.

Lijuan Duan,Changming Wang,Mengying Li,Yuanhua Qiao,Mingai Li,Sha Sha,Zeyu Wang

doi:10.3389/fnhum.2021.727139

Abstract

Sleep staging is one of the important methods to diagnosis and treatment of sleep diseases. However, it is laborious and time-consuming, therefore, computer assisted sleep staging is necessary. Most of the existing sleep staging researches using hand-engineered features rely on prior knowledges of sleep analysis, and usually single channel electroencephalogram (EEG) is used for sleep staging task. Prior knowledge is not always available, and single channel EEG signal cannot fully represent the patient’s sleeping physiological states. To tackle the above two problems, we propose an automatic sleep staging network model based on data adaptation and multimodal feature fusion using EEG and electrooculogram (EOG) signals. 3D-CNN is used to extract the time-frequency features of EEG at different time scales, and LSTM is used to learn the frequency evolution of EOG. The nonlinear relationship between the High-layer features of EEG and EOG is fitted by deep probabilistic network. Experiments on SLEEP-EDF and a private dataset show that the proposed model achieves state-of-the-art performance. Moreover, the prediction result is in accordance with that from the expert diagnosis.

Highlights

SLEEP is an important physiological requirement of human beings, which is essential for human health
We present a deep learning fusion network framework denoted as Multi Sensor Deep Fusion Network (MSDFN) for feature extraction, multimodal fusion and sleep stage classification
This paper proposes an automatic sleep stage classification framework based on HHT and deep multimodal feature fusion network, with the function of data adaptation and multimodal feature fusion

Summary

A Novel Sleep Staging Network Based on Data Adaptation and Multimodal Fusion

Lijuan Duan1,2,3* , Mengying Li1,2,3, Changming Wang4,5* , Yuanhua Qiao, Zeyu Wang, Sha Sha and Mingai Li8. Sleep staging is one of the important methods to diagnosis and treatment of sleep diseases. It is laborious and time-consuming, computer assisted sleep staging is necessary. Most of the existing sleep staging researches using handengineered features rely on prior knowledges of sleep analysis, and usually single channel electroencephalogram (EEG) is used for sleep staging task. Prior knowledge is not always available, and single channel EEG signal cannot fully represent the patient’s sleeping physiological states. To tackle the above two problems, we propose an automatic sleep staging network model based on data adaptation and multimodal feature fusion using EEG and electrooculogram (EOG) signals. Experiments on SLEEP-EDF and a private dataset show that the proposed model achieves state-ofthe-art performance.

INTRODUCTION

Evaluation

Method

20 SC 20 SC 20 SC 20 SC 20SC 20SC

DISCUSSION AND CONCLUSION

ETHICS STATEMENT