[1]CHEN Lingling,BI Xiaojun.Sleep staging model based on multimodal fusion[J].CAAI Transactions on Intelligent Systems,2022,17(6):1194-1200.[doi:10.11992/tis.202202018]
Copy

Sleep staging model based on multimodal fusion

CAAI Transactions on Intelligent Systems[ISSN 1673-4785/CN 23-1538/TP] Volume: 17 Number of periods: 2022 6 Page number: 1194-1200 Column: 学术论文—智能系统 Public date: 2022-11-05
Title:
Sleep staging model based on multimodal fusion
Author(s):
CHEN Lingling1;  BI Xiaojun2
1. College of Information and Communication Engineering, Harbin Engineering University, Harbin 150001, China;
2. School of Information Engineering, Minzu University of China, Beijing 100081, China
Keywords:
deep learning; sleep stage; channel attention mechanism; polysomnogram; multimodal; bidirectional long short-term memory; sleep-European data format dataset; residual shrinkage network
CLC:
TP18; TH79
DOI:
10.11992/tis.202202018
Abstract:
To solve the problem of insufficient feature use due to the differences in modal information in sleep polygraphs at different stages, this paper proposes a sleep staging method based on the channel attention mechanism and multimodal gating module. First, a residual shrinkage network is used to design each modal feature extraction network to extract the features of all modules, which are spliced and fused in the channel dimension, and the fused features are further recalibrated using the channel attention mechanism to obtain time-invariant polysomnography features. Then, a multimodal gating module based on an adaptive gating mechanism is proposed, which weighs and fuses all modal and time-invariant features according to their importance, thereby realizing the organic fusion of features. Finally, the time sequence characteristics of polysomnography are obtained using a bidirectional long short-term memory network. The experimental result shows that the accuracy of the sleep staging model proposed in this paper is 87.6% on the Sleep-EDF dataset, with an $M_{F_1} $ of 82.0%, thus achieving the best staging effect to date.
References:
[1] BERRY R B, BUDHIRAJA R, GOTTLIEB D J, et al. Rules for scoring respiratory events in sleep: update of the 2007 AASM manual for the scoring of sleep and associated events[J]. Journal of clinical sleep medicine, 2012, 8(5): 597–619.
[2] 王菡侨. 有关美国睡眠医学学会睡眠分期的最新判读标准指南解析[J]. 诊断学理论与实践, 2009, 8(6): 575–578
WANG Hanqiao. Analysis of the latest guidelines for sleep staging of the american academy of sleep medicine[J]. Journal of diagnostics concepts & practice, 2009, 8(6): 575–578
[3] ZHANG Linda, FABBRI D, UPENDER R, et al. Automated sleep stage scoring of the sleep heart health study using deep neural networks[J]. Sleep, 2019, 42(11): zsz159.
[4] ZHAO Dechun, WANG Yi, WANG Qiangqiang, et al. Comparative analysis of different characteristics of automatic sleep stages[J]. Computer methods and programs in biomedicine, 2019, 175: 53–72.
[5] FAUST O, RAZAGHI H, BARIKA R, et al. A review of automated sleep stage scoring based on physiological signals for the new millennia[J]. Computer methods and programs in biomedicine, 2019, 176: 81–91.
[6] VILAMALA A, MADSEN K H, HANSEN L K. Deep convolutional neural networks for interpretable analysis of EEG sleep stage scoring[C]//2017 IEEE 27th International Workshop on Machine Learning for Signal Processing. Tokyo: IEEE, 2017: 1-6.
[7] HUMAYUN A I, SUSHMIT A S, HASAN T, et al. End-to-end sleep staging with raw single channel EEG using deep residual ConvNets[C]//2019 IEEE EMBS International Conference on Biomedical & Health Informatics. Chicago: IEEE, 2019: 1-5.
[8] 贾子钰, 林友芳, 张宏钧, 等. 基于深度卷积神经网络的睡眠分期模型[J]. 浙江大学学报(工学版), 2020, 54(10): 1899–1905
JIA Ziyu, LIN Youfang, ZHANG Hongjun, et al. Sleep stage classification model based on deep convolutional neural network[J]. Journal of Zhejiang University (engineering science edition), 2020, 54(10): 1899–1905
[9] FERNANDEZ-BLANCO E, RIVERO D, PAZOS A. EEG signal processing with separable convolutional neural network for automatic scoring of sleeping stage[J]. Neurocomputing, 2020, 410: 220–228.
[10] PHAN H, ANDREOTTI F, COORAY N, et al. Joint classification and prediction CNN framework for automatic sleep stage classification[J]. IEEE transactions on biomedical engineering, 2019, 66(5): 1285–1296.
[11] JADHAV P, RAJGURU G, DATTA D, et al. Automatic sleep stage classification using time-frequency images of CWT and transfer learning using convolution neural network[J]. Biocybernetics and biomedical engineering, 2020, 40(1): 494–504.
[12] YüCELBA? C, YüCELBA? ?, ?Z?EN S, et al. Effect of the Hilbert-Huang transform method on sleep staging[C]//2017 25th Signal Processing and Communications Applications Conference (SIU). Antalya: IEEE, 2017: 1-4.
[13] LOH H W, OOI C P, VICNESH J, et al. Automated detection of sleep stages using deep learning techniques: a systematic review of the last decade (2010–2020)[J]. Applied sciences, 2020, 10(24): 8963.
[14] SOKOLOVSKY M, GUERRERO F, PAISARNSRISOMSUK S, et al. Deep learning for automated feature discovery and classification of sleep stages[J]. IEEE/ACM transactions on computational biology and bioinformatics, 2020, 17(6): 1835–1845.
[15] DUAN Lijuan, LI Mengying, WANG Changming, et al. A novel sleep staging network based on data adaptation and multimodal fusion[J]. Frontiers in human neuroscience, 2021, 15: 727139.
[16] ZHAO Minghang, ZHONG Shisheng, FU Xuyun, et al. Deep residual shrinkage networks for fault diagnosis[J]. IEEE transactions on industrial informatics, 2020, 16(7): 4681–4690.
[17] HU Jie, SHEN Li, SUN Gang. Squeeze-and-excitation networks[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 7132-7141.
[18] 张峰, 李希城, 董春茹, 等. 基于深度情感唤醒网络的多模态情感分析与情绪识别[J]. 控制与决策, 2022, 37(11): 2984–2992
ZHANG Feng, LI Xicheng, DONG Chunru, et al. Deep emotional arousal network for multimodal sentiment analysis and emotion recognition[J]. Control and decision, 2022, 37(11): 2984–2992
[19] WANG Yansen, SHEN Ying, LIU Zhun, et al. Words can shift: dynamically adjusting word representations using nonverbal behaviors[C]//Proceedings of the AAAI Conference on Artificial Intelligence. Honolulu: Assoc Advancement Artifical Intelligence, 2019: 7216-7223.
[20] ABDELHAMEED A M, DAOUD H G, BAYOUMI M. Deep convolutional bidirectional LSTM recurrent neural network for epileptic seizure detection[C]//2018 16th IEEE International New Circuits and Systems Conference. Montreal: IEEE, 2018: 139-143.
[21] KEMP B, ZWINDERMAN A H, TUK B, et al. Analysis of a sleep-dependent neuronal feedback loop: the slow-wave microcontinuity of the EEG[J]. IEEE transactions on bio-medical engineering, 2000, 47(9): 1185–1194.
[22] GOLDBERGER A L, AMARAL L A, GLASS L, et al. PhysioBank, physiotoolkit, and physionet: components of a new research resource for complex physiologic signals[J]. Circulation, 2000, 101(23): E215–E220.
[23] ZHANG Lei, CHEN Dan, CHEN Peilu, et al. Dual-CNN based multi-modal sleep scoring with temporal correlation driven fine-tuning[J]. Neurocomputing, 2021, 420: 317–328.
[24] PHAN H, CHéN O Y, KOCH P, et al. Towards more accurate automatic sleep staging via deep transfer learning[J]. IEEE transactions on biomedical engineering, 2021, 68(6): 1787–1798.
[25] YAN Rui, LI Fan, ZHOU Dong dong, et al. Automatic sleep scoring: a deep learning architecture for multi-modality time series[J]. Journal of neuroscience methods, 2021, 348: 108971.
Similar References:

Memo

-

Last Update: 1900-01-01

Copyright © CAAI Transactions on Intelligent Systems