多模态融合网络的睡眠分期研究

PDF下载HTML

Title:

Sleep staging model based on multimodal fusion

作者:

陈玲玲¹, 毕晓君²

1. 哈尔滨工程大学 信息与通信工程学院，黑龙江 哈尔滨 150001;
2. 中央民族大学 信息工程学院，北京 100081

Author(s):

CHEN Lingling¹, BI Xiaojun²

1. College of Information and Communication Engineering, Harbin Engineering University, Harbin 150001, China;
2. School of Information Engineering, Minzu University of China, Beijing 100081, China

关键词:

深度学习; 睡眠分期; 通道注意力机制; 睡眠多导图; 多模态; 双向长短时记忆网络; 欧洲数据格式睡眠数据集; 残差收缩网络

Keywords:

deep learning; sleep stage; channel attention mechanism; polysomnogram; multimodal; bidirectional long short-term memory; sleep-European data format dataset; residual shrinkage network

分类号:

TP18; TH79

DOI:

10.11992/tis.202202018

文献标志码:

2022-08-26

摘要:

针对睡眠多导图中各模态信息在睡眠各阶段存在差异性，而导致特征利用不充分的问题，本文提出了一种基于通道注意力机制和多模态门控机制的睡眠分期模型。首先利用残差收缩网络设计各模态特征提取网络用于提取各模态特征，并在通道维度上进行拼接融合，利用通道注意力机制进一步对融合特征进行重标定得到睡眠多导图的时不变特征；之后提出了一种基于自适应门控机制的多模态门控模块，对各模态特征及时不变特征按照重要程度进行加权融合，实现特征融合；最后利用双向长短时记忆网络提取睡眠多导图的时序特征。实验结果表明，本文提出的睡眠分期模型在欧洲数据格式睡眠数据集(sleep-European data format, sleep-EDF)上准确率为87.6%，$ {M_{{F_1}}} $为82.0%，取得了目前最好的分期效果。

Abstract:

To solve the problem of insufficient feature use due to the differences in modal information in sleep polygraphs at different stages, this paper proposes a sleep staging method based on the channel attention mechanism and multimodal gating module. First, a residual shrinkage network is used to design each modal feature extraction network to extract the features of all modules, which are spliced and fused in the channel dimension, and the fused features are further recalibrated using the channel attention mechanism to obtain time-invariant polysomnography features. Then, a multimodal gating module based on an adaptive gating mechanism is proposed, which weighs and fuses all modal and time-invariant features according to their importance, thereby realizing the organic fusion of features. Finally, the time sequence characteristics of polysomnography are obtained using a bidirectional long short-term memory network. The experimental result shows that the accuracy of the sleep staging model proposed in this paper is 87.6% on the Sleep-EDF dataset, with an $M_{F_1} $ of 82.0%, thus achieving the best staging effect to date.