[1]蒋云良,周阳,张雄涛,等.基于域间Mixup微调策略的跨被试运动想象脑电信号分类算法[J].智能系统学报,2024,19(4):909-919.[doi:10.11992/tis.202208017]
JIANG Yunliang,ZHOU Yang,ZHANG Xiongtao,et al.Cross-subject motor imagery EEG classification based on inter-domain Mixup fine-tuning strategy[J].CAAI Transactions on Intelligent Systems,2024,19(4):909-919.[doi:10.11992/tis.202208017]
点击复制
JIANG Yunliang,ZHOU Yang,ZHANG Xiongtao,et al.Cross-subject motor imagery EEG classification based on inter-domain Mixup fine-tuning strategy[J].CAAI Transactions on Intelligent Systems,2024,19(4):909-919.[doi:10.11992/tis.202208017]
基于域间Mixup微调策略的跨被试运动想象脑电信号分类算法
《智能系统学报》[ISSN 1673-4785/CN 23-1538/TP] 卷:
19
期数:
2024年第4期
页码:
909-919
栏目:
学术论文—机器感知与模式识别
出版日期:
2024-07-05
- Title:
- Cross-subject motor imagery EEG classification based on inter-domain Mixup fine-tuning strategy
- Keywords:
- inter-domain Mixup; pre-training; fine-tuning; electroencephalogram; motor imagery; cross-subject knowledge transfer; convolutional neural network; regularization
- 分类号:
- TP18
- 摘要:
- 为了缓解传统微调算法的灾难性遗忘问题，本文提出了一种基于域间Mixup微调策略的跨被试运动想象脑电信号分类算法Mix-Tuning。Mix-Tuning通过预训练、微调的二阶段训练方式，实现跨领域知识迁移。预训练阶段，Mix-Tuning使用源域数据初始化模型参数，挖掘源域数据潜在信息。微调阶段，Mix-Tuning通过域间Mixup，生成域间插值数据微调模型参数。域间Mixup数据增强策略引入源域数据潜在信息，缓解传统微调算法在样本稀疏场景下的灾难性遗忘问题，提高模型的泛化性能。Mix-Tuning被进一步应用于运动想象脑电信号分类任务，实现了跨被试正向知识迁移。Mix-Tuning在BMI数据集的运动想象任务达到了85.50%的平均分类准确率，相较于被试–依赖和被试–独立训练方式的预测准确率58.72%和84.01%，分别提高26.78%和1.49%。本文分析结果可为跨被试运动想象脑电信号分类算法提供参考。
- Abstract:
- In order to alleviate the catastrophic forgetting problem of vanilla fine-tuning algorithms, we propose a cross-subject motor imagery EEG classification method based on inter-domain Mixup fine-tuning strategy, i.e., Mix-Tuning. Mix-Tuning realizes cross-domain knowledge transfer through a two-stage training manner consisting of pre-training and fine-tuning. In the pre-training stage, Mix-Tuning uses the source domain data to initialize the model parameters and mine potential information of the source domain data. In the fine-tuning stage, Mix-Tuning generates inter-domain interpolation data to fine-tune the model parameters through inter-domain Mixup. Inter-domain Mixup data enhancement strategy introduces latent information of the source domain data, which alleviates the catastrophic forgetting problem of Vanilla Fine-tuning in sparse sample scenarios and improves the generalization performance of the model. Mix-Tuning is further applied to the motor imagery EEG classification task and achieves cross-subject positive knowledge transfer. Mix-Tuning achieved an average classification accuracy of 85.50% on motor imagery task BMIdataset. Compared with 58.72% and 84.01% for Subject-specific and Subject-independent training manner, Mix-Tuning increased by 26.78% and 1.49%, respectively. The analysis results in this paper can provide a reference for cross-subject motor imagery EEG classification algorithm.
- 参考文献/References:
-
[1] RAMOSER H, MULLER-GERKING J, PFURTSCHELLER G. Optimal spatial filtering of single trial EEG during imagined hand movement[J]. IEEE transactions on rehabilitation engineering, 2000, 8(4): 441–446.
[2] ANG Kaikeng, CHIN Z Y, WANG Chuanchu, et al. Filter bank common spatial pattern algorithm on BCI competition IV datasets 2a and 2b[J]. Frontiers in neuroscience, 2012, 6: 39.
[3] GAO Xiaorong, WANG Yijun, CHEN Xiaogang, et al. Interface, interaction, and intelligence in generalized brain-computer interfaces[J]. Trends in cognitive sciences, 2021, 25(8): 671–684.
[4] REZEIKA A, BENDA M, STAWICKI P, et al. Brain-computer interface spellers: a review[J]. Brain sciences, 2018, 8(4): 57.
[5] 李景聪, 潘伟健, 林镇远, 等. 采用多路图注意力网络的情绪脑电信号识别方法[J]. 智能系统学报, 2022, 17(3): 531–539
LI Jingcong, PAN Weijian, LIN Zhenyuan, et al. Emotional EEG signal recognition method using multi-path graph attention network[J]. CAAI transactions on intelligent systems, 2022, 17(3): 531–539
[6] CRAIK A, HE Yongtian, CONTRERAS-VIDAL J L. Deep learning for electroencephalogram (EEG) classification tasks: a review[J]. Journal of neural engineering, 2019, 16(3): 031001.
[7] ZHAO Liming, YAN Xu, LU Baoliang. Plug-and-play domain adaptation for cross-subject EEG-based emotion recognition[C]//Proceedings of the AAAI Conference on Artificial Intelligence. Vancouver: AAAI, 2021, 35(1): 863-870.
[8] NI Ziyi, XU Jiaming, WU Yuwei, et al. Improving cross-state and cross-subject visual ERP-based BCI with temporal modeling and adversarial training[J]. IEEE transactions on neural systems and rehabilitation engineering, 2022, 30: 369–379.
[9] YOSINSKI J, CLUNE J, BENGIO Y, et al. How transferable are features in deep neural networks?[C]//Proceedings of the Advances in Neural Information Processing Systems. Montréal: MIT Press, 2014, 27: 3320-3328.
[10] WANG Yaqing, YAO Quanming, KWOK J T, et al. Generalizing from a few examples: a survey on few-shot learning[J]. ACM computing surveys, 53(3): 1-34.
[11] LEE M H, KWON O Y, KIM Y J, et al. EEG dataset and OpenBMI toolbox for three BCI paradigms: an investigation into BCI illiteracy[J]. GigaScience, 2019, 8(5): giz002.
[12] VAPNIK V N. An overview of statistical learning theory[J]. IEEE transactions on neural networks, 1999, 10(5): 988–999.
[13] PERKONIGG M, HOFMANNINGER J, HEROLD C J, et al. Dynamic memory to alleviate catastrophic forgetting in continual learning with medical imaging[J]. Nature communications, 2021, 12: 5678.
[14] ZHANG Baosheng, GUO Yuchen, LI Yipeng, et al. Memory recall: a simple neural network training framework against catastrophic forgetting[J]. IEEE transactions on neural networks and learning systems, 2022, 33(5): 2010–2022.
[15] HOULSBY N, GIURGIU A, JASTRZEBSKI S, et al. Parameter-efficient transfer learning for NLP[C]//Proceedings of the 36th International Conference on Machine Learning. Long Beach: IMLS, 2019, 97: 2790-2799.
[16] LI Xuhong, GRANDVALET Y, DAVOINE F. Explicit inductive bias for transfer learning with convolutional networks[C]//Proceedings of the 35th International Conference on Machine Learning. Stockholmsm?ssan: IMLS, 2018, 80: 2825-2834.
[17] CHEN Sanyuan, HOU Yutai, CUI Yiming, et al. Recall and learn: fine-tuning deep pretrained language models with less forgetting[C]//Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing. Stroudsburg: Association for Computational Linguistics, 2020: 7870-7881.
[18] LEE C, CHO K, KANG Wanmo. Mixout: effective regularization to finetune large-scale pretrained language models[C]//Proceeding of the 8th International Conference on Learning Representations. Addis Ababa: ICLR. 2020: 1-17.
[19] XU Runxin, LUO Fuli, ZHANG Zhiyuan, et al. Raise a child in large language model: towards effective and generalizable fine-tuning[C]//Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing. Stroudsburg: Association for Computational Linguistics, 2021: 9514-9528.
[20] SRIVASTAVA N, HINTON G, KRIZHEVSKY A, et al. Dropout: a simple way to prevent neural networks from overfitting[J]. Journal of machine learning research, 2014, 15: 1929–1958.
[21] CHAPELLE O, WESTON J, BOTTOU L, et al. Vicinal risk minimization[C]//Proceedings of the Advances in Neural Information Processing Systems. Denver: MIT Press, 2000, 13: 1-7.
[22] ZHANG H Y, CISSE M, DAUPHIN Y N, et al. Mixup: beyond empirical risk minimization[C]//Proceedings of the International Conference on Learning Representations. Vancouver: ICLR, 2018: 1-13.
[23] LAWHERN V J, SOLON A J, WAYTOWICH N R, et al. EEGNet: a compact convolutional neural network for EEG-based brain-computer interfaces[J]. Journal of neural engineering, 2018, 15(5): 056013.
[24] LI Yang, GUO Lianghui, LIU Yu, et al. A temporal-spectral-based squeeze-and-excitation feature fusion network for motor imagery EEG decoding[J]. IEEE transactions on neural systems and rehabilitation engineering, 2021, 29: 1534–1545.
[25] CAI Siqi, SU Enze, XIE Longhan, et al. EEG-based auditory attention detection via frequency and channel neural attention[J]. IEEE transactions on human-machine systems, 2021, 52(2): 256–266.
[26] SCHIRRMEISTER R T, SPRINGENBERG J T, FIEDERER L D J, et al. Deep learning with convolutional neural networks for EEG decoding and visualization[J]. Human brain mapping, 2017, 38(11): 5391–5420.
[27] KOSTAS D, RUDZICZ F. Thinker invariance: enabling deep neural networks for BCI across more people[J]. Journal of neural engineering, 2020, 17(5): 056008.
[28] KWON O Y, LEE M H, GUAN Cuntai, et al. Subject-independent brain-computer interfaces based on deep convolutional neural networks[J]. IEEE transactions on neural networks and learning systems, 2019, 31(10): 3839–3852.
[29] ZHANG Kaishuo, ROBINSON N, LEE S W, et al. Adaptive transfer learning for EEG motor imagery classification with deep convolutional neural network[J]. Neural networks, 2021, 136: 1–10.
[30] SZEGEDY C, VANHOUCKE V, IOFFE S, et al. Rethinking the inception architecture for computer vision[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016: 2818-2826.
[31] HE Kaiming, ZHANG Xiangyu, REN Shaoqing, et al. Deep residual learning for image recognition[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016: 770-778.
[32] HUANG Gao, LIU Zhuang, VAN DER MAATEN L, et al. Densely connected convolutional networks[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE, 2017: 2261-2269.
[33] GOODFELLOW I, BENGIO Y, COURVILLE A. Deep learning[M]. Cambridge: MIT press, 2016.
[34] 章杭奎, 刘栋军, 孔万增. 面向跨被试RSVP的多特征低维子空间嵌入的ERP检测[J]. 智能系统学报, 2022, 17(5): 1054–1061
ZHANG Hangkui, LIU Dongjun, KONG Wanzeng. ERP detection of multi-feature embedding in the low-dimensional subspace for cross-subject RSVP[J]. CAAI transactions on intelligent systems, 2022, 17(5): 1054–1061
[35] KINGMA D P, BA J L. Adam: a method for stochastic optimization[C]//3rd International Conference on Learning Representations, ICLR 2015 - Conference Track Proceedings. San Diego: ICLR, 2015: 1-15.
- 相似文献/References:
-
[1]马志强,李图雅,杨双涛,等.基于深度神经网络的蒙古语声学模型建模研究[J].智能系统学报,2018,13(3):486.[doi:10.11992/tis.201710029]
MA Zhiqiang,LI Tuya,YANG Shuangtao,et al.Mongolian acoustic modeling based on deep neural network[J].CAAI Transactions on Intelligent Systems,2018,13():486.[doi:10.11992/tis.201710029]
[2]程鹏超,杜军平,薛哲.基于多路交叉的用户金融行为预测[J].智能系统学报,2021,16(2):378.[doi:10.11992/tis.202006054]
CHENG Pengchao,DU Junping,XUE Zhe.Prediction of user financial behavior based on multi-way crossing[J].CAAI Transactions on Intelligent Systems,2021,16():378.[doi:10.11992/tis.202006054]
[3]盛子旗,霍冠英.样本仿真结合迁移学习的声呐图像水雷检测[J].智能系统学报,2021,16(2):385.[doi:10.11992/tis.202101030]
SHENG Ziqi,HUO Guanying.Detection of underwater mine target in sidescan sonar image based on sample simulation and transfer learning[J].CAAI Transactions on Intelligent Systems,2021,16():385.[doi:10.11992/tis.202101030]
[4]王竣禾,姜勇.基于深度强化学习的动态装配算法[J].智能系统学报,2023,18(1):2.[doi:10.11992/tis.202201006]
WANG Junhe,JIANG Yong.Dynamic assembly algorithm based on deep reinforcement learning[J].CAAI Transactions on Intelligent Systems,2023,18():2.[doi:10.11992/tis.202201006]
[5]莫宏伟,孙琪,孙鹏,等.乳腺钼靶肿块自监督预训练迁移检测方法研究[J].智能系统学报,2024,19(5):1082.[doi:10.11992/tis.202304032]
MO Hongwei,SUN Qi,SUN Peng,et al.Self-supervised pretraining detection of mammographic mass targets in breast[J].CAAI Transactions on Intelligent Systems,2024,19():1082.[doi:10.11992/tis.202304032]
备注/Memo
收稿日期:2022-08-17。
基金项目:国家自然科学基金项目（61771193，62101189，62376094，U22A20102）；浙江省教育厅科研项目（Y202146028）.
作者简介:蒋云良，教授，博士生导师，博士，主要研究方向为智能信息处理、地理信息系统。享受国务院政府特殊津贴。获发明专利授权26项。发表学术论文63篇，出版学术著作2部，E-mail：jyl2022@zjnu.cn;周阳，硕士研究生。主要研究方向为深度学习、迁移学习、脑电信号处理。 E-mail：3189269614@qq.com;张雄涛，副教授，博士，主要研究方向为人工智能与模式识别、机器学习。承担国家和省部级科研课题10余项，获发明专利授权7项，发表学术论文20余篇。E-mail：02032@zjhu.edu.cn
通讯作者:张雄涛. E-mail: 02032@zjhu.edu.cn
更新日期/Last Update:
1900-01-01