[1]邓欣,肖立峰,杨鹏飞,等.融合运动想象脑电与眼电信号的机械臂控制系统开发[J].智能系统学报,2022,17(6):1163-1172.[doi:10.11992/tis.202107042]
DENG Xin,XIAO Lifeng,YANG Pengfei,et al.Development of a robot arm control system using motor imagery electroencephalography and electrooculography[J].CAAI Transactions on Intelligent Systems,2022,17(6):1163-1172.[doi:10.11992/tis.202107042]
点击复制
DENG Xin,XIAO Lifeng,YANG Pengfei,et al.Development of a robot arm control system using motor imagery electroencephalography and electrooculography[J].CAAI Transactions on Intelligent Systems,2022,17(6):1163-1172.[doi:10.11992/tis.202107042]
融合运动想象脑电与眼电信号的机械臂控制系统开发
《智能系统学报》[ISSN 1673-4785/CN 23-1538/TP] 卷:
17
期数:
2022年第6期
页码:
1163-1172
栏目:
学术论文—智能系统
出版日期:
2022-11-05
- Title:
- Development of a robot arm control system using motor imagery electroencephalography and electrooculography
- Keywords:
- brain computer interfaces; motor imagery; electrooculography; robot arm control; common spatial pattern; wavelet transform; support vector machines; independent medication
- 分类号:
- TP39
- 文献标志码:
- 2022-06-23
- 摘要:
- 脑机接口(brain computer interface, BCI)旨在通过脑电信号与外部设备通信，以实现对外部设备的控制。针对目前脑机接口系统中混合多种复杂生理电信号，并且输出控制指令较少的问题，本文提出融合运动想象(motor imagery, MI)脑电与眼电信号方法扩充控制指令的轻量级机械臂控制系统。该系统分阶段融合脑电和眼电信号两种生物信号，使用双次眼电作为任务开关，运动想象脑电信号控制机械臂运动，单次眼电控制阶段切换，实现了二分类运动想象生成多种控制指令，完成了对机械臂的连续控制。其中运动想象脑电信号使用提升小波变换(lifting wavelet transform, LWT)和共空间模式(common spatial pattern, CSP)结合的方法提取特征，并采用支持向量机(support vector machines, SVM)进行分类；眼电信号通过分析无意识眼电和有意识眼电的峰值来设置阈值进行区分。为了验证系统的可行性，设计了一项脑控机械臂自主服药实验，通过在线实验测试，被试通过使用脑电信号和眼电信号实现了机械臂控制，并完成了服药流程，有利于进一步推广脑机接口技术的实际应用。
- Abstract:
- A brain computer interface (BCI) aims to control external devices by communicating with them via brain electroencephalography (EEG) signals. To tackle the issue of complex mixed multiple physiological electrical signals and low output control instructions in BCI systems, a lightweight robot arm control system that integrates motion imagery (MI) and electrooculography (EOG) signals was proposed to expand the control instructions. In this system, two biological EEG and EOG signals were integrated gradually. Robot arm movement was controlled by MI using double EOG as the task switch, and the control stages were switched by single EOG. The dichotomous MI generated a variety of control instructions, completing continuous control over the robot arm. MI EEG signals are extracted using lifting wavelet transform coupled with common spatial patterns and classified using support vector machines. The EOG signals were distinguished by analyzing the peak values of unconscious and conscious EOG set to a specific threshold. To verify the feasibility of this system, this study designed an autonomous drug-taking experiment. In the experiment, the subjects completed the drug-taking process using the BCI system with the robotic arm control, which is conducive to further promoting the practical application of BCI technology.
- 参考文献/References:
-
[1] 尹锐, 曹勇, 姜劲, 等. 基于运动想象脑机接口的外骨骼控制系统研究[J]. 航天医学与医学工程, 2020, 33(3): 208–213
YIN Rui, CAO Yong, JIANG Jin, et al. Design and implementation of exoskeleton control system based on motor imagery brain computer interface[J]. Space medicine & medical engineering, 2020, 33(3): 208–213
[2] 王斐, 张育中, 宁廷会, 等. 脑–机接口研究进展[J]. 智能系统学报, 2011, 6(3): 189–199
WANG Fei, ZHANG Yuzhong, NING Tinghui, et al. Research progress in a brain-computer interface[J]. CAAI transactions on intelligent systems, 2011, 6(3): 189–199
[3] ZHANG Wenchang, SUN Fuchun, WU Hang, et al. Asynchronous brain-computer interface shared control of robotic grasping[J]. Tsinghua science and technology, 2019, 24(3): 360–370.
[4] ZHU Yuanlu, LI Ying, LU Jinling, et al. A hybrid BCI based on SSVEP and EOG for robotic arm control[J]. Frontiers in neurorobotics, 2020, 14: 583641.
[5] 徐宝国, 何小杭, 魏智唯, 等. 基于运动想象脑电的机器人连续控制系统研究[J]. 仪器仪表学报, 2018, 39(9): 10–19
XU Baoguo, HE Xiaohang, WEI Zhiwei, et al. Research on continuous control system for robot based on motor imagery EEG[J]. Chinese journal of scientific instrument, 2018, 39(9): 10–19
[6] JEONG J H, SHIM K H, KIM D J, et al. Brain-controlled robotic arm system based on multi-directional CNN-BiLSTM network using EEG signals[J]. IEEE transactions on neural systems and rehabilitation engineering, 2020, 28(5): 1226–1238.
[7] LI Ting, HONG Jun, ZHANG Jinhua, et al. Brain-machine interface control of a manipulator using small-world neural network and shared control strategy[J]. Journal of neuroscience methods, 2014, 224: 26–38.
[8] RAKSHIT A, KONAR A, NAGAR A K. A hybrid brain-computer interface for closed-loop position control of a robot arm[J]. IEEE/CAA journal of automatica sinica, 2020, 7(5): 1344–1360.
[9] GAO Qiang, DOU Lixiang, BELKACEM A N, et al. Noninvasive electroencephalogram based control of a robotic arm for writing task using hybrid BCI system[J]. BioMed research international, 2017, 2017: 8316485.
[10] BHATTACHARYYA S, KONAR A, TIBAREWALA D N. Motor imagery, P300 and error-related EEG-based robot arm movement control for rehabilitation purpose[J]. Medical & biological engineering & computing, 2014, 52(12): 1007–1017.
[11] 李丹妮. 基于运动想象脑电信号的特征提取和分类算法研究[D]. 重庆: 重庆邮电大学, 2019.
LI Danni. Research on feature extraction and classification algorithm of EEG based on motor imagery[D]. Chongqing: Chongqing University of Posts and Telecommunications, 2019.
[12] 张晓楠, 刘建平. Graz-脑机接口研究概述[J]. 智能系统学报, 2013, 8(5): 395–399
ZHANG Xiaonan, LIU Jianping. Summary on the research for Graz-brain-computer interface[J]. CAAI transactions on intelligent systems, 2013, 8(5): 395–399
[13] BAI Xiaoping, WANG Xiangzhou, ZHENG Shuhua, et al. The offline feature extraction of four-class motor imagery EEG based on ICA and Wavelet-CSP[C]//Proceedings of the 33rd Chinese Control Conference. Nanjing: IEEE, 2014: 7189-7194.
[14] 邓欣, 龙灿, 米建勋, 等. 基于多域特征与随机子空间集成的脑电分类[J]. 系统仿真学报, 2020, 32(9): 1787–1798
DENG Xin, LONG Can, MI Jianxun, et al. EEG classification based on multi-domain features and random subspace ensemble[J]. Journal of system simulation, 2020, 32(9): 1787–1798
[15] 刘柯, 张孝, 李沛洋, 等. 基于脑功能网络和共空间模式分析的脑电情绪识别[J]. 计算机应用研究, 2021, 38(5): 1344–1349
LIU Ke, ZHANG Xiao, LI Peiyang, et al. EEG emotion recognition based on common spatial pattern of brain functional network[J]. Application research of computers, 2021, 38(5): 1344–1349
[16] YIN Xu, MENG Ming, SHE Qingshan, et al. Optimal channel-based sparse time-frequency blocks common spatial pattern feature extraction method for motor imagery classification[J]. Mathematical biosciences and engineering:MBE, 2021, 18(4): 4247–4263.
[17] SWELDENS W. The lifting scheme: a custom-design construction of biorthogonal wavelets[J]. Applied and computational harmonic analysis, 1996, 3(2): 186–200.
[18] 罗志增, 鲁先举, 周莹. 基于脑功能网络和样本熵的脑电信号特征提取[J]. 电子与信息学报, 2021, 43(2): 412–418
LUO Zhizeng, LU Xianju, ZHOU Ying. EEG feature extraction based on brain function network and sample entropy[J]. Journal of electronics & information technology, 2021, 43(2): 412–418
[19] 杨建平, 刘明华, 吕敬祥, 等. 低觉醒脑电识别与唤醒的可穿戴系统研究[J]. 智能系统学报, 2019, 14(4): 787–792
YANG Jianping, LIU Minghua, LYU Jingxiang, et al. A wearable system to recognize and awaken low-arousal state[J]. CAAI transactions on intelligent systems, 2019, 14(4): 787–792
[20] 伍广腾. 基于运动想象的脑—机接口在线异步系统研究[D]. 哈尔滨: 哈尔滨工业大学, 2018.
WU Guangteng. Research on online asynchronous brain-computer interface system based on motor imagery[D]. Harbin: Harbin Institute of Technology, 2018.
[21] 孟建军, 盛鑫军, 姚林, 等. 基于共空域频谱模式的少通道运动想象分类[J]. 中国生物医学工程学报, 2013, 32(5): 553–561
MENG Jianjun, SHENG Xinjun, YAO Lin, et al. Common spatial spectral pattern for motor imagery tasks in small channel configuration[J]. Chinese journal of biomedical engineering, 2013, 32(5): 553–561
[22] BHATTACHARYYA S, KONAR A, TIBAREWALA D N. Motor imagery and error related potential induced position control of a robotic arm[J]. IEEE/CAA journal of automatica sinica, 2017, 4(4): 639–650.
[23] DENG Xin, ZHANG Boxian, YU Nian, et al. Advanced TSGL-EEGNet for motor imagery EEG-based brain-computer interfaces[J]. IEEE access, 2021, 9: 25118–25130.
[24] LI Ninghan, LIU Yongze. Multi-domain feature extraction method of motor imagery EEG signal based on DWT and CSP[C]//Thirteenth international conference on signal processing systems (ICSPS 2021).Shanghai: SPIE, 2022: 286-291.
[25] 周鹏. 基于运动想象的脑机接口的研究[D]. 天津: 天津大学, 2007.
ZHOU Peng. The research of brain-computer interface based on motor imagery[D]. Tianjin: Tianjin University, 2007.
- 相似文献/References:
-
[1]张晓楠,刘建平.Graz- 脑机接口研究概述[J].智能系统学报,2013,8(5):395.[doi:10.3969/j.issn.1673-4785.201304055]
ZHANG Xiaonan,LIU Jianping.Summary on the research for Graz-brain-computer interface[J].CAAI Transactions on Intelligent Systems,2013,8():395.[doi:10.3969/j.issn.1673-4785.201304055]
[2]程煜,张鸣宇,陶霖密.1-Bit人机交互系统[J].智能系统学报,2015,10(4):528.[doi:10.3969/j.issn.1673-4785.201501015]
CHENG Yu,ZHANG Mingyu,TAO Linmi.One-Bit human-computer interactive system[J].CAAI Transactions on Intelligent Systems,2015,10():528.[doi:10.3969/j.issn.1673-4785.201501015]
[3]李景聪,潘伟健,林镇远,等.采用多路图注意力网络的情绪脑电信号识别方法[J].智能系统学报,2022,17(3):531.[doi:10.11992/tis.202107003]
LI Jingcong,PAN Weijian,LIN Zhenyuan,et al.Emotional EEG signal recognition method using multi-path graph attention network[J].CAAI Transactions on Intelligent Systems,2022,17():531.[doi:10.11992/tis.202107003]
[4]蒋云良,周阳,张雄涛,等.基于域间Mixup微调策略的跨被试运动想象脑电信号分类算法[J].智能系统学报,2024,19(4):909.[doi:10.11992/tis.202208017]
JIANG Yunliang,ZHOU Yang,ZHANG Xiongtao,et al.Cross-subject motor imagery EEG classification based on inter-domain Mixup fine-tuning strategy[J].CAAI Transactions on Intelligent Systems,2024,19():909.[doi:10.11992/tis.202208017]
备注/Memo
收稿日期:2021-07-20。
基金项目:国家自然科学基金项目（61806033,61703065）；重庆市自然科学基金面上项目（cstc2020jcyj-msxmX0284)；重庆市教委科技项目(KJQN202000625).
作者简介:邓欣，副教授，主要研究方向为脑机接口、大数据分析。主持和主要参与国家和省部级科研项目10项。发表学术论文70篇;肖立峰，硕士研究生，主要研究方向为智能信息处理;杨鹏飞，硕士研究生，主要研究方向为智能信息处理
通讯作者:邓欣.E-mail:dengxin@cqupt.edu.cn
更新日期/Last Update:
1900-01-01