[1]魏柏淳,姜峰,张松涛,等.基于耳周肌电信号的默念口令识别方法[J].智能系统学报,2025,20(4):894-904.[doi:10.11992/tis.202406017]
WEI Baichun,JIANG Feng,ZHANG Songtao,et al.Method for silent command recognition based on periauricular EMG signals[J].CAAI Transactions on Intelligent Systems,2025,20(4):894-904.[doi:10.11992/tis.202406017]
点击复制
WEI Baichun,JIANG Feng,ZHANG Songtao,et al.Method for silent command recognition based on periauricular EMG signals[J].CAAI Transactions on Intelligent Systems,2025,20(4):894-904.[doi:10.11992/tis.202406017]
基于耳周肌电信号的默念口令识别方法
《智能系统学报》[ISSN 1673-4785/CN 23-1538/TP] 卷:
20
期数:
2025年第4期
页码:
894-904
栏目:
学术论文—机器学习
出版日期:
2025-08-05
- Title:
- Method for silent command recognition based on periauricular EMG signals
- Keywords:
- artificial intelligence; pattern recognition; human-computer interaction; neural human-machine interface; human intent decoding; silent command recognition; EMG processing; neural networks
- 分类号:
- TP181
- 文献标志码:
- 2025-4-8
- 摘要:
- 智能设备的普及促使可穿戴人机交互技术需求日益增加。为提高用户接受度，人机交互技术对交互易用性与隐蔽性要求较高。本文提出基于耳周肌电信号的默念口令识别方法。该方法易于与集成生理电采集的耳机设备结合，实现无声操控智能设备，减少社交尴尬。具体地，本文首先确定并构建口令经验原则，筛选最优口令集。其次，根据单通道信噪比和分类准确率选择最优耳周传感器位置。再次，提出基于CNN-Transformer结构的识别模型构建耳周肌电信号与默念口令的时空映射。最后，大量实验评估方法可行性和稳定性。结果表明，本文方法平均准确率91.18%，优于相关任务的先进模型，且在命令变形和头部运动下表现稳定。本文方法奠定了默念口令识别商业产品的技术基础。
- Abstract:
- The widespread use of smart devices has led to an increasing demand for wearable human–computer interaction technologies. To improve user acceptance, human–computer interaction technologies require high levels of interaction usability and concealment. This paper proposes a method for silent command recognition based on periauricular EMG signals. This method is easy to integrate with headphones equipped with integrated physiological signal acquisition, enables silent control of smart devices, and reduces social awkwardness. First, the command empirical principles are determined and constructed, and then the optimal command set is selected through screening. Second, the optimal periauricular sensor positions are chosen based on single-channel signal-to-noise ratio and classification accuracy. Third, a recognition model based on the CNN–Transformer structure is proposed to learn the spatiotemporal mapping between periauricular EMG signals and silent commands. Finally, extensive experiments evaluate the feasibility and stability of this method. Results demonstrate that the average accuracy of this method is 91.18%. The proposed method is superior to advanced models in similar tasks and is stable under command deformation and head motion. This method lays the technical foundation for commercial products of silent command recognition.
- 参考文献/References:
-
[1] NICHOLLS B, ANG C S, KANJO E, et al. An EMG-based eating behaviour monitoring system with haptic feedback to promote mindful eating[J]. Computers in biology and medicine, 2022, 149: 106068.
[2] RASHID N, DAUTTA M, TSENG P, et al. HEAR: fog-enabled energy-aware online human eating activity recognition[J]. IEEE internet of things journal, 2021, 8(2): 860-868.
[3] BALLI S, SA?BA? E A, PEKER M. Human activity recognition from smart watch sensor data using a hybrid of principal component analysis and random forest algorithm[J]. Measurement and control, 2019, 52(1/2): 37-45.
[4] BI Hongliang, SUN Yuanyuan, LIU Jiajia, et al. SmartEar: rhythm-based tap authentication using earphone in information-centric wireless sensor network[J]. IEEE internet of things journal, 2022, 9(2): 885-896.
[5] KHAJAVI M, AHMADYFARD A. Human face aging based on active appearance model using proper feature set[J]. Signal, image and video processing, 2023, 17(4): 1465-1473.
[6] IRAVANTCHI Y, ZHANG Yang, BERNITSAS E, et al. Interferi: gesture sensing using on-body acoustic interferometry[C]//Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. Glasgow Scotland: ACM, 2019: 1-13.
[7] WAN Jun, LI Jing, LAI Zhihui, et al. Robust face alignment by cascaded regression and de-occlusion[J]. Neural networks, 2020, 123: 261-272.
[8] 王贤兵. 人脸识别系统技术分析[J]. 中国科技信息, 2024(2): 74-76.
WANG Xianbing. Technical analysis of face recognition system[J]. China science and technology information, 2024(2): 74-76.
[9] 王海荣. 面部表情追踪技术在VR设备中的发展研究[J]. 电脑知识与技术, 2019, 15(32): 232-235.
WANG Hairong. Research on the development of facial expression tracking technology in VR equipment[J]. Computer knowledge and technology, 2019, 15(32): 232-235.
[10] MCMAHAN B, RAO D. Listening to the world improves speech command recognition[C]//Proceedings of the AAAI Conference on Artificial Intelligence. New York: AAAI, 2018, 32(1): 378-385.
[11] 张锋, 陶浩兵, 慕京生, 等. 基于Bi-LSTM的人机语音交互[J]. 传感器与微系统, 2022, 41(4): 104-107,116.
ZHANG Feng, TAO Haobing, MU Jingsheng, et al. Man-machine voice interaction based on Bi-LSTM[J]. Transducer and microsystem technologies, 2022, 41(4): 104-107, 116.
[12] GAO Yang, JIN Yincheng, LI Jiyang, et al. EchoWhisper[J]. Proceedings of the ACM on interactive, mobile, wearable and ubiquitous technologies, 2020, 4(3): 1-27.
[13] MU Ruojun, CHEN Jianshe. Oral bio-interfaces: properties and functional roles of salivary multilayer in food oral processing[J]. Trends in food science & technology, 2023, 132: 121-131.
[14] ROSTAMINIA S, LAMSON A, MAJI S, et al. W!NCE: unobtrusive sensing of upper facial action units with EOG-based eyewear[J]. Proceedings of the ACM on interactive, mobile, wearable and ubiquitous technologies, 2019, 3(1): 1-26.
[15] FAJARDO J M, GOMEZ O, PRIETO F. EMG hand gesture classification using handcrafted and deep features[J]. Biomedical signal processing and control, 2021, 63: 102210.
[16] SIM?O M, NETO P, GIBARU O. EMG-based online classification of gestures with recurrent neural networks[J]. Pattern recognition letters, 2019, 128: 45-51.
[17] 章毅, 吕嘉仪, 兰星, 等. 结合面部动作单元感知的三维人脸重建算法[J]. 软件学报, 2024, 35(5): 2176-2191.
ZHANG Yi, LYU Jiayi, LAN Xing, et al. AU-aware algorithm for 3D facial reconstruction[J]. Journal of software, 2024, 35(5): 2176-2191.
[18] KIM T, SHIN Y, KANG K, et al. Ultrathin crystalline-silicon-based strain gauges with deep learning algorithms for silent speech interfaces[J]. Nature communications, 2022, 13(1): 5815.
[19] 刘万阳, 李晓欧. 基于表面肌电信号的虚拟现实控制系统设计[J]. 计算机测量与控制, 2019, 27(3): 107-111.
LIU Wanyang, LI Xiaoou. Design of control system for virtual reality based on surface electromyography signal[J]. Computer measurement & control, 2019, 27(3): 107-111.
[20] 孙利, 李放, 王晓梅, 等. 脑电图、肌电图及临床症状学在双侧强直发作局灶性癫痫定侧中的价值评估[J]. 基础医学与临床, 2023, 43(8): 1289-1293.
SUN Li, LI Fang, WANG Xiaomei, et al. Evaluation of EEG, EMG and clinical semiology in the lateral location of bilateral extremities tonic seizures of focal epilepsy[J]. Basic and clinical medicine, 2023, 43(8): 1289-1293.
[21] 刘声中, 许德章. 一种基于改进sEMG活动段检测的手部运动意图识别新方法[J]. 嘉兴学院学报, 2022, 34(6): 105-112.
LIU Shengzhong, XU Dezhang. A new method for hand motion intent recognition based on improved sEMG signal active segment detection[J]. Journal of Jiaxing University, 2022, 34(6): 105-112.
[22] CHEN Sanyuan, WANG Chengyi, CHEN Zhengyang, et al. WavLM: large-scale self-supervised pre-training for full stack speech processing[J]. IEEE journal of selected topics in signal processing, 2022, 16(6): 1505-1518.
[23] HUEBER T, BENAROYA E L, CHOLLET G, et al. Development of a silent speech interface driven by ultrasound and optical images of the tongue and lips[J]. Speech communication, 2010, 52(4): 288-300.
[24] XU Sijia, YU Jiexiang, GUO Hongshuang, et al. Force-induced ion generation in zwitterionic hydrogels for a sensitive silent-speech sensor[J]. Nature communications, 2023, 14(1): 219.
[25] MELTZNER G S, HEATON J T, DENG Yunbin, et al. Silent speech recognition as an alternative communication device for persons with laryngectomy[J]. IEEE/ACM transactions on audio, speech, and language processing, 2017, 25(12): 2386-2398.
[26] WAND M, JANKE M, SCHULTZ T. Tackling speaking mode varieties in EMG-based speech recognition[J]. IEEE transactions on bio-medical engineering, 2014, 61(10): 2515-2526.
[27] MATTHIES D J C, STRECKER B A, URBAN B. EarFieldSensing: a novel in-ear electric field sensing to enrich wearable gesture input through facial expressions[C]//Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. Denver: ACM, 2017: 1911-1922.
[28] NGUYEN A, ALQURASHI R, RAGHEBI Z, et al. In-ear biosignal recording system: a wearable for automatic whole-night sleep staging[C]//Proceedings of the 2016 Workshop on Wearable Systems and Applications. Singapore: ACM, 2016: 19-24.
[29] WU Yi, KAKARAPARTHI V, LI Zhuohang, et al. BioFace-3D: continuous 3d facial reconstruction through lightweight single-ear biosensors[C]//Proceedings of the 27th Annual International Conference on Mobile Computing and Networking. New Orleans: ACM, 2021: 350-363.
[30] KIM M, CAO Beiming, MAU T, et al. Speaker-independent silent speech recognition from flesh-point articulatory movements using an LSTM neural network[J]. IEEE/ACM transactions on audio, speech, and language processing, 2017, 25(12): 2323-2336.
[31] ENGLEHART K, HUDGINS B. A robust, real-time control scheme for multifunction myoelectric control[J]. IEEE transactions on bio-medical engineering, 2003, 50(7): 848-854.
[32] DU Yichun, LIN C H, SHYU L Y, et al. Portable hand motion classifier for multi-channel surface electromyography recognition using grey relational analysis[J]. Expert systems with applications, 2010, 37(6): 4283-4291.
[33] WEI Wentao, DAI Qingfeng, WONG Y, et al. Surface-electromyography-based gesture recognition by multi-view deep learning[J]. IEEE transactions on bio-medical engineering, 2019, 66(10): 2964-2973.
[34] MONTAZERIN M, RAHIMIAN E, NADERKHANI F, et al. Transformer-based hand gesture recognition from instantaneous to fused neural decomposition of high-density EMG signals[J]. Scientific reports, 2023, 13(1): 11000.
[35] GAO Yang, WANG Wei, PHOHA V V, et al. EarEcho[J]. Proceedings of the ACM on interactive, mobile, wearable and ubiquitous technologies, 2019, 3(3): 1-24.
- 相似文献/References:
-
[1]李德毅.网络时代人工智能研究与发展[J].智能系统学报,2009,4(1):1.
LI De-yi.AI research and development in the network age[J].CAAI Transactions on Intelligent Systems,2009,4():1.
[2]赵克勤.二元联系数A+Bi的理论基础与基本算法及在人工智能中的应用[J].智能系统学报,2008,3(6):476.
ZHAO Ke-qin.The theoretical basis and basic algorithm of binary connection A+Bi and its application in AI[J].CAAI Transactions on Intelligent Systems,2008,3():476.
[3]徐玉如,庞永杰,甘?? 永,等.智能水下机器人技术展望[J].智能系统学报,2006,1(1):9.
XU Yu-ru,PANG Yong-jie,GAN Yong,et al.AUV—state-of-the-art and prospect[J].CAAI Transactions on Intelligent Systems,2006,1():9.
[4]王志良.人工心理与人工情感[J].智能系统学报,2006,1(1):38.
WANG Zhi-liang.Artificial psychology and artificial emotion[J].CAAI Transactions on Intelligent Systems,2006,1():38.
[5]赵克勤.集对分析的不确定性系统理论在AI中的应用[J].智能系统学报,2006,1(2):16.
ZHAO Ke-qin.The application of uncertainty systems theory of set pair analysis (SPU)in the artificial intelligence[J].CAAI Transactions on Intelligent Systems,2006,1():16.
[6]秦裕林,朱新民,朱? 丹.Herbert Simon在最后几年里的两个研究方向[J].智能系统学报,2006,1(2):11.
QIN Yu-lin,ZHU Xin-min,ZHU Dan.Herbert Simons two research directions in his lost years[J].CAAI Transactions on Intelligent Systems,2006,1():11.
[7]程显毅,陈小波.基于多Agent的模式识别框[J].智能系统学报,2006,1(2):89.
CHENG Xian-yi,CHEN Xiao-bo.Frame of pattern recognition based on multi-Agent[J].CAAI Transactions on Intelligent Systems,2006,1():89.
[8]谷文祥,李 丽,李丹丹.规划识别的研究及其应用[J].智能系统学报,2007,2(1):1.
GU Wen-xiang,LI Li,LI Dan-dan.Research and application of plan recognition[J].CAAI Transactions on Intelligent Systems,2007,2():1.
[9]杨春燕,蔡 文.可拓信息-知识-智能形式化体系研究[J].智能系统学报,2007,2(3):8.
YANG Chun-yan,CAI Wen.A formalized system of extension information-knowledge-intelligence[J].CAAI Transactions on Intelligent Systems,2007,2():8.
[10]赵克勤.SPA的同异反系统理论在人工智能研究中的应用[J].智能系统学报,2007,2(5):20.
ZHAO Ke-qin.The application of SPAbased identicaldiscrepancycontrary system theory in artificial intelligence research[J].CAAI Transactions on Intelligent Systems,2007,2():20.
[11]李德毅.AI——人类社会发展的加速器[J].智能系统学报,2017,12(5):583.[doi:10.11992/tis.201710016]
LI Deyi.Artificial intelligence:an accelerator for the development of human society[J].CAAI Transactions on Intelligent Systems,2017,12():583.[doi:10.11992/tis.201710016]
备注/Memo
收稿日期:2024-6-11。
基金项目:江苏省科技计划项目(BE2021086)；中央引导地方科技发展专项项目(2024ZYD0266).
作者简介:魏柏淳，助理研究员，博士，主要研究方向为人机交互、可穿戴计算与外骨骼机器人。发表学术论文17篇。E-mail：bcwei@hit.edu.cn。;姜峰，教授，博士，主要研究方向为视频压缩、可穿戴计算、多智能体博弈、外骨骼机器人。主持国家自然科学基金面上项目2项、省部级项目5项，发表学术论文40余篇。E-mail：fjiang@hit.edu.cn。;王修来，教授，博士，主要研究方向为人力资源管理与信息不对称、大数据挖掘与分析和数据智能应用。E-mail：wangxiulai@126.com。
通讯作者:王修来. E-mail：wangxiulai@126.com
更新日期/Last Update:
1900-01-01