[1]WEI Baichun,JIANG Feng,ZHANG Songtao,et al.Method for silent command recognition based on periauricular EMG signals[J].CAAI Transactions on Intelligent Systems,2025,20(4):894-904.[doi:10.11992/tis.202406017]
Copy

Method for silent command recognition based on periauricular EMG signals

CAAI Transactions on Intelligent Systems[ISSN 1673-4785/CN 23-1538/TP] Volume: 20 Number of periods: 2025 4 Page number: 894-904 Column: 学术论文—机器学习 Public date: 2025-08-05
Title:
Method for silent command recognition based on periauricular EMG signals
Author(s):
WEI Baichun1;  JIANG Feng2;  ZHANG Songtao1;  ZHANG Qi1;  DUAN Jinnan1;  WANG Xiulai3
1. Department of Life Science and Medicine, Harbin Institute of Technology, Harbin 150000, China;
2. Nanjing University of Information Science and Technology, School of Future Technology, Nanjing 211800, China;
3. General Hospital of Eastern Theater, Nanjing 210018, China
Keywords:
artificial intelligence; pattern recognition; human-computer interaction; neural human-machine interface; human intent decoding; silent command recognition; EMG processing; neural networks
CLC:
TP181
DOI:
10.11992/tis.202406017
Abstract:
The widespread use of smart devices has led to an increasing demand for wearable human–computer interaction technologies. To improve user acceptance, human–computer interaction technologies require high levels of interaction usability and concealment. This paper proposes a method for silent command recognition based on periauricular EMG signals. This method is easy to integrate with headphones equipped with integrated physiological signal acquisition, enables silent control of smart devices, and reduces social awkwardness. First, the command empirical principles are determined and constructed, and then the optimal command set is selected through screening. Second, the optimal periauricular sensor positions are chosen based on single-channel signal-to-noise ratio and classification accuracy. Third, a recognition model based on the CNN–Transformer structure is proposed to learn the spatiotemporal mapping between periauricular EMG signals and silent commands. Finally, extensive experiments evaluate the feasibility and stability of this method. Results demonstrate that the average accuracy of this method is 91.18%. The proposed method is superior to advanced models in similar tasks and is stable under command deformation and head motion. This method lays the technical foundation for commercial products of silent command recognition.
References:
[1] NICHOLLS B, ANG C S, KANJO E, et al. An EMG-based eating behaviour monitoring system with haptic feedback to promote mindful eating[J]. Computers in biology and medicine, 2022, 149: 106068.
[2] RASHID N, DAUTTA M, TSENG P, et al. HEAR: fog-enabled energy-aware online human eating activity recognition[J]. IEEE internet of things journal, 2021, 8(2): 860-868.
[3] BALLI S, SA?BA? E A, PEKER M. Human activity recognition from smart watch sensor data using a hybrid of principal component analysis and random forest algorithm[J]. Measurement and control, 2019, 52(1/2): 37-45.
[4] BI Hongliang, SUN Yuanyuan, LIU Jiajia, et al. SmartEar: rhythm-based tap authentication using earphone in information-centric wireless sensor network[J]. IEEE internet of things journal, 2022, 9(2): 885-896.
[5] KHAJAVI M, AHMADYFARD A. Human face aging based on active appearance model using proper feature set[J]. Signal, image and video processing, 2023, 17(4): 1465-1473.
[6] IRAVANTCHI Y, ZHANG Yang, BERNITSAS E, et al. Interferi: gesture sensing using on-body acoustic interferometry[C]//Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. Glasgow Scotland: ACM, 2019: 1-13.
[7] WAN Jun, LI Jing, LAI Zhihui, et al. Robust face alignment by cascaded regression and de-occlusion[J]. Neural networks, 2020, 123: 261-272.
[8] 王贤兵. 人脸识别系统技术分析[J]. 中国科技信息, 2024(2): 74-76.
WANG Xianbing. Technical analysis of face recognition system[J]. China science and technology information, 2024(2): 74-76.
[9] 王海荣. 面部表情追踪技术在VR设备中的发展研究[J]. 电脑知识与技术, 2019, 15(32): 232-235.
WANG Hairong. Research on the development of facial expression tracking technology in VR equipment[J]. Computer knowledge and technology, 2019, 15(32): 232-235.
[10] MCMAHAN B, RAO D. Listening to the world improves speech command recognition[C]//Proceedings of the AAAI Conference on Artificial Intelligence. New York: AAAI, 2018, 32(1): 378-385.
[11] 张锋, 陶浩兵, 慕京生, 等. 基于Bi-LSTM的人机语音交互[J]. 传感器与微系统, 2022, 41(4): 104-107,116.
ZHANG Feng, TAO Haobing, MU Jingsheng, et al. Man-machine voice interaction based on Bi-LSTM[J]. Transducer and microsystem technologies, 2022, 41(4): 104-107, 116.
[12] GAO Yang, JIN Yincheng, LI Jiyang, et al. EchoWhisper[J]. Proceedings of the ACM on interactive, mobile, wearable and ubiquitous technologies, 2020, 4(3): 1-27.
[13] MU Ruojun, CHEN Jianshe. Oral bio-interfaces: properties and functional roles of salivary multilayer in food oral processing[J]. Trends in food science & technology, 2023, 132: 121-131.
[14] ROSTAMINIA S, LAMSON A, MAJI S, et al. W!NCE: unobtrusive sensing of upper facial action units with EOG-based eyewear[J]. Proceedings of the ACM on interactive, mobile, wearable and ubiquitous technologies, 2019, 3(1): 1-26.
[15] FAJARDO J M, GOMEZ O, PRIETO F. EMG hand gesture classification using handcrafted and deep features[J]. Biomedical signal processing and control, 2021, 63: 102210.
[16] SIM?O M, NETO P, GIBARU O. EMG-based online classification of gestures with recurrent neural networks[J]. Pattern recognition letters, 2019, 128: 45-51.
[17] 章毅, 吕嘉仪, 兰星, 等. 结合面部动作单元感知的三维人脸重建算法[J]. 软件学报, 2024, 35(5): 2176-2191.
ZHANG Yi, LYU Jiayi, LAN Xing, et al. AU-aware algorithm for 3D facial reconstruction[J]. Journal of software, 2024, 35(5): 2176-2191.
[18] KIM T, SHIN Y, KANG K, et al. Ultrathin crystalline-silicon-based strain gauges with deep learning algorithms for silent speech interfaces[J]. Nature communications, 2022, 13(1): 5815.
[19] 刘万阳, 李晓欧. 基于表面肌电信号的虚拟现实控制系统设计[J]. 计算机测量与控制, 2019, 27(3): 107-111.
LIU Wanyang, LI Xiaoou. Design of control system for virtual reality based on surface electromyography signal[J]. Computer measurement & control, 2019, 27(3): 107-111.
[20] 孙利, 李放, 王晓梅, 等. 脑电图、肌电图及临床症状学在双侧强直发作局灶性癫痫定侧中的价值评估[J]. 基础医学与临床, 2023, 43(8): 1289-1293.
SUN Li, LI Fang, WANG Xiaomei, et al. Evaluation of EEG, EMG and clinical semiology in the lateral location of bilateral extremities tonic seizures of focal epilepsy[J]. Basic and clinical medicine, 2023, 43(8): 1289-1293.
[21] 刘声中, 许德章. 一种基于改进sEMG活动段检测的手部运动意图识别新方法[J]. 嘉兴学院学报, 2022, 34(6): 105-112.
LIU Shengzhong, XU Dezhang. A new method for hand motion intent recognition based on improved sEMG signal active segment detection[J]. Journal of Jiaxing University, 2022, 34(6): 105-112.
[22] CHEN Sanyuan, WANG Chengyi, CHEN Zhengyang, et al. WavLM: large-scale self-supervised pre-training for full stack speech processing[J]. IEEE journal of selected topics in signal processing, 2022, 16(6): 1505-1518.
[23] HUEBER T, BENAROYA E L, CHOLLET G, et al. Development of a silent speech interface driven by ultrasound and optical images of the tongue and lips[J]. Speech communication, 2010, 52(4): 288-300.
[24] XU Sijia, YU Jiexiang, GUO Hongshuang, et al. Force-induced ion generation in zwitterionic hydrogels for a sensitive silent-speech sensor[J]. Nature communications, 2023, 14(1): 219.
[25] MELTZNER G S, HEATON J T, DENG Yunbin, et al. Silent speech recognition as an alternative communication device for persons with laryngectomy[J]. IEEE/ACM transactions on audio, speech, and language processing, 2017, 25(12): 2386-2398.
[26] WAND M, JANKE M, SCHULTZ T. Tackling speaking mode varieties in EMG-based speech recognition[J]. IEEE transactions on bio-medical engineering, 2014, 61(10): 2515-2526.
[27] MATTHIES D J C, STRECKER B A, URBAN B. EarFieldSensing: a novel in-ear electric field sensing to enrich wearable gesture input through facial expressions[C]//Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. Denver: ACM, 2017: 1911-1922.
[28] NGUYEN A, ALQURASHI R, RAGHEBI Z, et al. In-ear biosignal recording system: a wearable for automatic whole-night sleep staging[C]//Proceedings of the 2016 Workshop on Wearable Systems and Applications. Singapore: ACM, 2016: 19-24.
[29] WU Yi, KAKARAPARTHI V, LI Zhuohang, et al. BioFace-3D: continuous 3d facial reconstruction through lightweight single-ear biosensors[C]//Proceedings of the 27th Annual International Conference on Mobile Computing and Networking. New Orleans: ACM, 2021: 350-363.
[30] KIM M, CAO Beiming, MAU T, et al. Speaker-independent silent speech recognition from flesh-point articulatory movements using an LSTM neural network[J]. IEEE/ACM transactions on audio, speech, and language processing, 2017, 25(12): 2323-2336.
[31] ENGLEHART K, HUDGINS B. A robust, real-time control scheme for multifunction myoelectric control[J]. IEEE transactions on bio-medical engineering, 2003, 50(7): 848-854.
[32] DU Yichun, LIN C H, SHYU L Y, et al. Portable hand motion classifier for multi-channel surface electromyography recognition using grey relational analysis[J]. Expert systems with applications, 2010, 37(6): 4283-4291.
[33] WEI Wentao, DAI Qingfeng, WONG Y, et al. Surface-electromyography-based gesture recognition by multi-view deep learning[J]. IEEE transactions on bio-medical engineering, 2019, 66(10): 2964-2973.
[34] MONTAZERIN M, RAHIMIAN E, NADERKHANI F, et al. Transformer-based hand gesture recognition from instantaneous to fused neural decomposition of high-density EMG signals[J]. Scientific reports, 2023, 13(1): 11000.
[35] GAO Yang, WANG Wei, PHOHA V V, et al. EarEcho[J]. Proceedings of the ACM on interactive, mobile, wearable and ubiquitous technologies, 2019, 3(3): 1-24.
Similar References:

Memo

-

Last Update: 1900-01-01

Copyright © CAAI Transactions on Intelligent Systems