[1]童庆耘,公沛良,张丽颖,等.时序注意力网络评估空管员认知负荷[J].智能系统学报,2026,21(3):792-801.[doi:10.11992/tis.202507018]
TONG Qingyun,GONG Peiliang,ZHANG Liying,et al.Temporal attention memory network for evaluating air traffic controller cognitive load[J].CAAI Transactions on Intelligent Systems,2026,21(3):792-801.[doi:10.11992/tis.202507018]
点击复制
TONG Qingyun,GONG Peiliang,ZHANG Liying,et al.Temporal attention memory network for evaluating air traffic controller cognitive load[J].CAAI Transactions on Intelligent Systems,2026,21(3):792-801.[doi:10.11992/tis.202507018]
时序注意力网络评估空管员认知负荷
《智能系统学报》[ISSN 1673-4785/CN 23-1538/TP] 卷:
21
期数:
2026年第3期
页码:
792-801
栏目:
人工智能院长论坛
出版日期:
2026-05-05
- Title:
- Temporal attention memory network for evaluating air traffic controller cognitive load
- Keywords:
- air traffic control; electroencephalography; time series analysis; attention mechanism; cognitive load evaluation; neural network; long short-term memory network; classification recognition; human-computer interaction
- 分类号:
- TP391
- 文献标志码:
- 2026-4-7
- 摘要:
- 为提升复杂工作情境下认知状态监测的准确性与实时性，本文围绕空中交通管制任务中认知负荷的识别问题，提出了一种时序注意力记忆网络(temporal attention memory network, TAM-Net)。该方法面向脑电信号的动态特征建模，融合代理注意力机制与注意力长短期记忆网络细胞单元。前者通过代理标记在保持全局建模能力的同时有效降低计算开销，后者则增强了模型对长时间序列及复杂依赖关系的记忆与表达能力。实验使用自采集的EEG数据集，结果显示，TAM-Net在两类任务中均显著优于对比模型。研究结果表明，TAM-Net能够有效捕捉复杂时序依赖特征，为认知负荷的精准监测和实时调控提供了新方法和技术支持。
- Abstract:
- To improve the accuracy and real-time performance of cognitive state monitoring in complex work scenarios, this paper focuses on cognitive load recognition in air traffic control tasks and proposes a temporal attention memory network (TAM-Net). The method models the dynamic features of electroencephalography (EEG) signals by integrating a surrogate attention mechanism and attentional long short-term memory (LSTM) units. The surrogate attention mechanism effectively reduces computational overhead while maintaining global modeling capabilities through surrogate tokens, and the attentional LSTM units enhance the model’s capacity for memory and representation of long sequences and complex dependencies. Experiments were conducted on a self-collected EEG dataset, and the results show that TAM-Net considerably outperforms comparative models in two types of tasks. These findings indicate that TAM-Net can effectively capture complex temporal dependency features, providing a new methodology and technical support for precise cognitive load monitoring and real-time regulation.
- 参考文献/References:
-
[1] DE JONG T. Cognitive load theory, educational research, and instructional design: some food for thought[J]. Instructional science, 2010, 38(2): 105-134
[2] 陈巧芬. 认知负荷理论及其发展[J]. 现代教育技术, 2007, 17(9): 16-19, 15 CHEN Qiaofen. Cognitive load theory and its development[J]. Modern educational technology, 2007, 17(9): 16-19, 15
[3] SWELLER J, VAN MERRI?NBOER J J G, PAAS F. Cognitive architecture and instructional design: 20Years later[J]. Educational psychology review, 2019, 31(2): 261-292
[4] 任立海, 聂珍龙, 于潇, 等. 基于脑电信号的不良驾驶状态识别研究综述[J]. 中国公路学报, 2024, 37(8): 216-230 REN Lihai, NIE Zhenlong, YU Xiao, et al. Review on recognition of unsatisfactory driving status based on electroencephalogram[J]. China journal of highway and transport, 2024, 37(8): 216-230
[5] ANTONENKO P, PAAS F, GRABNER R, et al. Using electroencephalography to measure cognitive load[J]. Educational psychology review, 2010, 22(4): 425-438
[6] 梁明晶, 王璐, 温昕, 等. 多特征融合的脑电情绪分类[J]. 计算机工程与应用, 2023, 59(5): 155-159 LIANG Mingjing, WANG Lu, WEN Xin, et al. EEG emotion classification based on multi-feature fusion[J]. Computer engineering and applications, 2023, 59(5): 155-159
[7] VANNESTE P, RAES A, MORTON J, et al. Towards measuring cognitive load through multimodal physiological data[J]. Cognition, technology & work, 2021, 23(3): 567-585
[8] BARUA S, AHMED M U, BEGUM S. Classifying drivers’ cognitive load using EEG signals[M]//pHealth 2017. Windsor： IOS Press, 2017: 99-106.
[9] AYARE P. Cognitive load detection for advanced driver assistance systems[D]. Canada: University of Windsor, 2019.
[10] YOO G, KIM H, HONG S. Prediction of cognitive load from electroencephalography signals using long short-term memory network[J]. Bioengineering, 2023, 10(3): 361
[11] KYNDT E, DOCHY F, STRUYVEN K, et al. The perception of workload and task complexity and its influence on students’ approaches to learning: a study in higher education[J]. European journal of psychology of education, 2011, 26(3): 393-415
[12] ELKERDAWY M, ELHALABY M, HASSAN A, et al. Building cognitive profiles of learners using EEG[C]//2020 11th International Conference on Information and Communication Systems. Piscataway: IEEE, 2020: 27-32.
[13] FRIEDMAN N, FEKETE T, GAL K, et al. EEG-based prediction of cognitive load in intelligence tests[J]. Frontiers in human neuroscience, 2019, 13: 191
[14] 许子明, 牛一帆, 温旭云, 等. 基于脑电信号的认知负荷评估综述[J]. 航天医学与医学工程, 2021, 34(4): 339-348 XU Ziming, NIU Yifan, WEN Xuyun, et al. A review of cognitive workload assessment based on EEG[J]. Space medicine & medical engineering, 2021, 34(4): 339-348
[15] 李葳宁, 韩宗昌, 邢晨光. 基于脑电信号的飞行员认知负荷实时监测评估系统[J]. 航空科学技术, 2024, 35(11): 95-103 LI Weining, HAN Zongchang, XING Chenguang. Real-time mental workload monitoring and evaluation system based on EEG signals of pilots[J]. Aeronautical science and technology, 2024, 35(11): 95-103
[16] WIEDIARTINI, CIPTOMULYONO U, DEWI R S. Physiological and biochemical measures of mental workload of air traffic controllers: a systematic literature review[C]//Proceedings of the International Conference on Industrial Engineering and Operations Management. [S.l.]: IEOM Society International, 2021: 2026-2037.
[17] HARRISON J, ?ZZETO?LU K, AYAZ H, et al. Cognitive workload and learning assessment during the implementation of a next-generation air traffic control technology using functional near-infrared spectroscopy[J]. IEEE transactions on human-machine systems, 2014, 44(4): 429-440
[18] TRUSCHZINSKI M, BETELLA A, BRUNNETT G, et al. Emotional and cognitive influences in air traffic controller tasks: an investigation using a virtual environment?[J]. Applied ergonomics, 2018, 69: 1-9
[19] 曾令李, 刘迎欣, 于扬, 等. 一种基于脑电信号特征融合的认知负荷评估方法: CN115054271B[P]. 2024-11-08.
[20] LIU Yang, SHI Shanshan, SONG Yu, et al. EEG based mental workload assessment by power spectral density feature[C]//2022 IEEE International Conference on Mechatronics and Automation. Piscataway: IEEE, 2022: 450-454.
[21] YEDUKONDALU J, SHARMA L D. Cognitive load detection using Binary salp swarm algorithm for feature selection[C]//2022 IEEE 6th Conference on Information and Communication Technology. Piscataway: IEEE, 2023: 1-5.
[22] TREMMEL C, HERFF C, SATO T, et al. Estimating cognitive workload in an interactive virtual reality environment using EEG[J]. Frontiers in human neuroscience, 2019, 13: 401
[23] 孟荣华, 李子奇, 吴正佳, 等. 基于装配任务与EEG功率信息特征的操作员认知负荷研究[J]. 机械设计, 2022, 39(11): 60-70 MENG Ronghua, LI Ziqi, WU Zhengjia, et al. Study on cognitive load of operators based on assembly task and EEG power information features[J]. Journal of machine design, 2022, 39(11): 60-70
[24] ZHAO Xinqiao, ZHANG Hongmiao, ZHU Guilin, et al. A multi-branch 3D convolutional neural network for EEG-based motor imagery classification[J]. IEEE transactions on neural systems and rehabilitation engineering, 2019, 27(10): 2164-2177
[25] JIN Lin, QU Hongquan, PANG Liping, et al. Identifying stable EEG patterns over time for mental workload recognition using transfer DS-CNN framework[J]. Biomedical signal processing and control, 2024, 89: 105662
[26] KUANAR S, ATHITSOS V, PRADHAN N, et al. Cognitive analysis of working memory load from eeg, by a deep recurrent neural network[C]//2018 IEEE International Conference on Acoustics, Speech and Signal Processing. Piscataway: IEEE, 2018: 2576-2580.
[27] LIU Y, TRAPSILAWATI F, HOU X, et al. EEG-based mental workload recognition in human factors evaluation of future air traffic control systems[M]//Transdisciplinary Engineering: A Paradigm Shift. [S.l.]: IOS Press, 2017: 357-364.
[28] ZHOU Yueying, JIANG Junji, WANG Lijun, et al. Enhanced cognitive load detection in air traffic control operators using EEG and a hybrid deep learning approach[J]. IEEE access, 2025, 13: 12127-12137
[29] BAHDANAU D, CHO K, BENGIO Y. Neural machine translation by jointly learning to align and translate[EB/OL]. (2014-09-01)[2025-01-01]. https://arxiv.org/abs/1409.0473.
[30] SHEN Zhuoran, ZHANG Mingyuan, ZHAO Haiyu, et al. Efficient attention: attention with linear complexities[C]//2021 IEEE Winter Conference on Applications of Computer Vision. Piscataway: IEEE, 2021: 3530-3538.
[31] GON?ALVES T, RIO-TORTO I, TEIXEIRA L F, et al. A survey on attention mechanisms for medical applications: are we moving toward better algorithms?[J]. IEEE access, 2022, 10: 98909-98935
[32] HAN Dongchen, YE Tianzhu, HAN Yizeng, et al. Agent Attention: On theIntegration ofSoftmax andLinear Attention[C]//Computer Vision–ECCV 2024. Cham: Springer, 2025: 124-140.
[33] BECK M, P?PPEL K, SPANRING M, et al. xLSTM: extended long short-term memory[EB/OL]. (2024-05-07)[2025-01-01]. https://arxiv.org/abs/2405.04517.
[34] CORTES C, VAPNIK V. Support-vector networks[J]. Machine learning, 1995, 20(3): 273-297
[35] LECUN Y, BOSER B, DENKER J S, et al. Backpropagation applied to handwritten zip code recognition[J]. Neural computation, 1989, 1(4): 541-551
[36] SCHIRRMEISTER R T, SPRINGENBERG J T, FIEDERER L D J, et al. Deep learning with convolutional neural networks for EEG decoding and visualization[J]. Human brain mapping, 2017, 38(11): 5391-5420
[37] MANE R, CHEW E, CHUA K, et al. FBCNet: a multi-view convolutional neural network for brain-computer interface[EB/OL]. (2021-05-17)[2025-01-01]. https://arxiv.org/abs/2104.01233.
[38] HOCHREITER S, SCHMIDHUBER J. Long short-term memory[J]. Neural computation, 1997, 9(8): 1735-1780
[39] LAWHERN V J, SOLON A J, WAYTOWICH N R, et al. EEGNet: a compact convolutional neural network for EEG-based brain–computer interfaces[J]. Journal of neural engineering, 2018, 15(5): 056013
[40] DE OLIVEIRA I H, RODRIGUES A C. Empirical comparison of deep learning methods for EEG decoding[J]. Frontiers in neuroscience, 2023, 16: 1003984
[41] SAFARI M, SHALBAF R, BAGHERZADEH S, et al. Classification of mental workload using brain connectivity and machine learning on electroencephalogram data[J]. Scientific reports, 2024, 14: 9153
[42] GHASIMI A, SHAMEKHI S. Enhanced EEG-based cognitive workload detection using RADWT and machine learning[J]. Neuroscience, 2025, 569: 231-244
- 相似文献/References:
-
[1]王斐,张育中,宁廷会,等.脑-机接口研究进展[J].智能系统学报,2011,6(3):189.
WANG Fei,ZHANG Yuzhong,NING Tinghui,et al.Research progress in a braincomputer interface[J].CAAI Transactions on Intelligent Systems,2011,6():189.
[2]张毅,罗明伟,罗元.脑电信号的小波变换和样本熵特征提取方法[J].智能系统学报,2012,7(4):339.
ZHANG Yi,LUO Mingwei,LUO Yuan.EEG feature extraction method based on wavelet transform and sample entropy[J].CAAI Transactions on Intelligent Systems,2012,7():339.
[3]杨建平,刘明华,吕敬祥,等.低觉醒脑电识别与唤醒的可穿戴系统研究[J].智能系统学报,2019,14(4):787.[doi:10.11992/tis.201806047]
YANG Jianping,LIU Minghua,LYU Jingxiang,et al.A wearable system to recognize and awaken low-arousal state[J].CAAI Transactions on Intelligent Systems,2019,14():787.[doi:10.11992/tis.201806047]
[4]李景聪,潘伟健,林镇远,等.采用多路图注意力网络的情绪脑电信号识别方法[J].智能系统学报,2022,17(3):531.[doi:10.11992/tis.202107003]
LI Jingcong,PAN Weijian,LIN Zhenyuan,et al.Emotional EEG signal recognition method using multi-path graph attention network[J].CAAI Transactions on Intelligent Systems,2022,17():531.[doi:10.11992/tis.202107003]
[5]蒋云良,周阳,张雄涛,等.基于域间Mixup微调策略的跨被试运动想象脑电信号分类算法[J].智能系统学报,2024,19(4):909.[doi:10.11992/tis.202208017]
JIANG Yunliang,ZHOU Yang,ZHANG Xiongtao,et al.Cross-subject motor imagery EEG classification based on inter-domain Mixup fine-tuning strategy[J].CAAI Transactions on Intelligent Systems,2024,19():909.[doi:10.11992/tis.202208017]
[6]蒋云良,印泽宗,张雄涛,等.高阶Takagi-Sugeno-Kang模糊知识蒸馏分类器及其在脑电信号分类中的应用[J].智能系统学报,2024,19(6):1419.[doi:10.11992/tis.202307029]
JIANG Yunliang,YIN Zezong,ZHANG Xiongtao,et al.TSK fuzzy distillation classifier with negative Euclidean probability and High-order fuzzy dark knowledge transfer and its application on EEG signals classification[J].CAAI Transactions on Intelligent Systems,2024,19():1419.[doi:10.11992/tis.202307029]
备注/Memo
收稿日期:2025-6-22。
基金项目:国家自然科学基金项目(62276130，62136004，62406131)；国家重点研发计划项目(2023YFF1204803)；江苏省重点研究发展计划项目(BE2022842).
作者简介:童庆耘，硕士研究生，主要研究方向为时间序列分析、脑机接口与深度学习。E-mail：qingyuntong@nuaa.edu.cn。;公沛良，博士研究生，主要研究方向为时间序列分析、脑机接口与深度学习。E-mail：plgong@nuaa.edu.cn。;张道强，教授，博士，南京航空航天大学人工智能学院院长，IEEE高级会员，主要研究方向为机器学习、模式识别、数据挖掘和医学图像分析。在Nature Communications、IEEE TMI/TPAMI/TIP、中国科学、NeurIPS、CVPR、KDD、MICCAI等重要国内外期刊和会议发表学术论文200余篇。连续12年入选 Elsevier 中国高被引学者榜，论文被引用20000余次，其中有3篇论文的单篇引用均超1000次。E-mail：dqzhang@nuaa.edu.cn。
通讯作者:张道强. E-mail：dqzhang@nuaa.edu.cn
更新日期/Last Update:
1900-01-01