[1]蒋云良,余梅丽,金森洋,等.基于深度模糊知识蒸馏的多变量时间序列预测模型[J].智能系统学报,2026,21(3):639-650.[doi:10.11992/tis.202508031]
JIANG Yunliang,YU Meili,JIN Senyang,et al.Multivariate time series forecasting model based on deep fuzzy knowledge distillation[J].CAAI Transactions on Intelligent Systems,2026,21(3):639-650.[doi:10.11992/tis.202508031]
点击复制
JIANG Yunliang,YU Meili,JIN Senyang,et al.Multivariate time series forecasting model based on deep fuzzy knowledge distillation[J].CAAI Transactions on Intelligent Systems,2026,21(3):639-650.[doi:10.11992/tis.202508031]
基于深度模糊知识蒸馏的多变量时间序列预测模型
《智能系统学报》[ISSN 1673-4785/CN 23-1538/TP] 卷:
21
期数:
2026年第3期
页码:
639-650
栏目:
学术论文—机器学习
出版日期:
2026-05-05
- Title:
- Multivariate time series forecasting model based on deep fuzzy knowledge distillation
- Keywords:
- attention mechanism; multivariate time series; knowledge distillation; time series forecasting; TSK fuzzy system; teacher-bounded loss; deep learning; temporal attention; spatial attention
- 分类号:
- TP181
- 文献标志码:
- 2026-3-6
- 摘要:
- 多变量时间序列在交通流量、气象监测等领域广泛存在，其特征间存在复杂的时空依赖关系和高度不确定性，传统机器学习模型难以有效捕获潜在模式。尽管近年来的深度学习方法在预测精度上取得了显著提升，但往往依赖于庞大的网络结构与高计算开销，限制了在实时或资源受限场景中的应用。为解决以上问题，提出了一种新的用于时序预测的轻量级深度模糊知识蒸馏模型(Takagi-Sugeno-Kang with deep fuzzy knowledge distillation, TSK-DFKD)。具有强大表达能力的教师模型将深度暗知识迁移到轻量级的学生模型，从而降低预测成本。在学生模型中，首次利用具有不确定知识处理能力的模糊推理网络，有效应对时序数据的不确定性。在蒸馏过程中，引入了教师有界损失来替代传统的交叉熵损失，以实现知识蒸馏下的高效时序数据预测。在5个公开数据集上进行了实验，与9个最先进的基线模型相比，本文所提方法预测性能更优，效率更高。
- Abstract:
- Multivariate time series are common in areas such as traffic flow and weather monitoring. Their features show complex spatiotemporal dependencies and high uncertainty. Traditional machine learning models cannot effectively capture these hidden patterns. In recent years, deep learning methods have improved prediction accuracy, but they often rely on large network structures and high computational costs, which limit their use in real-time or resource-limited settings. To address these issues, a novel lightweight deep fuzzy knowledge distillation framework (TSK-DFKD, Takagi-Sugeno-Kang with deep fuzzy knowledge distillation) is proposed for time series forecasting. By transferring deep dark knowledge from a powerful representative teacher model to a lightweight student model, prediction costs can be reduced. The student model utilizes a fuzzy reasoning network, which has strong capabilities in handling uncertain knowledge, to effectively tackle uncertainties in time series data. During distillation, a teacher’s bounded loss is introduced in place of traditional cross-entropy loss, enabling efficient knowledge distillation in time series data prediction. Experiments conducted on five open datasets demonstrate that TSK-DFKD outperforms nine state-of-the-art baselines in prediction performance and efficiency.
- 参考文献/References:
-
[1] 陈梅, 柳博雅, 王钰, 等. 基于时间序列形态的模糊聚类算法[J]. 控制与决策, 2025, 40(4): 1116-1126 CHEN Mei, LIU Boya, WANG Yu, et al. Fuzzy clustering algorithm based on time series morphology[J]. Control and decision, 2025, 40(4): 1116-1126
[2] 胡磊, 韩敏. 基于核共轭梯度演化模糊系统的混沌时间序列在线预测[J]. 控制与决策, 2024, 39(9): 3099-3107 HU Lei, HAN Min. Online prediction of chaotic time series based on kernel conjugate gradient evolving fuzzy system[J]. Control and decision, 2024, 39(9): 3099-3107
[3] SHIH S Y, SUN Fankeng, LEE H Y. Temporal pattern attention for multivariate time series forecasting[J]. Machine learning, 2019, 108(8): 1421-1441
[4] ZHANG Yuxin, CHEN Yiqiang, WANG Jindong, et al. Unsupervised deep anomaly detection for multi-sensor time-series signals[J]. IEEE transactions on knowledge and data engineering, 2023, 35(2): 2118-2132
[5] STANKEVICIūT? K, ALAA A M, VAN DER SCHAAR M. Conformal time-series forecasting[C]//Neural Information Processing Systems. online: NeurIPS. 2021: 6216-6228.
[6] ZHOU Erhao, VONG C M, NOJIMA Y, et al. A fully interpretable first-order TSK fuzzy system and its training with negative entropic and rule-stability-based regularization[J]. IEEE transactions on fuzzy systems, 2023, 31(7): 2305-2319
[7] BIAN Zekang, ZHANG Jin, NOJIMA Y, et al. Hybrid-ensemble-based interpretable TSK fuzzy classifier for imbalanced data[J]. Information fusion, 2023, 98: 101845
[8] 张雄涛, 李水苗, 翁江玮, 等. 基于视角-规则的深度TSK模糊分类器及其在多元癫痫脑电信号识别中的应用[J]. 控制与决策, 2024, 39(4): 1315-1324 ZHANG Xiongtao, LI Shuimiao, WENG Jiangwei, et al. Recognition of multivariate epilepsy EEG signals based on view-to-rule deep TSK fuzzy classifier[J]. Control and decision, 2024, 39(4): 1315-1324
[9] 蒋云良, 印泽宗, 张雄涛, 等. 高阶Takagi-Sugeno-Kang模糊知识蒸馏分类器及其在脑电信号分类中的应用[J]. 智能系统学报, 2024, 19(6): 1419-1427 JIANG Yunliang, YIN Zezong, ZHANG Xiongtao, et al. TSK fuzzy distillation classifier with negative Euclidean probability and High-order fuzzy dark knowledge transfer and its application on EEG signals classification[J]. CAAI transactions on intelligent systems, 2024, 19(6): 1419-1427
[10] 张雄涛, 陈天宇, 赵康, 等. 基于多教师自适应知识蒸馏的TSK模糊分类器[J]. 智能系统学报, 2025, 20(5): 1136-1147 ZHANG Xiongtao, CHEN Tianyu, ZHAO Kang, et al. TSK fuzzy classifier based on multi-teacher adaptive knowledge distillation[J]. CAAI transactions on intelligent systems, 2025, 20(5): 1136-1147
[11] JIANG Jiawei, HAN Chengkai, ZHAO W X, et al. PDFormer: propagation delay-aware dynamic long-range transformer for traffic flow prediction[J]. Proceedings of the AAAI conference on artificial intelligence, 2023, 37(4): 4365-4373
[12] PALEYES A, URMA R G, LAWRENCE N D. Challenges in deploying machine learning: a survey of case studies[J]. ACM computing surveys, 2022, 55(6): 1-29
[13] QIU Yaner, MA Liyun, PRIYADARSHI R. Deep learning challenges and prospects in wireless sensor network deployment[J]. Archives of computational methods in engineering, 2024, 31(6): 3231-3254
[14] CHEN Guobin, CHOI W, YU Xiang, et al. Learning efficient object detection models with knowledge distillation[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems. New York: ACM, 2017: 742-751.
[15] XU Qing, CHEN Zhenghua, WU Keyu, et al. KDnet-RUL: a knowledge distillation framework to compress deep neural networks for machine remaining useful life prediction[J]. IEEE transactions on industrial electronics, 2022, 69(2): 2022-2032
[16] LI Ying, LI Ping, YAN Doudou, et al. Deep knowledge distillation: a self-mutual learning framework for traffic prediction[J]. Expert systems with applications, 2024, 252: 124138
[17] YANG Xue, YANG Xiaojiang, YANG Jirui, et al. Learning high-precision bounding box for rotated object detection via kullback-leibler divergence[C]//Neural Information Processing Systems. online: NeurIPS. 2021: 18381-19394.
[18] HE Kaiming, ZHANG Xiangyu, REN Shaoqing, et al. Deep residual learning for image recognition[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2016: 770-778.
[19] XU Jingjing, SUN Xu, ZHANG Zhiyuan, et al. Understanding and improving layer normalization[C]//Neural Information Processing Systems.Vancouver: NeurIPS. 2019: 32.
[20] ZHANG Yuanpeng, WANG Guanjin, HUANG Xiuyu, et al. TSK fuzzy system fusion at sensitivity-ensemble-level for imbalanced data classification[J]. Information fusion, 2023, 92: 350-362
[21] JIANG Yunliang, WENG Jiangwei, ZHANG Xiongtao, et al. A CNN-based born-again TSK fuzzy classifier integrating soft label information and knowledge distillation[J]. IEEE transactions on fuzzy systems, 2023, 31(6): 1843-1854
[22] DENG Zhaohong, CHOI K S, CHUNG F L, et al. Scalable TSK fuzzy modeling for very large datasets using minimal-enclosing-ball approximation[J]. IEEE transactions on fuzzy systems, 2011, 19(2): 210-226
[23] BOTEV Z I, KROESE D P, RUBINSTEIN R Y, et al. The cross-entropy method for optimization[M]//Handbook of Statistics-Machine Learning: Theory and Applications. Amsterdam: Elsevier, 2013: 35-59.
[24] WU Wei, FAN Qinwei, ZURADA J M, et al. Batch gradient method with smoothing regularization for training of feedforward neural networks[J]. Neural networks, 2014, 50: 72-78
[25] REYAD M, SARHAN A M, ARAFA M. A modified Adam algorithm for deep neural network optimization[J]. Neural computing and applications, 2023, 35(23): 17095-17112
[26] MARDANI M, SUN Qingyun, VASAWANALA S, et al. Neural proximal gradient descent for compressive imaging[C]//Proceedings of the 32nd International Conference on Neural Information Processing Systems. New York: ACM, 2018: 9596-9606.
[27] ZENG Ailing, CHEN Muxi, ZHANG Lei, et al. Are transformers effective for time series forecasting?[J]. Proceedings of the AAAI conference on artificial intelligence, 2023, 37(9): 11121-11128
[28] DAS A, KONG Weihao, LEACH A, et al. Long-term forecasting with TiDE: time-series dense encoder[EB/OL]. (2023-04-17)[2025-01-01]. https://arxiv.org/abs/2304.08424.
[29] WU H, HU T, LIU Y, et al. Timesnet: Temporal 2d-variation modeling for general time series analysis[C]//Proceedings of the International Conference on Learning Representations. Kigali, 2023.
[30] ZHOU Haoyi, ZHANG Shanghang, PENG Jieqi, et al. Informer: beyond efficient transformer for long sequence time-series forecasting[J]. Proceedings of the AAAI conference on artificial intelligence, 2021, 35(12): 11106-11115
[31] ZHOU Tian, MA Ziqing, WANG Xue, et al. Fedformer: frequency enhanced decomposed transformer for long-term series forecasting[M]//AI for Time Series. Boca Raton: CRC Press, 2026: 10-34.
[32] NIE Yuqi, NGUYEN N H, SINTHONG P, et al. A time series is worth 64 words: long-term forecasting with transformers[EB/OL]. (2022-11-27)[2025-01-01]. https://arxiv.org/abs/2211.14730.
[33] ZHANG Yunhao, YAN Junchi. Crossformer: transformer utilizing cross-dimension dependency for multivariate time series forecasting[C]//International Conference on Learning Representations, 2023.
[34] LIU Yong, HU Tengge, ZHANG Haoran, et al. iTransformer: inverted transformers are effective forecasting[C]//Proceedings of International Conference on Learning Representations.Vienna： ICLR，2024.
- 相似文献/References:
-
[1]申凯,王晓峰,杨亚东.基于双向消息链路卷积网络的显著性物体检测[J].智能系统学报,2019,14(6):1152.[doi:10.11992/tis.201812003]
SHEN Kai,WANG Xiaofeng,YANG Yadong.Salient object detection based on bidirectional message link convolution neural network[J].CAAI Transactions on Intelligent Systems,2019,14():1152.[doi:10.11992/tis.201812003]
[2]赵文清,程幸福,赵振兵,等.注意力机制和Faster RCNN相结合的绝缘子识别[J].智能系统学报,2020,15(1):92.[doi:10.11992/tis.201907023]
ZHAO Wenqing,CHENG Xingfu,ZHAO Zhenbing,et al.Insulator recognition based on attention mechanism and Faster RCNN[J].CAAI Transactions on Intelligent Systems,2020,15():92.[doi:10.11992/tis.201907023]
[3]申翔翔,侯新文,尹传环.深度强化学习中状态注意力机制的研究[J].智能系统学报,2020,15(2):317.[doi:10.11992/tis.201809033]
SHEN Xiangxiang,HOU Xinwen,YIN Chuanhuan.State attention in deep reinforcement learning[J].CAAI Transactions on Intelligent Systems,2020,15():317.[doi:10.11992/tis.201809033]
[4]曾碧卿,韩旭丽,王盛玉,等.层次化双注意力神经网络模型的情感分析研究[J].智能系统学报,2020,15(3):460.[doi:10.11992/tis.201812017]
ZENG Biqing,HAN Xuli,WANG Shengyu,et al.Hierarchical double-attention neural networks for sentiment classification[J].CAAI Transactions on Intelligent Systems,2020,15():460.[doi:10.11992/tis.201812017]
[5]莫宏伟,田朋.基于注意力融合的图像描述生成方法[J].智能系统学报,2020,15(4):740.[doi:10.11992/tis.201910039]
MO Hongwei,TIAN Peng.An image caption generation method based on attention fusion[J].CAAI Transactions on Intelligent Systems,2020,15():740.[doi:10.11992/tis.201910039]
[6]鲍维克,袁春.面向推荐系统的分期序列自注意力网络[J].智能系统学报,2021,16(2):353.[doi:10.11992/tis.202005028]
BAO Weike,YUAN Chun.Recommendation system with long-term and short-term sequential self-attention network[J].CAAI Transactions on Intelligent Systems,2021,16():353.[doi:10.11992/tis.202005028]
[7]洪恺临,曹江涛,姬晓飞.改进Center-Net网络的自主喷涂机器人室内窗户检测[J].智能系统学报,2021,16(3):425.[doi:10.11992/tis.202005016]
HONG Kailin,CAO Jiangtao,JI Xiaofei.Indoor window detection of autonomous spraying robot based on improved CenterNet network[J].CAAI Transactions on Intelligent Systems,2021,16():425.[doi:10.11992/tis.202005016]
[8]张勇,高大林,巩敦卫,等.用于关系抽取的注意力图长短时记忆神经网络[J].智能系统学报,2021,16(3):518.[doi:10.11992/tis.202008036]
ZHANG Yong,GAO Dalin,GONG Dunwei,et al.Attention graph long short-term memory neural network for relation extraction[J].CAAI Transactions on Intelligent Systems,2021,16():518.[doi:10.11992/tis.202008036]
[9]陈新元,谢晟祎,陈庆强,等.结合卷积特征提取和路径语义的知识推理[J].智能系统学报,2021,16(4):729.[doi:10.11992/tis.202008007]
CHEN Xinyuan,XIE Shengyi,CHEN Qingqiang,et al.Knowledge-based inference on convolutional feature extraction and path semantics[J].CAAI Transactions on Intelligent Systems,2021,16():729.[doi:10.11992/tis.202008007]
[10]张恒,何文玢,何军,等.医学知识增强的肿瘤分期多任务学习模型[J].智能系统学报,2021,16(4):739.[doi:10.11992/tis.202010005]
ZHANG Heng,HE Wenbin,HE Jun,et al.Multi-task tumor stage learning model with medical knowledge enhancement[J].CAAI Transactions on Intelligent Systems,2021,16():739.[doi:10.11992/tis.202010005]
备注/Memo
收稿日期:2025-8-26。
基金项目:国家自然科学基金项目(62376094)；国家自然科学基金区域创新发展联合基金重点支持项目(U22A20102)；浙江全省智能教育技术与应用重点实验室开放研究基金项目(2025ZNJYKF003).
作者简介:蒋云良，教授，博士生导师，博士，主要研究方向为深度学习、智慧交通、智慧医疗和智能教育。先后主持和参与国家和省部级科研项目13项。发表学术论文63篇，出版学术著作2部，授权发明专利26项。E-mail：jyl@zjhu.edu.cn。;余梅丽，硕士研究生，主要研究方向为模糊系统、深度学习。E-mail：1937365006@qq.com。;张雄涛，副教授，博士，主要研究方向为深度学习、模糊系统、智慧交通和智慧医疗。E-mail：1047897965@qq.com。
通讯作者:张雄涛. E-mail：1047897965@qq.com
更新日期/Last Update:
1900-01-01