[1]JIANG Yunliang,YU Meili,JIN Senyang,et al.Multivariate time series forecasting model based on deep fuzzy knowledge distillation[J].CAAI Transactions on Intelligent Systems,2026,21(3):639-650.[doi:10.11992/tis.202508031]
Copy

Multivariate time series forecasting model based on deep fuzzy knowledge distillation

CAAI Transactions on Intelligent Systems[ISSN 1673-4785/CN 23-1538/TP] Volume: 21 Number of periods: 2026 3 Page number: 639-650 Column: 学术论文—机器学习 Public date: 2026-05-05
Title:
Multivariate time series forecasting model based on deep fuzzy knowledge distillation
Author(s):
JIANG Yunliang1; 2; 3;  YU Meili1; 2;  JIN Senyang1; 2;  SHEN Qing1; 2;  ZHANG Xiongtao1; 2
1. School of Information Engineering, Huzhou Normal University, Huzhou 313000, China;
2. Zhejiang Key Laboratory of Intelligent Education Technology and Application, Jinhua 321004, China;
3. School of Computer Science and Technology, Zhejiang Normal University, Jinhua 321004, China
Keywords:
attention mechanism; multivariate time series; knowledge distillation; time series forecasting; TSK fuzzy system; teacher-bounded loss; deep learning; temporal attention; spatial attention
CLC:
TP181
DOI:
10.11992/tis.202508031
Abstract:
Multivariate time series are common in areas such as traffic flow and weather monitoring. Their features show complex spatiotemporal dependencies and high uncertainty. Traditional machine learning models cannot effectively capture these hidden patterns. In recent years, deep learning methods have improved prediction accuracy, but they often rely on large network structures and high computational costs, which limit their use in real-time or resource-limited settings. To address these issues, a novel lightweight deep fuzzy knowledge distillation framework (TSK-DFKD, Takagi-Sugeno-Kang with deep fuzzy knowledge distillation) is proposed for time series forecasting. By transferring deep dark knowledge from a powerful representative teacher model to a lightweight student model, prediction costs can be reduced. The student model utilizes a fuzzy reasoning network, which has strong capabilities in handling uncertain knowledge, to effectively tackle uncertainties in time series data. During distillation, a teacher’s bounded loss is introduced in place of traditional cross-entropy loss, enabling efficient knowledge distillation in time series data prediction. Experiments conducted on five open datasets demonstrate that TSK-DFKD outperforms nine state-of-the-art baselines in prediction performance and efficiency.
References:
[1] 陈梅, 柳博雅, 王钰, 等. 基于时间序列形态的模糊聚类算法[J]. 控制与决策, 2025, 40(4): 1116-1126 CHEN Mei, LIU Boya, WANG Yu, et al. Fuzzy clustering algorithm based on time series morphology[J]. Control and decision, 2025, 40(4): 1116-1126
[2] 胡磊, 韩敏. 基于核共轭梯度演化模糊系统的混沌时间序列在线预测[J]. 控制与决策, 2024, 39(9): 3099-3107 HU Lei, HAN Min. Online prediction of chaotic time series based on kernel conjugate gradient evolving fuzzy system[J]. Control and decision, 2024, 39(9): 3099-3107
[3] SHIH S Y, SUN Fankeng, LEE H Y. Temporal pattern attention for multivariate time series forecasting[J]. Machine learning, 2019, 108(8): 1421-1441
[4] ZHANG Yuxin, CHEN Yiqiang, WANG Jindong, et al. Unsupervised deep anomaly detection for multi-sensor time-series signals[J]. IEEE transactions on knowledge and data engineering, 2023, 35(2): 2118-2132
[5] STANKEVICIūT? K, ALAA A M, VAN DER SCHAAR M. Conformal time-series forecasting[C]//Neural Information Processing Systems. online: NeurIPS. 2021: 6216-6228.
[6] ZHOU Erhao, VONG C M, NOJIMA Y, et al. A fully interpretable first-order TSK fuzzy system and its training with negative entropic and rule-stability-based regularization[J]. IEEE transactions on fuzzy systems, 2023, 31(7): 2305-2319
[7] BIAN Zekang, ZHANG Jin, NOJIMA Y, et al. Hybrid-ensemble-based interpretable TSK fuzzy classifier for imbalanced data[J]. Information fusion, 2023, 98: 101845
[8] 张雄涛, 李水苗, 翁江玮, 等. 基于视角-规则的深度TSK模糊分类器及其在多元癫痫脑电信号识别中的应用[J]. 控制与决策, 2024, 39(4): 1315-1324 ZHANG Xiongtao, LI Shuimiao, WENG Jiangwei, et al. Recognition of multivariate epilepsy EEG signals based on view-to-rule deep TSK fuzzy classifier[J]. Control and decision, 2024, 39(4): 1315-1324
[9] 蒋云良, 印泽宗, 张雄涛, 等. 高阶Takagi-Sugeno-Kang模糊知识蒸馏分类器及其在脑电信号分类中的应用[J]. 智能系统学报, 2024, 19(6): 1419-1427 JIANG Yunliang, YIN Zezong, ZHANG Xiongtao, et al. TSK fuzzy distillation classifier with negative Euclidean probability and High-order fuzzy dark knowledge transfer and its application on EEG signals classification[J]. CAAI transactions on intelligent systems, 2024, 19(6): 1419-1427
[10] 张雄涛, 陈天宇, 赵康, 等. 基于多教师自适应知识蒸馏的TSK模糊分类器[J]. 智能系统学报, 2025, 20(5): 1136-1147 ZHANG Xiongtao, CHEN Tianyu, ZHAO Kang, et al. TSK fuzzy classifier based on multi-teacher adaptive knowledge distillation[J]. CAAI transactions on intelligent systems, 2025, 20(5): 1136-1147
[11] JIANG Jiawei, HAN Chengkai, ZHAO W X, et al. PDFormer: propagation delay-aware dynamic long-range transformer for traffic flow prediction[J]. Proceedings of the AAAI conference on artificial intelligence, 2023, 37(4): 4365-4373
[12] PALEYES A, URMA R G, LAWRENCE N D. Challenges in deploying machine learning: a survey of case studies[J]. ACM computing surveys, 2022, 55(6): 1-29
[13] QIU Yaner, MA Liyun, PRIYADARSHI R. Deep learning challenges and prospects in wireless sensor network deployment[J]. Archives of computational methods in engineering, 2024, 31(6): 3231-3254
[14] CHEN Guobin, CHOI W, YU Xiang, et al. Learning efficient object detection models with knowledge distillation[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems. New York: ACM, 2017: 742-751.
[15] XU Qing, CHEN Zhenghua, WU Keyu, et al. KDnet-RUL: a knowledge distillation framework to compress deep neural networks for machine remaining useful life prediction[J]. IEEE transactions on industrial electronics, 2022, 69(2): 2022-2032
[16] LI Ying, LI Ping, YAN Doudou, et al. Deep knowledge distillation: a self-mutual learning framework for traffic prediction[J]. Expert systems with applications, 2024, 252: 124138
[17] YANG Xue, YANG Xiaojiang, YANG Jirui, et al. Learning high-precision bounding box for rotated object detection via kullback-leibler divergence[C]//Neural Information Processing Systems. online: NeurIPS. 2021: 18381-19394.
[18] HE Kaiming, ZHANG Xiangyu, REN Shaoqing, et al. Deep residual learning for image recognition[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2016: 770-778.
[19] XU Jingjing, SUN Xu, ZHANG Zhiyuan, et al. Understanding and improving layer normalization[C]//Neural Information Processing Systems.Vancouver: NeurIPS. 2019: 32.
[20] ZHANG Yuanpeng, WANG Guanjin, HUANG Xiuyu, et al. TSK fuzzy system fusion at sensitivity-ensemble-level for imbalanced data classification[J]. Information fusion, 2023, 92: 350-362
[21] JIANG Yunliang, WENG Jiangwei, ZHANG Xiongtao, et al. A CNN-based born-again TSK fuzzy classifier integrating soft label information and knowledge distillation[J]. IEEE transactions on fuzzy systems, 2023, 31(6): 1843-1854
[22] DENG Zhaohong, CHOI K S, CHUNG F L, et al. Scalable TSK fuzzy modeling for very large datasets using minimal-enclosing-ball approximation[J]. IEEE transactions on fuzzy systems, 2011, 19(2): 210-226
[23] BOTEV Z I, KROESE D P, RUBINSTEIN R Y, et al. The cross-entropy method for optimization[M]//Handbook of Statistics-Machine Learning: Theory and Applications. Amsterdam: Elsevier, 2013: 35-59.
[24] WU Wei, FAN Qinwei, ZURADA J M, et al. Batch gradient method with smoothing regularization for training of feedforward neural networks[J]. Neural networks, 2014, 50: 72-78
[25] REYAD M, SARHAN A M, ARAFA M. A modified Adam algorithm for deep neural network optimization[J]. Neural computing and applications, 2023, 35(23): 17095-17112
[26] MARDANI M, SUN Qingyun, VASAWANALA S, et al. Neural proximal gradient descent for compressive imaging[C]//Proceedings of the 32nd International Conference on Neural Information Processing Systems. New York: ACM, 2018: 9596-9606.
[27] ZENG Ailing, CHEN Muxi, ZHANG Lei, et al. Are transformers effective for time series forecasting?[J]. Proceedings of the AAAI conference on artificial intelligence, 2023, 37(9): 11121-11128
[28] DAS A, KONG Weihao, LEACH A, et al. Long-term forecasting with TiDE: time-series dense encoder[EB/OL]. (2023-04-17)[2025-01-01]. https://arxiv.org/abs/2304.08424.
[29] WU H, HU T, LIU Y, et al. Timesnet: Temporal 2d-variation modeling for general time series analysis[C]//Proceedings of the International Conference on Learning Representations. Kigali, 2023.
[30] ZHOU Haoyi, ZHANG Shanghang, PENG Jieqi, et al. Informer: beyond efficient transformer for long sequence time-series forecasting[J]. Proceedings of the AAAI conference on artificial intelligence, 2021, 35(12): 11106-11115
[31] ZHOU Tian, MA Ziqing, WANG Xue, et al. Fedformer: frequency enhanced decomposed transformer for long-term series forecasting[M]//AI for Time Series. Boca Raton: CRC Press, 2026: 10-34.
[32] NIE Yuqi, NGUYEN N H, SINTHONG P, et al. A time series is worth 64 words: long-term forecasting with transformers[EB/OL]. (2022-11-27)[2025-01-01]. https://arxiv.org/abs/2211.14730.
[33] ZHANG Yunhao, YAN Junchi. Crossformer: transformer utilizing cross-dimension dependency for multivariate time series forecasting[C]//International Conference on Learning Representations, 2023.
[34] LIU Yong, HU Tengge, ZHANG Haoran, et al. iTransformer: inverted transformers are effective forecasting[C]//Proceedings of International Conference on Learning Representations.Vienna： ICLR，2024.
Similar References:

Memo

-

Last Update: 1900-01-01

Copyright © CAAI Transactions on Intelligent Systems