[1]范航舟,梅红岩,赵勤,等.融合双注意力机制的GNN多维时间序列预测[J].智能系统学报,2024,19(5):1277-1286.[doi:10.11992/tis.202305020]
FAN Hangzhou,MEI Hongyan,ZHAO Qin,et al.Multivariate time series forecasting with a graph neural network and dual attention mechanism[J].CAAI Transactions on Intelligent Systems,2024,19(5):1277-1286.[doi:10.11992/tis.202305020]
点击复制
FAN Hangzhou,MEI Hongyan,ZHAO Qin,et al.Multivariate time series forecasting with a graph neural network and dual attention mechanism[J].CAAI Transactions on Intelligent Systems,2024,19(5):1277-1286.[doi:10.11992/tis.202305020]
融合双注意力机制的GNN多维时间序列预测
《智能系统学报》[ISSN 1673-4785/CN 23-1538/TP] 卷:
19
期数:
2024年第5期
页码:
1277-1286
栏目:
学术论文—人工智能基础
出版日期:
2024-09-05
- Title:
- Multivariate time series forecasting with a graph neural network and dual attention mechanism
- Keywords:
- multivariate time series forecasting; graph neural network; attention mechanism; feature fusion; temporal convolutional network; deep learning; convolutional neural network; spatiotemporal features
- 分类号:
- TP391
- 文献标志码:
- 2024-08-28
- 摘要:
- 针对现有多维时间序列数据(multivariate time series, MTS)预测中变量间依赖关系捕获能力不足和时间序列数据多通道信息利用不充分的问题，提出一种融合双注意力机制的多维时间序列预测模型(feature fusion and dual attention mechanism based GNN, FFDA-GNN)。该模型将图神经网络与空间注意力机制融合，用于增强多变量之间依赖关系捕获能力；利用并行的多层膨胀卷积和通道注意力机制，对时间序列数据进行多通道的特征提取，实现对时间序列数据多通道信息的充分利用，从而提升预测性能。在经济、电力、交通3个领域数据集上与基准模型进行对比实验，该模型预测精度优于其他基准方法，有良好的可行性。
- Abstract:
- To address the issues of insufficient capture of dependency relationships between variables and inadequate utilization of multichannel temporal information in multivariate time series, a forecasting model fused with a dual attention mechanism (FFDA-GNN) is proposed. This model integrates a graph neural network with a spatial attention mechanism to enhance the capture of dependencies among multiple variables. Additionally, parallel multilayer inflation convolution and channel attention mechanisms are used for multi-channel feature extraction from temporal data. This approach fully utilizes multi-channel information and enhances forecasting performance. Comparative experiments conducted on datasets from the fields of economics, electric power, and traffic show that the proposed model achieves better prediction accuracy than other benchmark methods and demonstrates superior feasibility.
- 参考文献/References:
-
[1] GASPARIN A, LUKOVIC S, ALIPPI C. Deep learning for time series forecasting: The electric load case[J]. CAAI transactions on intelligence technology, 2022, 7(1): 1-25.
[2] 王锋华, 成敬周, 文凡. 快速双非凸回归算法及其电力数据预测应用[J]. 智能系统学报, 2018, 13(4): 665-672.
WANG Fenghua, CHENG Jingzhou, WEN Fan. Fast double nonconvex regression algorithm for forecast of electric power data[J]. CAAI transactions on intelligent systems, 2018, 13(4): 665-672.
[3] 程鹏超, 杜军平, 薛哲. 基于多路交叉的用户金融行为预测[J]. 智能系统学报, 2021, 16(2): 378-384.
CHENG Pengchao, DU Junping, XUE Zhe. Prediction of user financial behavior based on multi-way crossing[J]. CAAI transactions on intelligent systems, 2021, 16(2): 378-384.
[4] FANG Weiwei, ZHUO Wenhao, YAN Jingwen, et al. Attention meets long short-term memory: a deep learning network for traffic flow forecasting[J]. Physica A: statistical mechanics and its applications, 2022, 587: 126485.
[5] 李伯涵, 郭茂祖, 赵玲玲. 基于分割注意力机制残差网络的城市区域客流量预测[J]. 智能系统学报, 2022, 17(4): 839-848.
LI Bohan, GUO Maozu, ZHAO Lingling. Passenger flow prediction in urban areas based on residual networks with split attention mechanism[J]. CAAI transactions on intelligent systems, 2022, 17(4): 839-848.
[6] YULE G U. On a method of investigating periodicities disturbed series, with special reference to Wolfer’s sunspot numbers[J]. Philosophical transactions of the royal society of London series A, containing papers of a mathematical or physical character, 1927, 226(1): 267-298.
[7] WALKER G T. On periodicity in series of related terms[J]. Proceedings of the royal society of London series A, containing papers of a mathematical and physical character, 1931, 131(818): 518-532.
[8] JENKINS G M, BOX G E P. Time series analysis: forecasting, and control[J]. Journal of time, 1976, 31(2): 238-242.
[9] 杨海民, 潘志松, 白玮. 时间序列预测方法综述[J]. 计算机科学, 2019, 46(1): 21-28.
YANG Haimin, PAN Zhisong, BAI Wei. Review of time series prediction methods[J]. Computer science, 2019, 46(1): 21-28.
[10] 陈艳, 王子健, 赵泽, 等. 传感器网络环境监测时间序列数据的高斯过程建模与多步预测[J]. 通信学报, 2015, 36(10): 252-262.
CHEN Yan, WANG Zijian, ZHAO Ze, et al. Gaussian process modeling and multi-step prediction for time series data in wireless sensor network environmental monitoring[J]. Journal on communications, 2015, 36(10): 252-262.
[11] LI Peixian, TAN Zhixiang, YAN Lili, et al. Time series prediction of mining subsidence based on a SVM[J]. Mining science and technology (China), 2011, 21(4): 557-562.
[12] GOSSé J B, GUILLAUMIN C. L’apport de la représentation VAR de Christopher A. Sims à la science économique[J]. L’économique en perspective, 2014, 89(4): 305-319.
[13] ELMAN J L. Finding structure in time[J]. Cognitive science, 1990, 14(2): 179-211.
[14] HOCHREITER S, SCHMIDHUBER J. Long short-term memory[J]. Neural computation, 1997, 9(8): 1735-1780.
[15] CHUNG J, GULCEHRE C, CHO K, et al. Empirical evaluation of gated recurrent neural networks on sequence modeling[EB/OL]. (2014-12-11)[2023-05-20]. https://arxiv.org/abs/1412.3555.
[16] QIN Yao, SONG Dongjin, CHEN Haifeng, et al. A dual-stage attention-based recurrent neural network for time series prediction[EB/OL]. (2017-04-07)[2023-05-20]. https://arxiv.org/abs/1704.02971.
[17] LAI Guokun, CHANG Weicheng, YANG Yiming, et al. Modeling long and short-term temporal patterns with deep neural networks[C]//The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval. Ann Arbor: ACM, 2018: 95-104.
[18] HUANG Siteng, WANG Donglin, WU Xuehan, et al. DSANet: dual self-attention network for multivariate time series forecasting[C]//Proceedings of the 28th ACM International Conference on Information and Knowledge Management. Beijing: ACM, 2019: 2129-2132.
[19] BAI Shaojie, KOLTER J Z, KOLTUN V, et al. An empirical evaluation of generic convolutional and recurrent networks for sequence modeling[EB/OL]. (2018-03-04)[2023-05-20]. https://arxiv.org/abs/1803.01271.
[20] LIU Minhao, ZENG Ailing, CHEN Muxi, et al. SCINet: Time series modeling and forecasting with sample convolution and interaction[J]. Advances in neural information processing systems, 2022, 35: 5816-5828.
[21] VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems. California: ACM, 2017: 6000–6010.
[22] WU N, GREEN B, BEN Xue, et al. Deep transformer models for time series forecasting: the influenza prevalence case[EB/OL]. (2020-01-23)[2023-05-20]. https://arxiv.org/abs/2001.08317.
[23] LI Shiyang, JIN Xiaoyong, XUAN Yao, et al. Enhancing the locality and breaking the memory bottleneck of transformer on time series forecasting[EB/OL]. (2019-07-29)[2023-05-20]. https://arxiv.org/abs/1907.00235.
[24] ZHOU Haoyi, ZHANG Shanghang, PENG Jieqi, et al. Informer: beyond efficient transformer for long sequence time-series forecasting[J]. Proceedings of the AAAI conference on artificial intelligence, 2021, 35(12): 11106-11115.
[25] WU Haixu, XU Jiehui, WANG Jianmin, et al. Autoformer: decomposition transformers with auto-correlation for long-term series forecasting[J]. Advances in neural information processing systems, 2021, 34: 22419-22430.
[26] ZHOU Tian, MA Ziqing, WEN Qingsong, et al. FEDformer: frequency enhanced decomposed transformer for long-term series forecasting[EB/OL]. (2022-01-30)[2023-05-20]. https://arxiv.org/abs/2201.12740.
[27] 吴博, 梁循, 张树森, 等. 图神经网络前沿进展与应用[J]. 计算机学报, 2022, 45(1): 35-68.
WU Bo, LIANG Xun, ZHANG Shusen, et al. Advances and applications in graph neural network[J]. Chinese journal of computers, 2022, 45(1): 35-68.
[28] WU Zonghan, PAN Shirui, LONG Guodong, et al. Graph WaveNet for deep spatial-temporal graph modeling[EB/OL]. (2019-05-31)[2023-05-20]. https://arxiv.org/abs/1906.00121.
[29] SATORRAS V G, RANGAPURAM S S, JANUSCHOWSKI T. Multivariate time series forecasting with latent graph inference[EB/OL]. (2022-03-07)[2023-05-20]. https://arxiv.org/abs/2203.03423.
[30] WU Zonghan, PAN Shirui, LONG Guodong, et al. Connecting the dots: multivariate time series forecasting with graph neural networks[C]//Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. Virtual Event: ACM, 2020: 753-763.
[31] WOO S, PARK J, LEE J Y, et al. CBAM: convolutional block attention module[C]//Lecture Notes in Computer Science. Cham: Springer, 2018: 3-19.
[32] ZHANG G P. Time series forecasting using a hybrid ARIMA and neural network model[J]. Neurocomputing, 2003, 50: 159-175.
[33] JIA Pengtao, LIU Hangduo, WANG Sujian, et al. Research on a mine gas concentration forecasting model based on a GRU network[J]. IEEE access, 2020, 8: 38023-38031.
[34] YOO J, KANG U. Attention-based autoregression for accurate and efficient multivariate time series forecasting[C]//Proceedings of the 2021 SIAM International Conference on Data Mining. Philadelphia: Society for Industrial and Applied Mathematics, 2021: 531-539.
- 相似文献/References:
-
[1]吴国栋,查志康,涂立静,等.图神经网络推荐研究进展[J].智能系统学报,2020,15(1):14.[doi:10.11992/tis.201908034]
WU Guodong,ZHA Zhikang,TU Lijing,et al.Research advances in graph neural network recommendation[J].CAAI Transactions on Intelligent Systems,2020,15():14.[doi:10.11992/tis.201908034]
[2]张勇,高大林,巩敦卫,等.用于关系抽取的注意力图长短时记忆神经网络[J].智能系统学报,2021,16(3):518.[doi:10.11992/tis.202008036]
ZHANG Yong,GAO Dalin,GONG Dunwei,et al.Attention graph long short-term memory neural network for relation extraction[J].CAAI Transactions on Intelligent Systems,2021,16():518.[doi:10.11992/tis.202008036]
[3]周俊,张志强,曹月恬,等.地理位置和时间感知的表示学习框架[J].智能系统学报,2021,16(5):909.[doi:10.11992/tis.202104011]
ZHOU Jun,ZHANG Zhiqiang,CAO Yuetian,et al.A geography and time aware representation learning framework[J].CAAI Transactions on Intelligent Systems,2021,16():909.[doi:10.11992/tis.202104011]
[4]卢毅,陈亚冉,赵冬斌,等.关键点图对比图像分类方法[J].智能系统学报,2023,18(1):36.[doi:10.11992/tis.202112001]
LU Yi,CHEN Yaran,ZHAO Dongbin,et al.Keypoint-based graph contrastive neural network for image classification[J].CAAI Transactions on Intelligent Systems,2023,18():36.[doi:10.11992/tis.202112001]
[5]车飞虎,张大伟,邵朋朋,等.基于四元数门控图神经网络的脚本事件预测[J].智能系统学报,2023,18(1):138.[doi:10.11992/tis.202203042]
CHE Feihu,ZHANG Dawei,SHAO Pengpeng,et al.Script event prediction based on a quaternion-gated graph neural network[J].CAAI Transactions on Intelligent Systems,2023,18():138.[doi:10.11992/tis.202203042]
[6]孟祥福,温晶,李子函,等.多重注意力指导下的异构图嵌入方法[J].智能系统学报,2023,18(4):688.[doi:10.11992/tis.202204006]
MENG Xiangfu,WEN Jing,LI Zihan,et al.Heterogeneous graph embedding method guided by the multi-attention mechanism[J].CAAI Transactions on Intelligent Systems,2023,18():688.[doi:10.11992/tis.202204006]
[7]周贤琛,马扬,程光权,等.战场目标实体类型识别的鲁棒图神经网络方法[J].智能系统学报,2023,18(6):1156.[doi:10.11992/tis.202111025]
ZHOU Xianchen,MA Yang,CHENG Guangquan,et al.Robust graph neural network method for target entity type recognition in a battlefield environment[J].CAAI Transactions on Intelligent Systems,2023,18():1156.[doi:10.11992/tis.202111025]
[8]曹汉童,陈璟.融合Doc2vec与GCN的多类型蛋白质相互作用预测方法[J].智能系统学报,2023,18(6):1165.[doi:10.11992/tis.202212029]
CAO Hantong,CHEN Jing.Prediction of multitype protein interactions combining Doc2vec and GCN[J].CAAI Transactions on Intelligent Systems,2023,18():1165.[doi:10.11992/tis.202212029]
[9]刘晓光,李梅.基于深度学习的药物-靶标相互作用预测研究综述[J].智能系统学报,2024,19(3):494.[doi:10.11992/tis.202308024]
LIU Xiaoguang,LI Mei.A survey of deep learning-based drug-target interaction prediction[J].CAAI Transactions on Intelligent Systems,2024,19():494.[doi:10.11992/tis.202308024]
[10]王闪闪,巩长庆,秦华锋,等.基于深度学习的K近邻图迭代静脉识别算法研究[J].智能系统学报,2024,19(5):1149.[doi:10.11992/tis.202307009]
WANG Shanshan,GONG Changqing,QIN Huafeng,et al.Vein recognition algorithm combining K-nearest neighbor and graph iterative based on deep learning[J].CAAI Transactions on Intelligent Systems,2024,19():1149.[doi:10.11992/tis.202307009]
备注/Memo
收稿日期:2023-5-20。
基金项目:辽宁省教育厅科学研究项目(JZL202015404，LJKZ0625，JYTMS20230869).
作者简介:范航舟，硕士研究生，主要研究方向为时序序列预测、大数据分析、深度学习。E-mail：fanhangzhou@163.com;梅红岩，教授，博士，中国计算机学会会员，主要研究方向为深度学习、数据挖掘与大数据分析、网络服务。发表学术论文40余篇。E-mail：715014795@qq.com;赵勤，硕士研究生，主要研究方向为时序序列预测、深度学习。E-mail：1139574265@qq.com。
通讯作者:梅红岩. E-mail：715014795@qq.com
更新日期/Last Update:
2024-09-05