[1]周俊,张志强,曹月恬,等.地理位置和时间感知的表示学习框架[J].智能系统学报,2021,16(5):909-917.[doi:10.11992/tis.202104011]
ZHOU Jun,ZHANG Zhiqiang,CAO Yuetian,et al.A geography and time aware representation learning framework[J].CAAI Transactions on Intelligent Systems,2021,16(5):909-917.[doi:10.11992/tis.202104011]
点击复制
ZHOU Jun,ZHANG Zhiqiang,CAO Yuetian,et al.A geography and time aware representation learning framework[J].CAAI Transactions on Intelligent Systems,2021,16(5):909-917.[doi:10.11992/tis.202104011]
地理位置和时间感知的表示学习框架
《智能系统学报》[ISSN 1673-4785/CN 23-1538/TP] 卷:
16
期数:
2021年第5期
页码:
909-917
栏目:
吴文俊人工智能科技进步奖一等奖
出版日期:
2021-09-05
- Title:
- A geography and time aware representation learning framework
- Keywords:
- semantic-space-time mode; time modelling; spatial modelling; attention mechanism; graph learning; graph neural network; user behavior modeling; user behavior representation
- 分类号:
- TP183
- 摘要:
- 现有时空感知的表示学习框架无法对强时空语义的实际场景存在的“When”、“Where”和“What”3个问题给出一个统一的解决方案。同时，现有的时间和空间建模上的研究方案也存在着一定的缺陷，无法在复杂的实际场景中取得最优的性能。为了解决这些问题，本文提出了一个统一的用户表示框架—GTRL (geography and time aware representation learning)，可以同时在时间和空间的维度上对用户的历史行为轨迹进行联合建模。在时间建模上，GTRL采用函数式的时间编码以及连续时间和上下文感知的图注意力网络，在动态的用户行为图上灵活地捕获高阶的结构化时序信息。在空间建模上，GTRL采用了层级化的地理编码和深度历史轨迹建模模块高效地刻画了用户的地理位置偏好。GTRL设计了统一的联合优化方案，同时在交互预测、交互时间预测以及交互位置3个任务上进行模型学习。最后，本文在公开数据集和工业数据集上设计了大量的实验，分别验证了GTRL相较学术界基线模型的优势，以及在实际业务场景中的有效性。
- Abstract:
- The existing geography and time aware representation learning framework fails to give a unified solution to three problems of “When”, “Where” and “What” in real-world scenarios with strong geography and time aware semantics, and the existing research schemes for time and spatial modeling also have certain defects and thus cannot achieve optimal performance in complex actual scenarios. To fill in these gaps, this paper proposes a unified user representation framework—GTRL (geography and time aware representation learning), which is capable of jointly modeling users’ historical behaviors in both time and spatial dimensions. In terms of time modeling, GTRL adopts a functional time encoding and a continuous time and context aware graph attention network to flexibly exploit high-order and structured temporal information from dynamic user behaviour graphs. In terms of spatial modeling, GTRL uses a hierarchical geography encoding and a deep historical trajectory modelling module to effectively depict users’ preference for geographical location. Moreover, a unified optimization strategy is carefully designed to carry out model learning on three tasks of interaction prediction task, interaction time stamp prediction task and interaction geography prediction. At last, extensive experiments on public and industrial data sets are carried out, demonstrating the advantages of GTUL over the academic baseline model and its effectiveness in actual business scenarios, respectively.
- 参考文献/References:
-
[1] DU Lun, WANG Yun, SONG Guojie, et al. Dynamic network embedding: an extended approach for skip-gram based network embedding[C]//Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence (IJCAI-18). Stockholm, Sweden, 2018: 2086-2092.
[2] SINGER U, GUY I, RADINSKY K. Node embedding over temporal graphs[C]//Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence. Macao, China, 2019: 4605-4612.
[3] GUO Shengnan, LIN Youfang, FENG Ning, et al. Attention based spatial-temporal graph convolutional networks for traffic flow forecasting[J]. Proceedings of the AAAI conference on artificial intelligence, 2019, 33(1): 922-929.
[4] YANG Shuo, ZHANG Zhiqiang, ZHOU Jun, et al. Financial risk analysis for SMEs with graph-based supply chain mining[C]//Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence (IJCAI-20). Yokohama, Japan, 2020: 4661-4667.
[5] XU Da, RUAN Chuanwei, KORPEOGLU E, et al. Inductive representation learning on temporal graphs[C]//2020 International Conference on Learning Representations. Addis Ababa, Ethiopia, 2020: 1-19.
[6] HUANG Hong, FANG Zixuan, WANG Xiao, et al. Motif-preserving temporal network embedding[C]//Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence (IJCAI-20). Yokohama, Japan, 2020: 1237-1243.
[7] MINH H Q, NIYOGI P, YAO Yuan. Mercer’s theorem, feature maps, and smoothing[C]//Proceedings of the 19th Annual Conference on Learning Theory (COLT 2006). Pittsburgh, USA, 2006: 154-168.
[8] LIAN Defu, WU Yongji, GE Yong, et al. Geography-aware sequential location recommendation[C]//Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. Virtual Event, USA, 2020: 2009-2019.
[9] OMI T, UEDA N, AIHARA K. Fully neural network based model for general temporal point processes[C]//Proceedings of the 33st International Conference on Neural Information Processing Systems. Vancouver, Canada, 2019: 2120-2129.
[10] ZHU Yu, LI Hao, LIAO Yikang, et al. What to do next: modeling user behaviors by time-LSTM[C]//Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence (IJCAI-17). Melbourne, Australia, 2017: 3602-3608.
[11] KUMAR S, ZHANG Xikun, LESKOVEC J. Predicting dynamic embedding trajectory in temporal interaction networks[C]//Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. Anchorage, USA, 2019: 1269-1278.
[12] HAMILTON W L, YING R, LESKOVEC J. Inductive representation learning on large graphs[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems. Long Beach, CA, USA, 2017: 1025-1035.
[13] VELI?KOVI? P, CUCURULL G, CASANOVA A, et al. Graph attention networks[C]//The 6th International Conference on Learning Representations. Vancouver, BC, Canada, 2018: 1-12.
[14] NGUYEN G H, LEE J B, ROSSI R A, et al. Continuous-time dynamic network embeddings[C]//Companion Proceedings of the the Web Conference 2018. Lyon, France, 2018: 969-976.
[15] LU Yuanfu, WANG Xiao, SHI Chuan, et al. Temporal network embedding with micro-and macro-dynamics[C]//Proceedings of the 28th ACM International Conference on Information and Knowledge Management. Beijing, China, 2019: 469-478.
[16] SEO Y, DEFFERRARD M, VANDERGHEYNST P, et al. Structured sequence modeling with graph convolutional recurrent networks[C]//Processing of the 25th International Conference (ICONIP 2018). Siem Reap, Cambodia, 2018: 362-373.
[17] GUO Huifeng, TANG Ruiming, YE Yunming, et al. DeepFM: a factorization-machine based neural network for CTR prediction[C]//Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence (IJCAI-17). Melbourne, Australia, 2017: 1725-1731.
[18] HIDASI B, KARATZOGLOU A, BALTRUNAS L, et al. Session-based recommendations with recurrent neural networks[C]//International Conference on Learning Representations. San Juan, Puerto Rico, 2016: 1-10.
[19] ZHOU Guorui, ZHU Xiaoqiang, SONG Chenru, et al. Deep interest network for click-through rate prediction[C]//Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. London, United Kingdom, 2018: 1059-1068.
[20] KANG Wangcheng, MCAULEY J. Self-attentive sequential recommendation[C]//2018 IEEE International Conference on Data Mining (ICDM). Singapore, Singapore, 2018: 197-206.
[21] ZHOU Guorui, MOU Na, FAN Ying, et al. Deep interest evolution network for click-through rate prediction[J]. Proceedings of the AAAI conference on artificial intelligence, 2019, 33(1): 5941–5948.
[22] MA Jiaqi, ZHAO Zhe, YI Xinyang, et al. Modeling task relationships in multi-task learning with multi-gate mixture-of-experts[C]//Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. London, United Kingdom, 2018: 1930-1939.
[23] HU Binbin, ZHANG Zhiqiang, ZHOU Jun, et al. Loan default analysis with multiplex graph learning[C]//Proceedings of the 29th ACM International Conference on Information & Knowledge Management. 2020: 2525-2532.
[24] WANG D, ZHANG Z, ZHOU J, et al. Temporal-aware graph neural network for credit risk prediction[C]//Proceedings of the 2021 SIAM International Conference on Data Mining (SDM). 2021: 702-710.
[25] HU B, ZHANG Z, SHI C, et al. Cash-out user detection based on attributed heterogeneous information network with a hierarchical attention mechanism[C]//Proceedings of the AAAI Conference on Artificial Intelligence. Hawaii, USA, 2019: 946-953.
备注/Memo
收稿日期:2021-04-07。
基金项目:国家重点研发计划项目（2018YFB1403001）
作者简介:周俊，蚂蚁集团研究员，主要研究方向为机器学习，获得了第十届吴文俊人工智能科技进步奖一等奖(第一完成人)。发表学术论文40余篇;张志强，蚂蚁集团高级算法专家，主要研究方向为图机器学习，发表学术论文20余篇;郑小林，教授，博士生导师，主要研究方向为人工智能、智能电商、金融智能、服务计算。主持国家自然科学基金重点项目等多项。发表学术论文50余篇
通讯作者:郑小林.E-mail:xlzheng@zju.edu.cn
更新日期/Last Update:
1900-01-01