[1]陈容珊,高淑萍,齐小刚.注意力机制和图卷积神经网络引导的谱聚类方法[J].智能系统学报,2023,18(5):936-944.[doi:10.11992/tis.202208041]
CHEN Rongshan,GAO Shuping,QI Xiaogang.A spectral clustering based on GCNs and attention mechanism[J].CAAI Transactions on Intelligent Systems,2023,18(5):936-944.[doi:10.11992/tis.202208041]
点击复制
CHEN Rongshan,GAO Shuping,QI Xiaogang.A spectral clustering based on GCNs and attention mechanism[J].CAAI Transactions on Intelligent Systems,2023,18(5):936-944.[doi:10.11992/tis.202208041]
注意力机制和图卷积神经网络引导的谱聚类方法
《智能系统学报》[ISSN 1673-4785/CN 23-1538/TP] 卷:
18
期数:
2023年第5期
页码:
936-944
栏目:
学术论文—机器学习
出版日期:
2023-09-05
- Title:
- A spectral clustering based on GCNs and attention mechanism
- Keywords:
- deep learning; spectral clustering; image classification; graph convolutional neural networks; attention mechanism; convolutional neural networks; image segmentation; deep clustering
- 分类号:
- TP391.1
- 摘要:
- 经典的谱聚类算法需要计算图拉普拉斯矩阵的特征分解，代价昂贵，学者们将深度学习模型引入谱聚类算法。然而，现有的方法存在一定的局限性。针对现有聚类模型忽略了节点间的关联度信息，导致聚类结果不准确；当图神经网络中的节点数目发生变化时，需要更新整张图，耗费大量存储空间的问题，本文将注意力机制与改进的图卷积神经网络架构相结合，提出了一种基于注意力和图卷积神经网络的谱聚类方法。该方法主要利用注意力机制引导节点聚类，然后建立相应的目标，通过训练神经网络计算出目标最优时对应的聚类分配，并在图重构过程中利用注意力信息和拓扑信息双重引导，从而提升重构的精确度。实验结果显示，本文提出的方法在图分类、图聚类及图重构中具有良好性能。
- Abstract:
- The classical spectral clustering algorithm often involves computationally expensive feature decomposition of the Laplacian matrix, prompting researchers to introduce a deep learning model into the spectral clustering algorithm. Nonetheless, the existing methods possess certain limitations. The existing clustering model neglects the information on the correlation degree between nodes, which leads to inaccurate clustering results. Additionally, when the number of nodes in the graph neural network changes, the whole graph requires updating, leading to extensive storage space consumption. By combining the attention mechanism with the improved architecture of graph convolutional neural networks, this paper proposes a spectral clustering method based on the attention and graph convolutional neural network to address these issues. The proposed method mainly utilizes the attention mechanism to guide node clustering, following which it establishes the corresponding target. The corresponding cluster allocation, when the target is optimal, is calculated by training the neural network. Attention information and topological information are used to guide the graph reconstruction process, thereby improving the accuracy of reconstruction. Experimental results reveal that the proposed method exhibits excellent performance in graph classification, graph clustering, and graph reconstruction tasks.
- 参考文献/References:
-
[1] SHI Na, LIU Xumin, GUAN Yong. Research on k-means clustering algorithm: an improved k-means clustering algorithm[C]//2010 Third International Symposium on Intelligent Information Technology and Security Informatics. Jian: IEEE, 2010: 63?67.
[2] FU Chuanyi, CHEN Huandong, XING Jieqing. Spectral clustering and its research progress[C]//2011 Seventh International Conference on Computational Intelligence and Security: IEEE, 2012: 1367?1369.
[3] AGARWAL P, ALAM M A, BISWAS R. A hierarchical clustering algorithm for categorical attributes[C]//2010 Second International Conference on Computer Engineering and Applications. Bali: IEEE, 2010: 365?368.
[4] STUDER L, WALLAU J, INGOLD R, et al. Effects of graph pooling layers on classification with graph neural networks[C]//2020 7th Swiss Conference on Data Science. Luzern: IEEE, 2020: 57?58.
[5] GAMA F, ISUFI E, LEUS G, et al. Graphs, convolutions, and neural networks: from graph filters to graph neural networks[C]//IEEE Signal Processing Magazine. [S.l.]: IEEE, 2020: 128?138.
[6] KHAN A, SOHAIL A, ZAHOORA U, et al. A survey of the recent architectures of deep convolutional neural networks[J]. Artificial intelligence review, 2020, 53(8): 5455-5516.
[7] BOULEMNADJEL A, HACHOUF F. An improved algorithm for subspace clustering applied to image segmentation[C]//2012 16th International Conference on Information Visualisation. Montpellier: IEEE, 2012: 297?301.
[8] XIA Zhile, LI Junmin, LI Jiangrong. Passivity-based resilient adaptive control for fuzzy stochastic delay systems with Markovian switching[J]. Journal of the franklin institute, 2014, 351(7): 3818-3836.
[9] YUAN Gonglin, XUE Bing, ZHANG Mengjie. A graph-based approach to automatic convolutional neural network construction for image classification[C]//2020 35th International Conference on Image and Vision Computing New Zealand. Wellington: IEEE, 2020: 1?6.
[10] WU Zonghan, PAN Shirui, CHEN Fengwen, et al. A comprehensive survey on graph neural networks[J]. IEEE transactions on neural networks and learning systems, 2021, 32(1): 4-24.
[11] LI Yang, ZHANG Jiangzhou, NIE Mingyu, et al. Hessian-regularized spectral clustering for behavior recognition[C]//2020 International Conference on Intelligent Computing and Human-Computer Interaction. Sanya: IEEE, 2021: 156?159.
[12] GAO Jianliang, GAO Jun, YING Xiaoting, et al. Higher-order interaction goes neural: a substructure assembling graph attention network for graph classification[J]. IEEE transactions on knowledge and data engineering, 2023, 35(2): 1594-1608.
[13] LI Qiuyan, SHANG Yanlei, QIAO Xiuquan, et al. Heterogeneous dynamic graph attention network[C]//2020 IEEE International Conference on Knowledge Graph (ICKG). Nanjing: IEEE, 2020: 404?411.
[14] PENG Xi, FENG Jiashi, ZHOU J T, et al. Deep subspace clustering[J]. IEEE transactions on neural networks and learning systems, 2020, 31(12): 5509-5521.
[15] XU Xiaoliang, GAO Tong, WANG Yuxiang, et al. Event temporal relation extraction with attention mechanism and graph neural network[J]. Tsinghua science and technology, 2021, 27(1): 79-90.
[16] KIM M, KIM T, KIM D. Spatio-temporal slowfast self-attention network for action recognition[C]//2020 IEEE International Conference on Image Processing. Abu Dhabi: IEEE, 2020: 2206?2210.
[17] ZHANG Xuming, SUN Genyun, JIA Xiuping, et al. Spectral-spatial self-attention networks for hyperspectral image classification[J]. IEEE transactions on geoscience and remote sensing, 2022, 60: 1-15.
[18] GU Ruijun, WANG Jiacai. An improved spectral clustering algorithm based on neighbour adaptive scale[C]//2009 International Conference on Business Intelligence and Financial Engineering. Beijing: IEEE, 2009: 233?236.
[19] NGO M, MAI An, BUI T. On an improvement of Graph Convolutional Network in semi-supervised learning[C]//2020 7th NAFOSTED Conference on Information and Computer Science (NICS). Ho Chi Minh City: IEEE, 2021: 114?117.
[20] RUIZ L, GAMA F, RIBEIRO A. Graph neural networks: architectures, stability, and transferability[J]. Proceedings of the IEEE, 2021, 109(5): 660-682.
[21] PENG Hongrui. Graph convolutional networks: an analysis of method and applications in different fields and systems[C]//2020 International Conference on Communications, Information System and Computer Engineering (CISCE). Kuala Lumpur: IEEE, 2020: 153?157.
[22] CAO Jing, XU Yi, SONG Xuening. First-order meta-learning in node classification with graph convolutional network[C]//2021 4th International Conference on Artificial Intelligence and Big Data. Chengdu: IEEE, 2021: 443?447.
[23] XIAO Yi, CHEN Siying, WANG Hao, et al. HSGACN: hyperspectral image classification algorithm based on graph convolutional network[C]//2021 IEEE International Geoscience and Remote Sensing Symposium IGARSS. Brussels: IEEE, 2021: 3677?3680.
[24] CAI Chengtao, GUO Dongning. CNN-self-attention-DNN architecture for mandarin recognition[C]//2020 Chinese Control and Decision Conference. Hefei: IEEE, 2020: 1190?1194.
[25] UEDA T, OKADA M, MORI N, et al. A method to estimate request sentences using LSTM with self-attention mechanism[C]//2019 8th International Congress on Advanced Applied Informatics. Toyama: IEEE, 2020: 7?10.
[26] HU Bin, GUO Kehua, WANG Xiaokang, et al. RRL-GAT: graph attention network-driven multilabel image robust representation learning[J]. IEEE Internet of Things journal, 2022, 9(12): 9167-9178.
[27] TIAN Long, DU Qian, KOPRIVA I, et al. Spatial-spectral based multi-view low-rank sparse sbuspace clustering for hyperspectral imagery[C]//IEEE International Geoscience and Remote Sensing Symposium. Valencia: IEEE, 2018: 8488?8491.
[28] LI He, ZHANG Shiyu, SU Liangcai, et al. GraphSANet: a graph neural network and self attention based approach for spatial temporal prediction in sensor network[C]//2020 IEEE International Conference on Big Data. Atlanta: IEEE, 2021: 5756?5758.
[29] ZHANG Jintao, XU Quan. Attention-aware heterogeneous graph neural network[J]. Big data mining and analytics, 2021, 4(4): 233-241.
[30] BERIKOV V. Autoencoder-based low-rank spectral ensemble clustering of biological data[C]//2020 Cognitive Sciences, Genomics and Bioinformatics. Novosibirsk: IEEE, 2020: 43?46.
[31] SHANTHAMALLU U S, THIAGARAJAN J J, SPANIAS A. A regularized attention mechanism for graph attention networks[C]//2020 IEEE International Conference on Acoustics, Speech and Signal Processing. Barcelona: IEEE, 2020: 3372?3376.
[32] CAO Meng, MA Xiying, ZHU Kai, et al. Heterogeneous information network embedding with convolutional graph attention networks[C]//2020 International Joint Conference on Neural Networks. Glasgow: IEEE, 2020: 1?8.
[33] 张铭泉, 周辉, 曹锦纲. 基于注意力机制的双BERT有向情感文本分类研究[J]. 智能系统学报, 2022, 17(6): 1220-1227
ZHANG Mingquan, ZHOU Hui, CAO Jingang. Dual BERT directed sentiment text classification based on attention mechanism[J]. CAAI transactions on intelligent systems, 2022, 17(6): 1220-1227
[34] HUANG Shuning, OTA K, DONG Mianxiong, et al. MultiSpectralNet: spectral clustering using deep neural network for multi-view data[J]. IEEE transactions on computational social systems, 2019, 6(4): 749-760.
[35] GüZEL K, KUR?UN O. Improving spectral clustering using path-based connectivity[C]//2015 23nd Signal Processing and Communications Applications Conference. Malatya: IEEE, 2015: 2110?2113.
[36] 于润羽, 李雅文, 李昂. 融合领域特征的科技学术会议语义相似性计算方法[J]. 智能系统学报, 2022, 17(4): 737-743
YU Runyu, LI Yawen, LI Ang. Semantic similarity computing for scientific and technological conferences[J]. CAAI transactions on intelligent systems, 2022, 17(4): 737-743
[37] 马甜甜, 杨长春, 严鑫杰, 等. 融合知识图谱和轻量级图卷积网络推荐系统的研究[J]. 智能系统学报, 2022, 17(4): 721-727
MA Tiantian, YANG Changchun, YAN Xinjie, et al. Research on the fusion of knowledge graph and lightweight graph convolutional network recommendation system[J]. CAAI transactions on intelligent systems, 2022, 17(4): 721-727
[38] LI Hongmin, YE Xiucai, IMAKURA A, et al. Ensemble learning for spectral clustering[C]//2020 IEEE International Conference on Data Mining. Sorrento: IEEE, 2021: 1094?1099.
[39] BIANCHI F M, GRATTAROLA D, ALIPPI C. Mincut pooling in graph neural networks[EB/OL]. (2020?12?29)[2022?01?01]. https://arxiv.org/abs/1907.00481v2.
[40] 杨梦茵, 陈俊芬, 翟俊海. 非对称卷积编码器的聚类算法[J]. 智能系统学报, 2022, 17(5): 900-907
YANG Mengyin, CHEN Junfen, ZHAI Junhai. A clustering method based on the asymmetric convolutional autoencoder[J]. CAAI transactions on intelligent systems, 2022, 17(5): 900-907
[41] SANDFELDER D, VIJAYAN P, HAMILTON W L. Ego-GNNs: exploiting ego structures in graph neural networks[C]//2021 IEEE International Conference on Acoustics, Speech and Signal Processing. Toronto: IEEE, 2021: 8523?8527.
[42] LU Runkun, LIU Jianwei, ZUO Xin. Attentive multi-view deep subspace clustering net[J]. Neurocomputing, 2021, 435: 186-196.
[43] PENG Zhihao, LIU Hui, JIA Yuheng, et al. Deep attention-guided graph clustering with dual self-supervision[J]. IEEE transactions on circuits and systems for video technology, 2022: 1.
[44] DONG Huihui, MA Wenping, JIAO Licheng, et al. A multiscale self-attention deep clustering for change detection in SAR images[J]. IEEE transactions on geoscience and remote sensing, 2022, 60: 1-16.
[45] MIN Erxue, GUO Xifeng, LIU Qiang, et al. A survey of clustering with deep learning: from the perspective of network architecture[J]. IEEE access, 2018, 6: 39501-39514.
[46] XU K, HU WEIHUA, LESKOVEC J, et al. How powerful are graph neural networks? [EB/OL]. (2019?02?22)[2022?01?01]. https://arxiv.org/abs/1810.00826.
[47] PETAR V, GUILLEM C, ARANTXA C, et al. Graph attention networks[EB/OL]. (2018?02?04)[2022?01?01].https://arxiv.org/abs/1710.10903.
[48] LI Meng, HSU W, XIE Xiaodong, et al. SACNN: self-attention convolutional neural network for low-dose CT denoising with self-supervised perceptual loss network[J]. IEEE transactions on medical imaging, 2020, 39(7): 2289-2301.
- 相似文献/References:
-
[1]汪? 中,刘贵全,陈恩红.基于模糊K-harmonic means的谱聚类算法[J].智能系统学报,2009,4(2):95.
WANG Zhong,LIU Gui-quan,CHEN En-hong.A spectral clustering algorithm based on fuzzy Kharmonic means[J].CAAI Transactions on Intelligent Systems,2009,4():95.
[2]张伟伟,薄 华,王晓峰.多特征-谱聚类的SAR图像溢油分割[J].智能系统学报,2010,5(6):551.
ZHANG Wei-wei,BO Hua,WANG Xiao-feng.SAR oil spill image segmentationbased on a multispectral clustering algorithm[J].CAAI Transactions on Intelligent Systems,2010,5():551.
[3]张媛媛,霍静,杨婉琪,等.深度信念网络的二代身份证异构人脸核实算法[J].智能系统学报,2015,10(2):193.[doi:10.3969/j.issn.1673-4785.201405060]
ZHANG Yuanyuan,HUO Jing,YANG Wanqi,et al.A deep belief network-based heterogeneous face verification method for the second-generation identity card[J].CAAI Transactions on Intelligent Systems,2015,10():193.[doi:10.3969/j.issn.1673-4785.201405060]
[4]丁科,谭营.GPU通用计算及其在计算智能领域的应用[J].智能系统学报,2015,10(1):1.[doi:10.3969/j.issn.1673-4785.201403072]
DING Ke,TAN Ying.A review on general purpose computing on GPUs and its applications in computational intelligence[J].CAAI Transactions on Intelligent Systems,2015,10():1.[doi:10.3969/j.issn.1673-4785.201403072]
[5]马晓,张番栋,封举富.基于深度学习特征的稀疏表示的人脸识别方法[J].智能系统学报,2016,11(3):279.[doi:10.11992/tis.201603026]
MA Xiao,ZHANG Fandong,FENG Jufu.Sparse representation via deep learning features based face recognition method[J].CAAI Transactions on Intelligent Systems,2016,11():279.[doi:10.11992/tis.201603026]
[6]刘帅师,程曦,郭文燕,等.深度学习方法研究新进展[J].智能系统学报,2016,11(5):567.[doi:10.11992/tis.201511028]
LIU Shuaishi,CHENG Xi,GUO Wenyan,et al.Progress report on new research in deep learning[J].CAAI Transactions on Intelligent Systems,2016,11():567.[doi:10.11992/tis.201511028]
[7]林大华,杨利锋,邓振云,等.稀疏样本自表达子空间聚类算法[J].智能系统学报,2016,11(5):696.[doi:10.11992/tis.201601005]
LIN Dahua,YANG Lifeng,DENG Zhenyun,et al.Sparse sample self-representation for subspace clustering[J].CAAI Transactions on Intelligent Systems,2016,11():696.[doi:10.11992/tis.201601005]
[8]马世龙,乌尼日其其格,李小平.大数据与深度学习综述[J].智能系统学报,2016,11(6):728.[doi:10.11992/tis.201611021]
MA Shilong,WUNIRI Qiqige,LI Xiaoping.Deep learning with big data: state of the art and development[J].CAAI Transactions on Intelligent Systems,2016,11():728.[doi:10.11992/tis.201611021]
[9]王亚杰,邱虹坤,吴燕燕,等.计算机博弈的研究与发展[J].智能系统学报,2016,11(6):788.[doi:10.11992/tis.201609006]
WANG Yajie,QIU Hongkun,WU Yanyan,et al.Research and development of computer games[J].CAAI Transactions on Intelligent Systems,2016,11():788.[doi:10.11992/tis.201609006]
[10]黄心汉.A3I:21世纪科技之光[J].智能系统学报,2016,11(6):835.[doi:10.11992/tis.201605022]
HUANG Xinhan.A3I: the star of science and technology for the 21st century[J].CAAI Transactions on Intelligent Systems,2016,11():835.[doi:10.11992/tis.201605022]
备注/Memo
收稿日期:2022-8-27。
基金项目:国家自然科学基金项目(61877067，91338115)；近地面探测技术重点实验室开放基金项目(6142414211503)；重点实验室稳定支持项目(6142414200511)；高等学校学科创新引智基地“111”计划(B08038).
作者简介:陈容珊， 硕士研究生，主要研究方向为深度学习、深度聚类;高淑萍，教授，博士，主要研究方向为数学与信息技术交叉研究、大数据分析与处理。曾获陕西省科学技术二等奖。主持、参与国家与省级基金项目、横向项目等16项，发表学术论文40余篇;齐小刚，教授，博士生导师，主要研究方向为复杂系统建模与仿真、网络算法设计与应用。申请专利 47 项 (授权 19 项)，登记软件著作权 4 项。发表学术论文 100 余篇
通讯作者:高淑萍.E-mail：gaosp@mail.xidian.edu.cn
更新日期/Last Update:
1900-01-01