[1]CHEN Rongshan,GAO Shuping,QI Xiaogang.A spectral clustering based on GCNs and attention mechanism[J].CAAI Transactions on Intelligent Systems,2023,18(5):936-944.[doi:10.11992/tis.202208041]
Copy

A spectral clustering based on GCNs and attention mechanism

CAAI Transactions on Intelligent Systems[ISSN 1673-4785/CN 23-1538/TP] Volume: 18 Number of periods: 2023 5 Page number: 936-944 Column: 学术论文—机器学习 Public date: 2023-09-05
Title:
A spectral clustering based on GCNs and attention mechanism
Author(s):
CHEN Rongshan;  GAO Shuping;  QI Xiaogang
School of Mathematics and Statistics, Xidian University, Xi’an 710071, China
Keywords:
deep learning; spectral clustering; image classification; graph convolutional neural networks; attention mechanism; convolutional neural networks; image segmentation; deep clustering
CLC:
TP391.1
DOI:
10.11992/tis.202208041
Abstract:
The classical spectral clustering algorithm often involves computationally expensive feature decomposition of the Laplacian matrix, prompting researchers to introduce a deep learning model into the spectral clustering algorithm. Nonetheless, the existing methods possess certain limitations. The existing clustering model neglects the information on the correlation degree between nodes, which leads to inaccurate clustering results. Additionally, when the number of nodes in the graph neural network changes, the whole graph requires updating, leading to extensive storage space consumption. By combining the attention mechanism with the improved architecture of graph convolutional neural networks, this paper proposes a spectral clustering method based on the attention and graph convolutional neural network to address these issues. The proposed method mainly utilizes the attention mechanism to guide node clustering, following which it establishes the corresponding target. The corresponding cluster allocation, when the target is optimal, is calculated by training the neural network. Attention information and topological information are used to guide the graph reconstruction process, thereby improving the accuracy of reconstruction. Experimental results reveal that the proposed method exhibits excellent performance in graph classification, graph clustering, and graph reconstruction tasks.
References:
[1] SHI Na, LIU Xumin, GUAN Yong. Research on k-means clustering algorithm: an improved k-means clustering algorithm[C]//2010 Third International Symposium on Intelligent Information Technology and Security Informatics. Jian: IEEE, 2010: 63?67.
[2] FU Chuanyi, CHEN Huandong, XING Jieqing. Spectral clustering and its research progress[C]//2011 Seventh International Conference on Computational Intelligence and Security: IEEE, 2012: 1367?1369.
[3] AGARWAL P, ALAM M A, BISWAS R. A hierarchical clustering algorithm for categorical attributes[C]//2010 Second International Conference on Computer Engineering and Applications. Bali: IEEE, 2010: 365?368.
[4] STUDER L, WALLAU J, INGOLD R, et al. Effects of graph pooling layers on classification with graph neural networks[C]//2020 7th Swiss Conference on Data Science. Luzern: IEEE, 2020: 57?58.
[5] GAMA F, ISUFI E, LEUS G, et al. Graphs, convolutions, and neural networks: from graph filters to graph neural networks[C]//IEEE Signal Processing Magazine. [S.l.]: IEEE, 2020: 128?138.
[6] KHAN A, SOHAIL A, ZAHOORA U, et al. A survey of the recent architectures of deep convolutional neural networks[J]. Artificial intelligence review, 2020, 53(8): 5455-5516.
[7] BOULEMNADJEL A, HACHOUF F. An improved algorithm for subspace clustering applied to image segmentation[C]//2012 16th International Conference on Information Visualisation. Montpellier: IEEE, 2012: 297?301.
[8] XIA Zhile, LI Junmin, LI Jiangrong. Passivity-based resilient adaptive control for fuzzy stochastic delay systems with Markovian switching[J]. Journal of the franklin institute, 2014, 351(7): 3818-3836.
[9] YUAN Gonglin, XUE Bing, ZHANG Mengjie. A graph-based approach to automatic convolutional neural network construction for image classification[C]//2020 35th International Conference on Image and Vision Computing New Zealand. Wellington: IEEE, 2020: 1?6.
[10] WU Zonghan, PAN Shirui, CHEN Fengwen, et al. A comprehensive survey on graph neural networks[J]. IEEE transactions on neural networks and learning systems, 2021, 32(1): 4-24.
[11] LI Yang, ZHANG Jiangzhou, NIE Mingyu, et al. Hessian-regularized spectral clustering for behavior recognition[C]//2020 International Conference on Intelligent Computing and Human-Computer Interaction. Sanya: IEEE, 2021: 156?159.
[12] GAO Jianliang, GAO Jun, YING Xiaoting, et al. Higher-order interaction goes neural: a substructure assembling graph attention network for graph classification[J]. IEEE transactions on knowledge and data engineering, 2023, 35(2): 1594-1608.
[13] LI Qiuyan, SHANG Yanlei, QIAO Xiuquan, et al. Heterogeneous dynamic graph attention network[C]//2020 IEEE International Conference on Knowledge Graph (ICKG). Nanjing: IEEE, 2020: 404?411.
[14] PENG Xi, FENG Jiashi, ZHOU J T, et al. Deep subspace clustering[J]. IEEE transactions on neural networks and learning systems, 2020, 31(12): 5509-5521.
[15] XU Xiaoliang, GAO Tong, WANG Yuxiang, et al. Event temporal relation extraction with attention mechanism and graph neural network[J]. Tsinghua science and technology, 2021, 27(1): 79-90.
[16] KIM M, KIM T, KIM D. Spatio-temporal slowfast self-attention network for action recognition[C]//2020 IEEE International Conference on Image Processing. Abu Dhabi: IEEE, 2020: 2206?2210.
[17] ZHANG Xuming, SUN Genyun, JIA Xiuping, et al. Spectral-spatial self-attention networks for hyperspectral image classification[J]. IEEE transactions on geoscience and remote sensing, 2022, 60: 1-15.
[18] GU Ruijun, WANG Jiacai. An improved spectral clustering algorithm based on neighbour adaptive scale[C]//2009 International Conference on Business Intelligence and Financial Engineering. Beijing: IEEE, 2009: 233?236.
[19] NGO M, MAI An, BUI T. On an improvement of Graph Convolutional Network in semi-supervised learning[C]//2020 7th NAFOSTED Conference on Information and Computer Science (NICS). Ho Chi Minh City: IEEE, 2021: 114?117.
[20] RUIZ L, GAMA F, RIBEIRO A. Graph neural networks: architectures, stability, and transferability[J]. Proceedings of the IEEE, 2021, 109(5): 660-682.
[21] PENG Hongrui. Graph convolutional networks: an analysis of method and applications in different fields and systems[C]//2020 International Conference on Communications, Information System and Computer Engineering (CISCE). Kuala Lumpur: IEEE, 2020: 153?157.
[22] CAO Jing, XU Yi, SONG Xuening. First-order meta-learning in node classification with graph convolutional network[C]//2021 4th International Conference on Artificial Intelligence and Big Data. Chengdu: IEEE, 2021: 443?447.
[23] XIAO Yi, CHEN Siying, WANG Hao, et al. HSGACN: hyperspectral image classification algorithm based on graph convolutional network[C]//2021 IEEE International Geoscience and Remote Sensing Symposium IGARSS. Brussels: IEEE, 2021: 3677?3680.
[24] CAI Chengtao, GUO Dongning. CNN-self-attention-DNN architecture for mandarin recognition[C]//2020 Chinese Control and Decision Conference. Hefei: IEEE, 2020: 1190?1194.
[25] UEDA T, OKADA M, MORI N, et al. A method to estimate request sentences using LSTM with self-attention mechanism[C]//2019 8th International Congress on Advanced Applied Informatics. Toyama: IEEE, 2020: 7?10.
[26] HU Bin, GUO Kehua, WANG Xiaokang, et al. RRL-GAT: graph attention network-driven multilabel image robust representation learning[J]. IEEE Internet of Things journal, 2022, 9(12): 9167-9178.
[27] TIAN Long, DU Qian, KOPRIVA I, et al. Spatial-spectral based multi-view low-rank sparse sbuspace clustering for hyperspectral imagery[C]//IEEE International Geoscience and Remote Sensing Symposium. Valencia: IEEE, 2018: 8488?8491.
[28] LI He, ZHANG Shiyu, SU Liangcai, et al. GraphSANet: a graph neural network and self attention based approach for spatial temporal prediction in sensor network[C]//2020 IEEE International Conference on Big Data. Atlanta: IEEE, 2021: 5756?5758.
[29] ZHANG Jintao, XU Quan. Attention-aware heterogeneous graph neural network[J]. Big data mining and analytics, 2021, 4(4): 233-241.
[30] BERIKOV V. Autoencoder-based low-rank spectral ensemble clustering of biological data[C]//2020 Cognitive Sciences, Genomics and Bioinformatics. Novosibirsk: IEEE, 2020: 43?46.
[31] SHANTHAMALLU U S, THIAGARAJAN J J, SPANIAS A. A regularized attention mechanism for graph attention networks[C]//2020 IEEE International Conference on Acoustics, Speech and Signal Processing. Barcelona: IEEE, 2020: 3372?3376.
[32] CAO Meng, MA Xiying, ZHU Kai, et al. Heterogeneous information network embedding with convolutional graph attention networks[C]//2020 International Joint Conference on Neural Networks. Glasgow: IEEE, 2020: 1?8.
[33] 张铭泉, 周辉, 曹锦纲. 基于注意力机制的双BERT有向情感文本分类研究[J]. 智能系统学报, 2022, 17(6): 1220-1227
ZHANG Mingquan, ZHOU Hui, CAO Jingang. Dual BERT directed sentiment text classification based on attention mechanism[J]. CAAI transactions on intelligent systems, 2022, 17(6): 1220-1227
[34] HUANG Shuning, OTA K, DONG Mianxiong, et al. MultiSpectralNet: spectral clustering using deep neural network for multi-view data[J]. IEEE transactions on computational social systems, 2019, 6(4): 749-760.
[35] GüZEL K, KUR?UN O. Improving spectral clustering using path-based connectivity[C]//2015 23nd Signal Processing and Communications Applications Conference. Malatya: IEEE, 2015: 2110?2113.
[36] 于润羽, 李雅文, 李昂. 融合领域特征的科技学术会议语义相似性计算方法[J]. 智能系统学报, 2022, 17(4): 737-743
YU Runyu, LI Yawen, LI Ang. Semantic similarity computing for scientific and technological conferences[J]. CAAI transactions on intelligent systems, 2022, 17(4): 737-743
[37] 马甜甜, 杨长春, 严鑫杰, 等. 融合知识图谱和轻量级图卷积网络推荐系统的研究[J]. 智能系统学报, 2022, 17(4): 721-727
MA Tiantian, YANG Changchun, YAN Xinjie, et al. Research on the fusion of knowledge graph and lightweight graph convolutional network recommendation system[J]. CAAI transactions on intelligent systems, 2022, 17(4): 721-727
[38] LI Hongmin, YE Xiucai, IMAKURA A, et al. Ensemble learning for spectral clustering[C]//2020 IEEE International Conference on Data Mining. Sorrento: IEEE, 2021: 1094?1099.
[39] BIANCHI F M, GRATTAROLA D, ALIPPI C. Mincut pooling in graph neural networks[EB/OL]. (2020?12?29)[2022?01?01]. https://arxiv.org/abs/1907.00481v2.
[40] 杨梦茵, 陈俊芬, 翟俊海. 非对称卷积编码器的聚类算法[J]. 智能系统学报, 2022, 17(5): 900-907
YANG Mengyin, CHEN Junfen, ZHAI Junhai. A clustering method based on the asymmetric convolutional autoencoder[J]. CAAI transactions on intelligent systems, 2022, 17(5): 900-907
[41] SANDFELDER D, VIJAYAN P, HAMILTON W L. Ego-GNNs: exploiting ego structures in graph neural networks[C]//2021 IEEE International Conference on Acoustics, Speech and Signal Processing. Toronto: IEEE, 2021: 8523?8527.
[42] LU Runkun, LIU Jianwei, ZUO Xin. Attentive multi-view deep subspace clustering net[J]. Neurocomputing, 2021, 435: 186-196.
[43] PENG Zhihao, LIU Hui, JIA Yuheng, et al. Deep attention-guided graph clustering with dual self-supervision[J]. IEEE transactions on circuits and systems for video technology, 2022: 1.
[44] DONG Huihui, MA Wenping, JIAO Licheng, et al. A multiscale self-attention deep clustering for change detection in SAR images[J]. IEEE transactions on geoscience and remote sensing, 2022, 60: 1-16.
[45] MIN Erxue, GUO Xifeng, LIU Qiang, et al. A survey of clustering with deep learning: from the perspective of network architecture[J]. IEEE access, 2018, 6: 39501-39514.
[46] XU K, HU WEIHUA, LESKOVEC J, et al. How powerful are graph neural networks? [EB/OL]. (2019?02?22)[2022?01?01]. https://arxiv.org/abs/1810.00826.
[47] PETAR V, GUILLEM C, ARANTXA C, et al. Graph attention networks[EB/OL]. (2018?02?04)[2022?01?01].https://arxiv.org/abs/1710.10903.
[48] LI Meng, HSU W, XIE Xiaodong, et al. SACNN: self-attention convolutional neural network for low-dose CT denoising with self-supervised perceptual loss network[J]. IEEE transactions on medical imaging, 2020, 39(7): 2289-2301.
Similar References:

Memo

-

Last Update: 1900-01-01

Copyright © CAAI Transactions on Intelligent Systems