[1]LU Yi,CHEN Yaran,ZHAO Dongbin,et al.Keypoint-based graph contrastive neural network for image classification[J].CAAI Transactions on Intelligent Systems,2023,18(1):36-46.[doi:10.11992/tis.202112001]
Copy

Keypoint-based graph contrastive neural network for image classification

CAAI Transactions on Intelligent Systems[ISSN 1673-4785/CN 23-1538/TP] Volume: 18 Number of periods: 2023 1 Page number: 36-46 Column: 学术论文—机器感知与模式识别 Public date: 2023-01-05
Title:
Keypoint-based graph contrastive neural network for image classification
Author(s):
LU Yi1; 2;  CHEN Yaran1;  ZHAO Dongbin1;  LIU Bao3; 4;  LAI Zhichao3; 4;  WANG Chaonan3; 4
1. The State Key Laboratory of Management and Control for Complex Systems , Institute of Automation, Chinese Academy of Sciences, Beijng 100190, China;
2. School of Information Science and Engineering, Shan Dong Normal University, Ji’nan 250358, China;
3. Chinese Academy of Medical Sciences, Peking Union Medical College Hospital, Beijing 100730, China;
4. Department of Vascular Surgery, Peking Union Medical College Hospital, Beijing 100730, China
Keywords:
keypoint detection; graph topological structure; image classification; graph contrastive learning; metric learning; graph neural network; siamese network; graph classification
CLC:
TP3
DOI:
10.11992/tis.202112001
Abstract:
At present, deep learning is one of the mainstream methods for image classification. It focuses more on local features in the receptive field than the prior information of topological structure of the category. In this paper, We propose a Keypoint-based Discriminator Graph neural network (Key-D-Graph) for image binary classification method, which is a graph comparison network method based on key points. It explicitly introduces the topology prior structure when identifying image categories. The method contains two main steps. The first step is to build the graph representation of an image with the keypoints, that is, identifying possible key points of the target category in the image by a deep learning method, and then using the coordinates of the key points to generate the topological representation of the image. The second step is to build a graph contrastive network based on the image representation of key points, so as to estimate the structural difference between the graph to be identified and the object graph, realizing object discrimination. In this step, the topological prior structure information of the object is used to realize object recognition based on the global structure information of the image. Especially, the intermediate output results of Key-D-Graph are the key points of categories containing explicit semantic information, which facilitates analysis and debugging of the algorithm step by step in practical application. Contrast experiments show that the proposed method outperforms the mainstream methods both in efficiency and precision. And the mechanism and effectiveness of topological structure in classification are verified by the ablation experiments.
References:
[1] 黄凯奇, 任伟强, 谭铁牛. 图像物体分类与检测算法综述[J]. 计算机学报, 2014, 37(6): 1225–1240
HUANG Kaiqi, REN Weiqiang, TAN Tieniu. A review on image object classification and detection[J]. Chinese journal of computers, 2014, 37(6): 1225–1240
[2] 王亮申, 欧宗瑛, 朱玉才, 等. 基于SVM的图像分类[J]. 计算机应用与软件, 2005, 22(5): 98–99,126
WANG Liangshen, OU Zongying, ZHU Yucai, et al. Classifying images with SVM method[J]. Computer applications and software, 2005, 22(5): 98–99,126
[3] 任越美, 张艳宁, 李映. 压缩感知及其图像处理应用研究进展与展望[J]. 自动化学报, 2014, 40(8): 1563–1574
REN Yuemei, ZHANG Yanning, LI Ying. LI Ying. Advances and perspective on compressed sensing an application on image processing[J]. Acta automaica sinica, 2014, 40(8): 1563–1574
[4] 马忠丽, 刘权勇, 武凌羽, 等. 一种基于联合表示的图像分类方法[J]. 智能系统学报, 2018, 13(2): 220–226
MA Zhongli, LIU Quanyong, WU Lingyu, et al. Syncretic representation method for image classification[J]. CAAI transactions on intelligent systems, 2018, 13(2): 220–226
[5] KRIZHEVSKY A, SUTSKEVER I, HINTON G E. ImageNet classification with deep convolutional neural networks[J]. Communications of the acm, 2012, 60: 84–90.
[6] SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[EB/OL]. (2014?09?04)[ 2021?12?01].https://arxiv.org/abs/1409.1556.
[7] SZEGEDY C, LIU Wei, JIA Yangqing, et al. Going deeper with convolutions[C]//2015 IEEE Conference on Computer Vision and Pattern Recognition. Boston: IEEE, 2015: 1?9.
[8] HE Kaiming, ZHANG Xiangyu, REN Shaoqing, et al. Deep residual learning for image recognition[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016: 770?778.
[9] HUANG Gao, LIU Zhuang, VAN DER MAATEN L, et al. Densely connected convolutional networks[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE, 2017: 2261?2269.
[10] BALDASSARRE F, MORíN D G, RODéS-GUIRAO L. Deep koalarization: image colorization using CNNs and inception-ResNet-v2[EB/OL]. (2017?12?09) [ 2021?12?01].https://arxiv.org/abs/1712.03400.
[11] CHEN Yaran, LI Haoran, GAO Ruiyuan, et al. Boost 3-D object detection via point clouds segmentation and fused 3-D GIoU-L1 loss[J]. IEEE transactions on neural networks and learning systems, 2022, 33(2): 762–773.
[12] CHEN Y, GAO R, LIU F, et al. ModuleNet: knowledge-inherited neural architecture search[J]. IEEE transactions on cybernetics, 2022, 52(11): 11661–11671.
[13] DING Zixiang, CHEN Yaran, LI Nannan, et al. BNAS: efficient neural architecture search using broad scalable architecture[J]. IEEE transactions on neural networks and learning systems, 2022, 33(9): 5004–5018.
[14] LI Nannan, PAN Yu, CHEN Yaran, et al. Heuristic rank selection with progressively searching tensor ring network[J]. Complex & intelligent systems, 2022, 8(2): 771–785.
[15] TAN Mingxing, LE Q V. EfficientNet: rethinking model scaling for convolutional neural networks[EB/OL]. (2019?05?28)[2021?12?01].https://arxiv.org/abs/1905.11946.
[16] ZEILER M D, FERGUS R. Visualizing and understanding convolutional networks[C]//European Conference on Computer Vision. Cham: Springer, 2014: 818?833.
[17] GEIRHOS R, RUBISCH P, MICHAELIS C, et al. ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness[EB/OL]. (2018?11?29)[2021?12?01].https://arxiv.org/abs/1811.12231.
[18] LOWE D G. Object recognition from local scale-invariant features[C]//Proceedings of the Seventh IEEE International Conference on Computer Vision. Kerkyra: IEEE, 1999: 1150?1157.
[19] WEI Shihen, RAMAKRISHNA V, KANADE T, et al. Convolutional pose machines[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016: 4724?4732.
[20] WANG Jingdong, SUN Ke, CHENG Tianheng, et al. Deep high-resolution representation learning for visual recognition[J]. IEEE transactions on pattern analysis and machine intelligence, 2021, 43(10): 3349–3364.
[21] LU Yi, CHEN Yaran, ZHAO Dongbin, et al. CNN-G: convolutional neural network combined with graph for image segmentation with theoretical analysis[J]. IEEE transactions on cognitive and developmental systems, 2021, 13(3): 631–644.
[22] LU Yi, CHEN Yaran, ZHAO Dongbin, et al. Graph-FCN for image semantic segmentation[C]//International Symposium on Neural Networks. Cham: Springer, 2019: 97?105.
[23] 张勇, 高大林, 巩敦卫, 等. 用于关系抽取的注意力图长短时记忆神经网络[J]. 智能系统学报, 2021, 16(3): 518–527
ZHANG Yong, GAO Dalin, GONG Dunwei, et al. Attention graph long short-term memory neural network for relation extraction[J]. CAAI transactions on intelligent systems, 2021, 16(3): 518–527
[24] XU K, HU Weihua, LESKOVEC J, et al. How powerful are graph neural networks? [EB/OL]. (2018?10?01)[2021?12-01].https://arxiv.org/abs/1810.00826.
[25] CHOPRA S, HADSELL R, LECUN Y. Learning a similarity metric discriminatively, with application to face verification[C]//2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. San Diego: IEEE, 2005: 539?546.
[26] SOHN K. Improved deep metric learning with multi-class N-pair loss objective[C]//NIPS’16: Proceedings of the 30th International Conference on Neural Information Processing Systems. New York: ACM, 2016: 1857-1865.
[27] TAIGMAN Y, YANG Ming, RANZATO M, et al. DeepFace: closing the gap to human-level performance in face verification[C]//2014 IEEE Conference on Computer Vision and Pattern Recognition. Columbus: IEEE, 2014: 1701?1708.
[28] CARON M, MISRA I, MAIRAL J, et al. Unsupervised learning of visual features by contrasting cluster assignments[EB/OL]. (2020?06?17)[2021?12?01].https://arxiv.org/abs/2006.09882.
[29] OORD A V D, LI YAZHE, VINYALS O. Representation learning with contrastive predictive coding[EB/OL]. (2018?07?10)[2021?12?01].https://arxiv.org/abs/1807.03748.
[30] CHEN Ting, KORNBLITH S, NOROUZI M, et al. A simple framework for contrastive learning of visual representations[EB/OL]. (2020?02?13)[2021?12?01].https://www.semanticscholar.org/paper/A-Simple-Framework-for-Contrastive-Learning-of-Chen-Kornblith/34733eaf66007516347a40ad5d9bbe1cc9dacb6b.
[31] HE Kaiming, FAN Haoqi, WU Yuxin, et al. Momentum contrast for unsupervised visual representation learning[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle: IEEE, 2020: 9726?9735.
[32] CHEN Xinlei, HE Kaiming. Exploring simple Siamese representation learning[C]//2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Nashville: IEEE, 2021: 15745?15753.
[33] ZHU Yanqiao, XU Yichen, YU Feng, et al. Graph contrastive learning with adaptive augmentation[C]//WWW ’21: Proceedings of the Web Conference 2021. New York: ACM, 2021: 2069-2080.
[34] WANG Runzhong, YAN Junchi, YANG Xiaokang. Learning combinatorial embedding networks for deep graph matching[C]//2019 IEEE/CVF International Conference on Computer Vision. Seoul: IEEE, 2019: 3056?3065.
[35] 张俊, 田慧敏. 一种基于边指针搜索及区域划分的三角剖分算法[J]. 自动化学报, 2021, 47(1): 100–107
ZHANG Jun, TIAN Huimin. A triangulation algorithm based on edge-pointer search and region-division[J]. Acta automatica sinica, 2021, 47(1): 100–107
[36] GUIBAS L J, KNUTH D E, SHARIR M. Randomized incremental construction of delaunay and Voronoi diagrams[M]//Automata, Languages and Programming. Berlin/Heidelberg: Springer-Verlag, 2005: 414?431.
[37] EVERINGHAM M, ESLAMI S, GOOL L, et al. The pascal visual object classes challenge: a retrospective[J]. International journal of computer vision, 2014, 111: 98–136.
Similar References:

Memo

-

Last Update: 1900-01-01

Copyright © CAAI Transactions on Intelligent Systems