[1]卢毅,陈亚冉,赵冬斌,等.关键点图对比图像分类方法[J].智能系统学报,2023,18(1):36-46.[doi:10.11992/tis.202112001]
LU Yi,CHEN Yaran,ZHAO Dongbin,et al.Keypoint-based graph contrastive neural network for image classification[J].CAAI Transactions on Intelligent Systems,2023,18(1):36-46.[doi:10.11992/tis.202112001]
点击复制
LU Yi,CHEN Yaran,ZHAO Dongbin,et al.Keypoint-based graph contrastive neural network for image classification[J].CAAI Transactions on Intelligent Systems,2023,18(1):36-46.[doi:10.11992/tis.202112001]
关键点图对比图像分类方法
《智能系统学报》[ISSN 1673-4785/CN 23-1538/TP] 卷:
18
期数:
2023年第1期
页码:
36-46
栏目:
学术论文—机器感知与模式识别
出版日期:
2023-01-05
- Title:
- Keypoint-based graph contrastive neural network for image classification
- Keywords:
- keypoint detection; graph topological structure; image classification; graph contrastive learning; metric learning; graph neural network; siamese network; graph classification
- 分类号:
- TP3
- 摘要:
- 深度学习是目前图像分类的主流方法之一，其重视感受野内的局部信息，却忽略了类别的先验拓扑结构信息。本文提出了一种新的图像分类方法，即Key-D-Graph，这是基于关键点的图对比网络方法，在识别图像类别时可以显式地考虑拓扑先验结构。具体地，图像分类需要2个步骤，第一步是基于关键点构建图像的图表达，即采用深度学习方法识别图像中目标类别的可能关键点，并采用关键点坐标生成图像的拓扑图表达；第二步基于关键点的图像图表达建立图对比网络，以估计待识别图与目标类别之间的结构差异，实现类别判断，该步骤利用了物体的拓扑先验结构信息，实现了基于图像全局结构信息的物体识别。特别的，Key-D-Graph的中间输出结果为类别关键点，具有语义可解释性，便于在实际应用中对算法逐步分析调试。实验结果表明，提出的方法可在效率和精度上超过主流方法，且通过消融实验分析验证了拓扑结构在分类中的作用机制和有效性。
- Abstract:
- At present, deep learning is one of the mainstream methods for image classification. It focuses more on local features in the receptive field than the prior information of topological structure of the category. In this paper, We propose a Keypoint-based Discriminator Graph neural network (Key-D-Graph) for image binary classification method, which is a graph comparison network method based on key points. It explicitly introduces the topology prior structure when identifying image categories. The method contains two main steps. The first step is to build the graph representation of an image with the keypoints, that is, identifying possible key points of the target category in the image by a deep learning method, and then using the coordinates of the key points to generate the topological representation of the image. The second step is to build a graph contrastive network based on the image representation of key points, so as to estimate the structural difference between the graph to be identified and the object graph, realizing object discrimination. In this step, the topological prior structure information of the object is used to realize object recognition based on the global structure information of the image. Especially, the intermediate output results of Key-D-Graph are the key points of categories containing explicit semantic information, which facilitates analysis and debugging of the algorithm step by step in practical application. Contrast experiments show that the proposed method outperforms the mainstream methods both in efficiency and precision. And the mechanism and effectiveness of topological structure in classification are verified by the ablation experiments.
- 参考文献/References:
-
[1] 黄凯奇, 任伟强, 谭铁牛. 图像物体分类与检测算法综述[J]. 计算机学报, 2014, 37(6): 1225–1240
HUANG Kaiqi, REN Weiqiang, TAN Tieniu. A review on image object classification and detection[J]. Chinese journal of computers, 2014, 37(6): 1225–1240
[2] 王亮申, 欧宗瑛, 朱玉才, 等. 基于SVM的图像分类[J]. 计算机应用与软件, 2005, 22(5): 98–99,126
WANG Liangshen, OU Zongying, ZHU Yucai, et al. Classifying images with SVM method[J]. Computer applications and software, 2005, 22(5): 98–99,126
[3] 任越美, 张艳宁, 李映. 压缩感知及其图像处理应用研究进展与展望[J]. 自动化学报, 2014, 40(8): 1563–1574
REN Yuemei, ZHANG Yanning, LI Ying. LI Ying. Advances and perspective on compressed sensing an application on image processing[J]. Acta automaica sinica, 2014, 40(8): 1563–1574
[4] 马忠丽, 刘权勇, 武凌羽, 等. 一种基于联合表示的图像分类方法[J]. 智能系统学报, 2018, 13(2): 220–226
MA Zhongli, LIU Quanyong, WU Lingyu, et al. Syncretic representation method for image classification[J]. CAAI transactions on intelligent systems, 2018, 13(2): 220–226
[5] KRIZHEVSKY A, SUTSKEVER I, HINTON G E. ImageNet classification with deep convolutional neural networks[J]. Communications of the acm, 2012, 60: 84–90.
[6] SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[EB/OL]. (2014?09?04)[ 2021?12?01].https://arxiv.org/abs/1409.1556.
[7] SZEGEDY C, LIU Wei, JIA Yangqing, et al. Going deeper with convolutions[C]//2015 IEEE Conference on Computer Vision and Pattern Recognition. Boston: IEEE, 2015: 1?9.
[8] HE Kaiming, ZHANG Xiangyu, REN Shaoqing, et al. Deep residual learning for image recognition[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016: 770?778.
[9] HUANG Gao, LIU Zhuang, VAN DER MAATEN L, et al. Densely connected convolutional networks[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE, 2017: 2261?2269.
[10] BALDASSARRE F, MORíN D G, RODéS-GUIRAO L. Deep koalarization: image colorization using CNNs and inception-ResNet-v2[EB/OL]. (2017?12?09) [ 2021?12?01].https://arxiv.org/abs/1712.03400.
[11] CHEN Yaran, LI Haoran, GAO Ruiyuan, et al. Boost 3-D object detection via point clouds segmentation and fused 3-D GIoU-L1 loss[J]. IEEE transactions on neural networks and learning systems, 2022, 33(2): 762–773.
[12] CHEN Y, GAO R, LIU F, et al. ModuleNet: knowledge-inherited neural architecture search[J]. IEEE transactions on cybernetics, 2022, 52(11): 11661–11671.
[13] DING Zixiang, CHEN Yaran, LI Nannan, et al. BNAS: efficient neural architecture search using broad scalable architecture[J]. IEEE transactions on neural networks and learning systems, 2022, 33(9): 5004–5018.
[14] LI Nannan, PAN Yu, CHEN Yaran, et al. Heuristic rank selection with progressively searching tensor ring network[J]. Complex & intelligent systems, 2022, 8(2): 771–785.
[15] TAN Mingxing, LE Q V. EfficientNet: rethinking model scaling for convolutional neural networks[EB/OL]. (2019?05?28)[2021?12?01].https://arxiv.org/abs/1905.11946.
[16] ZEILER M D, FERGUS R. Visualizing and understanding convolutional networks[C]//European Conference on Computer Vision. Cham: Springer, 2014: 818?833.
[17] GEIRHOS R, RUBISCH P, MICHAELIS C, et al. ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness[EB/OL]. (2018?11?29)[2021?12?01].https://arxiv.org/abs/1811.12231.
[18] LOWE D G. Object recognition from local scale-invariant features[C]//Proceedings of the Seventh IEEE International Conference on Computer Vision. Kerkyra: IEEE, 1999: 1150?1157.
[19] WEI Shihen, RAMAKRISHNA V, KANADE T, et al. Convolutional pose machines[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016: 4724?4732.
[20] WANG Jingdong, SUN Ke, CHENG Tianheng, et al. Deep high-resolution representation learning for visual recognition[J]. IEEE transactions on pattern analysis and machine intelligence, 2021, 43(10): 3349–3364.
[21] LU Yi, CHEN Yaran, ZHAO Dongbin, et al. CNN-G: convolutional neural network combined with graph for image segmentation with theoretical analysis[J]. IEEE transactions on cognitive and developmental systems, 2021, 13(3): 631–644.
[22] LU Yi, CHEN Yaran, ZHAO Dongbin, et al. Graph-FCN for image semantic segmentation[C]//International Symposium on Neural Networks. Cham: Springer, 2019: 97?105.
[23] 张勇, 高大林, 巩敦卫, 等. 用于关系抽取的注意力图长短时记忆神经网络[J]. 智能系统学报, 2021, 16(3): 518–527
ZHANG Yong, GAO Dalin, GONG Dunwei, et al. Attention graph long short-term memory neural network for relation extraction[J]. CAAI transactions on intelligent systems, 2021, 16(3): 518–527
[24] XU K, HU Weihua, LESKOVEC J, et al. How powerful are graph neural networks? [EB/OL]. (2018?10?01)[2021?12-01].https://arxiv.org/abs/1810.00826.
[25] CHOPRA S, HADSELL R, LECUN Y. Learning a similarity metric discriminatively, with application to face verification[C]//2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. San Diego: IEEE, 2005: 539?546.
[26] SOHN K. Improved deep metric learning with multi-class N-pair loss objective[C]//NIPS’16: Proceedings of the 30th International Conference on Neural Information Processing Systems. New York: ACM, 2016: 1857-1865.
[27] TAIGMAN Y, YANG Ming, RANZATO M, et al. DeepFace: closing the gap to human-level performance in face verification[C]//2014 IEEE Conference on Computer Vision and Pattern Recognition. Columbus: IEEE, 2014: 1701?1708.
[28] CARON M, MISRA I, MAIRAL J, et al. Unsupervised learning of visual features by contrasting cluster assignments[EB/OL]. (2020?06?17)[2021?12?01].https://arxiv.org/abs/2006.09882.
[29] OORD A V D, LI YAZHE, VINYALS O. Representation learning with contrastive predictive coding[EB/OL]. (2018?07?10)[2021?12?01].https://arxiv.org/abs/1807.03748.
[30] CHEN Ting, KORNBLITH S, NOROUZI M, et al. A simple framework for contrastive learning of visual representations[EB/OL]. (2020?02?13)[2021?12?01].https://www.semanticscholar.org/paper/A-Simple-Framework-for-Contrastive-Learning-of-Chen-Kornblith/34733eaf66007516347a40ad5d9bbe1cc9dacb6b.
[31] HE Kaiming, FAN Haoqi, WU Yuxin, et al. Momentum contrast for unsupervised visual representation learning[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle: IEEE, 2020: 9726?9735.
[32] CHEN Xinlei, HE Kaiming. Exploring simple Siamese representation learning[C]//2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Nashville: IEEE, 2021: 15745?15753.
[33] ZHU Yanqiao, XU Yichen, YU Feng, et al. Graph contrastive learning with adaptive augmentation[C]//WWW ’21: Proceedings of the Web Conference 2021. New York: ACM, 2021: 2069-2080.
[34] WANG Runzhong, YAN Junchi, YANG Xiaokang. Learning combinatorial embedding networks for deep graph matching[C]//2019 IEEE/CVF International Conference on Computer Vision. Seoul: IEEE, 2019: 3056?3065.
[35] 张俊, 田慧敏. 一种基于边指针搜索及区域划分的三角剖分算法[J]. 自动化学报, 2021, 47(1): 100–107
ZHANG Jun, TIAN Huimin. A triangulation algorithm based on edge-pointer search and region-division[J]. Acta automatica sinica, 2021, 47(1): 100–107
[36] GUIBAS L J, KNUTH D E, SHARIR M. Randomized incremental construction of delaunay and Voronoi diagrams[M]//Automata, Languages and Programming. Berlin/Heidelberg: Springer-Verlag, 2005: 414?431.
[37] EVERINGHAM M, ESLAMI S, GOOL L, et al. The pascal visual object classes challenge: a retrospective[J]. International journal of computer vision, 2014, 111: 98–136.
备注/Memo
收稿日期:2021-12-01。
基金项目:国家重点研发计划项目(2019YFB1311700);山东省自然科学基金青年项目(ZR2021QF085);国家自然基金青年基金项目(62006223,62006226);中国科学院战略重点研究项目(XDA27030400);中央级公益性科研院所基本科研业务费临床与转化医学研究基金项目(2019XK320004);中国医学科学院医学与健康科技创新工程医学人工智能科技先导专项(2018-12M-AI-004);中央高校基本科研业务费重点项目(3332020009)
作者简介:卢毅,讲师,博士,主要研究方向为计算机视觉、图神经网络。发表学术论文8篇;陈亚冉,副研究员,博士,主要研究方向为计算机视觉、智能驾驶、机器人。发表学术论文34篇,获得包括IEEE汇刊2020年度唯一优秀论文等多项论文奖励;获得2017年中国智能车未来挑战赛离线测试2项第一名,2020 IEEE ICRA DJI RoboMaster人工智能挑战赛感知/导航/决策多赛道3项第一名;赵冬斌,研究员,博士,IEEE Fellow,主要研究方向为强化学习、自适应动态规划、智能游戏、智能交通、机器人。发表学术论文300余篇
通讯作者:陈亚冉.E-mail:chenyaran2013@ia.ac.cn
更新日期/Last Update:
1900-01-01