[1]李小菲,苟光磊,韩岩奇,等.多尺度知识引导局部增强的小样本细粒度图像分类方法[J].智能系统学报,2024,19(5):1157-1167.[doi:10.11992/tis.202309003]
LI Xiaofei,GOU Guanglei,HAN Yanqi,et al.Multiscale knowledge-guided local enhancement method for few-shot fine-grained image classification[J].CAAI Transactions on Intelligent Systems,2024,19(5):1157-1167.[doi:10.11992/tis.202309003]
点击复制
LI Xiaofei,GOU Guanglei,HAN Yanqi,et al.Multiscale knowledge-guided local enhancement method for few-shot fine-grained image classification[J].CAAI Transactions on Intelligent Systems,2024,19(5):1157-1167.[doi:10.11992/tis.202309003]
多尺度知识引导局部增强的小样本细粒度图像分类方法
《智能系统学报》[ISSN 1673-4785/CN 23-1538/TP] 卷:
19
期数:
2024年第5期
页码:
1157-1167
栏目:
学术论文—机器感知与模式识别
出版日期:
2024-09-05
- Title:
- Multiscale knowledge-guided local enhancement method for few-shot fine-grained image classification
- Keywords:
- few-shot learning; fine-grained image; image classification; multi-scale feature; relation network; deep learning; data augmentation; semantical correlations
- 分类号:
- TP391
- 文献标志码:
- 2024-08-29
- 摘要:
- 小样本细粒度图像分类任务中，由于支持样本与查询样本之间缺少局部关联性，导致图像关键可区分区域不易精确定位。针对这一问题，提出了多尺度知识引导局部增强的小样本细粒度图像分类方法，采用图像金字塔向下降低采样率获得多个不同分辨率的子图作为输入图像，融合多尺度特征，丰富了样本信息，利用知识引导模块捕获支持样本和查询样本之间的语义相关性，增强了支持样本重要区域的特征表示，对支持样本和查询样本的嵌入特征图进行克罗内克积操作，生成的空间相关图，更精确地定位样本特征之间的位置对应关系，突出了辨别性区域。实验结果表明该方法在小样本细粒度图像分类任务中表现出较好的分类性能。
- Abstract:
- In few-shot fine-grained image classification tasks, the absence of local correlation between support and query samples causes difficulty in precisely identifying discriminative regions in images. To solve this problem, this study proposes a multiscale knowledge-guided local enhancement method for few-shot fine-grained image classification. Multiple sub-images with different resolutions are obtained as input images by reducing the sampling rate downward using the image pyramid and by integrating multi-scale features to enrich the sample information. A knowledge-guided module is employed to capture semantic correlations between support and query samples, which enhances the feature representation of important regions in support samples. Kronecker product operations are performed on the embedded feature maps of support and query samples to generate spatial correlation maps. This way enables more accurate localization of the positional correspondence between sample features and highlights the discriminative area. Experimental results confirm the robust classification performance of the method in the classification task of few-shot fine-grained images.
- 参考文献/References:
-
[1] HE Kaiming, ZHANG Xiangyu, REN Shaoqing, et al. Deep residual learning for image recognition[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016: 770-778.
[2] 葛轶洲, 刘恒, 王言, 等. 小样本困境下的深度学习图像识别综述[J]. 软件学报, 2022, 33(1): 193-210.
GE Yizhou, LIU Heng, WANG Yan, et al. Survey on deep learning image recognition in dilemma of small samples[J]. Journal of software, 2022, 33(1): 193-210.
[3] WEN Qi, LI Shuang, HAN Bingfeng, et al. ZiGAN: fine-grained Chinese calligraphy font generation via a few-shot style transfer approach[C]//Proceedings of the 29th ACM International Conference on Multimedia. Virtual Event: ACM, 2021: 621-629.
[4] ZHANG Bo, CHEN Tao, WANG Bin, et al. Joint distribution alignment via adversarial learning for domain adaptive object detection[J]. IEEE transactions on multimedia, 2022, 24: 4102-4112.
[5] 杨祺, 孙俊. 融合多粒度特征的细粒度图像分类网络[J]. 小型微型计算机系统, 2023, 44(4): 818-824.
YANG Qi, SUN Jun. Multi-granularity feature fusion network for fine-grained visual classification[J]. Journal of Chinese computer systems, 2023, 44(4): 818-824.
[6] HE Ju, CHEN Jieneng, LIU Shuai, et al. TransFG: a transformer architecture for fine-grained recognition[C]//Proceedings of the AAAI conference on artificial intelligence. Vancouver: AAAI, 2022: 852-860.
[7] 许可凡. 基于注意力机制的细粒度图像分类技术的研究与应用[D]. 北京: 北京邮电大学, 2023.
XU Kefan. Research and application of fine-grained image classification technology based on attention mechanism[D]. Beijing: Beijing University of Posts and Telecommunications, 2023.
[8] 苗培昊. 基于度量学习的小样本细粒度图像识别研究[D]. 徐州: 中国矿业大学, 2022.
MIAO Peihao. Research on small sample fine-grained image recognition based on metric learning[D]. Xuzhou: China University of Mining and Technology, 2022.
[9] 周凯锐, 刘鑫, 景丽萍, 等. 概念驱动的小样本判别特征学习方法[J]. 智能系统学报, 2023, 18(1): 162-172.
ZHOU Kairui, LIU Xin, JING Liping, et al. Concept-driven discriminative feature learning for few-shot learning[J]. CAAI transactions on intelligent systems, 2023, 18(1): 162-172.
[10] YANG Shuo, LIU Lu, XU Min. Free lunch for few-shot learning: distribution calibration[EB/OL]. (2021-01-16)[2023-09-01]. https://arxiv.org/abs/2101.06395.
[11] VINYALS O, BLUNDELL C, LILLICRAP T, et al. Matching networks for one shot learning[J]. Advances in neural information processing systems, 2016: 3637-3645.
[12] SNELL J, SWERSKY K, ZEMEL R. Prototypical networks for few-shot learning[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems. Long Beach: ACM. 2017: 4080-4090.
[13] SUNG F, YANG Yongxin, ZHANG Li, et al. Learning to compare: relation network for few-shot learning[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 1199-1208.
[14] LEE S, MOON W, HEO J P. Task discrepancy maximization for fine-grained few-shot classification[C]//2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New Orleans: IEEE, 2022: 5321-5330.
[15] HARIHARAN B, GIRSHICK R. Low-shot visual recognition by shrinking and hallucinating features[C]//2017 IEEE International Conference on Computer Vision. Venice: IEEE, 2017: 3037-3046.
[16] KOCH G, ZEMEL R, SALAKHUTDINOV R. Siamese neural networks for one-shot image recognition[C]//Proceedings of the 32 nd International Conference on Machine Learning-Volume 37. Lille: JMLR, 2015: 1-8.
[17] HOU Ruibing, CHANG Hong, MA Bingpeng, et al. Cross attention network for few-shot classification[EB/OL]. (2019-10-17)[2023-09-01]. https://arxiv.org/abs/1910.07677.
[18] LI Wenbin, WANG Lei, XU Jinglin, et al. Revisiting local descriptor based image-to-class measure for few-shot learning[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach: IEEE, 2019: 7253-7260.
[19] WU Ziyang, LI Yuwei, GUO Lihua, et al. PARN: position-aware relation networks for few-shot learning[C]//2019 IEEE/CVF International Conference on Computer Vision. Seoul: IEEE, 2019: 6658-6666.
[20] ABDELAZIZ M, ZHANG Zuping. Multi-scale kronecker-product relation networks for few-shot learning[J]. Multimedia tools and applications, 2022, 81(5): 6703-6722.
[21] XIE Lingxi, TIAN Qi, HONG Richang, et al. Hierarchical part matching for fine-grained visual categorization[C]//2013 IEEE International Conference on Computer Vision. Sydney: IEEE, 2013: 1641-1648.
[22] ZHANG Ning, DONAHUE J, GIRSHICK R, et al. Part-based R-CNNs for fine-grained category detection[C]//European Conference on Computer Vision. Cham: Springer, 2014: 834-849.
[23] ZHANG Yu, WEI Xiushen, WU Jianxin, et al. Weakly supervised fine-grained categorization with part-based image representation[J]. IEEE transactions on image processing, 2016, 25(4): 1713-1725.
[24] SONG Kaitao, WEI Xiushen, SHU Xiangbo, et al. Bi-modal progressive mask attention for fine-grained recognition[J]. IEEE transactions on image processing, 2020, 29: 7006-7018.
[25] TANG Hao, YUAN Chengcheng, LI Zechao, et al. Learning attention-guided pyramidal features for few-shot fine-grained recognition[J]. Pattern recognition, 2022, 130: 108792.
[26] WU Jijie, CHANG Dongliang, SAIN A, et al. Bi-directional feature reconstruction network for fine-grained few-shot image classification[C]//Proceedings of the AAAI conference on artificial intelligence. Washington: AAAI, 2023: 2821-2829.
[27] WEI Xiushen, WANG Peng, LIU Lingqiao, et al. Piecewise classifier mappings: learning fine-grained learners for novel categories with few examples[J]. IEEE transactions on image processing, 2019, 28(12): 6116-6125.
[28] RUAN Xiaoqian, LIN Guosheng, LONG Cheng, et al. Few-shot fine-grained classification with Spatial Attentive Comparison[J]. Knowledge-based systems, 2021, 218: 106840.
[29] XU Shulin, ZHANG Faen, WEI Xiushen, et al. Dual attention networks for few-shot fine-grained recognition[J]. Proceedings of the AAAI conference on artificial intelligence, 2022, 36(3): 2911-2919.
[30] MUNJAL B, FLABOREA A, AMIN S, et al. Query-guided networks for few-shot fine-grained classification and person search[J]. Pattern recognition, 2023, 133: 109049.
[31] 解耀华, 章为川, 任劼, 等. 基于自适应特征融合的小样本细粒度图像分类[J]. 计算机工程与应用, 2023, 59(3): 184-192.
XIE Yaohua, ZHANG Weichuan, REN Jie, et al. Adaptive feature fusion embedding network for few shot fine-grained image classification[J]. Computer engineering and applications, 2023, 59(3): 184-192.
[32] HUANG Huaxi, ZHANG Junjie, ZHANG Jian, et al. Low-rank pairwise alignment bilinear network for few-shot fine-grained image classification[J]. IEEE transactions on multimedia, 2020, 23: 1666-1680.
[33] ZHU Yaohui, LIU Chenlong, JIANG Shuqiang. Multi-attention meta learning for few-shot fine-grained image recognition[C]//Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence. Yokohama: ACM, 2020: 1090-1096.
[34] SHEN Yantao, XIAO Tong, LI Hongsheng, et al. End-to-end deep kronecker-product matching for person re-identification[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 6886-6895.
[35] LECUN Y, BOTTOU L, BENGIO Y, et al. Gradient-based learning applied to document recognition[J]. Proceedings of the IEEE, 1998, 86(11): 2278-2324.
[36] WAH C, BRANSON S, WELINDER P, et al. The Caltech-UCSD birds-200-2011 dataset[R]. California: California Institute of Technology, 2011.
[37] KRAUSE J, STARK M, JIA Deng, et al. 3D object representations for fine-grained categorization[C]//2013 IEEE International Conference on Computer Vision Workshops. Sydney: IEEE, 2013: 554-561.
[38] DENG Jia, DONG Wei, SOCHER R, et al. ImageNet: a large-scale hierarchical image database[C]//2009 IEEE Conference on Computer Vision and Pattern Recognition. Miami: IEEE, 2009: 248-255.
[39] GARCIA V, BRUNA J. Few-shot learning with graph neural networks[EB/OL]. (2017-11-10)[2023-09-01]. https://arxiv.org/abs/1711.04043.
[40] WANG Xiaoru, MA Bing, YU Zhihong, et al. Multi-scale decision network with feature fusion and weighting for few-shot learning[J]. IEEE access, 2020, 8: 92172-92181.
[41] XUE Zhiyu, XIE Zhenshan, XING Zheng, et al. Relative position and map networks in few-shot learning for image classification[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. Seattle: IEEE, 2020: 4032-4036.
[42] XUE Z, DUAN Lixin, LI Wen, et al. Region comparison network for interpretable few-shot image classification[EB/OL]. (2020-09-08)[2023-09-01]. https://arxiv.org/abs/2009.03558.
- 相似文献/References:
-
[1]马岭,鲁越,蒋慧琴,等.基于小样本学习的LCD产品缺陷自动检测方法[J].智能系统学报,2020,15(3):560.[doi:10.11992/tis.201904020]
MA Ling,LU Yue,JIANG Huiqin,et al.An automatic small sample learning-based detection method for LCD product defects[J].CAAI Transactions on Intelligent Systems,2020,15():560.[doi:10.11992/tis.201904020]
[2]郭茂祖,王偲佳,王鹏跃,等.基于卫星图的小样本街区品质评估[J].智能系统学报,2022,17(6):1254.[doi:10.11992/tis.202111049]
GUO Maozu,WANG Sijia,WANG Pengyue,et al.Small sample block quality evaluation based on satellite images[J].CAAI Transactions on Intelligent Systems,2022,17():1254.[doi:10.11992/tis.202111049]
[3]周凯锐,刘鑫,景丽萍,等.概念驱动的小样本判别特征学习方法[J].智能系统学报,2023,18(1):162.[doi:10.11992/tis.202203061]
ZHOU Kairui,LIU Xin,JING Liping,et al.Concept-driven discriminative feature learning for few-shot learning[J].CAAI Transactions on Intelligent Systems,2023,18():162.[doi:10.11992/tis.202203061]
[4]季友昌,袁伟伟,毛善斌,等.不平衡小样本基于局部域对抗适应网络的发动机振动预测模型[J].智能系统学报,2023,18(5):1005.[doi:10.11992/tis.202210030]
JI Youchang,YUAN Weiwei,MAO Shanbin,et al.Partial domain adversarial adaptation networks for imbalanced small samples in aeroengine vibration prediction[J].CAAI Transactions on Intelligent Systems,2023,18():1005.[doi:10.11992/tis.202210030]
备注/Memo
收稿日期:2023-9-1。
基金项目:国家自然科学基金项目(62141201)；重庆市教委科学技术研究项目(KJZD-M202201102).
作者简介:李小菲，硕士研究生，主要研究方向为计算机视觉、人工智能。E-mail：sofina612@163.com;苟光磊，副教授，主要研究方向为智能信息处理、人工智能。已主持重庆市科委基础科学与前沿技术研究项目、重庆市教委科学技术研究项目等科研项目5项，授权发明专利3项，发表学术论文20余篇，出版专著1部、教材2部。E-mail：ggl@cqut.edu.cn;韩岩奇，硕士研究生，中国计算机学会会员，主要研究方向为计算机视觉、小样本学习。E-mail：hanyanqi0408@163.com。
通讯作者:苟光磊. E-mail：ggl@cqut.edu.cn
更新日期/Last Update:
2024-09-05