[1]LI Xiaofei,GOU Guanglei,HAN Yanqi,et al.Multiscale knowledge-guided local enhancement method for few-shot fine-grained image classification[J].CAAI Transactions on Intelligent Systems,2024,19(5):1157-1167.[doi:10.11992/tis.202309003]
Copy

Multiscale knowledge-guided local enhancement method for few-shot fine-grained image classification

CAAI Transactions on Intelligent Systems[ISSN 1673-4785/CN 23-1538/TP] Volume: 19 Number of periods: 2024 5 Page number: 1157-1167 Column: 学术论文—机器感知与模式识别 Public date: 2024-09-05
Title:
Multiscale knowledge-guided local enhancement method for few-shot fine-grained image classification
Author(s):
LI Xiaofei;  GOU Guanglei;  HAN Yanqi;  ZHU Donghua
College of Computer Science and Engineering, Chongqing University of Technology, Chongqing 400054, China
Keywords:
few-shot learning; fine-grained image; image classification; multi-scale feature; relation network; deep learning; data augmentation; semantical correlations
CLC:
TP391
DOI:
10.11992/tis.202309003
Abstract:
In few-shot fine-grained image classification tasks, the absence of local correlation between support and query samples causes difficulty in precisely identifying discriminative regions in images. To solve this problem, this study proposes a multiscale knowledge-guided local enhancement method for few-shot fine-grained image classification. Multiple sub-images with different resolutions are obtained as input images by reducing the sampling rate downward using the image pyramid and by integrating multi-scale features to enrich the sample information. A knowledge-guided module is employed to capture semantic correlations between support and query samples, which enhances the feature representation of important regions in support samples. Kronecker product operations are performed on the embedded feature maps of support and query samples to generate spatial correlation maps. This way enables more accurate localization of the positional correspondence between sample features and highlights the discriminative area. Experimental results confirm the robust classification performance of the method in the classification task of few-shot fine-grained images.
References:
[1] HE Kaiming, ZHANG Xiangyu, REN Shaoqing, et al. Deep residual learning for image recognition[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016: 770-778.
[2] 葛轶洲, 刘恒, 王言, 等. 小样本困境下的深度学习图像识别综述[J]. 软件学报, 2022, 33(1): 193-210.
GE Yizhou, LIU Heng, WANG Yan, et al. Survey on deep learning image recognition in dilemma of small samples[J]. Journal of software, 2022, 33(1): 193-210.
[3] WEN Qi, LI Shuang, HAN Bingfeng, et al. ZiGAN: fine-grained Chinese calligraphy font generation via a few-shot style transfer approach[C]//Proceedings of the 29th ACM International Conference on Multimedia. Virtual Event: ACM, 2021: 621-629.
[4] ZHANG Bo, CHEN Tao, WANG Bin, et al. Joint distribution alignment via adversarial learning for domain adaptive object detection[J]. IEEE transactions on multimedia, 2022, 24: 4102-4112.
[5] 杨祺, 孙俊. 融合多粒度特征的细粒度图像分类网络[J]. 小型微型计算机系统, 2023, 44(4): 818-824.
YANG Qi, SUN Jun. Multi-granularity feature fusion network for fine-grained visual classification[J]. Journal of Chinese computer systems, 2023, 44(4): 818-824.
[6] HE Ju, CHEN Jieneng, LIU Shuai, et al. TransFG: a transformer architecture for fine-grained recognition[C]//Proceedings of the AAAI conference on artificial intelligence. Vancouver: AAAI, 2022: 852-860.
[7] 许可凡. 基于注意力机制的细粒度图像分类技术的研究与应用[D]. 北京: 北京邮电大学, 2023.
XU Kefan. Research and application of fine-grained image classification technology based on attention mechanism[D]. Beijing: Beijing University of Posts and Telecommunications, 2023.
[8] 苗培昊. 基于度量学习的小样本细粒度图像识别研究[D]. 徐州: 中国矿业大学, 2022.
MIAO Peihao. Research on small sample fine-grained image recognition based on metric learning[D]. Xuzhou: China University of Mining and Technology, 2022.
[9] 周凯锐, 刘鑫, 景丽萍, 等. 概念驱动的小样本判别特征学习方法[J]. 智能系统学报, 2023, 18(1): 162-172.
ZHOU Kairui, LIU Xin, JING Liping, et al. Concept-driven discriminative feature learning for few-shot learning[J]. CAAI transactions on intelligent systems, 2023, 18(1): 162-172.
[10] YANG Shuo, LIU Lu, XU Min. Free lunch for few-shot learning: distribution calibration[EB/OL]. (2021-01-16)[2023-09-01]. https://arxiv.org/abs/2101.06395.
[11] VINYALS O, BLUNDELL C, LILLICRAP T, et al. Matching networks for one shot learning[J]. Advances in neural information processing systems, 2016: 3637-3645.
[12] SNELL J, SWERSKY K, ZEMEL R. Prototypical networks for few-shot learning[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems. Long Beach: ACM. 2017: 4080-4090.
[13] SUNG F, YANG Yongxin, ZHANG Li, et al. Learning to compare: relation network for few-shot learning[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 1199-1208.
[14] LEE S, MOON W, HEO J P. Task discrepancy maximization for fine-grained few-shot classification[C]//2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New Orleans: IEEE, 2022: 5321-5330.
[15] HARIHARAN B, GIRSHICK R. Low-shot visual recognition by shrinking and hallucinating features[C]//2017 IEEE International Conference on Computer Vision. Venice: IEEE, 2017: 3037-3046.
[16] KOCH G, ZEMEL R, SALAKHUTDINOV R. Siamese neural networks for one-shot image recognition[C]//Proceedings of the 32 nd International Conference on Machine Learning-Volume 37. Lille: JMLR, 2015: 1-8.
[17] HOU Ruibing, CHANG Hong, MA Bingpeng, et al. Cross attention network for few-shot classification[EB/OL]. (2019-10-17)[2023-09-01]. https://arxiv.org/abs/1910.07677.
[18] LI Wenbin, WANG Lei, XU Jinglin, et al. Revisiting local descriptor based image-to-class measure for few-shot learning[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach: IEEE, 2019: 7253-7260.
[19] WU Ziyang, LI Yuwei, GUO Lihua, et al. PARN: position-aware relation networks for few-shot learning[C]//2019 IEEE/CVF International Conference on Computer Vision. Seoul: IEEE, 2019: 6658-6666.
[20] ABDELAZIZ M, ZHANG Zuping. Multi-scale kronecker-product relation networks for few-shot learning[J]. Multimedia tools and applications, 2022, 81(5): 6703-6722.
[21] XIE Lingxi, TIAN Qi, HONG Richang, et al. Hierarchical part matching for fine-grained visual categorization[C]//2013 IEEE International Conference on Computer Vision. Sydney: IEEE, 2013: 1641-1648.
[22] ZHANG Ning, DONAHUE J, GIRSHICK R, et al. Part-based R-CNNs for fine-grained category detection[C]//European Conference on Computer Vision. Cham: Springer, 2014: 834-849.
[23] ZHANG Yu, WEI Xiushen, WU Jianxin, et al. Weakly supervised fine-grained categorization with part-based image representation[J]. IEEE transactions on image processing, 2016, 25(4): 1713-1725.
[24] SONG Kaitao, WEI Xiushen, SHU Xiangbo, et al. Bi-modal progressive mask attention for fine-grained recognition[J]. IEEE transactions on image processing, 2020, 29: 7006-7018.
[25] TANG Hao, YUAN Chengcheng, LI Zechao, et al. Learning attention-guided pyramidal features for few-shot fine-grained recognition[J]. Pattern recognition, 2022, 130: 108792.
[26] WU Jijie, CHANG Dongliang, SAIN A, et al. Bi-directional feature reconstruction network for fine-grained few-shot image classification[C]//Proceedings of the AAAI conference on artificial intelligence. Washington: AAAI, 2023: 2821-2829.
[27] WEI Xiushen, WANG Peng, LIU Lingqiao, et al. Piecewise classifier mappings: learning fine-grained learners for novel categories with few examples[J]. IEEE transactions on image processing, 2019, 28(12): 6116-6125.
[28] RUAN Xiaoqian, LIN Guosheng, LONG Cheng, et al. Few-shot fine-grained classification with Spatial Attentive Comparison[J]. Knowledge-based systems, 2021, 218: 106840.
[29] XU Shulin, ZHANG Faen, WEI Xiushen, et al. Dual attention networks for few-shot fine-grained recognition[J]. Proceedings of the AAAI conference on artificial intelligence, 2022, 36(3): 2911-2919.
[30] MUNJAL B, FLABOREA A, AMIN S, et al. Query-guided networks for few-shot fine-grained classification and person search[J]. Pattern recognition, 2023, 133: 109049.
[31] 解耀华, 章为川, 任劼, 等. 基于自适应特征融合的小样本细粒度图像分类[J]. 计算机工程与应用, 2023, 59(3): 184-192.
XIE Yaohua, ZHANG Weichuan, REN Jie, et al. Adaptive feature fusion embedding network for few shot fine-grained image classification[J]. Computer engineering and applications, 2023, 59(3): 184-192.
[32] HUANG Huaxi, ZHANG Junjie, ZHANG Jian, et al. Low-rank pairwise alignment bilinear network for few-shot fine-grained image classification[J]. IEEE transactions on multimedia, 2020, 23: 1666-1680.
[33] ZHU Yaohui, LIU Chenlong, JIANG Shuqiang. Multi-attention meta learning for few-shot fine-grained image recognition[C]//Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence. Yokohama: ACM, 2020: 1090-1096.
[34] SHEN Yantao, XIAO Tong, LI Hongsheng, et al. End-to-end deep kronecker-product matching for person re-identification[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 6886-6895.
[35] LECUN Y, BOTTOU L, BENGIO Y, et al. Gradient-based learning applied to document recognition[J]. Proceedings of the IEEE, 1998, 86(11): 2278-2324.
[36] WAH C, BRANSON S, WELINDER P, et al. The Caltech-UCSD birds-200-2011 dataset[R]. California: California Institute of Technology, 2011.
[37] KRAUSE J, STARK M, JIA Deng, et al. 3D object representations for fine-grained categorization[C]//2013 IEEE International Conference on Computer Vision Workshops. Sydney: IEEE, 2013: 554-561.
[38] DENG Jia, DONG Wei, SOCHER R, et al. ImageNet: a large-scale hierarchical image database[C]//2009 IEEE Conference on Computer Vision and Pattern Recognition. Miami: IEEE, 2009: 248-255.
[39] GARCIA V, BRUNA J. Few-shot learning with graph neural networks[EB/OL]. (2017-11-10)[2023-09-01]. https://arxiv.org/abs/1711.04043.
[40] WANG Xiaoru, MA Bing, YU Zhihong, et al. Multi-scale decision network with feature fusion and weighting for few-shot learning[J]. IEEE access, 2020, 8: 92172-92181.
[41] XUE Zhiyu, XIE Zhenshan, XING Zheng, et al. Relative position and map networks in few-shot learning for image classification[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. Seattle: IEEE, 2020: 4032-4036.
[42] XUE Z, DUAN Lixin, LI Wen, et al. Region comparison network for interpretable few-shot image classification[EB/OL]. (2020-09-08)[2023-09-01]. https://arxiv.org/abs/2009.03558.
Similar References:

Memo

-

Last Update: 2024-09-05

Copyright © CAAI Transactions on Intelligent Systems