[1]MIAO Chunling,ZHANG Hongyun,WU Zhuojia,et al.Multi-granularity occlusion feature enhancement algorithm for person search[J].CAAI Transactions on Intelligent Systems,2025,20(1):230-242.[doi:10.11992/tis.202407031]
Copy

Multi-granularity occlusion feature enhancement algorithm for person search

CAAI Transactions on Intelligent Systems[ISSN 1673-4785/CN 23-1538/TP] Volume: 20 Number of periods: 2025 1 Page number: 230-242 Column: 吴文俊人工智能科学技术奖论坛 Public date: 2025-01-05
Title:
Multi-granularity occlusion feature enhancement algorithm for person search
Author(s):
MIAO Chunling1; 2;  ZHANG Hongyun1; 2;  WU Zhuojia1; 2;  ZHANG Qixian1; 2;  MIAO Duoqian1; 2
1. College of Electronic and Information Engineering, Tongji University, Shanghai 201804, China;
2. Key Laboratory of Embedded System and Service Computing Ministry of Education, Tongji University, Shanghai 201804, China
Keywords:
deep learning; computer vision; person search; object detection; granular computing; data processing; feature extraction; generative adversarial networks; alignment
CLC:
TP389.1
DOI:
10.11992/tis.202407031
Abstract:
Existing person search methods focus on efficiently learning pedestrian representations from limited labeled scene images. Although these methods have achieved good results, learning more identity-discriminative pedestrian representations usually relies on large-scale labeled images, while obtaining large-scale labeled data is a resource and labor intensive process. Therefore, we propose a novel multi-granularity occlusion feature enhancement algorithm for person search, which first performs multi-granularity random occlusion on original scene images to expand the training data, and then generates virtual features with diverse information from the occluded scene images. Finally, the generated virtual features are used to enhance the pedestrian representation in the real features. Furthermore, based on generative adversarial learning, a multi-granularity feature alignment module is designed to align the occluded image features and the original image features, and thereby maintain their semantic consistency. Experiments on CUHK-SYSU and PRW datasets show that the proposed algorithm can significantly improve the search accuracy of person search.
References:
[1] XIAO Tong, LI Shuang, WANG Bochao. Joint detection and identification feature learning for person search[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition. Los Alamitos: IEEE Computer Society, 2017: 3415-3424.
[2] REN Shaoqing, HE Kaiming, GIRSHICK R, et al. Faster R-CNN: towards real-time object detection with region proposal networks[J]. IEEE transactions on pattern analysis and machine intelligence, 2017, 39(6): 1137-1149.
[3] TIAN Zhi, SHEN Chunhua, CHEN Hao, et al. FCOS: Fully convolutional one-stage object detection[C]//2019 IEEE/CVF International Conference on Computer Vision. Los Alamitos: IEEE Computer Society, 2019: 9626-9635.
[4] CARION N, MASSA F, SYNNAEVE G, et al. End-to-end object detection with transformers[C]// Proceedings of the 16th European Conference on Computer Vision. Cham: Springer, 2020: 213-229.
[5] CHEN Di, ZHANG Shanshan, YANG Jian, et al. Norm-aware embedding for efficient person search[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Los Alamitos: IEEE Computer Society, 2020: 12612-12621.
[6] 张帅宇, 彭力, 戴菲菲. 基于渐进式注意力金字塔的行人重识别方法[J]. 计算机科学, 2023, 50(S1): 452-459.
ZHANG Shuaiyu, PENG Li, DAI Feifei. Person re-identification method based on progressive attention pyramid[J]. Computer science, 2023, 50(S1): 452-459.
[7] 杨静, 张灿龙, 李志欣, 等. 集成空间注意力和姿态估计的遮挡行人再辨识[J]. 计算机研究与发展, 2022, 59(7): 1522-1532.
YANG Jing, ZHANG Canlong, LI Zhixin, et al. Integrated spatial attention and pose estimation for occluded person re-identification[J]. Journal of computer research and development, 2022, 59(7): 1522-1532.
[8] JAFFE L, ZAKHOR A. Gallery filter network for person search[C]//2023 IEEE/CVF Winter Conference on Appli- cations of Computer Vision. Los Alamitos: IEEE Computer Society, 2023: 1684-1693.
[9] YU Rui, DU Dawei, LALONDE R, et al. Cascade transformers for end-to-end person search[C]//2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Los Alamitos: IEEE Computer Society, 2022: 7257-7266.
[10] ZHONG Zhun, ZHENG Liang, KANG Guoliang, et al. Random erasing data augmentation[C]//Proceedings of the 34th AAAI Conference on Artificial Intelligence. Palo Alto: AAAI: 2020, 13001-13008.
[11] DEVRIES T, TAYLOR G W. Improved regularization of convolutional neural networks with cutout[EB/OL]. (2017-08-15)[2024-07-10]. https://doi.org/10.48550/arXiv.1708.04552.
[12] CHEN Pengguang, LIU Shu, ZHAO Henghuang, et al. GridMask data augmentation[EB/OL]. (2017-01-17) [2024-07-10]. https://doi.org/10.48550/arXiv.2001.04086.
[13] LI Junjie, YAN Yichao, WANG Guanshuo, et al. Domain adaptive person search[C]//Proceedings of the 17th European Conference on Computer Vision. Cham: Springer, 2022: 302-318.
[14] WANG Wenhao, LIAO Shengcai, ZHAO Fang, et al. Domainmix: learning generalizable person re-identifica- tion without human annotations[EB/OL]. (2020-11-24) [2024-07-10]. https://doi.org/10.48550/arXiv.2011.11953.
[15] CHEN Weihua, XU Xianzhe, JIA Jian, et al. Beyond appearance a semantic controllable self-supervised learning framework for human-centric visual tasks[C]// 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Los Alamitos: IEEE Computer Society, 2023: 15050-15061.
[16] 程德强, 马尚, 寇旗旗, 等. 基于YOLOv4改进特征融合及全局感知的目标检测算法[J]. 智能系统学报, 2024, 19(2): 325-334.
CHENG Deqiang, MA Shang, KOU Qiqi, et al. Target detection algorithm for improving feature fusion and global perception based on YOLOv4[J]. CAAI transactions on intelligent systems, 2024, 19(2): 325-334.
[17] ZHENG Liang, ZHANG Hengheng, SUN Shaoyan, et al. Person re-identification in the wild[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition. Los Alamitos: IEEE Computer Society, 2017: 3346-3355.
[18] CHEN Di, ZHANG Shanshan, OUYANG Wanli, et al. Person search via a mask-guided two-stream CNN model[C]//Proceedings of the 15th European Conference on Computer Vision. Cham: Springer, 2018: 764-781.
[19] DONG Wenkai, ZHANG Zhaoxiang, SONG Chunfeng, et al. Instance guided proposal network for person search[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Los Alamitos: IEEE Computer Society, 2020: 2585-2594.
[20] LI Zhengjia, MIAO Duoqian. Sequential end-to-end network for efficient person search[C]// Proceedings of the 35th AAAI Conference on Artificial Intelligence. Palo Alto: AAAI, 2021: 2011-2019.
[21] ZHANG Qixian, WU Jun, MIAO Duoqian, et al. Attentive multi-granularity perception network for person search[J]. Information sciences, 2024, 681: 121191.
[22] ZHANG Qixian, MIAO Duoqian, ZHANG Qi, et al. Learning adaptive shift and task decoupling for discriminative one-step person search[J]. Knowledge-based systems, 2024, 304: 112483.
[23] YAN Yichao, LI Jinpeng, QIN Jie, et al. Anchor-free person search[C]//2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Los Alamitos: IEEE Computer Society, 2021: 7686-7695.
[24] 邓家欣. 基于深度神经网络的行人搜索方法研究[D]. 成都: 电子科技大学, 2024.
DENG Jiaxin. Research on person search method based on deep neural network[D]. Chengdu: University of Electronic Science and Technology of China, 2024.
[25] CAO Jiale, PANG Yanwei, ANWER R M, et al. PSTR: end-to-end one-step person search with transformers[C]// 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Los Alamitos: IEEE Computer Society, 2022: 9448-9457.
[26] 祁磊, 任子豪, 刘俊汐, 等. 虚实结合的行人重识别方法[J/OL]. 计算机研究与发展. [2024-07-10]. http://kns.cnki.net/kcms/detail/11.1777.TP.20240108.1607.014.html.
QI Lei, REN Zihao, LIU Junxi, et al. Person re-identification method based on hybrid real-synthetic data[J/OL]. Journal of computer research and development. [2024-07-10]. http://kns.cnki.net/kcms/detail/11.1777. TP.20240108.1607.014.html.
[27] YUN Sangdoo, HAN Dongyoon, CHUN Sanghyuk, et al. CutMix: regularization strategy to train strong classifiers with localizable features[C]//2019 International Confer- ence on Computer Vision. Los Alamitos: IEEE Computer Society, 2019: 6022-6031.
[28] 戴臣超, 王洪元, 倪彤光, 等. 基于深度卷积生成对抗网络和拓展近邻重排序的行人重识别[J]. 计算机研究与发展, 2019, 56(8): 1632-1641.
DAI Chenchao, WANG Hongyuan, NI Tongguang, et al. Person re-identification based on deep convolutional generative adversarial network and expanded neighbor reranking[J]. Journal of computer research and development, 2019, 56(8): 1632-1641.
[29] WEI Longhui, ZHANG Shiliang, GAO Wen, et al. Person transfer gan to bridge domain gap for person re-identification[C]//2018 IEEE/CVF Conference on Com- puter Vision and Pattern Recognition. Los Alamitos: IEEE Computer Society, 2018: 79-88.
[30] ZHENG Zhedong, YANG Xiaodong, YU Zhiding, et al. Joint discriminative and generative learning for person re-identification[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Los Alamitos: IEEE Computer Society, 2019: 2138-2147.
[31] 苗夺谦, 张清华, 钱宇华, 等. 从人类智能到机器实现模型-粒计算理论与方法[J]. 智能系统学报, 2016, 11(6): 743-757.
MIAO Duoqian, ZHANG Qinghua, QIAN Yuhua, et al. From human intelligence to machine implementation model: theories and applications based on granular computing[J]. CAAI transactions on intelligent systems, 2016, 11(6): 743-757.
[32] 郭豆豆, 徐伟华. R-FCCL: 一种面向高维数据的模糊概念认知学习方法[J/OL]. 计算机研究与发展. [2024-03-09]. http://kns.cnki.net/kcms/detail/11.1777.TP.20240307.1525.002.html.
GUO Doudou, XU Weihua. R-FCCL: a novel fuzzy-based concept-cognitive learning approach for high-dimensional data [J/OL]. Journal of computer research and development. [2024-03-09]. http://kns.cnki.net/kcms/detail/11.1777.TP.20240307.1525.002.html.
[33] LI Yanping, MIAO Duoqian, ZHANG Hongyun, et al. Multi-granularity cross transformer network for person re-identification[J]. Pattern recognition, 2024, 150: 110362.
[34] 杨玉婷, 苗夺谦. 基于多粒度匹配的行人搜索算法[J]. 智能系统学报, 2022, 17(2): 420-426.
YANG Yuting, MIAO Duoqian. Person search algorithm based on multi-granularity matching[J]. CAAI transactions on intelligent systems, 2022, 17(2): 420-426.
[35] LIU Zhuang, MAO Hanzi, WU Chaoyuan, et al. A convnet for the 2020s[C]//2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Los Alamitos: IEEE Computer Society, 2022: 11976-11986.
[36] VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems. New York: Curran Associates, 2017: 6000-6010.
[37] LIU Hao, FENG Jiashi, JIE Zequn, et al. Neural person search machines[C]//2017 IEEE/CVF International Conference on Computer Vision. Los Alamitos: IEEE Computer Society, 2017: 493-501.
[38] MUNJAL B, AMIN S, TOMBARI F, et al. Query-guided end-to-end person search[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Los Alamitos: IEEE Computer Society, 2019: 811-820.
[39] 赵才荣, 齐鼎, 窦曙光, 等. 智能视频监控关键技术: 行人再识别研究综述[J]. 中国科学: 信息科学, 2021, 52(12): 1979-2015.
ZHAO Cairong, QI Ding, DOU Shuguang, et al. Key technology for intelligent video surveillance: a review of person re-identification[J]. Scientia sinica informationis, 2021, 52(12): 1979-2015.
[40] LAN Xu, ZHU Xiatian, GONG Shaogang. Person search by multi-scale matching[C]//Proceedings of the 15th European Conference on Computer Vision. Cham: Springer, 2018: 536-552.
[41] HAN Chuchu, YE Jiacheng, ZHONG Yunshan, et al. Re-ID driven localization refinement for person search[C]//2019 IEEE/CVF International Conference on Computer Vision. Los Alamitos: IEEE Computer Society, 2019: 9813-9822.
[42] WANG Cheng, MA Bingpeng, CHEN Xilin, et al. TCTS: a task-consistent two-stage framework for person search[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Los Alamitos: IEEE Computer Society, 2020: 11949-11958.
[43] XIAO Jimin, XIE Yanchun, TILLO T, et al. IAN: the individual aggregation network for person search[J]. Pattern recognition, 2019, 87: 332-340.
[44] HAN B J, KO K, SIM J Y. End-to-end trainable trident person search network using adaptive gradient propagation[C]//2021 IEEE/CVF International Conference on Computer Vision. Los Alamitos: IEEE Computer Society, 2021: 905-913.
[45] LI Yang, XU Huahu, BIAN Minjie, et al. Cross-scale global attention feature pyramid network for person search[J]. Image and vision computing, 2021, 116: 104332.
Similar References:

Memo

-

Last Update: 2025-01-05

Copyright © CAAI Transactions on Intelligent Systems