[1]苗春玲,张红云,吴卓嘉,等.多粒度遮挡特征增强的行人搜索算法[J].智能系统学报,2025,20(1):230-242.[doi:10.11992/tis.202407031]
MIAO Chunling,ZHANG Hongyun,WU Zhuojia,et al.Multi-granularity occlusion feature enhancement algorithm for person search[J].CAAI Transactions on Intelligent Systems,2025,20(1):230-242.[doi:10.11992/tis.202407031]
点击复制
MIAO Chunling,ZHANG Hongyun,WU Zhuojia,et al.Multi-granularity occlusion feature enhancement algorithm for person search[J].CAAI Transactions on Intelligent Systems,2025,20(1):230-242.[doi:10.11992/tis.202407031]
多粒度遮挡特征增强的行人搜索算法
《智能系统学报》[ISSN 1673-4785/CN 23-1538/TP] 卷:
20
期数:
2025年第1期
页码:
230-242
栏目:
吴文俊人工智能科学技术奖论坛
出版日期:
2025-01-05
- Title:
- Multi-granularity occlusion feature enhancement algorithm for person search
- Keywords:
- deep learning; computer vision; person search; object detection; granular computing; data processing; feature extraction; generative adversarial networks; alignment
- 分类号:
- TP389.1
- 摘要:
- 现有行人搜索方法着重于从有限的标注场景图中学习有效的行人表征，虽然这些方法取得了一定的效果，但学习更具有身份辨别力的行人表征通常依赖于大规模的标注数据，而获取大规模的标注数据是一个资源、劳动密集型的过程。为此，该文提出了一种场景图多粒度遮挡特征增强算法，对原始场景图进行多粒度随机遮挡，扩充训练数据，并从遮挡后的场景图中生成具有多样化信息的虚拟特征，最后利用生成的虚拟特征增强真实特征中的行人表征。进一步，基于生成对抗学习，该文设计了多粒度特征对齐模块，用于对齐遮挡图像特征和原始图像特征，保持两者语义一致性。实验结果表明，在CUHK-SYSU和PRW数据集上，该算法能够显著提升行人搜索任务的搜索精度。
- Abstract:
- Existing person search methods focus on efficiently learning pedestrian representations from limited labeled scene images. Although these methods have achieved good results, learning more identity-discriminative pedestrian representations usually relies on large-scale labeled images, while obtaining large-scale labeled data is a resource and labor intensive process. Therefore, we propose a novel multi-granularity occlusion feature enhancement algorithm for person search, which first performs multi-granularity random occlusion on original scene images to expand the training data, and then generates virtual features with diverse information from the occluded scene images. Finally, the generated virtual features are used to enhance the pedestrian representation in the real features. Furthermore, based on generative adversarial learning, a multi-granularity feature alignment module is designed to align the occluded image features and the original image features, and thereby maintain their semantic consistency. Experiments on CUHK-SYSU and PRW datasets show that the proposed algorithm can significantly improve the search accuracy of person search.
- 参考文献/References:
-
[1] XIAO Tong, LI Shuang, WANG Bochao. Joint detection and identification feature learning for person search[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition. Los Alamitos: IEEE Computer Society, 2017: 3415-3424.
[2] REN Shaoqing, HE Kaiming, GIRSHICK R, et al. Faster R-CNN: towards real-time object detection with region proposal networks[J]. IEEE transactions on pattern analysis and machine intelligence, 2017, 39(6): 1137-1149.
[3] TIAN Zhi, SHEN Chunhua, CHEN Hao, et al. FCOS: Fully convolutional one-stage object detection[C]//2019 IEEE/CVF International Conference on Computer Vision. Los Alamitos: IEEE Computer Society, 2019: 9626-9635.
[4] CARION N, MASSA F, SYNNAEVE G, et al. End-to-end object detection with transformers[C]// Proceedings of the 16th European Conference on Computer Vision. Cham: Springer, 2020: 213-229.
[5] CHEN Di, ZHANG Shanshan, YANG Jian, et al. Norm-aware embedding for efficient person search[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Los Alamitos: IEEE Computer Society, 2020: 12612-12621.
[6] 张帅宇, 彭力, 戴菲菲. 基于渐进式注意力金字塔的行人重识别方法[J]. 计算机科学, 2023, 50(S1): 452-459.
ZHANG Shuaiyu, PENG Li, DAI Feifei. Person re-identification method based on progressive attention pyramid[J]. Computer science, 2023, 50(S1): 452-459.
[7] 杨静, 张灿龙, 李志欣, 等. 集成空间注意力和姿态估计的遮挡行人再辨识[J]. 计算机研究与发展, 2022, 59(7): 1522-1532.
YANG Jing, ZHANG Canlong, LI Zhixin, et al. Integrated spatial attention and pose estimation for occluded person re-identification[J]. Journal of computer research and development, 2022, 59(7): 1522-1532.
[8] JAFFE L, ZAKHOR A. Gallery filter network for person search[C]//2023 IEEE/CVF Winter Conference on Appli- cations of Computer Vision. Los Alamitos: IEEE Computer Society, 2023: 1684-1693.
[9] YU Rui, DU Dawei, LALONDE R, et al. Cascade transformers for end-to-end person search[C]//2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Los Alamitos: IEEE Computer Society, 2022: 7257-7266.
[10] ZHONG Zhun, ZHENG Liang, KANG Guoliang, et al. Random erasing data augmentation[C]//Proceedings of the 34th AAAI Conference on Artificial Intelligence. Palo Alto: AAAI: 2020, 13001-13008.
[11] DEVRIES T, TAYLOR G W. Improved regularization of convolutional neural networks with cutout[EB/OL]. (2017-08-15)[2024-07-10]. https://doi.org/10.48550/arXiv.1708.04552.
[12] CHEN Pengguang, LIU Shu, ZHAO Henghuang, et al. GridMask data augmentation[EB/OL]. (2017-01-17) [2024-07-10].https://doi.org/10.48550/arXiv.2001.04086 .
[13] LI Junjie, YAN Yichao, WANG Guanshuo, et al. Domain adaptive person search[C]//Proceedings of the 17th European Conference on Computer Vision. Cham: Springer, 2022: 302-318.
[14] WANG Wenhao, LIAO Shengcai, ZHAO Fang, et al. Domainmix: learning generalizable person re-identifica- tion without human annotations[EB/OL]. (2020-11-24) [2024-07-10].https://doi.org/10.48550/arXiv.2011.11953 .
[15] CHEN Weihua, XU Xianzhe, JIA Jian, et al. Beyond appearance a semantic controllable self-supervised learning framework for human-centric visual tasks[C]// 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Los Alamitos: IEEE Computer Society, 2023: 15050-15061.
[16] 程德强, 马尚, 寇旗旗, 等. 基于YOLOv4改进特征融合及全局感知的目标检测算法[J]. 智能系统学报, 2024, 19(2): 325-334.
CHENG Deqiang, MA Shang, KOU Qiqi, et al. Target detection algorithm for improving feature fusion and global perception based on YOLOv4[J]. CAAI transactions on intelligent systems, 2024, 19(2): 325-334.
[17] ZHENG Liang, ZHANG Hengheng, SUN Shaoyan, et al. Person re-identification in the wild[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition. Los Alamitos: IEEE Computer Society, 2017: 3346-3355.
[18] CHEN Di, ZHANG Shanshan, OUYANG Wanli, et al. Person search via a mask-guided two-stream CNN model[C]//Proceedings of the 15th European Conference on Computer Vision. Cham: Springer, 2018: 764-781.
[19] DONG Wenkai, ZHANG Zhaoxiang, SONG Chunfeng, et al. Instance guided proposal network for person search[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Los Alamitos: IEEE Computer Society, 2020: 2585-2594.
[20] LI Zhengjia, MIAO Duoqian. Sequential end-to-end network for efficient person search[C]// Proceedings of the 35th AAAI Conference on Artificial Intelligence. Palo Alto: AAAI, 2021: 2011-2019.
[21] ZHANG Qixian, WU Jun, MIAO Duoqian, et al. Attentive multi-granularity perception network for person search[J]. Information sciences, 2024, 681: 121191.
[22] ZHANG Qixian, MIAO Duoqian, ZHANG Qi, et al. Learning adaptive shift and task decoupling for discriminative one-step person search[J]. Knowledge-based systems, 2024, 304: 112483.
[23] YAN Yichao, LI Jinpeng, QIN Jie, et al. Anchor-free person search[C]//2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Los Alamitos: IEEE Computer Society, 2021: 7686-7695.
[24] 邓家欣. 基于深度神经网络的行人搜索方法研究[D]. 成都: 电子科技大学, 2024.
DENG Jiaxin. Research on person search method based on deep neural network[D]. Chengdu: University of Electronic Science and Technology of China, 2024.
[25] CAO Jiale, PANG Yanwei, ANWER R M, et al. PSTR: end-to-end one-step person search with transformers[C]// 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Los Alamitos: IEEE Computer Society, 2022: 9448-9457.
[26] 祁磊, 任子豪, 刘俊汐, 等. 虚实结合的行人重识别方法[J/OL]. 计算机研究与发展. [2024-07-10]. http://kns.cnki.net/kcms/detail/11.1777.TP.20240108.1607.014.html.
QI Lei, REN Zihao, LIU Junxi, et al. Person re-identification method based on hybrid real-synthetic data[J/OL]. Journal of computer research and development. [2024-07-10]. http://kns.cnki.net/kcms/detail/11.1777. TP.20240108.1607.014.html.
[27] YUN Sangdoo, HAN Dongyoon, CHUN Sanghyuk, et al. CutMix: regularization strategy to train strong classifiers with localizable features[C]//2019 International Confer- ence on Computer Vision. Los Alamitos: IEEE Computer Society, 2019: 6022-6031.
[28] 戴臣超, 王洪元, 倪彤光, 等. 基于深度卷积生成对抗网络和拓展近邻重排序的行人重识别[J]. 计算机研究与发展, 2019, 56(8): 1632-1641.
DAI Chenchao, WANG Hongyuan, NI Tongguang, et al. Person re-identification based on deep convolutional generative adversarial network and expanded neighbor reranking[J]. Journal of computer research and development, 2019, 56(8): 1632-1641.
[29] WEI Longhui, ZHANG Shiliang, GAO Wen, et al. Person transfer gan to bridge domain gap for person re-identification[C]//2018 IEEE/CVF Conference on Com- puter Vision and Pattern Recognition. Los Alamitos: IEEE Computer Society, 2018: 79-88.
[30] ZHENG Zhedong, YANG Xiaodong, YU Zhiding, et al. Joint discriminative and generative learning for person re-identification[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Los Alamitos: IEEE Computer Society, 2019: 2138-2147.
[31] 苗夺谦, 张清华, 钱宇华, 等. 从人类智能到机器实现模型-粒计算理论与方法[J]. 智能系统学报, 2016, 11(6): 743-757.
MIAO Duoqian, ZHANG Qinghua, QIAN Yuhua, et al. From human intelligence to machine implementation model: theories and applications based on granular computing[J]. CAAI transactions on intelligent systems, 2016, 11(6): 743-757.
[32] 郭豆豆, 徐伟华. R-FCCL: 一种面向高维数据的模糊概念认知学习方法[J/OL]. 计算机研究与发展. [2024-03-09]. http://kns.cnki.net/kcms/detail/11.1777.TP.20240307.1525.002.html.
GUO Doudou, XU Weihua. R-FCCL: a novel fuzzy-based concept-cognitive learning approach for high-dimensional data [J/OL]. Journal of computer research and development. [2024-03-09]. http://kns.cnki.net/kcms/detail/11.1777.TP.20240307.1525.002.html.
[33] LI Yanping, MIAO Duoqian, ZHANG Hongyun, et al. Multi-granularity cross transformer network for person re-identification[J]. Pattern recognition, 2024, 150: 110362.
[34] 杨玉婷, 苗夺谦. 基于多粒度匹配的行人搜索算法[J]. 智能系统学报, 2022, 17(2): 420-426.
YANG Yuting, MIAO Duoqian. Person search algorithm based on multi-granularity matching[J]. CAAI transactions on intelligent systems, 2022, 17(2): 420-426.
[35] LIU Zhuang, MAO Hanzi, WU Chaoyuan, et al. A convnet for the 2020s[C]//2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Los Alamitos: IEEE Computer Society, 2022: 11976-11986.
[36] VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems. New York: Curran Associates, 2017: 6000-6010.
[37] LIU Hao, FENG Jiashi, JIE Zequn, et al. Neural person search machines[C]//2017 IEEE/CVF International Conference on Computer Vision. Los Alamitos: IEEE Computer Society, 2017: 493-501.
[38] MUNJAL B, AMIN S, TOMBARI F, et al. Query-guided end-to-end person search[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Los Alamitos: IEEE Computer Society, 2019: 811-820.
[39] 赵才荣, 齐鼎, 窦曙光, 等. 智能视频监控关键技术: 行人再识别研究综述[J]. 中国科学: 信息科学, 2021, 52(12): 1979-2015.
ZHAO Cairong, QI Ding, DOU Shuguang, et al. Key technology for intelligent video surveillance: a review of person re-identification[J]. Scientia sinica informationis, 2021, 52(12): 1979-2015.
[40] LAN Xu, ZHU Xiatian, GONG Shaogang. Person search by multi-scale matching[C]//Proceedings of the 15th European Conference on Computer Vision. Cham: Springer, 2018: 536-552.
[41] HAN Chuchu, YE Jiacheng, ZHONG Yunshan, et al. Re-ID driven localization refinement for person search[C]//2019 IEEE/CVF International Conference on Computer Vision. Los Alamitos: IEEE Computer Society, 2019: 9813-9822.
[42] WANG Cheng, MA Bingpeng, CHEN Xilin, et al. TCTS: a task-consistent two-stage framework for person search[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Los Alamitos: IEEE Computer Society, 2020: 11949-11958.
[43] XIAO Jimin, XIE Yanchun, TILLO T, et al. IAN: the individual aggregation network for person search[J]. Pattern recognition, 2019, 87: 332-340.
[44] HAN B J, KO K, SIM J Y. End-to-end trainable trident person search network using adaptive gradient propagation[C]//2021 IEEE/CVF International Conference on Computer Vision. Los Alamitos: IEEE Computer Society, 2021: 905-913.
[45] LI Yang, XU Huahu, BIAN Minjie, et al. Cross-scale global attention feature pyramid network for person search[J]. Image and vision computing, 2021, 116: 104332.
- 相似文献/References:
-
[1]夏 凡,王 宏.基于局部异常行为检测的欺骗识别研究[J].智能系统学报,2007,2(5):12.
XIA Fan,WANG Hong.Methodologies for deception detection based on abnormal b ehavior[J].CAAI Transactions on Intelligent Systems,2007,2():12.
[2]杨 戈,刘 宏.视觉跟踪算法综述[J].智能系统学报,2010,5(2):95.
YANG Ge,LIU Hong.Survey of visual tracking algorithms[J].CAAI Transactions on Intelligent Systems,2010,5():95.
[3]刘宏,李哲媛,许超.视错觉现象的分类和研究进展[J].智能系统学报,2011,6(1):1.
LIU Hong,LI Zheyuan,XU Chao.The categories and research advances of visual illusions[J].CAAI Transactions on Intelligent Systems,2011,6():1.
[4]叶果,程洪,赵洋.电影中吸烟活动识别[J].智能系统学报,2011,6(5):440.
YE Guo,CHENG Hong,ZHAO Yang.moking recognition in movies[J].CAAI Transactions on Intelligent Systems,2011,6():440.
[5]史晓鹏,何为,韩力群.采用Hough变换的道路边界检测算法[J].智能系统学报,2012,7(1):81.
SHI Xiaopeng,HE Wei,HAN Liqun.A road edge detection algorithm based on the Hough transform[J].CAAI Transactions on Intelligent Systems,2012,7():81.
[6]张媛媛,霍静,杨婉琪,等.深度信念网络的二代身份证异构人脸核实算法[J].智能系统学报,2015,10(2):193.[doi:10.3969/j.issn.1673-4785.201405060]
ZHANG Yuanyuan,HUO Jing,YANG Wanqi,et al.A deep belief network-based heterogeneous face verification method for the second-generation identity card[J].CAAI Transactions on Intelligent Systems,2015,10():193.[doi:10.3969/j.issn.1673-4785.201405060]
[7]丁科,谭营.GPU通用计算及其在计算智能领域的应用[J].智能系统学报,2015,10(1):1.[doi:10.3969/j.issn.1673-4785.201403072]
DING Ke,TAN Ying.A review on general purpose computing on GPUs and its applications in computational intelligence[J].CAAI Transactions on Intelligent Systems,2015,10():1.[doi:10.3969/j.issn.1673-4785.201403072]
[8]顾照鹏,刘宏.单目视觉同步定位与地图创建方法综述[J].智能系统学报,2015,10(4):499.[doi:10.3969/j.issn.1673-4785.201503003]
GU Zhaopeng,LIU Hong.A survey of monocular simultaneous localization and mapping[J].CAAI Transactions on Intelligent Systems,2015,10():499.[doi:10.3969/j.issn.1673-4785.201503003]
[9]赵军,於俊,汪增福.基于改进逆向运动学的人体运动跟踪[J].智能系统学报,2015,10(4):548.[doi:10.3969/j.issn.1673-4785.201403032]
ZHAO Jun,YU Jun,WANG Zengfu.Human motion tracking based on an improved inverse kinematics[J].CAAI Transactions on Intelligent Systems,2015,10():548.[doi:10.3969/j.issn.1673-4785.201403032]
[10]姬晓飞,王昌汇,王扬扬.分层结构的双人交互行为识别方法[J].智能系统学报,2015,10(6):893.[doi:10.11992/tis.201505006]
JI Xiaofei,WANG Changhui,WANG Yangyang.Human interaction behavior-recognition method based on hierarchical structure[J].CAAI Transactions on Intelligent Systems,2015,10():893.[doi:10.11992/tis.201505006]
[11]王科俊,赵彦东,邢向磊.深度学习在无人驾驶汽车领域应用的研究进展[J].智能系统学报,2018,13(1):55.[doi:10.11992/tis.201609029]
WANG Kejun,ZHAO Yandong,XING Xianglei.Deep learning in driverless vehicles[J].CAAI Transactions on Intelligent Systems,2018,13():55.[doi:10.11992/tis.201609029]
[12]周彦,李雅芳,王冬丽,等.视觉同时定位与地图创建综述[J].智能系统学报,2018,13(1):97.[doi:10.11992/tis.201703006]
ZHOU Yan,LI Yafang,WANG Dongli,et al.A survey of VSLAM[J].CAAI Transactions on Intelligent Systems,2018,13():97.[doi:10.11992/tis.201703006]
[13]孙必慎,石武祯,姜峰.计算视觉核心问题：自然图像先验建模研究综述[J].智能系统学报,2019,14(1):71.[doi:10.11992/tis.201804019]
SUN Bishen,SHI Wuzhen,JIANG Feng.Core problem in computer vision: survey of natural image prior models[J].CAAI Transactions on Intelligent Systems,2019,14():71.[doi:10.11992/tis.201804019]
[14]刘彪,黄蓉蓉,林和,等.基于卷积神经网络的盲文音乐识别研究[J].智能系统学报,2019,14(1):186.[doi:10.11992/tis.201805002]
LIU Biao,HUANG Rongrong,LIN He,et al.Research on braille music recognition based on convolutional neural networks[J].CAAI Transactions on Intelligent Systems,2019,14():186.[doi:10.11992/tis.201805002]
[15]朱文霖,刘华平,王博文,等.基于视-触跨模态感知的智能导盲系统[J].智能系统学报,2020,15(1):33.[doi:10.11992/tis.201908015]
ZHU Wenlin,LIU Huaping,WANG Bowen,et al.An intelligent blind guidance system based on visual-touch cross-modal perception[J].CAAI Transactions on Intelligent Systems,2020,15():33.[doi:10.11992/tis.201908015]
[16]张新钰,邹镇洪,李志伟,等.面向自动驾驶目标检测的深度多模态融合技术[J].智能系统学报,2020,15(4):758.[doi:10.11992/tis.202002010]
ZHANG Xinyu,ZOU Zhenhong,LI Zhiwei,et al.Deep multi-modal fusion in object detection for autonomous driving[J].CAAI Transactions on Intelligent Systems,2020,15():758.[doi:10.11992/tis.202002010]
[17]王凯诚,鲁华祥,龚国良,等.基于注意力机制的显著性目标检测方法[J].智能系统学报,2020,15(5):956.[doi:10.11992/tis.201903001]
WANG Kaicheng,LU Huaxiang,GONG Guoliang,et al.Salient object detection method based on the attention mechanism[J].CAAI Transactions on Intelligent Systems,2020,15():956.[doi:10.11992/tis.201903001]
[18]王照国,张红云,苗夺谦.基于F1值的非极大值抑制阈值自动选取方法[J].智能系统学报,2020,15(5):1006.[doi:10.11992/tis.202006056]
WANG Zhaoguo,ZHANG Hongyun,MIAO Duoqian.Automatic selection method of non-maximum suppression threshold based on F1 score[J].CAAI Transactions on Intelligent Systems,2020,15():1006.[doi:10.11992/tis.202006056]
[19]付常洋,王瑜,肖洪兵,等.基于深度学习与结构磁共振成像的抑郁症辅助诊断[J].智能系统学报,2021,16(3):544.[doi:10.11992/tis.201912006]
FU Changyang,WANG Yu,XIAO Hongbing,et al.Assisted diagnosis of major depression disorder using deep learning and structural magnetic resonance imaging[J].CAAI Transactions on Intelligent Systems,2021,16():544.[doi:10.11992/tis.201912006]
[20]董俊杰,刘华平,谢珺,等.基于反馈注意力机制和上下文融合的非模式实例分割[J].智能系统学报,2021,16(4):801.[doi:10.11992/tis.202007042]
DONG Junjie,LIU Huaping,XIE Jun,et al.Feedback attention mechanism and context fusion based amodal instance segmentation[J].CAAI Transactions on Intelligent Systems,2021,16():801.[doi:10.11992/tis.202007042]
备注/Memo
收稿日期:2024-7-25。
基金项目:国家重点研发计划项目（2022YFB3104700）；国家自然科学基金项目（62376198，62163016）.
作者简介:苗春玲，硕士研究生，主要研究方向为行人搜索和深度学习。E-mail：miaochunling@tongji.edu.cn。;张红云，副教授，博士生导师，主要研究方向为粒计算和计算机视觉。E-mail：zhanghongyun@tongji.edu.cn。;苗夺谦，教授，博士生导师，国际粗糙集学会会士，中国人工智能学会会士，嵌入式系统与服务计算教育部重点实验室副主任，上海市计算机学会副理事长，上海市人工智能学会副理事长。主要研究方向为人工智能、机器学习、粒度计算、粗糙集。主持完成国家自然科学基金项目6项，主持并参与省部级自然科学基金项目与科技攻关项目30余项。获得教育部科技进步一等奖、上海市技术发明一等奖、重庆市自然科学一等奖和中国人工智能学会吴文俊人工智能自然科学二等奖。发表学术论文180余篇。E-mail：dqmiao@tongji.edu.cn。
通讯作者:苗夺谦. E-mail：dqmiao@tongji.edu.cn
更新日期/Last Update:
2025-01-05