[1]王路遥,王凤随,闫涛,等.结合多尺度特征与混淆学习的跨模态行人重识别[J].智能系统学报,2024,19(4):898-908.[doi:10.11992/tis.202304010]
WANG Luyao,WANG Fengsui,YAN Tao,et al.Cross-modal person re-identification combining multi-scale features and confusion learning[J].CAAI Transactions on Intelligent Systems,2024,19(4):898-908.[doi:10.11992/tis.202304010]
点击复制
WANG Luyao,WANG Fengsui,YAN Tao,et al.Cross-modal person re-identification combining multi-scale features and confusion learning[J].CAAI Transactions on Intelligent Systems,2024,19(4):898-908.[doi:10.11992/tis.202304010]
结合多尺度特征与混淆学习的跨模态行人重识别
《智能系统学报》[ISSN 1673-4785/CN 23-1538/TP] 卷:
19
期数:
2024年第4期
页码:
898-908
栏目:
学术论文—机器感知与模式识别
出版日期:
2024-07-05
- Title:
- Cross-modal person re-identification combining multi-scale features and confusion learning
- Keywords:
- machine vision; person re-identification; cross-modal; multi-scale characteristics; coarse-grain; fine-grain; confusion learning; modal independent attribute
- 分类号:
- TP391.4
- 摘要:
- 跨模态行人重识别研究的重难点主要来自于行人图像之间巨大的模态差异和模态内差异。针对这些问题，提出一种结合多尺度特征与混淆学习的网络结构。为实现高效的特征提取、缩小模态内差异，将网络设计为多尺度特征互补的形式，分别学习行人的局部细化特征与全局粗糙特征，从细粒度和粗粒度两方面来增强网络的特征表达能力。利用混淆学习策略，模糊网络的模态识别反馈，挖掘稳定且有效的模态无关属性应对模态差异，来提高特征对模态变化的鲁棒性。在大规模数据集SYSU-MM01的全搜索模式下该算法首位击中率和平均精度(mean average precision, mAP)的结果分别为76.69%和72.45%，在RegDB数据集的可见光到红外模式下该算法首位击中率和mAP的结果分别为94.62%和94.60%，优于现有的主要方法，验证了所提方法的有效性。
- Abstract:
- The difficulties of cross-modal person re-identification research mainly come from the huge modal differences and intra-modal differences between pedestrian images. To address these issues, a network structure combining multi-scale features with obfuscation learning is proposed. In order to achieve high-efficiency feature extraction and reduce intra-modal differences, the network is designed as a complementary form of multi-scale features to learn local refinement features and global rough features of pedestrians respectively. The feature expression ability of the network is enhanced from fine-grained and coarse-grained aspects. Confusion learning strategy is used to fuzzy the modal identification feedback of the network, and mine the stable and effective modal-independent attributes to cope with modal differences, so as to improve the robustness of features to modal changes. In the all-search mode of the large-scale data set SYSU-MM01, the results of the first hit rate and mean average precision (mAP) of the algorithm are 76.69% and 72.45%, respectively. In the Visible to Infrared mode of the RegDB data set, the results of the first hit rate and mAP of the algorithm are 94.62% and 94.60%, respectively, which are better than the main existing methods, verifying effectiveness of the proposed method.
- 参考文献/References:
-
[1] 宋婉茹, 赵晴晴, 陈昌红, 等. 行人重识别研究综述[J]. 智能系统学报, 2017, 12(6): 770–780
SONG Wanru, ZHAO Qingqing, CHEN Changhong, et al. Survey on pedestrian re-identification research[J]. CAAI transactions on intelligent systems, 2017, 12(6): 770–780
[2] 罗浩, 姜伟, 范星, 等. 基于深度学习的行人重识别研究进展[J]. 自动化学报, 2019, 45(11): 2032-2049.
LUO Hao, JIANG Wei, FAN Xing, et al. A survey on deep learning based person re-identification[J]. Acta automatica sinica, 2019, 45(11): 2032-2049.
[3] 刘帅师, 程曦, 郭文燕, 等. 深度学习方法研究新进展[J]. 智能系统学报, 2016, 11(5): 567–577
LIU Shuaishi, CHENG Xi, GUO Wenyan, et al. Progress report on new research in deep learning[J]. CAAI transactions on intelligent systems, 2016, 11(5): 567–577
[4] 邵晓雯, 帅惠, 刘青山. 融合属性特征的行人重识别方法[J]. 自动化学报, 2022, 48(2): 564–571
SHAO Xiaowen, SHUAI Hui, LIU Qingshan. Person re-identification based on fused attribute features[J]. Acta automatica sinica, 2022, 48(2): 564–571
[5] 石跃祥, 周玥. 基于阶梯型特征空间分割与局部注意力机制的行人重识别[J]. 电子与信息学报, 2022, 44(1): 195–202
SHI Yuexiang, ZHOU Yue. Person re-identification based on stepped feature space segmentation and local attention mechanism[J]. Journal of electronics & information technology, 2022, 44(1): 195–202
[6] 庄建军, 庄宇辰. 一种结构化双注意力混合通道增强的跨模态行人重识别方法[J]. 电子与信息学报, 2024, 46(2): 518–526
ZHUANG Jianjun, ZHUANG Yuchen. A cross-modal person re-identification method based on hybrid channel augmentation with structured dual attention[J]. Journal of electronics & information technology, 2024, 46(2): 518–526
[7] 孙锐, 张磊, 余益衡, 等. 基于局部异构聚合图卷积网络的跨模态行人重识别[J]. 电子学报, 2023, 51(4): 810–825
SUN Rui, ZHANG Lei, YU Yiheng, et al. Cross-modality person re-identification based on locally heterogeneous polymerization graph convolutional network[J]. Acta electronica sinica, 2023, 51(4): 810–825
[8] WU Ancong, ZHENG Weishi, YU Hongxing, et al. RGB-infrared cross-modality person re-identification[C]//The IEEE International Conference on Computer Vision. Venice: IEEE, 2017: 5390-5399.
[9] YE Mang, LAN Xiangyuan, LI Jiawei, et al. Hierarchical discriminative learning for visible thermal person re-identification[C]//Proceedings of the 32nd AAAI Conference on Artificial Intelligence. New Orleans: AAAI, 2018: 7501-7508.
[10] YE Mang, WANG Zheng, LAN Xiangyuan, et al. Visible thermal person re-identification via dual-constrained top-ranking[C]//Proceedings of the 27th International Joint Conference on Artificial Intelligence. Stockholm: Morgan Kaufmann, 2018: 1092-1099.
[11] ZHU Yuanxin, YANG Zhao, WANG Li, et al. Hetero-center loss for cross-modality person re-identification[J]. Neurocomputing, 2020, 386: 97–109.
[12] WANG Guanan, ZHANG Tianzhu, CHENG Jian, et al. RGB-infrared cross-modality person re-identification via joint pixel and feature alignment[C]//The IEEE/CVF International Conference on Computer Vision. Seoul: IEEE, 2019: 3622-3631.
[13] LI Diangang, WEI Xing, HONG Xiaopeng, et al. Infrared-visible cross-modal person re-identification with an x modality[C]//Proceedings of the 34th AAAI Conference on Artificial Intelligence. New York: AAAI, 2020: 4610-4617.
[14] ZHANG Yukang, YAN Yan, LU Yang, et al. Towards a unified middle modality learning for visible-infrared person re-identification[C]//Proceedings of the 29th ACM International Conference on Multimedia. Chengdu: ACM, 2021: 788-796.
[15] HE Kaiming, ZHANG Xiangyu, REN Shaoqing, et al. Deep residual learning for image recognition[C]//The IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016: 770-778.
[16] SUN Yifan, ZHENG Liang, YANG Yi, et al. Beyond part models: person retrieval with refined part pooling[C]//Proceedings of the European Conference on Computer Vision. Munich: IEEE, 2018: 480-496.
[17] RADENOVI? F, TOLIAS G, CHUM O. Fine-tuning CNN image retrieval with no human annotation[J]. IEEE transactions on pattern analysis and machine intelligence, 2019, 41(7): 1655–1668.
[18] HAO Xin, ZHAO Sanyuan, YE Mang, et al. Cross-modality person re-identification via modality confusion and center aggregation[C]//The IEEE/CVF International Conference on Computer Vision. Montreal: IEEE, 2021: 16383-16392.
[19] YE Mang, SHEN Jianbing, LIN Gaojie, et al. Deep learning for person re-identification: a survey and outlook[J]. IEEE transactions on pattern analysis and machine intelligence, 2022, 44(6): 2872–2893.
[20] LIU Haijun, TAN Xiaoheng, ZHOU Xichuan. Parameter sharing exploration and hetero-center triplet loss for visible-thermal person re-identification[J]. IEEE transactions on multimedia, 2021, 23: 4414–4425.
[21] LIU Jialun, SUN Yifan, ZHU Feng, et al. Learning memory-augmented unidirectional metrics for cross-modality person re-Identification[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. New Orleans: IEEE, 2022: 19366-19375.
[22] YE Mang, LAN Xiangyan, WANG Zheng, et al. Bi-directional center-constrained top-ranking for visible thermal person re-identification[J]. IEEE transactions on information forensics and security, 2020, 15: 407–419.
[23] WANG Zhixiang, WANG Zheng, ZHENG Yinqiang, et al. Learning to reduce dual-level discrepancy for infrared-visible person re-identification[C]//The IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach: IEEE, 2019: 618-626.
[24] WU Ancong, ZHENG Weishi, GONG Shaogang, et al. RGB-IR person re-identification by cross-modality similarity preservation[J]. International journal of computer vision, 2020, 128(6): 1765–1785.
[25] XIANG Xuezhi, LYU Ning, YU Zeting, et al. Cross-modality person re-identification based on dual-path multi-branch network[J]. IEEE sensors journal, 2019, 19(23): 11706–11713.
[26] BASARAN E, GOKMEN M, KAMASAK M E. An efficient framework for visible-infrared cross modality person re-identification[J]. Signal processing: image communication, 2020, 87: 115933.
[27] YE Mang, SHEN Jianbing, CRANDALL D J, et al. Dynamic dual-attentive aggregation learning for visible-infrared person re-identification[C]//The 16th European Conference on Computer Vision. Glasgow: Springer, 2020: 229-247.
[28] PU Nan, CHEN Wei, LIU Yu, et al. Dual gaussian-based variational subspace disentanglement for visible-infrared person re-identification[C]//Proceedings of the 28th ACM International Conference on Multimedia. Seattle: ACM, 2020: 2149-2158.
[29] LU Yan, WU Yue, LIU Bin, et al. Cross-modality person re-identification with shared-specific feature transfer[C]//The IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle: IEEE, 2020: 13376-13386.
[30] WU Qiong, DAI Pingyang, CHEN Jie, et al. Discover cross-modality nuances for visible-infrared person re-identification[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Nashville: IEEE, 2021: 4330-4339.
[31] 石林波, 李华锋, 张亚飞, 等. 模态不变性特征学习和一致性细粒度信息挖掘的跨模态行人重识别[J]. 模式识别与人工智能, 2022, 35(12): 1064–1077
SHI Linbo, LI Huafeng, ZHANG Yafei, et al. Modal invariance feature learning and consistent fine-grained information mining based cross-modal person re-identification[J]. Pattern recognition and artificial intelligence, 2022, 35(12): 1064–1077
[32] ZHONG Zhun, ZHENG Liang, CAO Donglin, et al. Re-ranking person re-identification with k-reciprocal encoding[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE, 2017: 3652-3661.
- 相似文献/References:
-
[1]田国会,吉艳青,李晓磊.家庭智能空间下基于场景的人的行为理解[J].智能系统学报,2010,5(1):57.
TIAN Guo-hui,JI Yan-qing,LI Xiao-lei.Human behaviors understanding based on scene knowledge in home intelligent space[J].CAAI Transactions on Intelligent Systems,2010,5():57.
[2]梁义辉,战强.一种面向无线图像传输的视觉平台[J].智能系统学报,2016,11(5):608.[doi:10.11992/tis.201512014]
LIANG Yihui,ZHAN Qiang.A visual platform for wireless image transmission[J].CAAI Transactions on Intelligent Systems,2016,11():608.[doi:10.11992/tis.201512014]
[3]李霞丽,吴立成,樊艳明.易于硬件实现的压缩感知观测矩阵的研究与构造[J].智能系统学报,2017,12(3):279.[doi:10.11992/tis.201606037]
LI Xiali,WU Licheng,FAN Yanming.Study and construction of a compressed sensing measurement matrix that is easy to implement in hardware[J].CAAI Transactions on Intelligent Systems,2017,12():279.[doi:10.11992/tis.201606037]
[4]宋婉茹,赵晴晴,陈昌红,等.行人重识别研究综述[J].智能系统学报,2017,12(6):770.[doi:10.11992/tis.201706084]
SONG Wanru,ZHAO Qingqing,CHEN Changhong,et al.Survey on pedestrian re-identification research[J].CAAI Transactions on Intelligent Systems,2017,12():770.[doi:10.11992/tis.201706084]
[5]郭晓峰,王耀南,周显恩,等.中国象棋机器人棋子定位与识别方法[J].智能系统学报,2018,13(4):517.[doi:10.11992/tis.201709020]
GUO Xiaofeng,WANG Yaonan,ZHOU Xianen,et al.Chess-piece localization and recognition method for Chinese chess robot[J].CAAI Transactions on Intelligent Systems,2018,13():517.[doi:10.11992/tis.201709020]
[6]安果维,王耀南,周显恩,等.基于显著性检测的双目测距系统[J].智能系统学报,2018,13(6):913.[doi:10.11992/tis.201712005]
AN Guowei,WANG Yaonan,ZHOU Xianen,et al.Binocular distance measurement system based on saliency detection[J].CAAI Transactions on Intelligent Systems,2018,13():913.[doi:10.11992/tis.201712005]
[7]赵立明,龙大周,徐晓东,等.工业机器人加工轨迹双目3D激光扫描成像修正方法[J].智能系统学报,2021,16(4):690.[doi:10.11992/tis.202008008]
ZHAO Liming,LONG Dazhou,XU Xiaodong,et al.Binocular 3D laser scanning imaging-based industrial robot machining trajectory correction method[J].CAAI Transactions on Intelligent Systems,2021,16():690.[doi:10.11992/tis.202008008]
[8]朱齐丹,李小铜,郑天昊.舰载机位姿实时视觉测量算法研究[J].智能系统学报,2021,16(6):1045.[doi:10.11992/tis.202103014]
ZHU Qidan,LI Xiaotong,ZHENG Tianhao.Research on real-time vision measurement algorithm of shipborne aircraft pose[J].CAAI Transactions on Intelligent Systems,2021,16():1045.[doi:10.11992/tis.202103014]
[9]谢家阳,王行健,史治国,等.动态云台摄像机无人机检测与跟踪算法[J].智能系统学报,2021,16(5):858.[doi:10.11992/tis.202103032]
XIE Jiayang,WANG Xingjian,SHI Zhiguo,et al.Drone detection and tracking in dynamic pan-tilt-zoom cameras[J].CAAI Transactions on Intelligent Systems,2021,16():858.[doi:10.11992/tis.202103032]
[10]王凤随,陈金刚,王启胜,等.自适应上下文特征的多尺度目标检测算法[J].智能系统学报,2022,17(2):276.[doi:10.11992/tis.202101029]
WANG Fengsui,CHEN Jingang,WANG Qisheng,et al.Multi-scale target detection algorithm based on adaptive context features[J].CAAI Transactions on Intelligent Systems,2022,17():276.[doi:10.11992/tis.202101029]
[11]张智,毕晓君.基于风格转换的无监督聚类行人重识别[J].智能系统学报,2021,16(1):48.[doi:10.11992/tis.202012014]
ZHANG Zhi,BI Xiaojun.Clustering approach based on style transfer for unsupervised person re-identification[J].CAAI Transactions on Intelligent Systems,2021,16():48.[doi:10.11992/tis.202012014]
备注/Memo
收稿日期:2023-04-06。
基金项目:安徽省自然科学基金项目（2108085MF197）；安徽高校省级自然科学研究重点项目（KJ2019A0162）；安徽工程大学国家自然科学基金预研项目（Xjky2022040）.
作者简介:王路遥，硕士研究生，主要研究方向为深度学习、行人重识别方面的研究。E-mail：578651059@qq.com;王凤随，教授，博士，主要研究方向为图像与视频信息处理、视觉计算与智能分析、视觉目标跟踪、智能计算与多目标优化视频通信。主持省级自然科学研究重点项目2项，省级自然科学基金项目2项、省教育厅省级质量工程虚拟仿真实验教学项目1项、省教育厅质量工程“六卓越、一拔尖”卓越人才培养创新项目1项，获国家专利授权16项，发表学术论文40余篇。E-mail：fswang@ahpu.edu.ac.cn;闫涛，硕士研究生，主要研究方向为深度学习、行人重识别方面的研究，发表学术论文2篇。E-mail：1847026840@qq.com
通讯作者:王凤随. E-mail：fswang@ahpu.edu.ac.cn
更新日期/Last Update:
1900-01-01