[1]刘智,李广登,李诗雨,等.用于可见–红外行人重识别的模态共享门控双流Transformer[J].智能系统学报,2026,21(4):979-987.[doi:10.11992/tis.202511021]
LIU Zhi,LI Guangdeng,LI Shiyu,et al.Modality-shared gated two-stream Transformer for visibleinfrared person re-identification[J].CAAI Transactions on Intelligent Systems,2026,21(4):979-987.[doi:10.11992/tis.202511021]
点击复制
LIU Zhi,LI Guangdeng,LI Shiyu,et al.Modality-shared gated two-stream Transformer for visibleinfrared person re-identification[J].CAAI Transactions on Intelligent Systems,2026,21(4):979-987.[doi:10.11992/tis.202511021]
用于可见–红外行人重识别的模态共享门控双流Transformer
《智能系统学报》[ISSN 1673-4785/CN 23-1538/TP] 卷:
21
期数:
2026年第4期
页码:
979-987
栏目:
学术论文—机器感知与模式识别
出版日期:
2026-07-05
- Title:
- Modality-shared gated two-stream Transformer for visibleinfrared person re-identification
- 关键词:
- 可见–红外行人重识别; 多模态学习; 视觉Transformer; 对比语言-图像预训练; 模态共享特征提取; 模态共享门控
- Keywords:
- visible-infrared person re-identification; multi-modal learning; vision Transformer; contrastive language–image pre-training; modality-shared features extraction; modality-shared gated
- 分类号:
- TP391
- 摘要:
- 可见光–红外行人重识别旨在匹配不同模态下的同一身份行人图像，是计算机视觉领域一项极具挑战性的任务。在现实场景中，光照不足往往导致识别困难，严重限制了行人重识别系统的实用性。尽管基于卷积神经网络的传统方法已得到广泛研究，但受限于局部感受野及下采样操作，这类方法容易造成模态信息的丢失。针对上述问题，本文提出了一种基于vision Transformer(ViT)的可见光–红外行人重识框架——模态共享门控双流Transformer。针对红外与RGB图像间的模态差异，采用灰度图像增强策略，将RGB图像转换为灰度图以最小化模态间隙。设计模态特有嵌入模块，引导ViT更有效地提取模态共享特征。随后，利用模态共享模块充分挖掘RGB、灰度及红外3类图像中的模态无关特征。最后，引入模态共享门控层对特征进行精细筛选，保留高鉴别力特征以用于后续重识别任务。本文框架能够全面提取多模态信息，并有效保留模态无关特征。在SYSU-MM01和RegDB数据集上的大量实验表明，该方法优于当前主流技术。
- Abstract:
- Visible-infrared person re-identification (VI-ReID) aims to match person images of the same identity across different modalities, representing a highly challenging task in the field of computer vision. In real-world scenarios, poor illumination often leads to recognition difficulties, severely limiting the practicality of person re-identification systems. Although traditional methods based on convolutional neural networks have been widely investigated, they are limited by local receptive fields and down-sampling operations, which tend to result in the loss of key modality information. To address these issues, this paper proposes a vision Transformer-based VI-ReID framework, termed modality-shared gating two-stream Transformer (MgtFormer). First, aiming at the modality discrepancy between infrared and RGB images, a grayscale image augmentation strategy is adopted to convert RGB images into grayscale ones to minimize the modality gap. Second, a modality-specific embedding module is designed to guide the ViT to extract modality-shared features more effectively. Subsequently, a modality-shared module is utilized to fully mine modality-invariant features from RGB, grayscale, and infrared images. Finally, a modality-shared gating layer is introduced to finely screen the features, retaining highly discriminative ones for subsequent re-identification tasks. MgtFormer is capable of comprehensively extracting multi-modal information and effectively preserving modality-invariant features. Extensive experiments on the SYSU-MM01 and RegDB datasets demonstrate that the proposed method outperforms current state-of-the-art techniques.
- 参考文献/References:
-
[1] ZHENG Liang, YANG Yi, HAUPTMANN A G. Person re-identification: past, present and future[EB/OL]. (2016-10-10)[2025-11-15]. https://arxiv.org/abs/1610.02984.
[2] SUN Yifan, ZHENG Liang, YANG Yi, et al. Beyond part models: person retrieval with refined part pooling (and a strong convolutional baseline)[C]//Computer Vision–ECCV 2018. Cham: Springer International Publishing, 2018: 501-518.
[3] ZHENG Feng, DENG Cheng, SUN Xing, et al. Pyramidal person re-IDentification via multi-loss dynamic training[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach: IEEE, 2020: 8506-8514.
[4] JIN Xin, LAN Cuiling, ZENG Wenjun, et al. Style normalization and restitution for generalizable person re-identification[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle: IEEE, 2020: 3140-3149.
[5] SUN Yifan, CHENG Changmao, ZHANG Yuhan, et al. Circle loss: a unified perspective of pair similarity optimization[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle: IEEE, 2020: 6397-6406.
[6] NGUYEN B X, NGUYEN B D, DO T, et al. Graph-based person signature for person re-identifications[C]//2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. Nashville: IEEE, 2021: 3487-3496.
[7] HE Shuting, LUO Hao, WANG Pichao, et al. TransReID: transformer-based object re-identification[C]//2021 IEEE/CVF International Conference on Computer Vision. Montreal: IEEE, 2022: 14993-15002.
[8] REN Min, HE Lingxiao, LIAO Xingyu, et al. Learning instance-level spatial-temporal patterns for person re-identification[C]//2021 IEEE/CVF International Conference on Computer Vision. Montreal: IEEE, 2022: 14910-14919.
[9] LI Dengjie, CHEN Siyu, ZHONG Yujie, et al. DiP: learning discriminative implicit parts for person re-identification[EB/OL]. (2022-12-24)[2025-11-15]. https://arxiv.org/abs/2212.13906.
[10] LI Siyuan, SUN Li, LI Qingli. CLIP-ReID: exploiting vision-language model for image re-identification without concrete text labels[J]. Proceedings of the AAAI conference on artificial intelligence, 2023, 37(1): 1405-1413
[11] SOMERS V, DE VLEESCHOUWER C, ALAHI A. Body part-based representation learning for occluded person re-identification[C]//2023 IEEE/CVF Winter Conference on Applications of Computer Vision. Waikoloa: IEEE, 2023: 1613-1623.
[12] TIAN Xudong, ZHANG Zhizhong, WANG Cong, et al. Variational distillation for multi-view learning[EB/OL]. (2022-06-20)[2025-11-15]. https://arxiv.org/abs/2206.09548.
[13] JOSI A, ALEHDAGHI M, CRUZ R M O, et al. Multimodal data augmentation for visual-infrared person ReID with corrupted data[C]//2023 IEEE/CVF Winter Conference on Applications of Computer Vision Workshops. Waikoloa: IEEE, 2023: 1-10.
[14] ALEHDAGHI M, JOSI A, CRUZ R M O, et al. Visible-infrared person re-identification using privileged intermediate information[C]//Computer Vision–ECCV 2022 Workshops. Cham: Springer Nature Switzerland, 2023: 720-737.
[15] FENG Jiawei, WU Ancong, ZHENG Weishi. Shape-erased feature learning for visible-infrared person re-identification[C]//2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Vancouver: IEEE, 2023: 22752-22761.
[16] YE Mang, WANG Zheng, LAN Xiangyuan, et al. Visible thermal person re-identification via dual-constrained top-ranking[C]//Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence Main track. Stockholm: IJCAI, 2018: 1092-1099.
[17] HAO Yi, WANG Nannan, LI Jie, et al. HSME: hypersphere manifold embedding for visible thermal person re-identification[C]//Proceedings of the AAAI conference on artificial intelligence, Honolulu: ACM, 2019: 8385-8392.
[18] LU Yan, WU Yue, LIU Bin, et al. Cross-modality person re-identification with shared-specific feature transfer[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle: IEEE, 2020: 13376-13386.
[19] LUO Hao, WANG Pichao, XU Yi, et al. Self-supervised pre-training for Transformer-based person re-identification[EB/OL]. (2021-11-23)[2025-11-15]. https://arxiv.org/abs/2111.12084.
[20] JIA Mengxi, CHENG Xinhua, LU Shijian, et al. Learning disentangled representation implicitly via Transformer for occluded person re-identification[J]. IEEE transactions on multimedia, 2023, 25: 1294-1305
[21] DOU Shuguang, ZHAO Cairong, JIANG Xinyang, et al. Human co-parsing guided alignment for occluded person re-identification[J]. IEEE transactions on image processing, 2023, 32: 458-470
[22] TAN Lei, XIA Jiaer, LIU Wenfeng, et al. Occluded person re-identification via saliency-guided patch transfer[C]//Proceedings of the 2024 AAAI Conference on Artificial Intelligence. Vancouver: AAAI, 2024: 5070-5078.
[23] XIA Jiaer, TAN Lei, DAI Pingyang, et al. Attention disturbance and dual-path constraint network for occluded person re-identification[C]//Proceedings of the 2024 AAAI Conference on Artificial Intelligence. Vancouver: AAAI, 2024: 6198-6206.
[24] GAO Liying, JIAO Bingliang, LONG Yuzhou, et al. Contrastive pedestrian attentive and correlation learning network for occluded person re-identification[J]. IEEE transactions on circuits and systems for video technology, 2024, 34(9): 8862-8880
[25] WANG Tao, LIU Mengyuan, LIU Hong, et al. Feature completion transformer for occluded person re-identification[J]. IEEE transactions on multimedia, 2024, 26: 8529-8542
[26] LI Yanping, LIU Yizhang, ZHANG Hongyun, et al. Occlusion-aware transformer with second-order attention for person re-identification[J]. IEEE transactions on image processing, 2024, 33: 3200-3211
[27] YE Mang, RUAN Weijian, DU Bo, et al. Channel augmented joint learning for visible-infrared recognition[C]//2021 IEEE/CVF International Conference on Computer Vision. Montreal: IEEE, 2022: 13547-13556.
[28] WU Ancong, ZHENG Weishi, YU Hongxing, et al. RGB-infrared cross-modality person re-identification[C]//2017 IEEE International Conference on Computer Vision. Venice: IEEE, 2017: 5390-5399.
[29] 王晋溪, 鲁鸣鸣. 基于场景图知识的文本到图像行人重识别[J]. 模式识别与人工智能, 2024, 37(11): 947-959 WANG Jinxi, LU Mingming. Scene graph knowledge based text-to-image person re-identification[J]. Pattern recognition and artificial intelligence, 2024, 37(11): 947-959
[30] 石瑞鑫, 智敏, 殷雁君. 多模态行人重识别研究综述[J]. 计算机应用研究, 2025, 42(7): 1921-1929 SHI Ruixin, ZHI Min, YIN Yanjun. Review of multimodal pedestrian re-identification[J]. Application research of computers, 2025, 42(7): 1921-1929
[31] 李俊峰, 楼琼, 钱亚冠, 等. 基于像素对齐和特征对齐的跨模态行人重识别[J]. 浙江科技学院学报, 2022(3): 251-260 LI Junfeng, LOU Qiong, QIAN Yaguan, et al. Cross-modality person re-identification based onpixel alignment and feature alignment[J]. Journal of Zhejiang University of Science and Technology, 2022(3): 251-260
[32] 冯展祥, 赖剑煌, 袁藏, 等. 走向通用行人重识别: 预训练大模型技术在行人重识别的应用综述[J]. 中国图象图形学报, 2025, 30(6): 1638-1660 FENG Zhanxiang, LAI Jianhuang, YUAN Zang, et al. Advancing universal person reidentification: a survey on the applications of large-scale, pretraining models for identifying individuals[J]. Journal of image and graphics, 2025, 30(6): 1638-1660
[33] 孙锐, 杜云, 陈龙, 等. 隐式多尺度对齐与交互的文本-图像行人重识别方法[J]. 软件学报, 2025, 36(10): 4846-4863 SUN Rui, DU Yun, CHEN Long, et al. Implicit multi-scale alignment and interaction for text-image person re-identification method[J]. Journal of software, 2025, 36(10): 4846-4863
[34] 金昌胜, 王海瑞. 基于关系挖掘的跨模态行人重识别[J]. 空军工程大学学报, 2024, 25(1): 106-114 JIN Changsheng, WANG Hairui. A cross-modal person re-identification based on relationship mining[J]. Journal of Air Force Engineering University, 2024, 25(1): 106-114
[35] NGUYEN D T, HONG H G, KIM K W, et al. Person recognition system based on a combination of body images from visible light and thermal cameras[J]. Sensors, 2017, 17(3): 605
[36] YE Mang, LAN Xiangyuan, WANG Zheng, et al. Bi-directional center-constrained top-ranking for visible thermal person re-identification[J]. IEEE transactions on information forensics and security, 2020, 15: 407-419
[37] CHOI S, LEE S, KIM Y, et al. Hi-CMD: hierarchical cross-modality disentanglement for visible-infrared person re-identification[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle: IEEE, 2020: 10254-10263.
[38] WANG Pingyu, ZHAO Zhicheng, SU Fei, et al. Deep multi-patch matching network for visible thermal person re-identification[J]. IEEE transactions on multimedia, 2021, 23: 1474-1488
[39] PARK H, LEE S, LEE J, et al. Learning by aligning: visible-infrared person re-identification using cross-modal correspondences[C]//2021 IEEE/CVF International Conference on Computer Vision. Montreal: IEEE, 2022: 12026-12035.
[40] CHEN Y, WAN Lin, LI Zhihang, et al. Neural feature search for RGB-infrared person re-identification[C]//2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Nashville: IEEE, 2021: 587-597.
[41] YE Mang, SHEN Jianbing, LIN Gaojie, et al. Deep learning for person re-identification: a survey and outlook[J]. IEEE transactions on pattern analysis and machine intelligence, 2022, 44(6): 2872-2893
[42] WANG Pingyu, SU Fei, ZHAO Zhicheng, et al. Deep hard modality alignment for visible thermal person re-identification[J]. Pattern recognition letters, 2020, 133: 195-201
[43] YE Mang, CHEN Cuiqun, SHEN Jianbing, et al. Dynamic tri-level relation mining with attentive graph for visible infrared re-identification[J]. IEEE transactions on information forensics and security, 2022, 17: 386-398
[44] ZHAO Jiaqi, WANG Hanzheng, ZHOU Yong, et al. Spatial-channel enhanced transformer for visible-infrared person re-identification[J]. IEEE transactions on multimedia, 2023, 25: 3668-3680
[45] WANG Guanan, ZHANG Tianzhu, CHENG Jian, et al. RGB-infrared cross-modality person re-identification via joint pixel and feature alignment[C]//2019 IEEE/CVF International Conference on Computer Vision. Seoul: IEEE, 2019: 3622-3631.
[46] WANG Guanan, ZHANG Tianzhu, YANG Yang, et al. Cross-modality paired-images generation for RGB-infrared person re-identification[C]//The AAAI 2020 proceedings include all papers presented at the 34th AAAI Conference. New York: AAAI, 2020: 12144-12151.
[47] WANG Xiaogang, DORETTO G, SEBASTIAN T, et al. Shape and appearance context modeling[C]//2007 IEEE 11th International Conference on Computer Vision. Rio de Janeiro: IEEE, 2007: 1-8.
[48] ZHENG Liang, SHEN Liyue, TIAN Lu, et al. Scalable person re-identification: a benchmark[C]//2015 IEEE International Conference on Computer Vision. Santiago: IEEE, 2016: 1116-1124.
[49] CHENG De, HUANG Xiaojian, WANG Nannan, et al. Unsupervised visible-infrared person ReID by collaborative learning with neighbor-guided label refinement[C]//Proceedings of the 31st ACM International Conference on Multimedia. Ottawa: ACM, 2023: 7085-7093.
[50] ZHANG Xu, LIU Yinghui, GUO Liangchen, et al. Zero-shot infrared domain adaptation for pedestrian re-identification via deep learning[J]. Electronics, 2025, 14(14): 2784
[51] 张红颖, 樊世钰, 罗谦, 等. 结合视觉文本匹配和图嵌入的可见光-红外行人重识别[J]. 电子与信息学报, 2024, 46(9): 3662-3671 ZHANG Hongying, FAN Shiyu, LUO Qian, et al. Visible-infrared pedestrian recognition combined with visual text matching and graph embedding[J]. Journal of electronics & information technology, 2024, 46(9): 3662-3671
- 相似文献/References:
-
[1]肖建力,黄星宇,姜飞.智慧教育中的大语言模型综述[J].智能系统学报,2025,20(5):1054.[doi:10.11992/tis.202406040]
XIAO Jianli,HUANG Xingyu,JIANG Fei.A survey of large language models in smart education[J].CAAI Transactions on Intelligent Systems,2025,20():1054.[doi:10.11992/tis.202406040]
[2]王忠美,敖文秀,刘建华,等.基于自适应梯度调制的音视频多模态平衡学习方法[J].智能系统学报,2025,20(5):1217.[doi:10.11992/tis.202412009]
WANG Zhongmei,AO Wenxiu,LIU Jianhua,et al.An audio-visual multimodal balanced learning method based on adaptive gradient modulation[J].CAAI Transactions on Intelligent Systems,2025,20():1217.[doi:10.11992/tis.202412009]
备注/Memo
收稿日期:2025-11-15。
作者简介:刘智，副教授，主要研究方向为计算机视觉、机器学习、视频与信号分析。主持或参与国家自然科学基金、重庆市自然科学基金等纵向及企业横向项目20余项。获国家发明专利授权4项，以第一或通讯作者发表学术论文20余篇。E-mail：liuzhi@cqut.edu.cn。;李广登，硕士，主要研究方向为计算机视觉和行人重识别。E-mail：guangdeng.lee@gmail.com。;张小川，教授，CAAI杰出会员，CAAI机器博弈专委会主任委员，重庆工程学院智能系统工程中心主任。主要研究方向为软件工程、机器博弈、自然语言处理、机器学习和智能机器人。主持和参与纵向项目38项，获省部级自然科学奖等2项，出版专著和 教材5部，发表论文100余篇。E-mail： cqpczxc@qq.com。
通讯作者:张小川. E-mail：cqpczxc@qq.com
更新日期/Last Update:
1900-01-01