基于多粒度匹配的行人搜索算法

PDF下载HTML

Title:

Person search algorithm based on multi-granularity matching

作者:

杨玉婷¹, 苗夺谦²

1. 同济大学 电子与信息工程学院, 上海 201804;
2. 同济大学 嵌入式系统与服务计算教育部重点实验室, 上海 201804

Author(s):

YANG Yuting¹, MIAO Duoqian²

1. College of Electronic and Information Engineering, Tongji University, Shanghai 201804, China;
2. Key Laboratory of Embedded System and Service Computing Ministry of Education, Tongji University, Shanghai 201804, China

关键词:

行人搜索; 行人检测; 行人重识别; 多粒度; 特征融合; 深度学习; 鲁棒性; 计算机视觉

Keywords:

person search; person detection; person re-identification; multi-granularity; multi-feature fusion; deep learning; robustness; computer vision

分类号:

TP389.1

DOI:

10.11992/tis.202105038

摘要:

行人搜索旨在从一系列未经裁剪的图像中对行人进行定位与识别，融合了行人检测和行人重识别两个子任务。现有的方法设计了基于Faster R-CNN的端到端框架来解决此任务，但是行人检测和重识别两个子任务之间存在特征优化目标粒度不一致问题。为了解决这一问题，提出一种双全局池化结构，使用全局平均池化提取检测分支的共性特征，使用基于注意力机制的全局K最大池化提取re-ID分支的特性特征，为两个子任务提取符合各自粒度特性的特征。同时由于re-ID子任务的细粒度特性，还提出一种改善粒度匹配的画廊边界框加权算法，把查询人和画廊边界框的分辨率差异纳入相似度计算。实验证明融入多粒度的方法有效地提高了单阶段算法在CHUK-SYSU和PRW数据集上的性能。

Abstract:

Person search aims to locate and recognize a specified person from a series of uncropped images, which combines Pedestrian Detection and Person Re-identification (re-ID). Existing methods based on Faster R-CNN have been widely used to solve the two subtasks jointly. However, the optimization goals of the two subtasks are inconsistent. To alleviate this issue, we propose a dual global pooling structure, which applies Global Average Pooling to extract common features in detection branch and applies Global K-Max Pooling to extract discriminative features in re-ID branch. In this way, our method successfully extracts features that conform to the granularity characteristics of the two subtasks. In addition, to relieve the granularity mismatch problem, we propose a multi-granularity gallery boxes re-weighting algorithm, which incorporates granularity difference into similarity measurement. Extensive experiments show that our method greatly improves the performance of the end-to-end framework on two widely used person search datasets, CUHK-SYSU and PRW.