[1]张少乐,雷涛,王营博,等.基于多尺度金字塔Transformer的人群计数方法[J].智能系统学报,2024,19(1):67-78.[doi:10.11992/tis.202304044]
ZHANG Shaole,LEI Tao,WANG Yingbo,et al.A crowd counting network based on multi-scale pyramid Transformer[J].CAAI Transactions on Intelligent Systems,2024,19(1):67-78.[doi:10.11992/tis.202304044]
点击复制
ZHANG Shaole,LEI Tao,WANG Yingbo,et al.A crowd counting network based on multi-scale pyramid Transformer[J].CAAI Transactions on Intelligent Systems,2024,19(1):67-78.[doi:10.11992/tis.202304044]
基于多尺度金字塔Transformer的人群计数方法
《智能系统学报》[ISSN 1673-4785/CN 23-1538/TP] 卷:
19
期数:
2024年第1期
页码:
67-78
栏目:
学术论文—机器学习
出版日期:
2024-01-05
- Title:
- A crowd counting network based on multi-scale pyramid Transformer
- Keywords:
- dense crowd; crowd counting; multi-scale; pyramid; Transformer; self-attention; density map; deep supervision
- 分类号:
- TP391.41
- 文献标志码:
- 2024-01-03
- 摘要:
- 针对密集人群场景中背景复杂、目标尺度变化较大导致人群计数精度较低的问题，本文提出一种基于多尺度金字塔Transformer的人群计数方法(multi-scale pyramid transformer network, MSPT-Net)。在特征提取阶段设计了一种基于深度可分离自注意力的金字塔Transformer主干网络结构，该网络结构能有效捕获图像的局部和全局信息，从而有效解决人群密度图像背景复杂导致计数精度低的问题；设计了一种特征金字塔融合模块及多尺度感受野的回归头，实现了密集人群图像浅层细节特征和深层语义特征的高效融合，增强了网络对不同尺度目标的捕获能力；采用深度监督的训练方法在3个公开数据集上对提出的方法进行验证。实验结果表明，本文方法在全监督与弱监督学习策略中，与目前主流的人群计数方法相比，实现了更高精度的人群计数，克服了主流方法对背景复杂、目标尺度变化大的密集人群图像计数精度低的问题，同时本文方法保持着更小的参数量与计算量。
- Abstract:
- A crowd counting network based on multi-scale pyramid Transformer (MSPT-Net) is proposed to address the problem of low accuracy in crowd counting in dense crowd scenes caused by complex backgrounds and large target scale variations. A pyramid transformer backbone network structure based on depth separable self-attention is designed in the feature extraction phase to effectively capture local as well as global information of the image, thereby effectively addressing the problem of low counting accuracy in crowd density images caused by complex backgrounds. A feature pyramid fusion module and a regression head with multi-scale receptive fields are designed to efficiently integrate shallow detail features and deep semantic features in dense crowd scenes, enhancing the network’s ability to capture targets of different scales. Lastly, the proposed model is validated using a deep supervision training method on three publicly available datasets. The experimental results show that the proposed MSPT-Net achieves higher crowd counting accuracy in the fully supervised and weakly supervised learning strategies as compared to mainstream crowd counting networks, overcoming the issue of low counting accuracy in dense crowd images with complex backgrounds and significant changes in target scales. At the same time, the method in this paper keeps the parameter number and calculation amount smaller.
- 参考文献/References:
-
[1] LIU Hong, XU Bin, LU Dianjie, et al. A path planning approach for crowd evacuation in buildings based on improved artificial bee colony algorithm[J]. Applied soft computing, 2018, 68: 360–376.
[2] KHAN M A, MENOUAR H, HAMILA R. Revisiting crowd counting: state-of-the-art, trends, and future perspectives[J]. Image and vision computing, 2023, 129: 104597.
[3] DOLLAR P, WOJEK C, SCHIELE B, et al. Pedestrian detection: an evaluation of the state of the art[J]. IEEE transactions on pattern analysis and machine intelligence, 2011, 34(4): 743–761.
[4] 齐鹏宇, 王洪元, 张继, 等. 基于改进FCOS的拥挤行人检测算法[J]. 智能系统学报, 2021, 16(4): 811–818
QI Pengyu, WANG Hongyuan, ZHANG Ji, et al. Crowded pedestrian detection algorithm based on improved FCOS[J]. CAAI transactions on intelligent systems, 2021, 16(4): 811–818
[5] CHEN Ke, GONG Shaogang, XIANG Tao, et al. Cumulative attribute space for age and crowd density estimation[C]//2013 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2013: 2467?2474.
[6] LEMPITSKY V, ZISSERMAN A. Learning to count objects in images advances in neural information processing systems, 2010, 23: 1324–1332.
[7] ZHANG Yingying, ZHOU Desen, CHEN Siqin, et al. Single-image crowd counting via multi-column convolutional neural network[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition . Piscataway: IEEE, 2016: 589?597.
[8] LI Yuhong, ZHANG Xiaofan, CHEN Deming. CSRNet: dilated convolutional neural networks for understanding the highly congested scenes[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2018: 1091?1100.
[9] SINDAGI V A, PATEL V M. Generating high-quality crowd density maps using contextual pyramid CNNs[C]//2017 IEEE International Conference on Computer Vision. Piscataway: IEEE, 2017: 1879?1888.
[10] HOSSAIN M, HOSSEINZADEH M, CHANDA O, et al. Crowd counting using scale-aware attention networks[C]//2019 IEEE Winter Conference on Applications of Computer Vision . Piscataway : IEEE, 2019: 1280?1288.
[11] DAI Jifeng, QI Haozhi, XIONG Yuwen, et al. Deformable convolutional networks[C]//2017 IEEE International Conference on Computer Vision . Piscataway: IEEE, 2017: 764?773.
[12] SHEN Zan, XU Yi, NI Bingbing, et al. Crowd counting via adversarial cross-scale consistency pursuit[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2018: 5245?5254.
[13] LIANG Dingkang, CHEN Xiwu, XU Wei, et al. TransCrowd: weakly-supervised crowd counting with transformers[J]. Science China information sciences, 2022, 65(6): 1–14.
[14] JIANG Xiaoheng, ZHANG Li, ZHANG Tianzhu, et al. Density-aware multi-task learning for crowd counting[J]. IEEE transactions on multimedia, 2020, 23: 443–453.
[15] CHAUDHARI S, MITHAL V, POLATKAN G, et al. An attentive survey of attention models[J]. ACM transactions on intelligent systems and technology, 2021, 12(5): 1–32.
[16] LIU Ning, LONG Yongchao, ZOU Changqing, et al. ADCrowdNet: an attention-injective deformable convolutional network for crowd understanding[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE, 2020: 3220?3229.
[17] JIANG Xiaoheng, ZHANG Li, XU Mingliang, et al. Attention scaling for crowd counting[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE, 2020: 4705?4714.
[18] LEI Tao, ZHANG Dong, WANG Risheng, et al. MFP-Net: multi-scale feature pyramid network for crowd counting[J]. IET image processing, 2021, 15(14): 3522–3533.
[19] LIAN Dongze, LI Jing, ZHENG Jia, et al. Density map regression guided detection network for RGB-D crowd counting and localization[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE, 2020: 1821?1830.
[20] WANG Boyu, LIU Huidong, SAMARAS D, et al. Distribution matching for crowd counting[EB/OL]. (2020?09?28)[2023?04?30]. 2009.13077. https://arxiv.org/abs/2009.13077.pdf.
[21] VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems. New York: ACM, 2017: 6000?6010.
[22] DOSOVITSKIY A, BEYER L, KOLESNIKOV A, et al. An image is worth 16x16 words: transformers for image recognition at scale[EB/OL]. (2020?10?22)[2023?04?30]. https://arxiv.org/abs/2010.11929.pdf.
[23] WANG Wenhai, XIE Enze, LI Xiang, et al. Pyramid vision transformer: a versatile backbone for dense prediction without convolutions[C]//2021 IEEE/CVF International Conference on Computer Vision . Piscataway: IEEE, 2022: 548?558.
[24] CHOLLET F. Xception: deep learning with depthwise separable convolutions[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition . Piscataway: IEEE, 2017: 1800?1807.
[25] 雷涛, 王洁, 薛丁华, 等. 差异特征融合的无监督SAR图像变化检测[J]. 智能系统学报, 2021, 16(3): 595–604
LEI Tao, WANG Jie, XUE Dinghua, et al. Unsupervised SAR image change detection based on difference feature fusion[J]. CAAI transactions on intelligent systems, 2021, 16(3): 595–604
[26] YU F, KOLTUN V. Multi-scale context aggregation by dilated convolutions[EB/OL]. (2015?11?23)[2023?04?30]. https://arxiv.org/abs/1511.07122.pdf.
[27] LEE Chenyu, XIE Saining, GALLAGHER P, et al. Deeply-supervised nets[EB/OL]. (2014?09?18)[2023?04?30]. https://arxiv.org/abs/1409.5185.pdf.
[28] IDREES H, TAYYAB M, ATHREY K, et al. Composition loss for counting, density map estimation and localization in dense crowds[C]//Computer Vision - ECCV 2018: 15th European Conference, Munich, Germany, September 8-14, 2018, Proceedings, Part II. New York: ACM, 2018: 544?559.
[29] WANG Qi, GAO Junyu, LIN Wei, et al. NWPU-crowd: a large-scale benchmark for crowd counting and localization[J]. IEEE transactions on pattern analysis and machine intelligence, 2021, 43(6): 2141–2149.
[30] MENG Yanda, ZHANG Hongrun, ZHAO Yitian, et al. Spatial uncertainty-aware semi-supervised crowd counting[C]//2021 IEEE/CVF International Conference on Computer Vision . Piscataway: IEEE, 2022: 15529?15539.
[31] CHEN Qi, LEI Tao, GENG Xinzhe, et al. DLMP-net: a dynamic yet lightweight multi-pyramid network for crowd density estimation[C]//Chinese Conference on Pattern Recognition and Computer Vision . Cham: Springer, 2022: 27?39.
[32] LIANG Lanjun, ZHAO Huailin, ZHOU Fangbo, et al. SC2Net: scale-aware crowd counting network with pyramid dilated convolution[J]. Applied intelligence, 2023, 53(5): 5146–5159.
[33] LIANG Dingkang, XU Wei, ZHU Yingying, et al. Focal inverse distance transform maps for crowd localization[J]. IEEE transactions on multimedia, 2023, 25: 6040–6052.
[34] YANG Yifan, LI Guorong, WU Zhe, et al. Weakly-supervised crowd counting learns from sorting rather than locations[C]//Vedaldi A, Bischof H, Brox T, et al. European Conference on Computer Vision. Cham: Springer, 2020: 1?17.
[35] LEI Yinjie, LIU Yan, ZHANG Pingping, et al. Towards using count-level weak supervision for crowd counting[J]. Pattern recognition, 2021, 109: 107616.
[36] ZAND M, DAMIRCHI H, FARLEY A, et al. Multiscale crowd counting and localization by multitask point supervision[C]//ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing . Piscataway: IEEE, 2022: 1820?1824.
备注/Memo
收稿日期:2023-04-30。
基金项目:国家自然科学基金项目 (62271296，62201334)；陕西省重点研发计划项目(2021ZDLGY08-07)；陕西省杰出青年科学基金项目(2021JC-47).
作者简介:张少乐，硕士研究生，主要研究方向为计算机视觉、机器学习。E-mail：210612054@sust.edu.cn;雷涛，教授，博士生导师，陕西科技大学电子信息与人工智能学院副院长、IEEE 高级会员，主要研究方向为计算机视觉、机器学习。主持国家自然科学基金项目 5 项、陕西省重点研发计划、中国博士后科学基金等6项，授权发明专利 15 项，获陕西省科学技术二等奖 1 项（自然科学奖）。发表学术论文 90 余篇。E-mail：leitao@sust.edu.cn;王营博，讲师，主要研究方向为散射环境下图像复原与场景感知。参与国家自然科学基金面上项目、高分重大专项等项目 5 项，授权发明专利 8 项，授权软件著作权 1 项。发表学术论文 20 余篇。E-mail：wangyingbo@sust.edu.cn
通讯作者:雷涛. E-mail：leitao@sust.edu.cn
更新日期/Last Update:
1900-01-01