[1]ZHANG Shaole,LEI Tao,WANG Yingbo,et al.A crowd counting network based on multi-scale pyramid Transformer[J].CAAI Transactions on Intelligent Systems,2024,19(1):67-78.[doi:10.11992/tis.202304044]
Copy

A crowd counting network based on multi-scale pyramid Transformer

CAAI Transactions on Intelligent Systems[ISSN 1673-4785/CN 23-1538/TP] Volume: 19 Number of periods: 2024 1 Page number: 67-78 Column: 学术论文—机器学习 Public date: 2024-01-05
Title:
A crowd counting network based on multi-scale pyramid Transformer
Author(s):
ZHANG Shaole1;  LEI Tao2; 3;  WANG Yingbo2;  ZHOU Qiang1;  XUE Mingyuan2;  ZHAO Weiqiang4
1. School of Electrical and Control Engineering, Shaanxi University of Science and Technology, Xi’an 710021, China;
2. School of Electronic Information and Artificial Intelligence, Shaanxi University of Science and Technology, Xi’an 710021, China;
3. Shaanxi Joint Laboratory of Artificial Intelligence, Shaanxi University of Science and Technology, Xi’an 710021, China;
4. China Electronics Technology Group Corporation Northwest Group Corporation Xi’an Branch, Xi’an 710065, China
Keywords:
dense crowd; crowd counting; multi-scale; pyramid; Transformer; self-attention; density map; deep supervision
CLC:
TP391.41
DOI:
10.11992/tis.202304044
Abstract:
A crowd counting network based on multi-scale pyramid Transformer (MSPT-Net) is proposed to address the problem of low accuracy in crowd counting in dense crowd scenes caused by complex backgrounds and large target scale variations. A pyramid transformer backbone network structure based on depth separable self-attention is designed in the feature extraction phase to effectively capture local as well as global information of the image, thereby effectively addressing the problem of low counting accuracy in crowd density images caused by complex backgrounds. A feature pyramid fusion module and a regression head with multi-scale receptive fields are designed to efficiently integrate shallow detail features and deep semantic features in dense crowd scenes, enhancing the network’s ability to capture targets of different scales. Lastly, the proposed model is validated using a deep supervision training method on three publicly available datasets. The experimental results show that the proposed MSPT-Net achieves higher crowd counting accuracy in the fully supervised and weakly supervised learning strategies as compared to mainstream crowd counting networks, overcoming the issue of low counting accuracy in dense crowd images with complex backgrounds and significant changes in target scales. At the same time, the method in this paper keeps the parameter number and calculation amount smaller.
References:
[1] LIU Hong, XU Bin, LU Dianjie, et al. A path planning approach for crowd evacuation in buildings based on improved artificial bee colony algorithm[J]. Applied soft computing, 2018, 68: 360–376.
[2] KHAN M A, MENOUAR H, HAMILA R. Revisiting crowd counting: state-of-the-art, trends, and future perspectives[J]. Image and vision computing, 2023, 129: 104597.
[3] DOLLAR P, WOJEK C, SCHIELE B, et al. Pedestrian detection: an evaluation of the state of the art[J]. IEEE transactions on pattern analysis and machine intelligence, 2011, 34(4): 743–761.
[4] 齐鹏宇, 王洪元, 张继, 等. 基于改进FCOS的拥挤行人检测算法[J]. 智能系统学报, 2021, 16(4): 811–818
QI Pengyu, WANG Hongyuan, ZHANG Ji, et al. Crowded pedestrian detection algorithm based on improved FCOS[J]. CAAI transactions on intelligent systems, 2021, 16(4): 811–818
[5] CHEN Ke, GONG Shaogang, XIANG Tao, et al. Cumulative attribute space for age and crowd density estimation[C]//2013 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2013: 2467?2474.
[6] LEMPITSKY V, ZISSERMAN A. Learning to count objects in images advances in neural information processing systems, 2010, 23: 1324–1332.
[7] ZHANG Yingying, ZHOU Desen, CHEN Siqin, et al. Single-image crowd counting via multi-column convolutional neural network[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition . Piscataway: IEEE, 2016: 589?597.
[8] LI Yuhong, ZHANG Xiaofan, CHEN Deming. CSRNet: dilated convolutional neural networks for understanding the highly congested scenes[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2018: 1091?1100.
[9] SINDAGI V A, PATEL V M. Generating high-quality crowd density maps using contextual pyramid CNNs[C]//2017 IEEE International Conference on Computer Vision. Piscataway: IEEE, 2017: 1879?1888.
[10] HOSSAIN M, HOSSEINZADEH M, CHANDA O, et al. Crowd counting using scale-aware attention networks[C]//2019 IEEE Winter Conference on Applications of Computer Vision . Piscataway : IEEE, 2019: 1280?1288.
[11] DAI Jifeng, QI Haozhi, XIONG Yuwen, et al. Deformable convolutional networks[C]//2017 IEEE International Conference on Computer Vision . Piscataway: IEEE, 2017: 764?773.
[12] SHEN Zan, XU Yi, NI Bingbing, et al. Crowd counting via adversarial cross-scale consistency pursuit[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2018: 5245?5254.
[13] LIANG Dingkang, CHEN Xiwu, XU Wei, et al. TransCrowd: weakly-supervised crowd counting with transformers[J]. Science China information sciences, 2022, 65(6): 1–14.
[14] JIANG Xiaoheng, ZHANG Li, ZHANG Tianzhu, et al. Density-aware multi-task learning for crowd counting[J]. IEEE transactions on multimedia, 2020, 23: 443–453.
[15] CHAUDHARI S, MITHAL V, POLATKAN G, et al. An attentive survey of attention models[J]. ACM transactions on intelligent systems and technology, 2021, 12(5): 1–32.
[16] LIU Ning, LONG Yongchao, ZOU Changqing, et al. ADCrowdNet: an attention-injective deformable convolutional network for crowd understanding[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE, 2020: 3220?3229.
[17] JIANG Xiaoheng, ZHANG Li, XU Mingliang, et al. Attention scaling for crowd counting[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE, 2020: 4705?4714.
[18] LEI Tao, ZHANG Dong, WANG Risheng, et al. MFP-Net: multi-scale feature pyramid network for crowd counting[J]. IET image processing, 2021, 15(14): 3522–3533.
[19] LIAN Dongze, LI Jing, ZHENG Jia, et al. Density map regression guided detection network for RGB-D crowd counting and localization[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE, 2020: 1821?1830.
[20] WANG Boyu, LIU Huidong, SAMARAS D, et al. Distribution matching for crowd counting[EB/OL]. (2020?09?28)[2023?04?30]. 2009.13077. https://arxiv.org/abs/2009.13077.pdf.
[21] VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems. New York: ACM, 2017: 6000?6010.
[22] DOSOVITSKIY A, BEYER L, KOLESNIKOV A, et al. An image is worth 16x16 words: transformers for image recognition at scale[EB/OL]. (2020?10?22)[2023?04?30]. https://arxiv.org/abs/2010.11929.pdf.
[23] WANG Wenhai, XIE Enze, LI Xiang, et al. Pyramid vision transformer: a versatile backbone for dense prediction without convolutions[C]//2021 IEEE/CVF International Conference on Computer Vision . Piscataway: IEEE, 2022: 548?558.
[24] CHOLLET F. Xception: deep learning with depthwise separable convolutions[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition . Piscataway: IEEE, 2017: 1800?1807.
[25] 雷涛, 王洁, 薛丁华, 等. 差异特征融合的无监督SAR图像变化检测[J]. 智能系统学报, 2021, 16(3): 595–604
LEI Tao, WANG Jie, XUE Dinghua, et al. Unsupervised SAR image change detection based on difference feature fusion[J]. CAAI transactions on intelligent systems, 2021, 16(3): 595–604
[26] YU F, KOLTUN V. Multi-scale context aggregation by dilated convolutions[EB/OL]. (2015?11?23)[2023?04?30]. https://arxiv.org/abs/1511.07122.pdf.
[27] LEE Chenyu, XIE Saining, GALLAGHER P, et al. Deeply-supervised nets[EB/OL]. (2014?09?18)[2023?04?30]. https://arxiv.org/abs/1409.5185.pdf.
[28] IDREES H, TAYYAB M, ATHREY K, et al. Composition loss for counting, density map estimation and localization in dense crowds[C]//Computer Vision - ECCV 2018: 15th European Conference, Munich, Germany, September 8-14, 2018, Proceedings, Part II. New York: ACM, 2018: 544?559.
[29] WANG Qi, GAO Junyu, LIN Wei, et al. NWPU-crowd: a large-scale benchmark for crowd counting and localization[J]. IEEE transactions on pattern analysis and machine intelligence, 2021, 43(6): 2141–2149.
[30] MENG Yanda, ZHANG Hongrun, ZHAO Yitian, et al. Spatial uncertainty-aware semi-supervised crowd counting[C]//2021 IEEE/CVF International Conference on Computer Vision . Piscataway: IEEE, 2022: 15529?15539.
[31] CHEN Qi, LEI Tao, GENG Xinzhe, et al. DLMP-net: a dynamic yet lightweight multi-pyramid network for crowd density estimation[C]//Chinese Conference on Pattern Recognition and Computer Vision . Cham: Springer, 2022: 27?39.
[32] LIANG Lanjun, ZHAO Huailin, ZHOU Fangbo, et al. SC2Net: scale-aware crowd counting network with pyramid dilated convolution[J]. Applied intelligence, 2023, 53(5): 5146–5159.
[33] LIANG Dingkang, XU Wei, ZHU Yingying, et al. Focal inverse distance transform maps for crowd localization[J]. IEEE transactions on multimedia, 2023, 25: 6040–6052.
[34] YANG Yifan, LI Guorong, WU Zhe, et al. Weakly-supervised crowd counting learns from sorting rather than locations[C]//Vedaldi A, Bischof H, Brox T, et al. European Conference on Computer Vision. Cham: Springer, 2020: 1?17.
[35] LEI Yinjie, LIU Yan, ZHANG Pingping, et al. Towards using count-level weak supervision for crowd counting[J]. Pattern recognition, 2021, 109: 107616.
[36] ZAND M, DAMIRCHI H, FARLEY A, et al. Multiscale crowd counting and localization by multitask point supervision[C]//ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing . Piscataway: IEEE, 2022: 1820?1824.
Similar References:

Memo

-

Last Update: 1900-01-01

Copyright © CAAI Transactions on Intelligent Systems