[1]孟月波,张娅琳,王宙.比例融合与多层规模感知的人群计数方法[J].智能系统学报,2024,19(2):307-315.[doi:10.11992/tis.202208048]
MENG Yuebo,ZHANG Yalin,WANG Zhou.Crowd counting method based on proportion fusion and multilayer scale-aware[J].CAAI Transactions on Intelligent Systems,2024,19(2):307-315.[doi:10.11992/tis.202208048]
点击复制
MENG Yuebo,ZHANG Yalin,WANG Zhou.Crowd counting method based on proportion fusion and multilayer scale-aware[J].CAAI Transactions on Intelligent Systems,2024,19(2):307-315.[doi:10.11992/tis.202208048]
比例融合与多层规模感知的人群计数方法
《智能系统学报》[ISSN 1673-4785/CN 23-1538/TP] 卷:
19
期数:
2024年第2期
页码:
307-315
栏目:
学术论文—机器感知与模式识别
出版日期:
2024-03-05
- Title:
- Crowd counting method based on proportion fusion and multilayer scale-aware
- Keywords:
- crowd density estimation and counting; convolutional neural network; multilayer scale-aware; proportional fusion; local consistency loss; density map regression; multiscale information; dilated convolution
- 分类号:
- TP391
- 文献标志码:
- 2023-11-16
- 摘要:
- 针对密集场景下人群图像拍摄视角或距离多变造成的多尺度特征获取不足、融合不佳和全局特征利用不充分等问题，提出一种比例融合与多层规模感知的人群计数网络。首先采用骨干网络VGG16提取人群密度初始特征；其次，设计多层规模感知模块，获得人群多尺度信息的丰富表达；再次，提出比例融合策略，根据卷积层捕获的特征权重重构多尺度信息，提取显著性人群特征；最后，采用卷积回归策略进行密度图的回归。同时，提出一种局部一致性损失函数，通过区域化密度图的方式增强生成密度图与真实密度图的相似度，提高计数性能。在多个人群数据集上的试验结果表明，所提模型优于近年人群计数的先进方法，且在车辆计数上有较好推广性。
- Abstract:
- To deal with the problems of insufficient multiscale feature acquisition, poor fusion, and insufficient utilization of global features as a result of the changing view angles or distances of crowd images in dense scenes, we propose a crowd counting network based on proportion fusion and multilayer scale-aware. First, the backbone network VGG16 is employed to extract the initial characteristics of the population density. Subsequently, a multilayer scale-aware module is developed to acquire a rich expression of multiscale information from the crowd. Afterward, a proportional fusion strategy is designed to reconstruct the multiscale information based on the feature weights captured by the convolution layer and extract the significant crowd features. Lastly, convolution regression is utilized to regress the density map. Concurrently, a local consistency loss function is proposed, which improves the similarity between the generated density map and the real density map by regionalizing the density map and enhances the counting performance. The results of the experiments on multiple population datasets exhibit that the model proposed here surpasses the existing state-of-the-art methods of population density counting and has good generalization in vehicle counting.
- 参考文献/References:
-
[1] CHEN Ke, LOY C C, GONG Shaogang, et al. Feature mining for localised crowd counting[C]//Proceedings ofthe British Machine Vision Conference 2012. Surrey. British Machine Vision Association, 2012: 1-11.
[2] 向飞宇, 张秀伟. 基于卷积神经网络的人群计数算法研究[J]. 计算机技术与发展, 2021, 31(7): 42–46
XIANG Feiyu, ZHANG Xiuwei. Research on crowd counting algorithm based on convolution neural network[J]. Computer technology and development, 2021, 31(7): 42–46
[3] SOURTZINOS P, VELASTIN S A, JARA M, et al. People counting in videos by fusing temporal cues from spatial context-aware convolutional neural networks[M]//Lecture Notes in Computer Science. Cham: Springer International Publishing, 2016: 655-667.
[4] ZHANG Yingying, ZHOU Desen, CHEN Siqin, et al. Single-image crowd counting via multi-column convolutional neural network[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016: 589-597.
[5] SAM D B, SURYA S, BABU R V. Switching convolutional neural network for crowd counting[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE, 2017: 4031-4039.
[6] 孟月波, 纪拓, 刘光辉, 等. 编码-解码多尺度卷积神经网络人群计数方法[J]. 西安交通大学学报, 2020, 54(5): 149–157
MENG Yuebo, JI Tuo, LIU Guanghui, et al. Encoding-decoding multi-scale convolutional neural network for crowd counting[J]. Journal of Xi’an Jiaotong university, 2020, 54(5): 149–157
[7] SHEN Zan, XU Yi, NI Bingbing, et al. Crowd counting via adversarial cross-scale consistency pursuit[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 5245-5254.
[8] JIANG Xiaoheng, ZHANG Li, XU Mingliang, et al. Attention scaling for crowd counting[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle: IEEE, 2020: 4705-4714.
[9] LI Yuhong, ZHANG Xiaofan, CHEN Deming. CSRNet: dilated convolutional neural networks for understanding the highly congested scenes[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 1091-1100.
[10] XU Chenfeng, QIU Kai, FU Jianlong, et al. Learn to scale: generating multipolar normalized density maps for crowd counting[C]//2019 IEEE/CVF International Conference on Computer Vision. Seoul: IEEE, 2020: 8381-8389.
[11] JIANG Xiaolong, XIAO Zehao, ZHANG Baochang, et al. Crowd counting and density estimation by trellis encoder-decoder networks[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach: IEEE, 2020: 6126-6135.
[12] LIU Weizhe, SALZMANN M, FUA P. Context-aware crowd counting[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach: IEEE, 2020: 5094-5103.
[13] LIU Xiyang, YANG Jie, DING Wenrui, et al. Adaptive mixture regression network with local counting map for crowd counting[M]//Computer Vision-ECCV 2020. Cham: Springer International Publishing, 2020: 241-257.
[14] SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[J]. 3rd international conference on learning representations, ICLR 2015-conference track proceedings, 2015: 1-14.
[15] SINDAGI V A, PATEL V M. Generating high-quality crowd density maps using contextual pyramid CNNs[C]//2017 IEEE International Conference on Computer Vision. Venice: IEEE, 2017: 1879-1888.
[16] 刘万军, 佟畅, 曲海成. 空洞卷积与注意力融合的对抗式图像阴影去除算法[J]. 智能系统学报, 2021, 16(6): 1081–1089
LIU Wanjun, TONG Chang, QU Haicheng. An antagonistic image shadow removal algorithm based on dilated convolution and attention mechanism[J]. CAAI transactions on intelligent systems, 2021, 16(6): 1081–1089
[17] ZHANG Cong, LI Hongsheng, WANG Xiaogang, et al. Cross-scene crowd counting via deep convolutional neural networks[C]//2015 IEEE Conference on Computer Vision and Pattern Recognition. Boston: IEEE, 2015: 833-841.
[18] IDREES H, SALEEMI I, SEIBERT C, et al. Multi-source multi-scale counting in extremely dense crowd images[C]//2013 IEEE Conference on Computer Vision and Pattern Recognition. Portland: IEEE, 2013: 2547-2554.
[19] IDREES H, TAYYAB M, ATHREY K, et al. Composition loss for counting, density map estimation and localization in dense crowds[M]//Computer Vision-ECCV 2018. Cham: Springer International Publishing, 2018: 544-559.
[20] ZHU Pengfei, WEN Longyin, DU Dawei, et al. Vision Meets Drones: Past, Present and Future[EB/OL]. (2020-01-16)[2022-01-01]. https://arxiv.org/pdf/2001.06303.pdf.
[21] XIONG Feng, SHI Xingjian, YEUNG D Y. Spatiotemporal modeling for crowd counting in videos[C]//2017 IEEE International Conference on Computer Vision. Venice: IEEE, 2017: 5161-5169.
[22] 张宇倩, 李国辉, 雷军, 等. FF-CAM: 基于通道注意机制前后端融合的人群计数[J]. 计算机学报, 2021, 44(2): 304–317
ZHANG Yuqian, LI Guohui, LEI Jun, et al. FF-CAM: crowd counting based on frontend-backend fusion through channel-attention mechanism[J]. Chinese journal of computers, 2021, 44(2): 304–317
[23] ZENG Xin, WU Yunpeng, HU Shizhe, et al. DSPNet: deep scale purifier network for dense crowd counting[J]. Expert systems with applications, 2020, 141: 112977.
[24] 翁佳鑫, 仝明磊. 基于Triangle Net的密集人群计数[J]. 科技创新与应用, 2021(9): 38–40,44
WENG Jiaxin, TONG Minglei. Dense crowd counting based on Triangle Net[J]. Technology innovation and application, 2021(9): 38–40,44
[25] ZHANG Jun, LIU Jiaze, WANG Zhizhong. Convolutional neural network for crowd counting on metro platforms[J]. Symmetry, 2021, 13(4): 703.
- 相似文献/References:
-
[1]殷瑞,苏松志,李绍滋.一种卷积神经网络的图像矩正则化策略[J].智能系统学报,2016,11(1):43.[doi:10.11992/tis.201509018]
YIN Rui,SU Songzhi,LI Shaozi.Convolutional neural network’s image moment regularizing strategy[J].CAAI Transactions on Intelligent Systems,2016,11():43.[doi:10.11992/tis.201509018]
[2]龚震霆,陈光喜,任夏荔,等.基于卷积神经网络和哈希编码的图像检索方法[J].智能系统学报,2016,11(3):391.[doi:10.11992/tis.201603028]
GONG Zhenting,CHEN Guangxi,REN Xiali,et al.An image retrieval method based on a convolutional neural network and hash coding[J].CAAI Transactions on Intelligent Systems,2016,11():391.[doi:10.11992/tis.201603028]
[3]刘帅师,程曦,郭文燕,等.深度学习方法研究新进展[J].智能系统学报,2016,11(5):567.[doi:10.11992/tis.201511028]
LIU Shuaishi,CHENG Xi,GUO Wenyan,et al.Progress report on new research in deep learning[J].CAAI Transactions on Intelligent Systems,2016,11():567.[doi:10.11992/tis.201511028]
[4]师亚亭,李卫军,宁欣,等.基于嘴巴状态约束的人脸特征点定位算法[J].智能系统学报,2016,11(5):578.[doi:10.11992/tis.201602006]
SHI Yating,LI Weijun,NING Xin,et al.A facial feature point locating algorithmbased on mouth-state constraints[J].CAAI Transactions on Intelligent Systems,2016,11():578.[doi:10.11992/tis.201602006]
[5]宋婉茹,赵晴晴,陈昌红,等.行人重识别研究综述[J].智能系统学报,2017,12(6):770.[doi:10.11992/tis.201706084]
SONG Wanru,ZHAO Qingqing,CHEN Changhong,et al.Survey on pedestrian re-identification research[J].CAAI Transactions on Intelligent Systems,2017,12():770.[doi:10.11992/tis.201706084]
[6]杨晓兰,强彦,赵涓涓,等.基于医学征象和卷积神经网络的肺结节CT图像哈希检索[J].智能系统学报,2017,12(6):857.[doi:10.11992/tis.201706035]
YANG Xiaolan,QIANG Yan,ZHAO Juanjuan,et al.Hashing retrieval for CT images of pulmonary nodules based on medical signs and convolutional neural networks[J].CAAI Transactions on Intelligent Systems,2017,12():857.[doi:10.11992/tis.201706035]
[7]王科俊,赵彦东,邢向磊.深度学习在无人驾驶汽车领域应用的研究进展[J].智能系统学报,2018,13(1):55.[doi:10.11992/tis.201609029]
WANG Kejun,ZHAO Yandong,XING Xianglei.Deep learning in driverless vehicles[J].CAAI Transactions on Intelligent Systems,2018,13():55.[doi:10.11992/tis.201609029]
[8]莫凌飞,蒋红亮,李煊鹏.基于深度学习的视频预测研究综述[J].智能系统学报,2018,13(1):85.[doi:10.11992/tis.201707032]
MO Lingfei,JIANG Hongliang,LI Xuanpeng.Review of deep learning-based video prediction[J].CAAI Transactions on Intelligent Systems,2018,13():85.[doi:10.11992/tis.201707032]
[9]王成济,罗志明,钟准,等.一种多层特征融合的人脸检测方法[J].智能系统学报,2018,13(1):138.[doi:10.11992/tis.201707018]
WANG Chengji,LUO Zhiming,ZHONG Zhun,et al.Face detection method fusing multi-layer features[J].CAAI Transactions on Intelligent Systems,2018,13():138.[doi:10.11992/tis.201707018]
[10]葛园园,许有疆,赵帅,等.自动驾驶场景下小且密集的交通标志检测[J].智能系统学报,2018,13(3):366.[doi:10.11992/tis.201706040]
GE Yuanyuan,XU Youjiang,ZHAO Shuai,et al.Detection of small and dense traffic signs in self-driving scenarios[J].CAAI Transactions on Intelligent Systems,2018,13():366.[doi:10.11992/tis.201706040]
备注/Memo
收稿日期:2022-08-30。
基金项目:陕西省重点研发计划项目（2021SF-429）.
作者简介:孟月波，教授，博士生导师，博士，主要研究方向为机器视觉信息处理与分析、建筑智能化。近年来主持/参与国家自然科学基金项目、国家重点研发计划项目、陕西省基础研究项目和陕西省重点研发项目10项。发表学术论文30余篇。 E-mail：mengyuebo@ 163.com;张娅琳，硕士研究生，主要研究方向为计算机视觉理解、建筑智能化技术。E-mail：1243697118@qq.com;王宙，硕士研究生，主要研究方向为深度学习、计算机视觉。E-mail：1119307454@qq.com
通讯作者:孟月波. E-mail：mengyuebo@163.com
更新日期/Last Update:
1900-01-01