[1]邢天祎,郭茂祖,陈加栋,等.基于空时对抗变分自编码器的人群异常行为检测[J].智能系统学报,2023,18(5):994-1004.[doi:10.11992/tis.202303002]
XING Tianyi,GUO Maozu,CHEN Jiadong,et al.Detection of abnormal crowd behavior based on spatial-temporal adversarial variational autoencoder[J].CAAI Transactions on Intelligent Systems,2023,18(5):994-1004.[doi:10.11992/tis.202303002]
点击复制
XING Tianyi,GUO Maozu,CHEN Jiadong,et al.Detection of abnormal crowd behavior based on spatial-temporal adversarial variational autoencoder[J].CAAI Transactions on Intelligent Systems,2023,18(5):994-1004.[doi:10.11992/tis.202303002]
基于空时对抗变分自编码器的人群异常行为检测
《智能系统学报》[ISSN 1673-4785/CN 23-1538/TP] 卷:
18
期数:
2023年第5期
页码:
994-1004
栏目:
学术论文—机器学习
出版日期:
2023-09-05
- Title:
- Detection of abnormal crowd behavior based on spatial-temporal adversarial variational autoencoder
- Keywords:
- detection on abnormal crowd behavior; variational autoencoder; autoencoder; long short-term memory network; adversarial network; spatiotemporal adversarial variational autoencoder; reconstruction errors; abnormal escape behavior
- 分类号:
- TP181
- 摘要:
- 基于视频的人群异常行为检测对提前发现安全风险、预防群体安全事故发生具有重要价值。针对人群异常行为事件的稀少性导致的无法直接充分学习异常样本的表示、异常事件检测精度低的问题，在变分自编码器基础上，提出一种基于预测的空时对抗变分自编码器（spatial-temporal adversarial variational autoencoder, ST-AVAE）视频异常检测模型，通过引入长短期记忆网络（long short-term memory，LSTM）和对抗网络模块，对正常样本视频序列的时间维度与空间维度进行联合特征表示与重构，减少了正常样本重建过程中的特征损失进而扩大了异常样本的预测损失，避免了对异常样本的依赖，实现了基于模型重构误差的人群逃散异常行为检测。在公开数据集UMN及采集视频数据集上进行对比实验，证明ST-AVAE模型在基于监控视频的人群异常逃散行为检测中均具有最优的检测精度和召回率，对抗网络模块显著提升了异常检测的性能。
- Abstract:
- Video-based detection of abnormal crowd behavior is important for the early discovery of safety risks and the prevention of group safety accidents. To address insufficient direct learning of the representation of abnormal samples because of the scarcity of abnormal crowd behavior events and the low detection accuracy of abnormal events, this study proposed a predictive spatiotemporal adversarial variational autoencoder (ST-AVAE) video anomaly detection model based on variational autoencoder, by adding the long short-term memory and adversarial network modules. Joint feature representation and reconstruction of the temporal and spatial dimensions of normal sample video sequences were performed, which reduced the feature loss in the reconstruction process of normal samples, thereby expanding the prediction loss of abnormal samples, avoiding dependence on abnormal samples, and realizing the detection of the abnormal behavior of crowd dispersal based on model reconstruction errors. Comparative experiments were conducted on the public dataset UMN and captured video datasets to prove that the ST-AVAE model has the optimal detection accuracy and recall rate in the detection of abnormal crowd escape behavior based on surveillance video, and the adversarial network module significantly improves the performance of anomaly detection.
- 参考文献/References:
-
[1] LI Weixin, MAHADEVAN V, VASCONCELOS N. Anomaly detection and localization in crowded scenes[J]. IEEE transactions on pattern analysis and machine intelligence, 2014, 36(1): 18-32.
[2] XIE Shaoci, ZHANG Xiaohong, CAI Jing. Video crowd detection and abnormal behavior model detection based on machine learning method[J]. Neural computing and applications, 2019, 31(1): 175-184.
[3] HASAN M, CHOI J, NEUMANN J, et al. Learning temporal regularity in video sequences[C]//IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016: 733?742.
[4] CHONG Y S, TAY Y H. Abnormal event detection in videos using spatiotemporal autoencoder[C]//CONG F, LEUNG A, WEI Q. International Symposium on Neural Networks. Cham: Springer, 2017: 189?196.
[5] LUO Weixin, LIU Wen, GAO Shenghua. Remembering history with convolutional LSTM for anomaly detection[C]//2017 IEEE International Conference on Multimedia and Expo. Hong Kong: IEEE, 2017: 439?444.
[6] 杨彪, 曹金梦, 张御宇, 等. 加权卷积自编码长短期记忆网络人群异常检测方法: CN108805015B[P]. 2021-09-03.
YANG Biao, CAO Jinmeng, ZHANG Yuyu, et al. Weighted convolutional autoencoder-long short-term memory network-based crowd anomaly detection method: CN108805015B[P]. 2021-09-03.
[7] YAN Shiyang, SMITH J S, LU Wenjin, et al. Abnormal event detection from videos using a two-stream recurrent variational autoencoder[J]. IEEE transactions on cognitive and developmental systems, 2020, 12(1): 30-42.
[8] LIU Jie, SONG Kechen, FENG Mingzheng, et al. Semi-supervised anomaly detection with dual prototypes autoencoder for industrial surface inspection[J]. Optics and lasers in engineering, 2021, 136: 106324.
[9] LIU Wen, LUO Weixin, LIAN Dongze, et al. Future frame prediction for anomaly detection-A new baseline[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 6536?6545.
[10] DONG Fei, ZHANG Yu, NIE Xiushan. Dual discriminator generative adversarial network for video anomaly detection[J]. IEEE access, 2020, 8: 88170-88176.
[11] NGUYEN T N, MEUNIER J. Anomaly detection in video sequence with appearance-motion correspondence[C]//2019 IEEE/CVF International Conference on Computer Vision. Seoul: IEEE, 2020: 1273?1283.
[12] MAKHZANI A, SHLENS J, JAITLY N, et al. Adversarial autoencoders[EB/OL]. (2015-11-18)[2020-01-01]. https://arxiv.org/abs/1511.05644.
[13] 唐浩漾, 张小媛, 王燕, 等. 基于生成对抗网络的人体异常行为检测算法[J]. 西安邮电大学学报, 2020, 25(3): 92-97
TANG Haoyang, ZHANG Xiaoyan, WANG Yan, et al. Human abnormal behaviour detection algorithm based on generative adversarial nets[J]. Journal of Xi’an University of Posts and Telecommunications, 2020, 25(3): 92-97
[14] LI Nanjun, CHANG Faliang, LIU Chunsheng. Spatial-temporal cascade autoencoder for video anomaly detection in crowded scenes[J]. IEEE transactions on multimedia, 2021, 23: 203-215.
[15] 张冀, 曹艺, 王亚茹, 等. 融合VAE和StackGAN的零样本图像分类方法[J]. 智能系统学报, 2022, 17(3): 593-601
ZHANG Ji, CAO Yi, WANG Yaru, et al. Zero-shot image classification method combining VAE and StackGAN[J]. CAAI transactions on intelligent systems, 2022, 17(3): 593-601
[16] PARK H, NOH J, HAM B. Learning memory-guided normality for anomaly detection[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle: IEEE, 2020: 14360?14369.
[17] MARKOVITZ A, SHARIR G, FRIEDMAN I, et al. Graph embedded pose clustering for anomaly detection[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle: IEEE, 2020: 10536?10544.
[18] GOYAL S, RAGHUNATHAN A, JAIN M, et al. DROCC: Deep robust one-class classification[C]//International Conference on Machine Learning. [S.l.]: PMLR, 2020: 3711?3721.
[19] LI Chunliang, SOHN K, YOON J, et al. CutPaste: self-supervised learning for anomaly detection and localization[C]//2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Nashville: IEEE, 2021: 9659?9669.
[20] RUDOLPH M, WEHRBEIN T, ROSENHAHN B, et al. Fully convolutional cross-scale-flows for image-based defect detection[EB/OL]. (2021-10-06)[2022-12-01]. https://arxiv.org/abs/2110.02855.
[21] CARRARA F, AMATO G, BROMBIN L, et al. Combining GANs and AutoEncoders for efficient anomaly detection[C]//2020 25th International Conference on Pattern Recognition. Milan: IEEE, 2021: 3939?3946.
[22] HE Kaiming, ZHANG Xiangyu, REN Shaoqing, et al. Deep residual learning for image recognition[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016: 770?778.
[23] UMN. University of minnesota dataset for detection of unusual crowd activity[EB/OL]. (2006-05-30)[2020-01-01]. http://mha.cs.umn.edu/proj_events.shtml#crowd.
[24] WANG Lin, ZHOU Fuqiang, LI Zuoxin, et al. Abnormal event detection in videos using hybrid spatio-temporal autoencoder[C]// 25th IEEE International Conference on Image Processing. Athens: IEEE, 2018: 2276?2280.
[25] AN J, CHO S. Variational autoencoder based anomaly detection using reconstruction probability[J]. Special lecture on IE, 2015, 2(1): 1-18.
[26] DAI Feng, LIU Hao, MA Yike, et al. Dense scale network for crowd counting[EB/OL]. (2019-06-24)[2020-01-01]. https://arxiv.org/abs/1906.09707.
- 相似文献/References:
-
[1]王德文,魏波涛.基于孪生变分自编码器的小样本图像分类方法[J].智能系统学报,2021,16(2):254.[doi:10.11992/tis.201906022]
WANG Dewen,WEI Botao.A small-sample image classification method based on a Siamese variational auto-encoder[J].CAAI Transactions on Intelligent Systems,2021,16():254.[doi:10.11992/tis.201906022]
[2]张冀,曹艺,王亚茹,等.融合VAE和StackGAN的零样本图像分类方法[J].智能系统学报,2022,17(3):593.[doi:10.11992/tis.202107012]
ZHANG Ji,CAO Yi,WANG Yaru,et al.Zero-shot image classification method combining VAE and StackGAN[J].CAAI Transactions on Intelligent Systems,2022,17():593.[doi:10.11992/tis.202107012]
备注/Memo
收稿日期:2023-3-1。
基金项目:国家自然科学基金项目（62271036，61871020）；国家重点研发计划项目（2021YFF0306303）；北京市属高校高水平创新团队建设计划项目（IDHT20190506）.
作者简介:邢天祎，硕士研究生，主要研究方向为模式识别与智能系统;郭茂祖，教授，博士生导师，博士，中国人工智能学会机器学习专委会常委、中国建筑学会计算性设计学术委员会常委，主要研究方向为机器学习、智慧城市、计算生物学。北京建筑大学电气与信息工程学院院长，2019 年以第一完成人获吴文俊人工智能自然 科学奖二等奖。发表学术论文100余篇。;赵玲玲，副教授，博士，中国计算机学会生物信息学专委会委员、中国建筑学会计算性设计专委会委员，主要研究方向为机器学习、城市计算、生物信息学。主持和参与国家自然科学基金青年基金、面上项目、重点项目8 项。发表学术论文40 余篇
通讯作者:赵玲玲.E-mail：Zhaoll@hit.edu.cn
更新日期/Last Update:
1900-01-01