[1]XING Tianyi,GUO Maozu,CHEN Jiadong,et al.Detection of abnormal crowd behavior based on spatial-temporal adversarial variational autoencoder[J].CAAI Transactions on Intelligent Systems,2023,18(5):994-1004.[doi:10.11992/tis.202303002]
Copy

Detection of abnormal crowd behavior based on spatial-temporal adversarial variational autoencoder

CAAI Transactions on Intelligent Systems[ISSN 1673-4785/CN 23-1538/TP] Volume: 18 Number of periods: 2023 5 Page number: 994-1004 Column: 学术论文—机器学习 Public date: 2023-09-05
Title:
Detection of abnormal crowd behavior based on spatial-temporal adversarial variational autoencoder
Author(s):
XING Tianyi1;  GUO Maozu1;  CHEN Jiadong1;  ZHAO Lingling2;  CHEN Linxin2;  TIAN Le1
1. School of Electrical and Information Engineering, Beijing, University of Civil Engineering and Architecture, Beijing 100044 China;
2. Faculty of Computing, Harbin Institute of Technology, Harbin 150001, China
Keywords:
detection on abnormal crowd behavior; variational autoencoder; autoencoder; long short-term memory network; adversarial network; spatiotemporal adversarial variational autoencoder; reconstruction errors; abnormal escape behavior
CLC:
TP181
DOI:
10.11992/tis.202303002
Abstract:
Video-based detection of abnormal crowd behavior is important for the early discovery of safety risks and the prevention of group safety accidents. To address insufficient direct learning of the representation of abnormal samples because of the scarcity of abnormal crowd behavior events and the low detection accuracy of abnormal events, this study proposed a predictive spatiotemporal adversarial variational autoencoder (ST-AVAE) video anomaly detection model based on variational autoencoder, by adding the long short-term memory and adversarial network modules. Joint feature representation and reconstruction of the temporal and spatial dimensions of normal sample video sequences were performed, which reduced the feature loss in the reconstruction process of normal samples, thereby expanding the prediction loss of abnormal samples, avoiding dependence on abnormal samples, and realizing the detection of the abnormal behavior of crowd dispersal based on model reconstruction errors. Comparative experiments were conducted on the public dataset UMN and captured video datasets to prove that the ST-AVAE model has the optimal detection accuracy and recall rate in the detection of abnormal crowd escape behavior based on surveillance video, and the adversarial network module significantly improves the performance of anomaly detection.
References:
[1] LI Weixin, MAHADEVAN V, VASCONCELOS N. Anomaly detection and localization in crowded scenes[J]. IEEE transactions on pattern analysis and machine intelligence, 2014, 36(1): 18-32.
[2] XIE Shaoci, ZHANG Xiaohong, CAI Jing. Video crowd detection and abnormal behavior model detection based on machine learning method[J]. Neural computing and applications, 2019, 31(1): 175-184.
[3] HASAN M, CHOI J, NEUMANN J, et al. Learning temporal regularity in video sequences[C]//IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016: 733?742.
[4] CHONG Y S, TAY Y H. Abnormal event detection in videos using spatiotemporal autoencoder[C]//CONG F, LEUNG A, WEI Q. International Symposium on Neural Networks. Cham: Springer, 2017: 189?196.
[5] LUO Weixin, LIU Wen, GAO Shenghua. Remembering history with convolutional LSTM for anomaly detection[C]//2017 IEEE International Conference on Multimedia and Expo. Hong Kong: IEEE, 2017: 439?444.
[6] 杨彪, 曹金梦, 张御宇, 等. 加权卷积自编码长短期记忆网络人群异常检测方法: CN108805015B[P]. 2021-09-03.
YANG Biao, CAO Jinmeng, ZHANG Yuyu, et al. Weighted convolutional autoencoder-long short-term memory network-based crowd anomaly detection method: CN108805015B[P]. 2021-09-03.
[7] YAN Shiyang, SMITH J S, LU Wenjin, et al. Abnormal event detection from videos using a two-stream recurrent variational autoencoder[J]. IEEE transactions on cognitive and developmental systems, 2020, 12(1): 30-42.
[8] LIU Jie, SONG Kechen, FENG Mingzheng, et al. Semi-supervised anomaly detection with dual prototypes autoencoder for industrial surface inspection[J]. Optics and lasers in engineering, 2021, 136: 106324.
[9] LIU Wen, LUO Weixin, LIAN Dongze, et al. Future frame prediction for anomaly detection-A new baseline[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 6536?6545.
[10] DONG Fei, ZHANG Yu, NIE Xiushan. Dual discriminator generative adversarial network for video anomaly detection[J]. IEEE access, 2020, 8: 88170-88176.
[11] NGUYEN T N, MEUNIER J. Anomaly detection in video sequence with appearance-motion correspondence[C]//2019 IEEE/CVF International Conference on Computer Vision. Seoul: IEEE, 2020: 1273?1283.
[12] MAKHZANI A, SHLENS J, JAITLY N, et al. Adversarial autoencoders[EB/OL]. (2015-11-18)[2020-01-01]. https://arxiv.org/abs/1511.05644.
[13] 唐浩漾, 张小媛, 王燕, 等. 基于生成对抗网络的人体异常行为检测算法[J]. 西安邮电大学学报, 2020, 25(3): 92-97
TANG Haoyang, ZHANG Xiaoyan, WANG Yan, et al. Human abnormal behaviour detection algorithm based on generative adversarial nets[J]. Journal of Xi’an University of Posts and Telecommunications, 2020, 25(3): 92-97
[14] LI Nanjun, CHANG Faliang, LIU Chunsheng. Spatial-temporal cascade autoencoder for video anomaly detection in crowded scenes[J]. IEEE transactions on multimedia, 2021, 23: 203-215.
[15] 张冀, 曹艺, 王亚茹, 等. 融合VAE和StackGAN的零样本图像分类方法[J]. 智能系统学报, 2022, 17(3): 593-601
ZHANG Ji, CAO Yi, WANG Yaru, et al. Zero-shot image classification method combining VAE and StackGAN[J]. CAAI transactions on intelligent systems, 2022, 17(3): 593-601
[16] PARK H, NOH J, HAM B. Learning memory-guided normality for anomaly detection[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle: IEEE, 2020: 14360?14369.
[17] MARKOVITZ A, SHARIR G, FRIEDMAN I, et al. Graph embedded pose clustering for anomaly detection[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle: IEEE, 2020: 10536?10544.
[18] GOYAL S, RAGHUNATHAN A, JAIN M, et al. DROCC: Deep robust one-class classification[C]//International Conference on Machine Learning. [S.l.]: PMLR, 2020: 3711?3721.
[19] LI Chunliang, SOHN K, YOON J, et al. CutPaste: self-supervised learning for anomaly detection and localization[C]//2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Nashville: IEEE, 2021: 9659?9669.
[20] RUDOLPH M, WEHRBEIN T, ROSENHAHN B, et al. Fully convolutional cross-scale-flows for image-based defect detection[EB/OL]. (2021-10-06)[2022-12-01]. https://arxiv.org/abs/2110.02855.
[21] CARRARA F, AMATO G, BROMBIN L, et al. Combining GANs and AutoEncoders for efficient anomaly detection[C]//2020 25th International Conference on Pattern Recognition. Milan: IEEE, 2021: 3939?3946.
[22] HE Kaiming, ZHANG Xiangyu, REN Shaoqing, et al. Deep residual learning for image recognition[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016: 770?778.
[23] UMN. University of minnesota dataset for detection of unusual crowd activity[EB/OL]. (2006-05-30)[2020-01-01]. http://mha.cs.umn.edu/proj_events.shtml#crowd.
[24] WANG Lin, ZHOU Fuqiang, LI Zuoxin, et al. Abnormal event detection in videos using hybrid spatio-temporal autoencoder[C]// 25th IEEE International Conference on Image Processing. Athens: IEEE, 2018: 2276?2280.
[25] AN J, CHO S. Variational autoencoder based anomaly detection using reconstruction probability[J]. Special lecture on IE, 2015, 2(1): 1-18.
[26] DAI Feng, LIU Hao, MA Yike, et al. Dense scale network for crowd counting[EB/OL]. (2019-06-24)[2020-01-01]. https://arxiv.org/abs/1906.09707.
Similar References:

Memo

-

Last Update: 1900-01-01

Copyright © CAAI Transactions on Intelligent Systems