[1]JI Xiaofei,XIE Xuan,REN Yan.Human interaction recognition and prediction algorithm based on deep learning[J].CAAI Transactions on Intelligent Systems,2020,15(3):484-490.[doi:10.11992/tis.201812029]
Copy

Human interaction recognition and prediction algorithm based on deep learning

CAAI Transactions on Intelligent Systems[ISSN 1673-4785/CN 23-1538/TP] Volume: 15 Number of periods: 2020 3 Page number: 484-490 Column: 学术论文—智能系统 Public date: 2020-05-05
Title:
Human interaction recognition and prediction algorithm based on deep learning
Author(s):
JI Xiaofei;  XIE Xuan;  REN Yan
School of Automation, Shenyang Aerospace University, Shenyang 110136, China
Keywords:
video analysis; action recognition; action prediction; deep learning; convolutional neural network; long short term memory; UT-interaction dataset; SBU Kinect interaction dataset
CLC:
TP391.4
DOI:
10.11992/tis.201812029
Abstract:
A drawback of the human interaction recognition algorithm based on a convolutional neural network (CNN) is that the extracted depth features cannot effectively represent the characteristics of interaction sequences. Instead, this paper proposes a human interaction recognition and prediction algorithm based on deep learning, by combining the Long Short-Term Memory (LSTM) network with the CNN model. In the process, video images of unknown action categories of different time lengths are sent to a trained CNN model to extract the depth features. The depth features are then sent to a trained LSTM model to complete the recognition and prediction of the interaction behavior. When tested on the UT-interaction human interaction behavior dataset, the algorithm demonstrates a 92.31% correct human interaction recognition rate and can complete the preliminary prediction of unknown actions.
References:
[1] RYOO M S. Human activity prediction: Early recognition of ongoing activities from streaming videos[C]//Proceedings of 2011 International Conference on Computer Vision. Barcelona, Spain, 2011: 1036-1043.
[2] XU Kaiping, QIN Zheng, WANG Guolong. Human activities prediction by learning combinatorial sparse representations[C]//Proceedings of 2016 IEEE International Conference on Image Processing. Phoenix, USA, 2016: 724-728.
[3] RAPTIS M, SIGAL L. Poselet key-framing: a model for human activity recognition[C]//Proceedings of 2013 IEEE Conference on Computer Vision and Pattern Recognition. Portland, USA, 2013: 2650-2657.
[4] KONG Yu, FU Yun. Max-margin action prediction machine[J]. IEEE transactions on pattern analysis and machine intelligence, 2016, 38(9): 1844-1858.
[5] KUNZE K, LUKOWICZ P. Dealing with sensor displacement in motion-based onbody activity recognition systems[C]//Proceedings of the 10th International Conference on Ubiquitous Computing. Seoul, South Korea, 2008: 20-29.
[6] BULLING A, ROGGEN D. Recognition of visual memory recall processes using eye movement analysis[C]//Proceedings of the 13th International Conference on Ubiquitous Computing. New York, USA, 2011: 455-464.
[7] VAN KASTEREN T, NOULAS A, ENGLEBIENNE G, et al. Accurate activity recognition in a home setting[C]//Proceedings of the 10th International Conference on Ubiquitous Computing. Seoul, South Korea, 2008: 1-9.
[8] CHUNG P C, LIU C D. A daily behavior enabled hidden Markov model for human behavior understanding[J]. Pattern recognition, 2008, 41(5): 1572-1580.
[9] TANG K, LI Feifei, KOLLER D. Learning latent temporal structure for complex event detection[C]//Proceedings of 2012 IEEE Conference on Computer Vision and Pattern Recognition. Providence, USA, 2012: 1025-1257.
[10] LAFFERTY J D, MCCALLUM A, PEREIRA F C N. Conditional random fields: probabilistic models for segmenting and labeling sequence data[C]//Proceedings of the 18th International Conference on Machine Learning. San Francisco, USA, 2001: 282-289.
[11] ZHANG Jianguo, GONG Shaogang. Action categorization with modified hidden conditional random field[J]. Pattern recognition, 2010, 43(1): 197-203.
[12] SONG Yale, MORENCY L P, DAVIS R. Action recognition by hierarchical sequence summarization[C]//IEEE Conference on Computer Vision and Pattern Recognition. Portland, USA, 2013: 3563-3569.
[13] KE Qiuhong, BENNAMOUN M, AN Senjian, et al. Human interaction prediction using deep temporal features [C]//Proceedings of European Conference on Computer Vision. Amsterdam, The Netherlands, 2016: 403-414.
[14] SIMONYAN K, ZISSERMAN A. Two-stream convolutional networks for action recognition in videos[C]//Proceedings of the 27th International Conference on Neural Information Processing Systems. Montreal, Canada, 2014: 568-576.
[15] HOCHREITER S, SCHMIDHUBER J. Long short-term memory[J]. Neural computation, 1997, 9(8): 1735-1780.
[16] BACCOUCHE M, MAMALET F, WOLF C, et al. Sequential deep learning for human action recognition[C]//Proceedings of the 2nd International Workshop on Human Behavior Understanding. Amsterdam, The Netherlands, 2011: 29-39.
[17] SZEGEDY C, LIU Wei, JIA Yangqing, et al. Going deeper with convolutions[C]//Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition. Boston, USA, 2015: 1-9.
[18] SZEGEDY C, VANHOUCKE V, IOFFE S, et al. Rethinking the inception architecture for computer vision[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, USA, 2016: 2818-2826.
[19] RYOO M S, AGGARWAL J K. Spatio-temporal relationship match: video structure comparison for recognition of complex human activities[C]//Proceedings of 2009 IEEE 12th International Conference on Computer Vision. Kyoto, Japan, 2009: 1593-1600.
Similar References:

Memo

-

Last Update: 1900-01-01

Copyright © CAAI Transactions on Intelligent Systems