[1]WANG Zuowei,XU Zheng,ZHANG Rubo,et al.Research progress and application of memory neural network in robot navigation[J].CAAI Transactions on Intelligent Systems,2020,15(5):835-846.[doi:10.11992/tis.202002020]
Copy

Research progress and application of memory neural network in robot navigation

CAAI Transactions on Intelligent Systems[ISSN 1673-4785/CN 23-1538/TP] Volume: 15 Number of periods: 2020 5 Page number: 835-846 Column: 综述 Public date: 2020-09-05
Title:
Research progress and application of memory neural network in robot navigation
Author(s):
WANG Zuowei1; 2;  XU Zheng3; 4;  ZHANG Rubo5;  HONG Caisen1;  WANG Shu1
1. School of Computer Science and Technology, Tianjin Polytechnic University, Tianjin 300387, China;
2. College of Mechanical Engineering Post-doctoral Research Station, Tianjin Polytechnic University, Tianjin 300387, China;
3. DongHexin Technology Co., Ltd., Tianjin 300350, China;
4. College of Automobile and Transportation, Tianjin University of Technology and Education, Tianjin 300222, China;
5. College of Mechanical and Electrical Engineering, Dalian Minzu University, Dalian 116600, China
Keywords:
memory neural network; robot navigation; deep reinforcement learning; differentiable neural computer; differentiable neural dictionary; deep learning; reinforcement learning; memory networks
CLC:
TP183
DOI:
10.11992/tis.202002020
Abstract:
Memory networks are a relatively new class of models designed to alleviate the problem of learning long-term dependencies in sequential data, by providing an explicit memory representation for each token in the sequence, and they can be used for learning navigation policies in an unstructured terrain, which is a complex task. Memory neural networks are highly suitable for solving time series decision-making problems, and their application in robot navigation is a very promising and emerging research field. The research progress of memory neural networks in the field of robot navigation is primarily discussed in this paper. First, the working mechanism of several basic memory neural networks used for robot navigationis introduced, and the advantages and disadvantages of different models are summarized. Then, the research progress of memory neural network in navigation field is briefly reviewed, and the development of navigation verification environment is discussed. Finally, the complex challenges faced by memory neural networks in navigation are summarized, and the future development of memory neural networks in navigation field is predicted.
References:
[1] 刘强, 段富海, 桑勇. 复杂环境下视觉SLAM闭环检测方法综述[J]. 机器人, 2019, 41(1): 112-123, 136
LIU Qiang, DUAN Fuhai, SANG Yong. A survey of loop-closure detection method of visual SLAM in complex environments[J]. Robot, 2019, 41(1): 112-123, 136
[2] KULKARNI T D, SAEEDI A, GAUTAM S, et al. Deep successor reinforcement learning[J]. arXiv preprint arXiv: 1606.02396v1, 2016.
[3] MNIH V, BADIA A P, MIRZA M, et al. Asynchronous methods for deep reinforcement learning[C]//Proceedings of the 33rd International Conference on International Conference on Machine Learning. New York, USA, 2016: 1928-1937.
[4] ZHU Yuke, MOTTAGHI R, KOLVE E, et al. Target-driven visual navigation in indoor scenes using deep reinforcement learning[C]//Proceedings of 2017 IEEE International Conference on Robotics and Automation (ICRA). Singapore, 2016.
[5] MIROWSKI P, PASCANU R, VIOLA F, et al. Learning to navigate in complex environments[C]//Proceedings of the 5th International Conference on Learning Representations. Toulon, France, 2017.
[6] JADERBERG M, MNIH V, CZARNECKI W M, et al. Reinforcement learning with unsupervised auxiliary tasks[C]//Proceedings of the 5th International Conference on Learning Representations. Toulon, France, 2016.
[7] HEESS N, HUNT J J, LILLICRAP T P, et al. Memory-based control with recurrent neural networks[C]//Proceedings of the Workshops of Advances in Neural Information Processing Systems. Montreal, Canada, 2015: 301-312.
[8] RAMANI D. A short survey on memory based reinforcement learning[J]. arXiv preprint arXiv:1904.06736v1, 2019.
[9] SAVINOV N, DOSOVITSKIY A, KOLTUN V. Semi-parametric topological memory for navigation[C]//Proceedings of the 6th International Conference on Learning Representations. Vancouver, Canada, 2018.
[10] SUKHBAATAR A, WESTON J, FERGUS R, et al. End-to-end memory networks[C]//Proceedings of the 28th International Conference on Neural Information Processing Systems. Montreal, Canada, 2015: 2440-2448.
[11] ZHANG Lei, WANG Shuai, LIU Bing. Deep learning for sentiment analysis: A survey[J]. Wiley interdisciplinary reviews: data mining and knowledge discovery, 2018, 8(4): e1253.
[12] YOUNG T, HAZARIKA D, PORIA S, et al. Recent trends in deep learning based natural language processing[J]. IEEE computational intelligence magazine, 2018, 13(3): 55-75.
[13] OH J, CHOCKALINGAM V, SINGH S, et al. Control of memory, active perception, and action in minecraft[C]//Proceedings of the 33nd International Conference on Machine Learning. New York, USA, 2016: 2790-2799.
[14] BOTHE C, MAGG S, WEBER C, et al. Conversational analysis using utterance-level attention-based bidirectional recurrent neural networks[C]//Proceedings of the 19th Annual Conference of the International Speech Communication Association. Hyderabad, India, 2018.
[15] 张新生, 高腾. 多头注意力记忆网络的对象级情感分类[J]. 模式识别与人工智能, 2019, 32(11): 997-1005
ZHANG Xinsheng, GAO Teng. Aspect level sentiment classification with multiple-head attention memory network[J]. Pattern recognition and artificial intelligence, 2019, 32(11): 997-1005
[16] BAHDANAU D, CHOROWSKI J, SERDYUK D, et al. End-to-end attention-based large vocabulary speech recognition[C]//Proceedings of 2016 IEEE International Conference on Acoustics, Speech and Signal Processing. Shanghai, China, 2016: 4945-4949.
[17] JETLEY S, LORD N A, LEE N, et al. Learn to pay attention[C]//Proceedings of the 6th International Conference on Learning Representations. Vancouver, Canada, 2018.
[18] 梁天新, 杨小平, 王良, 等. 记忆神经网络的研究与发展[J]. 软件学报, 2017, 28(11): 2905-2924
LIANG Tianxin, YANG Xiaoping, WANG Liang, et al. Review on research and development of memory neural networks[J]. Journal of software, 2017, 28(11): 2905-2924
[19] TANG Duyu, QIN Bing, LIU Ting. Aspect level sentiment classification with deep memory network[C]//Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. Austin, USA, 2016.
[20] GRAVES A, WAYNE G, REYNOLDS M, et al. Hybrid computing using a neural network with dynamic external memory[J]. Nature, 2016, 538(7626): 471-476.
[21] YANG Feng, ZHANG Shiyue, ZHANG Andi, et al. Memory-augmented neural machine translation[C]//Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. Copenhagen, Denmark, 2017.
[22] CINGILLIOGLU N, RUSSO A. DeepLogic: towards end-to-end differentiable logical reasoning[C]//Proceedings of the AAAI 2019 Spring Symposium on Combining Machine Learning with Knowledge Engineering. Palo Alto, USA, 2019.
[23] 廖祥文, 林威, 吴运兵, 等. 基于辅助记忆循环神经网络的视角级情感分析[J]. 模式识别与人工智能, 2019, 32(11): 987-996
LIAO Xiangwen, LIN Wei, WU Yunbing, et al. Aspect level sentiment analysis based on recurrent neural network with auxiliary memory[J]. Pattern recognition and artificial intelligence, 2019, 32(11): 987-996
[24] DAI Hanjun, KHALIL E B, ZHANG Yuyu, et al. Learning combinatorial optimization algorithms over graphs[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems. Long Beach, USA, 2017: 6351-6361.
[25] PARISI G I, KEMKER R, PART J L, et al. Continual lifelong learning with neural networks: a review[J]. Neural networks, 2019, 113: 54-71.
[26] PARISOTTO E, SALAKHUTDINOV R. Neural map: Structured memory for deep reinforcement learning[C]//Proceedings of the 6th International Conference on Learning Representations. Vancouver, Canada, 2018.
[27] PRITZEL A, URIA B, SRINIVASAN S, et al. Neural episodic control[C]//Proceedings of the 34th International Conference on Machine Learning. Sydney, Australia, 2017: 2827-2836.
[28] BENTLEY J L. Multidimensional binary search trees used for associative searching[J]. Communications of the ACM, 1975, 18(9): 509-517.
[29] ZHANG Jingwei, TAI Lei, BOEDECKER J, et al. Neural SLAM: learning to explore with external memory[J]. arXiv:1706.09520v4, 2017.
[30] CHAPLOT D S, PARISOTTO E, SALAKHUTDINOV R. Active neural localization[C]//Proceedings of the 6th International Conference on Learning Representations. Vancouver, Canada, 2018.
[31] CHUNG J, GüLCEHRE C, CHO K, et al. Empirical evaluation of gated recurrent neural networks on sequence modeling[J]. arXiv:1412.3555v1, 2014.
[32] TAMAR A, WU Yi, THOMAS G, et al. Value iteration networks[C]//Proceedings of the 26th International Joint Conference on Artificial Intelligence. Melbourne, Australia, 2017: 4949-4953.
[33] GUPTA S, DAVIDSON J, LEVINE S, et al. Cognitive mapping and planning for visual navigation[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, USA, 2017: 7272-7281.
[34] KHAN A, ZHANG C, ATANASOV N, et al. Memory augmented control networks[C]//Proceedings of the 6th International Conference on Learning Representations. Vancouver, Canada, 2018.
[35] SUTTON R S, PRECUP D, SINGH S. Between MDPs and semi-MDPs: A framework for temporal abstraction in reinforcement learning[J]. Artificial intelligence, 1999, 112(1/2): 181-211.
[36] RABINER L R. A tutorial on hidden markov models and selected applications in speech recognition[J]. Proceedings of the IEEE, 1989, 77(2): 257-286.
[37] KALMAN R E. A new approach to linear filtering and prediction problems[J]. Journal of basic engineering, 1960, 82(1): 35-45.
[38] CHUNG J, KASTNER K, DINH L, et al. A recurrent latent variable model for sequential data[C]//Proceedings of the 28th International Conference on Neural Information Processing Systems. Montreal, Canada, 2015: 2962-2970.
[39] KRISHNAN R G, SHALIT U, SONTAG D. Deep Kalman filters[J]. arXiv preprint arXiv:1511.05121v2, 2015.
[40] GEMICI M, HUNG C C, SANTORO A, et al. Generative temporal models with memory[J]. arXiv preprint arXiv:1702.04649v2, 2017.
[41] FRACCARO M, REZENDE D J, ZWOLS Y, et al. Generative temporal models with spatial memory for partially observed environments[C]//Proceedings of the 35th International Conference on Machine Learning. Stockholm, Sweden, 2018.
[42] WAYNE G, HUNG C C, AMOS D, et al. Unsupervised predictive memory in a goal-directed agent[J]. arXiv preprint arXiv:1803.10760v1, 2018.
[43] BEATTIE C, LEIBO J Z, TEPLYASHIN D, et al. Deepmind lab[J]. arXiv:1612.03801v2, 2016.
[44] JOHNSON M, HOFMANN K, HUTTON T, et al. The Malmo platform for artificial intelligence experimentation[C]//Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence. New York, American, 2016.
[45] KEMPKA M, WYDMUCH M, RUNC G, et al. Vizdoom: A doom-based AI research platform for visual reinforcement learning[C]//Proceedings of IEEE Conference on Computational Intelligence and Games. Santorini, Greece, 2016.
[46] WU Yi, WU Yuxin, GKIOXARI G, et al. Building generalizable agents with a realistic and rich 3D environment[C]//Proceedings of the 6th International Conference on Learning Representations. Vancouver, Canada, 2018.
[47] SONG Shuran, YU F, ZENG A, et al. Semantic scene completion from a single depth image[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, USA, 2017: 190-198.
[48] BRODEUR S, PEREZ E, ANAND A, et al. Home: a household multimodal environment[C]//Proceedings of the 6th International Conference on Learning Representations. Vancouver, Canada, 2018.
[49] SAVVA M, CHANG A X, DOSOVITSKIY A, et al. MINOS: Multimodal indoor simulator for navigation in complex environments[J]. arXiv:1712.03931v1, 2017.
[50] KOLVE E, MOTTAGHI R, HAN W, et al. AI2-thor: An interactive 3D environment for visual AI[J]. /arXiv:1712.05474v3, 2019.
[51] YANG Wei, WANG Xiaolong, FARHADI A, et al. Visual semantic navigation using scene priors[C]//Proceedings of the 7th International Conference on Learning Representations. New Orleans, USA, 2019.
[52] MIROWSKI P, GRIMES M K, MALINOWSKI M, et al. Learning to navigate in cities without a map[C]// Proceedings of the 32nd International Conference on Neural Information Processing Systems. Montreal, Canada, 2018: 2424-2435.
Similar References:

Memo

-

Last Update: 2021-01-15

Copyright © CAAI Transactions on Intelligent Systems