[1]于建均,姚红柯,左国玉,等.基于动态系统的机器人模仿学习方法研究[J].智能系统学报,2019,14(5):1026-1034.[doi:10.11992/tis.201807018]
YU Jianjun,YAO Hongke,ZUO Guoyu,et al.Research on robot imitation learning method based on dynamical system[J].CAAI Transactions on Intelligent Systems,2019,14(5):1026-1034.[doi:10.11992/tis.201807018]
点击复制
YU Jianjun,YAO Hongke,ZUO Guoyu,et al.Research on robot imitation learning method based on dynamical system[J].CAAI Transactions on Intelligent Systems,2019,14(5):1026-1034.[doi:10.11992/tis.201807018]
基于动态系统的机器人模仿学习方法研究
《智能系统学报》[ISSN 1673-4785/CN 23-1538/TP] 卷:
14
期数:
2019年第5期
页码:
1026-1034
栏目:
学术论文—智能系统
出版日期:
2019-09-05
- Title:
- Research on robot imitation learning method based on dynamical system
- Keywords:
- robot; imitation learning; trajectory level; Gaussian mixture model; dynamical system; parameter learning; 7bot manipulator; generalization performance
- 分类号:
- TP242.6
- 摘要:
- 针对当前机器人模仿学习过程中，运动模仿存在无法收敛到目标点以及泛化能力差的问题，引入一种基于动态系统（dynamical system，DS）的模仿学习方法。该方法通过高斯混合模型（gaussian mixture model，GMM）将示教运动数据建模为一非线性动态系统；将DS全局稳定的充分条件作为约束，以保证DS所生成的所有轨迹收敛到目标点；将动态系统模型的参数学习问题转化为求解一个约束优化问题，从而得到模型参数。以7bot机械臂为实验对象，进行仿真实验和机器人实验，实验结果表明：该方法学习的DS模型从不同起点生成的所有轨迹都收敛到目标点，轨迹平滑，泛化能力好。
- Abstract:
- In the current robot imitation learning process, the motion imitation cannot converge to the target point, and the generalization ability is poor. To solve this problem, an imitation learning method based on dynamical system (DS) is introduced. First, the demonstration motion data is modeled as a nonlinear DS by Gaussian mixture model (GMM). Second, the sufficient condition of DS global stability is used as a constraint to ensure that all the DS-generated trajectories converge to the target. Finally, the parameter learning problem of the DS model is transformed into seeking for a solution to a constrained optimization problem to obtain the model parameters. Simulation experiments and robot experiments were carried out using the 7bot manipulator. The experimental results show that all the trajectories generated by the DS model from different starting points converged to the target point, and the trajectory was smooth and the generalization performance was improved.
- 参考文献/References:
-
[1] SCHAAL S. Is imitation learning the route to humanoid robots?[J]. Trends in cognitive sciences, 1999, 3(6):233-242.
[2] 杨俊友, 马乐, 白殿春, 等. 机器人模仿学习的非接触观测控制图模型[J]. 机器人, 2014, 36(3):309-315 YANG Junyou, MA Le, BAI Dianchun, et al. Cybernetic-graphic model for robot imitation learning based on non-contact observation[J]. Robot, 2014, 36(3):309-315
[3] 曾华琳, 黄雨轩, 晁飞, 等. 书写机器人研究综述[J]. 智能系统学报, 2016, 11(1):15-26 ZENG Hualin, HUANG Yuxuan, CHAO Fei, et al. Survey of robotic calligraphy research[J]. CAAI transactions on intelligent systems, 2016, 11(1):15-26
[4] UGUR E, NAGAI Y, SAHIN E, et al. Staged development of robot skills:behavior formation, affordance learning and imitation with motionese[J]. IEEE transactions on autonomous mental development, 2015, 7(2):119-139.
[5] BOBOC R G, TOMA M I, PANFIR A N, et al. Learning new skills by a humanoid robot through imitation[C]//Proceedings of the 2013 IEEE 14th International Symposium on Computational Intelligence and Informatics. Budapest, Hungary, 2013:515-519.
[6] 于建均, 门玉森, 阮晓钢, 等. 基于Kinect的Nao机器人动作模仿系统的研究与实现[J]. 智能系统学报, 2016, 11(2):180-187 YU Jianjun, MEN Yusen, RUAN Xiaogang, et al. CAAI transactions on intelligent systems[J]. CAAI transactions on intelligent systems, 2016, 11(2):180-187
[7] BILLARD A, CALINON S, DILLMANN R, et al. Robot programming by demonstration[M]//SICILIANO B, KHATIB O. Springer Handbook of Robotics. Berlin, Heidelberg:Springer, 2008:1371?1394.
[8] ATKESON C G, SCHAAL S. Learning tasks from a single demonstration[C]//Proceedings of International Conference on Robotics and Automation. Albuquerque, NM, USA, 1997:1706?1712.
[9] MAAREF M, REZAZADEH A, SHAMAEI K, et al. A gaussian mixture framework for co-operative rehabilitation therapy in assistive impedance-based tasks[J]. IEEE journal of selected topics in signal processing, 2016, 10(5):904-913.
[10] KULI? D, TAKANO W, NAKAMURA Y. Incremental learning, clustering and hierarchy formation of whole body motion patterns using adaptive hidden markov chains[J]. The international journal of robotics research, 2008, 27(7):761-784.
[11] ANTONELO E A, SCHRAUWEN B. Supervised learning of internal models for autonomous goal-oriented robot navigation using reservoir computing[C]//Proceedings of 2010 IEEE International Conference on Robotics and Automation. Anchorage, AK, USA, 2010:2959-2964.
[12] CALINON S, BILLARD A. Incremental learning of gestures by imitation in a humanoid robot[C]//Proceedings of the ACM/IEEE international conference on Human-robot interaction. Arlington, VA, USA, 2007:255-262.
[13] HERSCH M, GUENTER F, CALINON S, et al. Dynamical system modulation for robot learning via kinesthetic demonstrations[J]. IEEE transactions on robotics, 2008, 24(6):1463-1467.
[14] SCHAAL S, ATKESON C, VIJAYAKUMAR S. Scalable locally weighted statistical techniques for real time robot learning[J]. Applied intelligence-special issue on scalable robotic applications of neural networks, 2002, 17(1):49-60.
[15] PETERNEL L, OZTOP E, BABI? J. A shared control method for online human-in-the-loop robot learning based on locally weighted regression[C]//Proceedings of 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems. Daejeon, South Korea, 2016:3900-3906.
[16] MCCORMICK J, VINCS K, NAHAVANDI S, et al. Teaching a digital performing agent:Artificial neural network and hidden markov model for recognising and performing dance movement[C]//Proceedings of the 2014 International Workshop on Movement and Computing. Paris, France, 2014:70.
[17] 于建均, 吴鹏申, 左国玉, 等. 基于RNN的机械臂任务模仿系统[J]. 北京工业大学学报, 2018, 44(11):1401-1408 YU Jianjun, WU Pengshen, ZUO Guoyu, et al. Robot arm task imitation system based on RNN[J]. Journal of Beijing University of Technology, 2018, 44(11):1401-1408
[18] 于建均, 门玉森, 阮晓钢, 等. 在书写任务中的基于轨迹匹配的模仿学习[J]. 北京工业大学学报, 2016, 42(8):1144-1152 YU Jianjun, MEN Yusen, RUAN Xiaogang, et al. Imitation learning based on trajectory matching in the writing task[J]. Journal of Beijing University of Technology, 2016, 42(8):1144-1152
[19] IJSPEERT A J, NAKANISHI J, HOFFMANN H, et al. Dynamical movement primitives:learning attractor models for motor behaviors[J]. Neural computation, 2013, 25(2):328-373.
[20] PARASCHOS A, RUECKERT E, PETERS J, et al. Model-free probabilistic movement primitives for physical interaction[C]//Proceedings of 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems. Hamburg, Germany, 2015:2860-2866.
[21] KOCH K H, CLEVER D, MOMBAUR K, et al. Learning movement primitives from optimal and dynamically feasible trajectories for humanoid walking[C]//Proceedings of 2015 IEEE-RAS 15th International Conference on Humanoid Robots. Seoul, South Korea, 2015:866-873.
[22] KOBER J, GIENGER M, STEIL J J. Learning movement primitives for force interaction tasks[C]//Proceedings of 2015 IEEE International Conference on Robotics and Automation. Seattle, USA, 2015:3192-3199.
[23] WIGGINS S. Introduction to applied nonlinear dynamical systems and chaos[M]. 2nd ed. New York:Springer Science & Business Media, 2003.
[24] SEEGER M. Gaussian processes for machine learning[J]. International journal of neural systems, 2004, 14(2):69-106.
[25] KHANSARI-ZADEH S M, BILLARD A. BM:an iterative algorithm to learn stable non-linear dynamical systems with gaussian mixture models[C]//Proceedings of 2010 IEEE International Conference on Robotics and Automation. Anchorage, USA, 2010:2381-2388.
- 相似文献/References:
-
[1]方勇纯.机器人视觉伺服研究综述[J].智能系统学报,2008,3(2):109.
FANG Yong-chun.A survey of robot visual servoing[J].CAAI Transactions on Intelligent Systems,2008,3():109.
[2]王立权,刘秉昊,吴健荣,等.6R关节型机器人运动学建模[J].智能系统学报,2010,5(2):156.
WANG Li-quan,LIU Bing-hao,WU Jian-rong,et al.Modeling and implementing the inverse kinematics ofa six revolute joint robot[J].CAAI Transactions on Intelligent Systems,2010,5():156.
[3]孙宁,方勇纯.一类欠驱动系统的控制方法综述[J].智能系统学报,2011,6(3):200.
SUN Ning,FANG Yongchun.A review for the control of a class of underactuated systems[J].CAAI Transactions on Intelligent Systems,2011,6():200.
[4]蒲兴成,张军,张毅.基于神经网络的改进行为协调控制及其在智能轮椅路径规划中的应用[J].智能系统学报,2011,6(5):456.
PU Xingcheng,ZHANG Jun,ZHANG Yi.Modified behavior coordination for intelligent wheelchair path planning based on a neural network[J].CAAI Transactions on Intelligent Systems,2011,6():456.
[5]李晓飞,刘宏.机器人听觉声源定位研究综述[J].智能系统学报,2012,7(1):9.
LI Xiaofei,LIU Hong.A survey of sound source localization for robot audition[J].CAAI Transactions on Intelligent Systems,2012,7():9.
[6]孙凤池,宋萌,刘光.一种无线传感器信号衰减自适应测距模型[J].智能系统学报,2012,7(3):214.
SUN Fengchi,SONG Meng,LIU Guang.An adaptive ranging model based on energy distance loss of wireless sensors[J].CAAI Transactions on Intelligent Systems,2012,7():214.
[7]孙玉香,曹会彬,冯勇,等.具有拨土功能的轮腿一体化机器人结构设计[J].智能系统学报,2012,7(5):409.
SUN Yuxiang,CAO Huibin,FENG Yong,et al.Structure design on a legwheeledintegration robotwith an earthmoving function[J].CAAI Transactions on Intelligent Systems,2012,7():409.
[8]伍明,李琳琳,李承剑.基于协方差交集的多机器人协作目标跟踪算法[J].智能系统学报,2013,8(1):66.[doi:10.3969/j.issn.1673-4785.201204022]
WU Ming,LI Linlin,LI Chengjian.An algorithm of multi robot cooperative object tracking based on covariance intersection[J].CAAI Transactions on Intelligent Systems,2013,8():66.[doi:10.3969/j.issn.1673-4785.201204022]
[9]肖国宝,严宣辉.一种基于改进Theta *的机器人路径规划算法[J].智能系统学报,2013,8(1):58.[doi:10.3969/j.issn.1673-4785.201208032]
XIAO Guobao,YAN Xuanhui.A path planning algorithm based on improved Theta * for mobile robot[J].CAAI Transactions on Intelligent Systems,2013,8():58.[doi:10.3969/j.issn.1673-4785.201208032]
[10]李大伟,贾鹏飞,李卫国,等.一种基于卡尔曼滤波与模糊算法的变电站机器人组合导航及控制系统设计[J].智能系统学报,2013,8(3):226.
LI Dawei,JIA Pengfei,LI Weiguo,et al.A kind of integrated navigation and control system design for substation robot based on the Kalman filtering and fuzzy algorithm[J].CAAI Transactions on Intelligent Systems,2013,8():226.
[11]曾华琳,黄雨轩,晁飞,等.书写机器人研究综述[J].智能系统学报,2016,11(1):15.[doi:10.11992/tis.201507067]
ZENG Hualin,HUANG Yuxuan,CHAO Fei,et al.Survey of robotic calligraphy research[J].CAAI Transactions on Intelligent Systems,2016,11():15.[doi:10.11992/tis.201507067]
备注/Memo
收稿日期:2018-07-18。
基金项目:国家自然科学基金项目（61773027）；北京市自然科学基金项目（4182008）；北京市自然科学基金项目/北京市教育委员会科技计划重点项目（KZ201610005010）.
作者简介:于建均,女,1965年生,副教授,主要研究方向为智能机器人的仿生自主控制、智能计算与智能优化控制、复杂过程建模、优化与控制。主持或参与国家"863"计划项目、国家自然科学基金等省部级科研项目以及横向科研课题多项。取得国家发明专利、实用新型专利、国家软件著作权10余项。发表SCI、EI、ISTP收录论文40余篇;姚红柯,男,1991年生,硕士研究生,主要研究方向为机器学习、机器人行为模仿和控制;左国玉,男,1971年生,副教授,博士,主要研究方向为机器人学习与控制。主持科研项目10余项,取得国家发明专利20余项。发表学术论文40余篇。
通讯作者:左国玉.E-mail:zuoguoyu@bjut.edu.cn
更新日期/Last Update:
1900-01-01