[1]段肖,马钢,危辉.基于3D目标跟踪算法的机器人手眼协调研究[J].智能系统学报,2022,17(5):941-950.[doi:10.11992/tis.202107037]
DUAN Xiao,MA Gang,WEI Hui.Hand-eye coordination of robots based on the 3D object tracking algorithm[J].CAAI Transactions on Intelligent Systems,2022,17(5):941-950.[doi:10.11992/tis.202107037]
点击复制
DUAN Xiao,MA Gang,WEI Hui.Hand-eye coordination of robots based on the 3D object tracking algorithm[J].CAAI Transactions on Intelligent Systems,2022,17(5):941-950.[doi:10.11992/tis.202107037]
基于3D目标跟踪算法的机器人手眼协调研究
《智能系统学报》[ISSN 1673-4785/CN 23-1538/TP] 卷:
17
期数:
2022年第5期
页码:
941-950
栏目:
学术论文—智能系统
出版日期:
2022-09-05
- Title:
- Hand-eye coordination of robots based on the 3D object tracking algorithm
- Keywords:
- pose tracking; pose optimization; hand-eye coordination; visual perception; visual feedback; robot operating system; motion planning; robot simulation; decision control
- 分类号:
- TP391.41
- 文献标志码:
- 2022-06-22
- 摘要:
- 为了适应环境的复杂性和多样性，增强机器人抓取任务的鲁棒性，本文从3D目标跟踪算法出发，提出了一种实现机器人手眼协调的新方法。该方法采用改进的基于区域的位姿追踪算法同时跟踪机械臂夹持器和目标物体的位姿，根据二者的相对位置关系引导机械臂运动。对基于区域的位姿跟踪算法，本文提出根据局部区域分割线构建分割模型并改进模型颜色似然的线性更新方式，使得算法能够准确跟踪机械臂夹持器与目标物体。基于ROS平台搭建了一套仿真实验环境，并分别在仿真环境和真实环境下验证了此手眼协调系统的有效性和鲁棒性。这种方式不仅不需要手眼标定，更接近于人类“Sensor-Actor”带反馈的闭环控制方式，同时赋予了机器人足够的灵活性来应对弹性的任务和多变的环境。
- Abstract:
- To adapt to the complexity and diversity of an environment and enhance the robustness of the robot grasping tasks, this paper suggests a novel method to realize the hand-eye coordination of a robot based on the 3D object tracking algorithm. In this method, an improved region-based pose tracking algorithm is used to track the poses of the manipulator and the target and simultaneously guide the manipulator movement according to its relative position. For the region-based pose tracking algorithm, a segmentation model is constructed as per the segmentation lines of the local region, and the linear updating method of the model color likelihoods is improved, to accurately track the gripper and the target. Simulation experiments are conducted based on the ROS platform, and the effectiveness and robustness of the hand-eye coordination system are verified in both the simulation and real environments. This method does not need hand-eye calibration and is closer to the human closed-loop control of the “Sensor-Actor” mode with feedback. Concurrently, it provides robots with enough flexibility to deal with versatile tasks in a modifiable environment.
- 参考文献/References:
-
[1] HUTCHINSON S, HAGER G D, CORKE P I. A tutorial on visual servo control[J]. IEEE transactions on robotics and automation, 1996, 12(5): 651–670.
[2] HAGER G D, CHANG W C, MORSE A S. Robot hand-eye coordination based on stereo vision[J]. IEEE control systems magazine, 1995, 15(1): 30–39.
[3] HELLER J, HAVLENA M, PAJDLA T. Globally optimal hand-eye calibration using branch-and-bound[J]. IEEE transactions on pattern analysis and machine intelligence, 2016, 38(5): 1027–1033.
[4] LI Wenlong, XIE He, ZHANG Gang, et al. Hand-eye calibration in visually-guided robot grinding[J]. IEEE transactions on cybernetics, 2016, 46(11): 2634–2642.
[5] GOODALE M A, MILNER A D. Separate visual pathways for perception and action[J]. Trends in neurosciences, 1992, 15(1): 20–25.
[6] GOODALE M A. Transforming vision into action[J]. Vision research, 2011, 51(13): 1567–1587.
[7] YILMAZ A, JAVED O, SHAH M. Object tracking[J]. ACM computing surveys, 2006, 38(4): 13.
[8] KLEIN G, MURRAY D W. Full-3D edge tracking with a particle filter[C]//Proceedings ofthe British Machine Vision Conference 2006. Edinburgh. British Machine Vision Association, 2006: 1119–1128.
[9] WAGNER D, REITMAYR G, MULLONI A, et al. Real-time detection and tracking for augmented reality on mobile phones[J]. IEEE transactions on visualization and computer graphics, 2010, 16(3): 355–368.
[10] KIM K, LEPETIT V, WOO W. Keyframe-based modeling and tracking of multiple 3D objects[C]//2010 IEEE International Symposium on Mixed and Augmented Reality. Seoul, IEEE, 2010: 193–198.
[11] CRIVELLARO A, RAD M, VERDIE Y, et al. Robust 3D object tracking from monocular images using stable parts[J]. IEEE transactions on pattern analysis and machine intelligence, 2018, 40(6): 1465–1479.
[12] HINTERSTOISSER S, LEPETIT V, ILIC S, et al. Model based training, detection and pose estimation of texture-less 3D objects in heavily cluttered scenes[M]//Computer Vision – ACCV 2012. Berlin: Springer Berlin Heidelberg, 2013: 548–562.
[13] WANG Chen, XU Danfei, ZHU Yuke, et al. DenseFusion: 6D object pose estimation by iterative dense fusion[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Long Beach, IEEE, 2019: 3338–3347.
[14] PENG Sida, LIU Yuan, HUANG Qixing, et al. PVNet: pixel-wise voting network for 6DoF pose estimation[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Long Beach, IEEE, 2019: 4556–4565.
[15] PRISACARIU V A, REID I D. PWP3D: real-time segmentation and tracking of 3D objects[J]. International journal of computer vision, 2012, 98(3): 335–354.
[16] TJADEN H, SCHWANECKE U, SCH?MER E. Real-time monocular segmentation and pose tracking of multiple objects[M]//Computer Vision – ECCV 2016. Cham: Springer International Publishing, 2016: 423–438.
[17] TJADEN H, SCHWANECKE U, SCH?MER E. Real-time monocular pose estimation of 3D objects using temporally consistent local color histograms[C]//2017 IEEE International Conference on Computer Vision. Venice, IEEE, 2017: 124–132.
[18] TJADEN H, SCHWANECKE U, SCH?MER E, et al. A region-based Gauss-Newton approach to real-time monocular multiple object tracking[J]. IEEE transactions on pattern analysis and machine intelligence, 2019, 41(8): 1797–1812.
[19] HEXNER J, HAGEGE R R. 2D-3D pose estimation of heterogeneous objects using a region based approach[J]. International journal of computer vision, 2016, 118(1): 95–112.
[20] 张雪华, 刘华平, 孙富春, 等. 采用Kinect的移动机器人目标跟踪[J]. 智能系统学报, 2014, 9(1): 34–39
ZHANG Xuehua, LIU Huaping, SUN Fuchun, et al. Target tracking of mobile robot using Kinect[J]. CAAI transactions on intelligent systems, 2014, 9(1): 34–39
[21] 贺超, 刘华平, 孙富春, 等. 采用Kinect的移动机器人目标跟踪与避障[J]. 智能系统学报, 2013, 8(5): 426–432
HE Chao, LIU Huaping, SUN Fuchun, et al. Target tracking and obstacle avoidance of mobile robot using Kinect[J]. CAAI transactions on intelligent systems, 2013, 8(5): 426–432
[22] 韩峥, 刘华平, 黄文炳, 等. 基于Kinect的机械臂目标抓取[J]. 智能系统学报, 2013, 8(2): 149–155
HAN Zheng, LIU Huaping, HUANG Wenbing, et al. Kinect-based object grasping by manipulator[J]. CAAI transactions on intelligent systems, 2013, 8(2): 149–155
[23] 徐呈艺, 刘英, 贾民平, 等. 木板抓取冗余机械臂逆运动学求解[J]. 计算机集成制造系统, 2020, 26(12): 3368–3374
XU Chengyi, LIU Ying, JIA Minping, et al. Inverse kinematics solution of redundant robot arm in picking board[J]. Computer integrated manufacturing systems, 2020, 26(12): 3368–3374
[24] 臧雨飞. 基于机器视觉的机械臂半自主控制系统的研究[D]. 南京: 东南大学, 2018.
ZANG Yufei. The research on semi-autonomous control system of mechanical arm based on machine vision[D]. Nanjing: Southeast University, 2018.
[25] DU Guoguang, WANG Kai, LIAN Shiguo, et al. Vision-based robotic grasping from object localization, object pose estimation to grasp estimation for parallel grippers: a review[J]. Artificial intelligence review, 2021, 54(3): 1677–1734.
[26] FERR?O J, DIAS P, NEVES A J R. Detection of aruco markers using the quadrilateral sum conjuncture[M]//Lecture Notes in Computer Science. Cham: Springer International Publishing, 2018: 363–369.
[27] LEPETIT V, MORENO-NOGUER F, FUA P. EPnP: an accurate O(n) solution to the PnP problem[J]. International journal of computer vision, 2008, 81(2): 155–166.
[28] MISRA A, SINGH G. Kinematic and dynamic analysis of an industrial six-axis robotic manipulator[C]// International Conference on Robotics, Automation and Non-destructive Evaluation. [S.l.]: 2019.
备注/Memo
收稿日期:2021-07-19。
基金项目:国家自然科学基金项目(61771146, 61375122).
作者简介:段肖，硕士研究生，主要研究方向为智能机器人;马钢，博士研究生，主要研究方向为3D场景理解和智能机器人;危辉，教授，博士生导师，博士，主要研究方向为认知科学与人工智能。发表学术论文70余篇；出版学术专著《人工智能形式概念系统》《认知相关性与智能模型构造的系统观点》和《类脑计算》
通讯作者:危辉. E-mail：weihui@fudan.edu.cn
更新日期/Last Update:
1900-01-01