[1]DUAN Xiao,MA Gang,WEI Hui.Hand-eye coordination of robots based on the 3D object tracking algorithm[J].CAAI Transactions on Intelligent Systems,2022,17(5):941-950.[doi:10.11992/tis.202107037]
Copy

Hand-eye coordination of robots based on the 3D object tracking algorithm

CAAI Transactions on Intelligent Systems[ISSN 1673-4785/CN 23-1538/TP] Volume: 17 Number of periods: 2022 5 Page number: 941-950 Column: 学术论文—智能系统 Public date: 2022-09-05
Title:
Hand-eye coordination of robots based on the 3D object tracking algorithm
Author(s):
DUAN Xiao;  MA Gang;  WEI Hui
Laboratory of Algorithms for Cognitive Models, School of Computer Science, Fudan University, Shanghai 200433, China
Keywords:
pose tracking; pose optimization; hand-eye coordination; visual perception; visual feedback; robot operating system; motion planning; robot simulation; decision control
CLC:
TP391.41
DOI:
10.11992/tis.202107037
Abstract:
To adapt to the complexity and diversity of an environment and enhance the robustness of the robot grasping tasks, this paper suggests a novel method to realize the hand-eye coordination of a robot based on the 3D object tracking algorithm. In this method, an improved region-based pose tracking algorithm is used to track the poses of the manipulator and the target and simultaneously guide the manipulator movement according to its relative position. For the region-based pose tracking algorithm, a segmentation model is constructed as per the segmentation lines of the local region, and the linear updating method of the model color likelihoods is improved, to accurately track the gripper and the target. Simulation experiments are conducted based on the ROS platform, and the effectiveness and robustness of the hand-eye coordination system are verified in both the simulation and real environments. This method does not need hand-eye calibration and is closer to the human closed-loop control of the “Sensor-Actor” mode with feedback. Concurrently, it provides robots with enough flexibility to deal with versatile tasks in a modifiable environment.
References:
[1] HUTCHINSON S, HAGER G D, CORKE P I. A tutorial on visual servo control[J]. IEEE transactions on robotics and automation, 1996, 12(5): 651–670.
[2] HAGER G D, CHANG W C, MORSE A S. Robot hand-eye coordination based on stereo vision[J]. IEEE control systems magazine, 1995, 15(1): 30–39.
[3] HELLER J, HAVLENA M, PAJDLA T. Globally optimal hand-eye calibration using branch-and-bound[J]. IEEE transactions on pattern analysis and machine intelligence, 2016, 38(5): 1027–1033.
[4] LI Wenlong, XIE He, ZHANG Gang, et al. Hand-eye calibration in visually-guided robot grinding[J]. IEEE transactions on cybernetics, 2016, 46(11): 2634–2642.
[5] GOODALE M A, MILNER A D. Separate visual pathways for perception and action[J]. Trends in neurosciences, 1992, 15(1): 20–25.
[6] GOODALE M A. Transforming vision into action[J]. Vision research, 2011, 51(13): 1567–1587.
[7] YILMAZ A, JAVED O, SHAH M. Object tracking[J]. ACM computing surveys, 2006, 38(4): 13.
[8] KLEIN G, MURRAY D W. Full-3D edge tracking with a particle filter[C]//Proceedings ofthe British Machine Vision Conference 2006. Edinburgh. British Machine Vision Association, 2006: 1119–1128.
[9] WAGNER D, REITMAYR G, MULLONI A, et al. Real-time detection and tracking for augmented reality on mobile phones[J]. IEEE transactions on visualization and computer graphics, 2010, 16(3): 355–368.
[10] KIM K, LEPETIT V, WOO W. Keyframe-based modeling and tracking of multiple 3D objects[C]//2010 IEEE International Symposium on Mixed and Augmented Reality. Seoul, IEEE, 2010: 193–198.
[11] CRIVELLARO A, RAD M, VERDIE Y, et al. Robust 3D object tracking from monocular images using stable parts[J]. IEEE transactions on pattern analysis and machine intelligence, 2018, 40(6): 1465–1479.
[12] HINTERSTOISSER S, LEPETIT V, ILIC S, et al. Model based training, detection and pose estimation of texture-less 3D objects in heavily cluttered scenes[M]//Computer Vision – ACCV 2012. Berlin: Springer Berlin Heidelberg, 2013: 548–562.
[13] WANG Chen, XU Danfei, ZHU Yuke, et al. DenseFusion: 6D object pose estimation by iterative dense fusion[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Long Beach, IEEE, 2019: 3338–3347.
[14] PENG Sida, LIU Yuan, HUANG Qixing, et al. PVNet: pixel-wise voting network for 6DoF pose estimation[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Long Beach, IEEE, 2019: 4556–4565.
[15] PRISACARIU V A, REID I D. PWP3D: real-time segmentation and tracking of 3D objects[J]. International journal of computer vision, 2012, 98(3): 335–354.
[16] TJADEN H, SCHWANECKE U, SCH?MER E. Real-time monocular segmentation and pose tracking of multiple objects[M]//Computer Vision – ECCV 2016. Cham: Springer International Publishing, 2016: 423–438.
[17] TJADEN H, SCHWANECKE U, SCH?MER E. Real-time monocular pose estimation of 3D objects using temporally consistent local color histograms[C]//2017 IEEE International Conference on Computer Vision. Venice, IEEE, 2017: 124–132.
[18] TJADEN H, SCHWANECKE U, SCH?MER E, et al. A region-based Gauss-Newton approach to real-time monocular multiple object tracking[J]. IEEE transactions on pattern analysis and machine intelligence, 2019, 41(8): 1797–1812.
[19] HEXNER J, HAGEGE R R. 2D-3D pose estimation of heterogeneous objects using a region based approach[J]. International journal of computer vision, 2016, 118(1): 95–112.
[20] 张雪华, 刘华平, 孙富春, 等. 采用Kinect的移动机器人目标跟踪[J]. 智能系统学报, 2014, 9(1): 34–39
ZHANG Xuehua, LIU Huaping, SUN Fuchun, et al. Target tracking of mobile robot using Kinect[J]. CAAI transactions on intelligent systems, 2014, 9(1): 34–39
[21] 贺超, 刘华平, 孙富春, 等. 采用Kinect的移动机器人目标跟踪与避障[J]. 智能系统学报, 2013, 8(5): 426–432
HE Chao, LIU Huaping, SUN Fuchun, et al. Target tracking and obstacle avoidance of mobile robot using Kinect[J]. CAAI transactions on intelligent systems, 2013, 8(5): 426–432
[22] 韩峥, 刘华平, 黄文炳, 等. 基于Kinect的机械臂目标抓取[J]. 智能系统学报, 2013, 8(2): 149–155
HAN Zheng, LIU Huaping, HUANG Wenbing, et al. Kinect-based object grasping by manipulator[J]. CAAI transactions on intelligent systems, 2013, 8(2): 149–155
[23] 徐呈艺, 刘英, 贾民平, 等. 木板抓取冗余机械臂逆运动学求解[J]. 计算机集成制造系统, 2020, 26(12): 3368–3374
XU Chengyi, LIU Ying, JIA Minping, et al. Inverse kinematics solution of redundant robot arm in picking board[J]. Computer integrated manufacturing systems, 2020, 26(12): 3368–3374
[24] 臧雨飞. 基于机器视觉的机械臂半自主控制系统的研究[D]. 南京: 东南大学, 2018.
ZANG Yufei. The research on semi-autonomous control system of mechanical arm based on machine vision[D]. Nanjing: Southeast University, 2018.
[25] DU Guoguang, WANG Kai, LIAN Shiguo, et al. Vision-based robotic grasping from object localization, object pose estimation to grasp estimation for parallel grippers: a review[J]. Artificial intelligence review, 2021, 54(3): 1677–1734.
[26] FERR?O J, DIAS P, NEVES A J R. Detection of aruco markers using the quadrilateral sum conjuncture[M]//Lecture Notes in Computer Science. Cham: Springer International Publishing, 2018: 363–369.
[27] LEPETIT V, MORENO-NOGUER F, FUA P. EPnP: an accurate O(n) solution to the PnP problem[J]. International journal of computer vision, 2008, 81(2): 155–166.
[28] MISRA A, SINGH G. Kinematic and dynamic analysis of an industrial six-axis robotic manipulator[C]// International Conference on Robotics, Automation and Non-destructive Evaluation. [S.l.]: 2019.
Similar References:

Memo

-

Last Update: 1900-01-01

Copyright © CAAI Transactions on Intelligent Systems