[1]QIU Jielu,WANG Mingming.Flexible wire reconstructing and gripping verification under the image information input[J].CAAI Transactions on Intelligent Systems,2025,20(2):435-444.[doi:10.11992/tis.202403021]
Copy

Flexible wire reconstructing and gripping verification under the image information input

CAAI Transactions on Intelligent Systems[ISSN 1673-4785/CN 23-1538/TP] Volume: 20 Number of periods: 2025 2 Page number: 435-444 Column: 学术论文—智能系统 Public date: 2025-03-05
Title:
Flexible wire reconstructing and gripping verification under the image information input
Author(s):
QIU Jielu1; 2;  WANG Mingming1; 2
1. School of Aeronautics, Northwestern Polytechnical University, Xi’an 710072, China;
2. Research & Shenzhen Research Institute of Northwestern Polytechnical University, Shenzhen 518057, China
Keywords:
flexible wire; visual sensing; depth camera; image processing; coordinates calculation; curve reconstruction; pose transformation; robot arm grasping
CLC:
TP24
DOI:
10.11992/tis.202403021
Abstract:
To address the complex task of operating flexible objects with a robot, a concise and efficient method for grasping soft wires is designed. First, the soft wire is measured using a stereo camera and image processing techniques. By combining the transformation relationships between coordinate systems, three-dimensional coordinate information is reconstructed from the image data. The soft wire is then modeled and reconstructed using the Frenet frame, enabling the acquisition of positional and bending direction information at any arbitrary point along the wire. The representations and transformation methods for the six-DOF robotic arm pose, along with the camera calibration process, are provided. These enable the conversion of the Frenet frame’s coordinates and directional data for any point along the curve into the robotic gripper’s pose. Finally, a ground experiment was conducted to demonstrate the robot’s ability to grasp the soft wire along the bending direction of a target point, verifying the feasibility of this method.
References:
[1] 计时鸣, 黄希欢. 工业机器人技术的发展与应用综述[J]. 机电工程, 2015, 32(1): 1-13.
JI Shiming, HUANG Xihuan. Review of development and application of industrial robot technology[J]. Journal of mechanical & electrical engineering, 2015, 32(1): 1-13.
[2] 潘广耀. 基于机器视觉的工业机器人目标定位及检测系统研究[D]. 青岛: 青岛科技大学, 2022.
PAN Guangyao. Research on target location and detection system of industrial robot based on machine vision[D]. Qingdao: Qingdao University of Science & Technology, 2022.
[3] 刘文龙. 面向智能制造的视觉识别技术及其应用研究[D]. 哈尔滨: 哈尔滨工业大学, 2021.
LIU Wenlong. Research on visual identification technology for intelligent manufacturing and its application[D]. Harbin: Harbin Institute of Technology, 2021.
[4] 李泽辰. 基于双目视觉的工业机器人目标识别与定位系统研究[D]. 青岛: 青岛科技大学, 2021.
LI Zechen. Research on industrial robot target recognition and positioning system based on binocular vision[D]. Qingdao: Qingdao University of Science & Technology, 2021.
[5] 陈英. 机器视觉技术的发展现状与应用动态研究[J]. 无线互联科技, 2018, 15(19): 147-148.
CHEN Ying. Study on development status and application trends of machine vision technology[J]. Wireless Internet technology, 2018, 15(19): 147-148.
[6] 王飞. 机器视觉技术发展及其工业应用[J]. 电子技术与软件工程, 2018(16): 246.
WANG Fei. Development of machine vision technology and its industrial application[J]. Electronic technology & software engineering, 2018(16): 246.
[7] 滕婷婷. 面向交通复杂目标场景的机器视觉检测技术研究[D]. 南京: 南京邮电大学, 2022.
TENG Tingting. Research on machine vision detection technologiesfor complex traffic target scenarios[D]. Nanjing: Nanjing University of Posts and Telecommunications, 2022.
[8] 王瑞阳, 徐洋. 机器视觉技术在农业机器人定位中的作用[J]. 南方农机, 2023, 54(13): 90-92.
WANG Ruiyang, XU Yang. The role of machine vision technology in the positioning of agricultural robots[J]. China southern agricultural machinery, 2023, 54(13): 90-92.
[9] 唐浩然. 机器视觉的研究及应用进展[J]. 科技与创新, 2024(16): 52-55.
TANG Haoran. Research and application progress of machine vision[J]. Science and technology & innovation, 2024(16): 52-55.
[10] 颉永鹏. 基于机器视觉的工业机器人目标识别和定位研究[D]. 沈阳: 沈阳工业大学, 2022.
XIE Yongpeng. Research on target recognition and location of industrial robots based on machine vision[D]. Shenyang: Shenyang University of Technology, 2022.
[11] 刘远桃, 秦龙, 杨海能. 基于机器视觉的工业机器人分拣技术研究[J]. 机电工程技术, 2024, 53(9): 173-176.
LIU Yuantao, QIN Long, YANG Haineng. Research on sorting technology of industrial robots based on machine vision[J]. Mechanical & electrical engineering technology, 2024, 53(9): 173-176.
[12] 程家浩.基于机器视觉的工业机器人定位与抓取方法研究[D]. 天津: 天津理工大学, 2024.
CHENG Jiahao. Research on positioning and grasping methods for industrial robots based on machine vision[D]. Tianjin: Tianjin University of Technology, 2024.
[13] LYU Naijing, LIU Jianhua, JIA Yunyi. Dynamic modeling and control of deformable linear objects for single-arm and dual-arm robot manipulations[J]. IEEE transactions on robotics, 2022, 38(4): 2341-2353.
[14] SANCHEZ J, CORRALES J A, BOUZGARROU B C, et al. Robotic manipulation and sensing of deformable objects in domestic and industrial applications: a survey[J]. The international journal of robotics research, 2018, 37(7): 688-716.
[15] INOUE H. Hand eye coordination in rope handling[C]//Proceedings of International Symposium of Robotics Research. Cambridge: MIT Press, 1984: 163-174.
[16] LUI W H, SAXENA A. Tangled: learning to untangle ropes with RGB-D perception[C]//2013 IEEE/RSJ International Conference on Intelligent Robots and Systems. Tokyo: IEEE, 2013: 837-844.
[17] KUMRA S, KANAN C. Robotic grasp detection using deep convolutional neural networks[C]//2017 IEEE/RSJ International Conference on Intelligent Robots and Systems. Vancouver: IEEE, 2017: 769-776.
[18] SCHULMAN J, LEE A, HO J, et al. Tracking deformable objects with point clouds[C]//2013 IEEE International Conference on Robotics and Automation. Karlsruhe: IEEE, 2013: 1130-1137.
[19] YAMAKAWA Y, NAMIKI A, ISHIKAWA M. Dynamic high-speed knotting of a rope by a manipulator[J]. International journal of advanced robotic systems, 2013, 10(10): 361.
[20] LEVIN D. The approximation power of moving least-squares[J]. Mathematics of computation, 1998, 67(224): 1517-1531.
[21] AMENTA N, BERN M, EPPSTEIN D. The crust and the β-skeleton: combinatorial curve reconstruction[J]. Graphical models and image processing, 1998, 60(2): 125-135.
[22] YAN Dongming, WINTZ J, MOURRAIN B, et al. Efficient and robust tree model reconstruction from laser scanned data points[C]//Proceedings of the 11th IEEE International conference on Computer-Aided Design and Computer Graphics. Washington: IEEE, 2009: 572-576.
[23] 朱东方. 基于复杂拓扑结构点云的曲线拟合研究与应用[D]. 济南: 山东大学, 2015.
ZHU Dongfang. Research and application of curve fitting based on complex topological structure point cloud[D]. Jinan: Shandong University, 2015.
[24] 林思宇. 受限空间下基于视觉的线缆装配方法研究[D]. 哈尔滨: 哈尔滨工业大学, 2021.
LIN Siyu. Research on cable assembly method based on vision in confined space[D]. Harbin: Harbin Institute of Technology, 2021.
[25] PARK J, BRUNEL N, CHASSAT P. Curvature and Torsion estimation of 3D functional data: a geometric approach to build the mean shape under the Frenet Serret framework[EB/OL]. (2022-03-04)[2024-3-12]. https://arxiv.org/abs/2203.02398v1.
[26] ZHANG Zhengyou. Flexible camera calibration by viewing a plane from unknown orientations[C]//Proceedings of the Seventh IEEE International Conference on Computer Vision. Kerkyra: IEEE, 2002: 666-673.
[27] TSAI R Y, LENZ R K. A new technique for fully autonomous and efficient 3D robotics hand/eye calibration[J]. IEEE transactions on robotics and automation, 1989, 5(3): 345-358.
Similar References:

Memo

-

Last Update: 2025-03-05

Copyright © CAAI Transactions on Intelligent Systems