[1]ZHAI Yongjie,WANG Jiahao,ZHANG Xin,et al.Design of automatic picking system for seedlings based on semantic segmentation visual servo[J].CAAI Transactions on Intelligent Systems,2023,18(6):1259-1267.[doi:10.11992/tis.202212026]
Copy

Design of automatic picking system for seedlings based on semantic segmentation visual servo

CAAI Transactions on Intelligent Systems[ISSN 1673-4785/CN 23-1538/TP] Volume: 18 Number of periods: 2023 6 Page number: 1259-1267 Column: 学术论文—机器人 Public date: 2023-11-05
Title:
Design of automatic picking system for seedlings based on semantic segmentation visual servo
Author(s):
ZHAI Yongjie1;  WANG Jiahao1;  ZHANG Xin1;  HU Dongyang1;  WANG Qianming1;  XU Dawei1; 2;  LIU Yajun3
1. Department of Automation, North China Electric Power University, Baoding 071003, China;
2. The State Key Laboratory of Management and Control for Complex Systems, Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China;
3. Hubei Artisan Biotech Co., Ltd, Zhongxiang 431900, China
Keywords:
semantic segmentation; visual servo; seedling; grasping point localization; grasping system; robot arm; intelligent robotic grasping; deep learning
CLC:
TP241.2；S126
DOI:
10.11992/tis.202212026
Abstract:
Modern plant tissue culture is a time-consuming and labor-intensive task with monotonous work. An automatic seedling clamping system based on semantic segmentation visual servo was designed and tested to reduce labor costs and increase production. First, a vision localization method was proposed based on the DP-BiSeNetV2 semantic segmentation algorithm to determine the appropriate clamping point on the root. Further, a clamping device suitable for the actual working environment was designed, developed, and tested. Finally, an automatic seedling clamping system was constructed by integrating the vision localization algorithm with the robot clamping device. In the experimental session, tests were conducted using the Phalaenopsis seedling dataset. In the semantic segmentation experiment, the mIoU and pixel accuracy of the DP-BiSeNetV2 model were 63.51% and 98.25%, respectively. Furthermore, the success rate was 81.7% in the clamping experiment. Experimental results show that the automatic clamping system has a large potential to meet the transplantation requirements of plant tissue culture production lines.
References:
[1] DEBNATH M, MALIK C, BISEN P. Micropropagation: a tool for the production of high quality plant-based medicines[J]. Current pharmaceutical biotechnology, 2006, 7(1): 33–49.
[2] LEE T J, ZOBAYED S, FIRMANI F, et al. A novel automated transplanting system for plant tissue culture[J]. Biosystems engineering, 2019, 181: 63–72.
[3] 翟永杰, 胡东阳, 苑朝, 等. 基于视觉伺服的蝴蝶兰种苗切割系统设计与试验[J]. 农业工程学报, 2022, 38(6): 148–156
ZHAI Yongjie, HU Dongyang, YUAN Chao, et al. Design and experiments of phalaenopsis seedling cutting system using visual servo[J]. Transactions of the Chinese society of agricultural engineering, 2022, 38(6): 148–156
[4] NING Zhengtong, LUO Lufeng, DING Xinming, et al. Recognition of sweet peppers and planning the robotic picking sequence in high-density orchards[J]. Computers and electronics in agriculture, 2022, 196: 106878.
[5] RINGDAHL O, KURTSER P, EDAN Y. Evaluation of approach strategies for harvesting robots: case study of sweet pepper harvesting[J]. Journal of intelligent & robotic systems, 2019, 95(1): 149–164.
[6] VAN H E J, HEMMING J, VAN TUIJL B A J, et al. Collision-free motion planning for a cucumber picking robot[J]. Biosystems engineering, 2003, 86(2): 135–144.
[7] WANG Guohua, YU Yabo, FENG Qingchun. Design of end-effector for tomato robotic harvesting[J]. IFAC-Papers OnLine, 2016, 49(16): 190–193.
[8] MU Longtao, LIU Yadong, CUI Yongjie, et al. Design of end-effector for kiwifruit harvesting robot experiment[C]//2017 ASABE Annual International Meeting. Washington: American Society of Agricultural and Biological Engineers, 2017.
[9] SARABU H, AHLIN K, HU Aiping. Leveraging deep learning and RGB-D cameras for cooperative apple-picking robot arms[C]// 2019 ASABE Annual International Meeting. Boston: American Society of Agricultural and Biological Engineers, 2019.
[10] 李天华, 孙萌, 丁小明, 等. 基于YOLO v4+HSV的成熟期番茄识别方法[J]. 农业工程学报, 2021, 37(21): 183–190
LI Tianhua, SUN Meng, DING Xiaoming, et al. Tomato recognition method at the ripening stage based on YOLO v4 and HSV[J]. Transactions of the Chinese society of agricultural engineering, 2021, 37(21): 183–190
[11] 孙红, 乔金博, 李松, 等. 基于深度学习的玉米拔节期冠层识别[J]. 农业工程学报, 2021, 37(21): 53–61
SUN Hong, QIAO Jinbo, LI Song, et al. Recognition of the maize canopy at the jointing stage based on deep learning[J]. Transactions of the Chinese society of agricultural engineering, 2021, 37(21): 53–61
[12] QUAN Longzhe, FENG Huaiqu, LYU Yingjie, et al. Maize seedling detection under different growth stages and complex field environments based on an improved Faster R-CNN[J]. Biosystems engineering, 2019, 184: 1–23.
[13] ZAHID A, MAHMUD M S, HE Long, et al. Technological advancements towards developing a robotic pruner for apple trees: a review[J]. Computers and electronics in agriculture, 2021, 189: 106383.
[14] LOTTES P, BEHLEY J, CHEBROLU N, et al. Robust joint stem detection and crop-weed classification using image sequences for plant-specific treatment in precision farming[J]. Journal of field robotics, 2020, 37(1): 20–34.
[15] SHI Weinan, VAN DE ZEDDE R, JIANG Huanyu, et al. Plant-part segmentation using deep learning and multi-view vision[J]. Biosystems engineering, 2019, 187: 81–95.
[16] GUO W J, LIN Yuzu, LEE N. Photosynthetic light requirements and effects of low irradiance and daylength on phalaenopsis amabilis[J]. Journal of the American society for horticultural science, 2012, 137(6): 465–472.
[17] LIN Mingju, HSU B D. Photosynthetic plasticity of Phalaenopsis in response to different light environments[J]. Journal of plant physiology, 2004, 161(11): 1259–1268.
[18] LIU Y C, TSENG K M, CHEN C C, et al. Warm-night temperature delays spike emergence and alters carbon pool metabolism in the stem and leaves of Phalaenopsis aphroide[J]. Scientia horticulturae, 2013, 161: 198–203.
[19] XU De, LU Jinyan, WANG Peng, et al. Partially decoupled image-based visual servoing using different sensitive features[J]. IEEE transactions on systems, man, and cybernetics:systems, 2017, 47(8): 2233–2243.
[20] VICENTE P, JAMONE L, BERNARDINO A. Towards markerless visual servoing of grasping tasks for humanoid robots[C]//2017 IEEE International Conference on Robotics and Automation. Singapore: IEEE, 2017: 3811-3816.
[21] CHANG W C. Robotic assembly of smartphone back shells with eye-in-hand visual servoing[J]. Robotics and computer-integrated manufacturing, 2018, 50: 102–113.
[22] 杨月全, 秦瑞康, 李福东, 等. 机器人视觉伺服控制研究进展与挑战[J]. 郑州大学学报(理学版), 2018, 50(2): 41–48
YANG Yuequan, QIN Ruikang, LI Fudong, et al. The development and challenges of studies on robot visual servo control[J]. Journal of Zhengzhou university (natural science edition), 2018, 50(2): 41–48
[23] YU Changqian, WANG Jingbo, PENG Chao, et al. BiSeNet: bilateral segmentation network for real-time semantic segmentation[C]//European Conference on Computer Vision. Cham: Springer, 2018: 334-349.
[24] 李国进, 黄晓洁, 李修华, 等. 采用轻量级网络MobileNetV2的酿酒葡萄检测模型[J]. 农业工程学报, 2021, 37(17): 168–176,317
LI Guojin, HUANG Xiaojie, LI Xiuhua, et al. Detection model for wine grapes using MobileNetV2 lightweight network[J]. Transactions of the Chinese society of agricultural engineering, 2021, 37(17): 168–176,317
[25] 宁政通, 罗陆锋, 廖嘉欣, 等. 基于深度学习的葡萄果梗识别与最优采摘定位[J]. 农业工程学报, 2021, 37(9): 222–229
NING Zhengtong, LUO Lufeng, LIAO Jiaxin, et al. Recognition and the optimal picking point location of grape stems based on deep learning[J]. Transactions of the Chinese society of agricultural engineering, 2021, 37(9): 222–229
[26] 赵德安, 吴任迪, 刘晓洋, 等. 基于YOLO深度卷积神经网络的复杂背景下机器人采摘苹果定位[J]. 农业工程学报, 2019, 35(3): 164–173
ZHAO Dean, WU Rendi, LIU Xiaoyang, et al. Apple positioning based on YOLO deep convolutional neural network for picking robot in complex background[J]. Transactions of the Chinese society of agricultural engineering, 2019, 35(3): 164–173
[27] YU Changqian, GAO Changxin, WANG Jingbo, et al. BiSeNet V2: bilateral network with guided aggregation for real-time semantic segmentation[J]. International journal of computer vision, 2021, 129(11): 3051–3068.
[28] CHEN L C, ZHU Yukun, PAPANDREOU G, et al. Encoder-decoder with atrous separable convolution for semantic image segmentation[C]//European Conference on Computer Vision. Cham: Springer, 2018: 833-851.
[29] LI Hanchao, XIONG Pengfei, FAN Haoqiang, et al. DFANet: deep feature aggregation for real-time semantic segmentation[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach: IEEE, 2020: 9514-9523.
[30] LI Gen, YUN I, KIM J, et al. DABNet: depth-wise asymmetric bottleneck for real-time semantic segmentation[EB/OL]. (2019-07-26)[2022-12-26]. https://arxiv.org/abs/1907.11357.
[31] POUDEL R P K, LIWICKI S, CIPOLLA R. Fast-SCNN: fast semantic segmentation network[EB/OL]. (2019-02-12) [2022-12-26]. https://arxiv.org/abs/1902.04502.
Similar References:

Memo

-

Last Update: 1900-01-01

Copyright © CAAI Transactions on Intelligent Systems