[1]翟永杰,王家豪,张鑫,等.基于语义分割视觉伺服的种苗自动夹取系统设计[J].智能系统学报,2023,18(6):1259-1267.[doi:10.11992/tis.202212026]
ZHAI Yongjie,WANG Jiahao,ZHANG Xin,et al.Design of automatic picking system for seedlings based on semantic segmentation visual servo[J].CAAI Transactions on Intelligent Systems,2023,18(6):1259-1267.[doi:10.11992/tis.202212026]
点击复制
ZHAI Yongjie,WANG Jiahao,ZHANG Xin,et al.Design of automatic picking system for seedlings based on semantic segmentation visual servo[J].CAAI Transactions on Intelligent Systems,2023,18(6):1259-1267.[doi:10.11992/tis.202212026]
基于语义分割视觉伺服的种苗自动夹取系统设计
《智能系统学报》[ISSN 1673-4785/CN 23-1538/TP] 卷:
18
期数:
2023年第6期
页码:
1259-1267
栏目:
学术论文—机器人
出版日期:
2023-11-05
- Title:
- Design of automatic picking system for seedlings based on semantic segmentation visual servo
- Keywords:
- semantic segmentation; visual servo; seedling; grasping point localization; grasping system; robot arm; intelligent robotic grasping; deep learning
- 分类号:
- TP241.2；S126
- 摘要:
- 现代植物组织培养是一项耗时费力的工作，工作强度大，工作内容单调，为了减少劳动力成本并提高产量，设计了一种基于语义分割视觉伺服的种苗自动夹取系统并进行了测试。首先，提出了一种基于DP-BiSeNetV2语义分割算法的视觉定位方法，确定了根上合适的夹取点；之后，设计开发并测试了一种适合于实际工作环境的夹取装置；最后，将视觉定位算法与机器人夹取装置集成，构建了一个种苗自动夹取系统。在实验环节，使用蝴蝶兰种苗数据集进行了测试，在语义分割实验中，DP-BiSeNetV2模型的平均交并比为63.51%，像素准确度为98.25%；在夹取实验中，夹取成功率为81.7%。实验结果表明，该自动夹取系统具有很高的潜力，可以满足植物组织培养生产线的移植需求。
- Abstract:
- Modern plant tissue culture is a time-consuming and labor-intensive task with monotonous work. An automatic seedling clamping system based on semantic segmentation visual servo was designed and tested to reduce labor costs and increase production. First, a vision localization method was proposed based on the DP-BiSeNetV2 semantic segmentation algorithm to determine the appropriate clamping point on the root. Further, a clamping device suitable for the actual working environment was designed, developed, and tested. Finally, an automatic seedling clamping system was constructed by integrating the vision localization algorithm with the robot clamping device. In the experimental session, tests were conducted using the Phalaenopsis seedling dataset. In the semantic segmentation experiment, the mIoU and pixel accuracy of the DP-BiSeNetV2 model were 63.51% and 98.25%, respectively. Furthermore, the success rate was 81.7% in the clamping experiment. Experimental results show that the automatic clamping system has a large potential to meet the transplantation requirements of plant tissue culture production lines.
- 参考文献/References:
-
[1] DEBNATH M, MALIK C, BISEN P. Micropropagation: a tool for the production of high quality plant-based medicines[J]. Current pharmaceutical biotechnology, 2006, 7(1): 33–49.
[2] LEE T J, ZOBAYED S, FIRMANI F, et al. A novel automated transplanting system for plant tissue culture[J]. Biosystems engineering, 2019, 181: 63–72.
[3] 翟永杰, 胡东阳, 苑朝, 等. 基于视觉伺服的蝴蝶兰种苗切割系统设计与试验[J]. 农业工程学报, 2022, 38(6): 148–156
ZHAI Yongjie, HU Dongyang, YUAN Chao, et al. Design and experiments of phalaenopsis seedling cutting system using visual servo[J]. Transactions of the Chinese society of agricultural engineering, 2022, 38(6): 148–156
[4] NING Zhengtong, LUO Lufeng, DING Xinming, et al. Recognition of sweet peppers and planning the robotic picking sequence in high-density orchards[J]. Computers and electronics in agriculture, 2022, 196: 106878.
[5] RINGDAHL O, KURTSER P, EDAN Y. Evaluation of approach strategies for harvesting robots: case study of sweet pepper harvesting[J]. Journal of intelligent & robotic systems, 2019, 95(1): 149–164.
[6] VAN H E J, HEMMING J, VAN TUIJL B A J, et al. Collision-free motion planning for a cucumber picking robot[J]. Biosystems engineering, 2003, 86(2): 135–144.
[7] WANG Guohua, YU Yabo, FENG Qingchun. Design of end-effector for tomato robotic harvesting[J]. IFAC-Papers OnLine, 2016, 49(16): 190–193.
[8] MU Longtao, LIU Yadong, CUI Yongjie, et al. Design of end-effector for kiwifruit harvesting robot experiment[C]//2017 ASABE Annual International Meeting. Washington: American Society of Agricultural and Biological Engineers, 2017.
[9] SARABU H, AHLIN K, HU Aiping. Leveraging deep learning and RGB-D cameras for cooperative apple-picking robot arms[C]// 2019 ASABE Annual International Meeting. Boston: American Society of Agricultural and Biological Engineers, 2019.
[10] 李天华, 孙萌, 丁小明, 等. 基于YOLO v4+HSV的成熟期番茄识别方法[J]. 农业工程学报, 2021, 37(21): 183–190
LI Tianhua, SUN Meng, DING Xiaoming, et al. Tomato recognition method at the ripening stage based on YOLO v4 and HSV[J]. Transactions of the Chinese society of agricultural engineering, 2021, 37(21): 183–190
[11] 孙红, 乔金博, 李松, 等. 基于深度学习的玉米拔节期冠层识别[J]. 农业工程学报, 2021, 37(21): 53–61
SUN Hong, QIAO Jinbo, LI Song, et al. Recognition of the maize canopy at the jointing stage based on deep learning[J]. Transactions of the Chinese society of agricultural engineering, 2021, 37(21): 53–61
[12] QUAN Longzhe, FENG Huaiqu, LYU Yingjie, et al. Maize seedling detection under different growth stages and complex field environments based on an improved Faster R-CNN[J]. Biosystems engineering, 2019, 184: 1–23.
[13] ZAHID A, MAHMUD M S, HE Long, et al. Technological advancements towards developing a robotic pruner for apple trees: a review[J]. Computers and electronics in agriculture, 2021, 189: 106383.
[14] LOTTES P, BEHLEY J, CHEBROLU N, et al. Robust joint stem detection and crop-weed classification using image sequences for plant-specific treatment in precision farming[J]. Journal of field robotics, 2020, 37(1): 20–34.
[15] SHI Weinan, VAN DE ZEDDE R, JIANG Huanyu, et al. Plant-part segmentation using deep learning and multi-view vision[J]. Biosystems engineering, 2019, 187: 81–95.
[16] GUO W J, LIN Yuzu, LEE N. Photosynthetic light requirements and effects of low irradiance and daylength on phalaenopsis amabilis[J]. Journal of the American society for horticultural science, 2012, 137(6): 465–472.
[17] LIN Mingju, HSU B D. Photosynthetic plasticity of Phalaenopsis in response to different light environments[J]. Journal of plant physiology, 2004, 161(11): 1259–1268.
[18] LIU Y C, TSENG K M, CHEN C C, et al. Warm-night temperature delays spike emergence and alters carbon pool metabolism in the stem and leaves of Phalaenopsis aphroide[J]. Scientia horticulturae, 2013, 161: 198–203.
[19] XU De, LU Jinyan, WANG Peng, et al. Partially decoupled image-based visual servoing using different sensitive features[J]. IEEE transactions on systems, man, and cybernetics:systems, 2017, 47(8): 2233–2243.
[20] VICENTE P, JAMONE L, BERNARDINO A. Towards markerless visual servoing of grasping tasks for humanoid robots[C]//2017 IEEE International Conference on Robotics and Automation. Singapore: IEEE, 2017: 3811-3816.
[21] CHANG W C. Robotic assembly of smartphone back shells with eye-in-hand visual servoing[J]. Robotics and computer-integrated manufacturing, 2018, 50: 102–113.
[22] 杨月全, 秦瑞康, 李福东, 等. 机器人视觉伺服控制研究进展与挑战[J]. 郑州大学学报(理学版), 2018, 50(2): 41–48
YANG Yuequan, QIN Ruikang, LI Fudong, et al. The development and challenges of studies on robot visual servo control[J]. Journal of Zhengzhou university (natural science edition), 2018, 50(2): 41–48
[23] YU Changqian, WANG Jingbo, PENG Chao, et al. BiSeNet: bilateral segmentation network for real-time semantic segmentation[C]//European Conference on Computer Vision. Cham: Springer, 2018: 334-349.
[24] 李国进, 黄晓洁, 李修华, 等. 采用轻量级网络MobileNetV2的酿酒葡萄检测模型[J]. 农业工程学报, 2021, 37(17): 168–176,317
LI Guojin, HUANG Xiaojie, LI Xiuhua, et al. Detection model for wine grapes using MobileNetV2 lightweight network[J]. Transactions of the Chinese society of agricultural engineering, 2021, 37(17): 168–176,317
[25] 宁政通, 罗陆锋, 廖嘉欣, 等. 基于深度学习的葡萄果梗识别与最优采摘定位[J]. 农业工程学报, 2021, 37(9): 222–229
NING Zhengtong, LUO Lufeng, LIAO Jiaxin, et al. Recognition and the optimal picking point location of grape stems based on deep learning[J]. Transactions of the Chinese society of agricultural engineering, 2021, 37(9): 222–229
[26] 赵德安, 吴任迪, 刘晓洋, 等. 基于YOLO深度卷积神经网络的复杂背景下机器人采摘苹果定位[J]. 农业工程学报, 2019, 35(3): 164–173
ZHAO Dean, WU Rendi, LIU Xiaoyang, et al. Apple positioning based on YOLO deep convolutional neural network for picking robot in complex background[J]. Transactions of the Chinese society of agricultural engineering, 2019, 35(3): 164–173
[27] YU Changqian, GAO Changxin, WANG Jingbo, et al. BiSeNet V2: bilateral network with guided aggregation for real-time semantic segmentation[J]. International journal of computer vision, 2021, 129(11): 3051–3068.
[28] CHEN L C, ZHU Yukun, PAPANDREOU G, et al. Encoder-decoder with atrous separable convolution for semantic image segmentation[C]//European Conference on Computer Vision. Cham: Springer, 2018: 833-851.
[29] LI Hanchao, XIONG Pengfei, FAN Haoqiang, et al. DFANet: deep feature aggregation for real-time semantic segmentation[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach: IEEE, 2020: 9514-9523.
[30] LI Gen, YUN I, KIM J, et al. DABNet: depth-wise asymmetric bottleneck for real-time semantic segmentation[EB/OL]. (2019-07-26)[2022-12-26]. https://arxiv.org/abs/1907.11357.
[31] POUDEL R P K, LIWICKI S, CIPOLLA R. Fast-SCNN: fast semantic segmentation network[EB/OL]. (2019-02-12) [2022-12-26]. https://arxiv.org/abs/1902.04502.
- 相似文献/References:
-
[1]方勇纯.机器人视觉伺服研究综述[J].智能系统学报,2008,3(2):109.
FANG Yong-chun.A survey of robot visual servoing[J].CAAI Transactions on Intelligent Systems,2008,3():109.
[2]张雪华,刘华平,孙富春,等.采用Kinect的多臂协调操作系统[J].智能系统学报,2014,9(3):307.[doi:10.3969/j.issn.1673-4785.201308019]
ZHANG Xuehua,LIU Huaping,SUN Fuchun,et al.Kinect-based dobby coordinate operating system[J].CAAI Transactions on Intelligent Systems,2014,9():307.[doi:10.3969/j.issn.1673-4785.201308019]
[3]黄心汉.微装配机器人：关键技术、发展与应用[J].智能系统学报,2020,15(3):413.[doi:10.11992/tis.201809031]
HUANG Xinhan.Microassembly robot: key technology, development, and applications[J].CAAI Transactions on Intelligent Systems,2020,15():413.[doi:10.11992/tis.201809031]
[4]吴涛,董肖莉,孟伟,等.基于语义分割的简洁线条肖像画生成方法[J].智能系统学报,2021,16(1):134.[doi:10.11992/tis.202101003]
WU Tao,DONG Xiaoli,MENG Wei,et al.Concise line portrait generation method based on semantic segmentation[J].CAAI Transactions on Intelligent Systems,2021,16():134.[doi:10.11992/tis.202101003]
[5]王兴武,雷涛,王营博,等.基于多模态互补特征学习的遥感影像语义分割[J].智能系统学报,2022,17(6):1123.[doi:10.11992/tis.202201025]
WANG Xingwu,LEI Tao,WANG Yingbo,et al.Semantic segmentation of remote sensing image based on multimodal complementary feature learning[J].CAAI Transactions on Intelligent Systems,2022,17():1123.[doi:10.11992/tis.202201025]
[6]王潇棠,闫河,刘建骐,等.一种边缘梯度插值的双分支deeplabv3+语义分割模型[J].智能系统学报,2023,18(3):604.[doi:10.11992/tis.202111023]
WANG Xiaotang,YAN He,LIU Jianqi,et al.A new deeplabv3+ semantic segmentation model of edge gradient interpolation with double branch structure[J].CAAI Transactions on Intelligent Systems,2023,18():604.[doi:10.11992/tis.202111023]
[7]庞荣,杨燕,冷雄进,等.基于双分支点流语义先验的路面病害分割模型[J].智能系统学报,2024,19(1):153.[doi:10.11992/tis.202306037]
PANG Rong,YANG Yan,LENG Xiongjin,et al.Segmentation model of pavement diseases based on semantic priori of double-branched point flow[J].CAAI Transactions on Intelligent Systems,2024,19():153.[doi:10.11992/tis.202306037]
备注/Memo
收稿日期:2022-12-26。
基金项目:国家自然科学基金联合基金重点支持项目（U21A20486）；中国科学院自动化研究所复杂系统管理与控制国家重点实验室开放课题（20220102）.
作者简介:翟永杰,教授,博士,主要研究方向为电力视觉。主持国家自然科学基金面上项目2项、河北省自然科学基金项目2项、横向科研项目20余项。编著1部,参编教材1部、著作3部。发表学术论文30余篇;王家豪,硕士研究生,主要研究方向为电力视觉与智能机器人;徐大伟,讲师,博士,主要研究方向为机器人。参与国家重点研发计划项目1项、国家自然科学基金面上项目1项,主持横向科研项目2项
通讯作者:徐大伟.E-mail:xudawei@ncepu.edu.cn
更新日期/Last Update:
1900-01-01