<-上一篇/Previous Article 下一篇/Next Article->

[1]董俊杰,刘华平,谢珺,等.基于反馈注意力机制和上下文融合的非模式实例分割[J].智能系统学报,2021,16(4):801-810.[doi:10.11992/tis.202007042]
　DONG Junjie,LIU Huaping,XIE Jun,et al.Feedback attention mechanism and context fusion based amodal instance segmentation[J].CAAI Transactions on Intelligent Systems,2021,16(4):801-810.[doi:10.11992/tis.202007042]

点击复制

基于反馈注意力机制和上下文融合的非模式实例分割

PDF下载 HTML

《智能系统学报》[ISSN 1673-4785/CN 23-1538/TP] 卷: 16 期数: 2021年第4期页码: 801-810 栏目: 吴文俊人工智能科学技术奖论坛出版日期: 2021-07-05

Title:: Feedback attention mechanism and context fusion based amodal instance segmentation

作者:: 董俊杰¹, 刘华平², 谢珺¹, 续欣莹³, 孙富春²; 1. 太原理工大学信息与计算机学院，山西晋中 030600;
2. 清华大学智能技术与系统国家重点实验室，北京 100084;
3. 太原理工大学电气与动力工程学院，山西太原 030024

Author(s):: DONG Junjie¹, LIU Huaping², XIE Jun¹, XU Xinying³, SUN Fuchun²; 1. College of Information and Computer, Taiyuan University of Technology, Jinzhong 030600, China;
2. State Key Lab. of Intelligent Technology and Systems, Tsinghua University, Beijing 100084, China;
3. College of Electrical and Power Engineering, Taiyuan University of Technology, Taiyuan 030024, China

关键词:: 非模式实例分割; 遮挡预测; 反馈连接; 注意力机制; 上下文信息; 深度学习; 神经网络; 计算机视觉

Keywords:: amodal instance segmentation; occlusion prediction; feedback connection; attention mechanism; context information; deep learning; neural network; computer vision

分类号:: TP183

DOI:: 10.11992/tis.202007042

摘要:: 非模式实例分割是最近提出的对实例分割的扩展，其任务是对每个对象实例的可见区域和被遮挡区域都进行预测，感知完整的物理结构和语义概念。在预测对象被遮挡部分的形状和语义时，往往由于特征表示的识别能力不够和对上下文信息缺乏而导致对遮挡区域预测欠拟合甚至错误。针对这个问题，提出一个上下文注意模块和反馈注意力机制的特征金字塔结构，引入反馈连接进行再学习。该方法能够有效捕获全局语义信息和精细的空间细节，通过在COCO-amodal数据集训练和验证，非模式实例分割掩码平均精确率从8.4%提高到14.3%，平均召回率从16.6%提高到20.8%。实验结果表明，该方法能够显著提高对物体被遮挡部分预测的准确率，有效解决欠拟合问题。

Abstract:: Recently, model instance segmentation has been proposed as an extension of instance segmentation to predict the visible and occluded areas of each object instance and perceive the complete physical structure and semantic concepts. When the shapes and meanings of occluded objects are being predicted, underfitting or even wrong results are obtained in the occlusion prediction due to the insufficient recognition capability of feature representation and the lack of contextual information. To solve this problem, this paper proposes a contextual attention module and feature pyramid structure of feedback attention mechanism and introduces feedback connections for relearning. The proposed method can effectively capture global semantic information and fine spatial details. Through training and verification in the COCO-amodal dataset, the average precision of the amodal instance segmentation mask increases from 8.4% to 14.3%, and the average recall rate increases from 16.6% to 20.8%. Experimental results show that this method can significantly improve the accuracy of occlusion prediction and effectively end underfitting.

参考文献/References:: [1] SIMONYAN K, ZISSERMAN A. Very deep convolution- al networks for large-scale image recognition[EB/OL]. (20 15-04-10)[2020-07-21] https://arxiv.org/abs/1409.1556.
[2] HE Kaiming, ZHANG Xiangyu, REN Shaoqing, et al. Deep residual learning for image recognition[C]//Pro- ceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas, NV, USA, 2016: 770-778.
[3] LIU Wei, ANGUELOV D, ERHAN D, et al. SSD: single shot MultiBox detector[C]//Proceedings of the 14th Euro- pean Conference on Computer Vision. Amsterdam, The Netherlands, 2016: 21-37.
[4] REN Shaoqing, HE Kaiming, GIRSHICK R, et al. Faster R-CNN: towards real-time object detection with region proposal networks[J]. IEEE transactions on pattern analysis and machine intelligence, 2017, 39(6): 1137-1149.
[5] LIN T Y, GOYAL P, GIRSHICK R, et al. Focal loss for dense object detection[C]//Proceedings of 2017 IEEE International Conference on Computer Vision. Venice, Italy, 2017: 2999-3007.
[6] GIRSHICK R. Fast R-CNN[C]//Proceedings of 2015 IEEE International Conference on Computer Vision (ICCV). Santiago, Chile, 2015: 1440-1448.
[7] LONG J, SHELHAMER E, DARRELL T. Fully convolutional networks for semantic segmentation[C]//Pr- oceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition. Boston, America, 2015: 3431-3440.
[8] CHEN L C, PAPANDREOU G, SCHROFF F, et al. Rethinking atrous convolution for semantic image segme- ntation[EB/OL]. (2017-10-05)[2020-07-21] https://arxiv.org/abs/1706.05587.
[9] HE Kaiming, GKIOXARI G, DOLLáR P, et al. Mask R-CNN[C]//Proceedings of 2017 IEEE International Conference on Computer Vision. Venice, Italy, 2017: 2980-2988.
[10] BOLYA D, ZHOU Chong, XIAO Fanyi, et al. YOLACT: real-time instance segmentation[C]//Proceedings of 2019 IEEE/CVF International Conference on Computer Vision. Seoul, South Korea, 2019: 9156-9165.
[11] ZHU Yan, TIAN Yuandong, METAXAS D, et al. Semantic amodal segmentation[C]//Proceedings of the 30th IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, United States, 2017: 3001-3009.
[12] ZHANG Ziheng, CHEN Aapei, XIE Ling, et al. Learning semantics-aware distance map with semantics layering network for amodal instance segmentation[C]//Pro- ceedings of the 27th ACM International Conference on Multimedia. Nice, France, 2019: 2124-2132.
[13] QIAO Siyuan, CHEN L C, YUILLE A. DetectoRS: detecting objects with recursive feature pyramid and swi- tchable atrous convolution[EB/OL]. (2020-06-03)[2020-07- 21] https://arxiv.org/abs/2006.02334.
[14] LI Yi, QI Haozhi, DAI Jifeng, et al. Fully convolutional instance-aware semantic segmentation[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, United States, 2017: 4438-4446.
[15] LIU Shu, QI Lu, QIN Haifang, et al. Path aggregation network for instance segmentation[C]//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, United States, 2018: 8759-8768.
[16] CHEN Xinlei, GIRSHICK R, HE Kaiming, et al. Tensormask: a foundation for dense object segmenta- tion[C]//Proceedings of 2019 IEEE/CVF International Conference on Computer Vision. Seoul, South Korea, 2019: 2061-2069.
[17] XIE Enze, SUN Peize, SONG Xiaoge, et al. PolarMask: single shot instance segmentation with polar representa- tion[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle, United States, 2020: 12190-12199.
[18] LI Ke, MALIK J. Amodal instance segmentation[C]//Pro- ceedings of the 14th European Conference on Computer Vision. Amsterdam, The Netherlands, 2016: 677-693.
[19] FOLLMANN P, K?NIG R, H?RTINGER P, et al. Learning to see the invisible: end-to-end trainable amodal instance segmentation[C]//2019 IEEE Winter Conference on Applications of Computer Vision (WACV). Waikoloa, United States, 2019: 1328-1336.
[20] EHSANI K, MOTTAGHI R, FARHADI A. SeGAN: segmenting and generating the invisible[C]//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, United States, 2018: 6144-6153.
[21] HU Jie, SHEN Li, SUN Gang. Squeeze-and-excitation networks[C]//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, United States, 2018: 7132-7141.
[22] WANG Xiaolong, GIRSHICK R, GUPTA A, et al. Non-local neural networks[C]//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, United States, 2018: 7794-7803.
[23] WOO S, PARK J, LEE J Y, et al. CBAM: convolutional block attention module[C]//Proceedings of the 15th Euro- pean Conference on Computer Vision (ECCV). Munich, Germany, 2018: 3-19.
[24] CAO Yun, XU Jiarui, LIN S, et al. GCNet: non-local networks meet squeeze-excitation networks and beyo- nd[C]//Proceedings of 2019 IEEE/CVF International Conference on Computer Vision Workshops. Seoul, South Korea, 2019: 1971-1980.
[25] FAN Zhibo, YU Jingang, LIANG Zhihao, et al. FGN: fully guided network for few-shot instance segmenta- tion[C]//Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle, United States, 2020: 9169-9178.
[26] LIU Dongnan, ZHANG Donghao, SONG Yang, et al. Cell R-CNN v3: a novel panoptic paradigm for instance segmentation in biomedical images[EB/OL]. (2020-02-15) [2020-07-21] https://arxiv.org/abs/2002.06345.
[27] FU Jun, LIU Jing, TIAN Haijie, et al. Dual attention network for scene segmentation[C]//Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach, United States, 2019: 3141-3149.
[28] CAO Junxu, CHEN Qi, GUO Jun, et al. Attention-guided context feature pyramid network for object detecti- on[EB/OL]. (2020-05-23)[2020-07-21] https://arxiv.org/abs/2005.11475.

备注/Memo

收稿日期:2020-07-24。
基金项目:山西省自然科学基金项目(201801D121144，201801D221190)；辽宁省科技厅机器人技术国家重点实验室联合基金项目(2020-KF-22-06)
作者简介:董俊杰，硕士研究生，主要研究方向为智能信息处理、计算机视觉和图像识别;刘华平，副教授，博士生导师，IEEE Senior Member、中国人工智能学会理事、中国人工智能学会认知系统与信息处理专业委员会秘书长。主要研究方向为机器人感知、学习与控制、多模态信息融合。发表学术论文340余篇;谢珺，副教授，主要研究方向为粗糙集、粒计算、数据挖掘和智能信息处理.
通讯作者:刘华平.E-mail:hpliu@tsinghua.edu.cn

更新日期/Last Update: 1900-01-01

基于反馈注意力机制和上下文融合的非模式实例分割 PDF下载HTML

备注/Memo

基于反馈注意力机制和上下文融合的非模式实例分割

PDF下载 HTML