[1]DONG Junjie,LIU Huaping,XIE Jun,et al.Feedback attention mechanism and context fusion based amodal instance segmentation[J].CAAI Transactions on Intelligent Systems,2021,16(4):801-810.[doi:10.11992/tis.202007042]
Copy

Feedback attention mechanism and context fusion based amodal instance segmentation

CAAI Transactions on Intelligent Systems[ISSN 1673-4785/CN 23-1538/TP] Volume: 16 Number of periods: 2021 4 Page number: 801-810 Column: 吴文俊人工智能科学技术奖论坛 Public date: 2021-07-05
Title:
Feedback attention mechanism and context fusion based amodal instance segmentation
Author(s):
DONG Junjie1;  LIU Huaping2;  XIE Jun1;  XU Xinying3;  SUN Fuchun2
1. College of Information and Computer, Taiyuan University of Technology, Jinzhong 030600, China;
2. State Key Lab. of Intelligent Technology and Systems, Tsinghua University, Beijing 100084, China;
3. College of Electrical and Power Engineering, Taiyuan University of Technology, Taiyuan 030024, China
Keywords:
amodal instance segmentation; occlusion prediction; feedback connection; attention mechanism; context information; deep learning; neural network; computer vision
CLC:
TP183
DOI:
10.11992/tis.202007042
Abstract:
Recently, model instance segmentation has been proposed as an extension of instance segmentation to predict the visible and occluded areas of each object instance and perceive the complete physical structure and semantic concepts. When the shapes and meanings of occluded objects are being predicted, underfitting or even wrong results are obtained in the occlusion prediction due to the insufficient recognition capability of feature representation and the lack of contextual information. To solve this problem, this paper proposes a contextual attention module and feature pyramid structure of feedback attention mechanism and introduces feedback connections for relearning. The proposed method can effectively capture global semantic information and fine spatial details. Through training and verification in the COCO-amodal dataset, the average precision of the amodal instance segmentation mask increases from 8.4% to 14.3%, and the average recall rate increases from 16.6% to 20.8%. Experimental results show that this method can significantly improve the accuracy of occlusion prediction and effectively end underfitting.
References:
[1] SIMONYAN K, ZISSERMAN A. Very deep convolution- al networks for large-scale image recognition[EB/OL]. (20 15-04-10)[2020-07-21] https://arxiv.org/abs/1409.1556.
[2] HE Kaiming, ZHANG Xiangyu, REN Shaoqing, et al. Deep residual learning for image recognition[C]//Pro- ceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas, NV, USA, 2016: 770-778.
[3] LIU Wei, ANGUELOV D, ERHAN D, et al. SSD: single shot MultiBox detector[C]//Proceedings of the 14th Euro- pean Conference on Computer Vision. Amsterdam, The Netherlands, 2016: 21-37.
[4] REN Shaoqing, HE Kaiming, GIRSHICK R, et al. Faster R-CNN: towards real-time object detection with region proposal networks[J]. IEEE transactions on pattern analysis and machine intelligence, 2017, 39(6): 1137-1149.
[5] LIN T Y, GOYAL P, GIRSHICK R, et al. Focal loss for dense object detection[C]//Proceedings of 2017 IEEE International Conference on Computer Vision. Venice, Italy, 2017: 2999-3007.
[6] GIRSHICK R. Fast R-CNN[C]//Proceedings of 2015 IEEE International Conference on Computer Vision (ICCV). Santiago, Chile, 2015: 1440-1448.
[7] LONG J, SHELHAMER E, DARRELL T. Fully convolutional networks for semantic segmentation[C]//Pr- oceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition. Boston, America, 2015: 3431-3440.
[8] CHEN L C, PAPANDREOU G, SCHROFF F, et al. Rethinking atrous convolution for semantic image segme- ntation[EB/OL]. (2017-10-05)[2020-07-21] https://arxiv.org/abs/1706.05587.
[9] HE Kaiming, GKIOXARI G, DOLLáR P, et al. Mask R-CNN[C]//Proceedings of 2017 IEEE International Conference on Computer Vision. Venice, Italy, 2017: 2980-2988.
[10] BOLYA D, ZHOU Chong, XIAO Fanyi, et al. YOLACT: real-time instance segmentation[C]//Proceedings of 2019 IEEE/CVF International Conference on Computer Vision. Seoul, South Korea, 2019: 9156-9165.
[11] ZHU Yan, TIAN Yuandong, METAXAS D, et al. Semantic amodal segmentation[C]//Proceedings of the 30th IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, United States, 2017: 3001-3009.
[12] ZHANG Ziheng, CHEN Aapei, XIE Ling, et al. Learning semantics-aware distance map with semantics layering network for amodal instance segmentation[C]//Pro- ceedings of the 27th ACM International Conference on Multimedia. Nice, France, 2019: 2124-2132.
[13] QIAO Siyuan, CHEN L C, YUILLE A. DetectoRS: detecting objects with recursive feature pyramid and swi- tchable atrous convolution[EB/OL]. (2020-06-03)[2020-07- 21] https://arxiv.org/abs/2006.02334.
[14] LI Yi, QI Haozhi, DAI Jifeng, et al. Fully convolutional instance-aware semantic segmentation[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, United States, 2017: 4438-4446.
[15] LIU Shu, QI Lu, QIN Haifang, et al. Path aggregation network for instance segmentation[C]//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, United States, 2018: 8759-8768.
[16] CHEN Xinlei, GIRSHICK R, HE Kaiming, et al. Tensormask: a foundation for dense object segmenta- tion[C]//Proceedings of 2019 IEEE/CVF International Conference on Computer Vision. Seoul, South Korea, 2019: 2061-2069.
[17] XIE Enze, SUN Peize, SONG Xiaoge, et al. PolarMask: single shot instance segmentation with polar representa- tion[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle, United States, 2020: 12190-12199.
[18] LI Ke, MALIK J. Amodal instance segmentation[C]//Pro- ceedings of the 14th European Conference on Computer Vision. Amsterdam, The Netherlands, 2016: 677-693.
[19] FOLLMANN P, K?NIG R, H?RTINGER P, et al. Learning to see the invisible: end-to-end trainable amodal instance segmentation[C]//2019 IEEE Winter Conference on Applications of Computer Vision (WACV). Waikoloa, United States, 2019: 1328-1336.
[20] EHSANI K, MOTTAGHI R, FARHADI A. SeGAN: segmenting and generating the invisible[C]//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, United States, 2018: 6144-6153.
[21] HU Jie, SHEN Li, SUN Gang. Squeeze-and-excitation networks[C]//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, United States, 2018: 7132-7141.
[22] WANG Xiaolong, GIRSHICK R, GUPTA A, et al. Non-local neural networks[C]//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, United States, 2018: 7794-7803.
[23] WOO S, PARK J, LEE J Y, et al. CBAM: convolutional block attention module[C]//Proceedings of the 15th Euro- pean Conference on Computer Vision (ECCV). Munich, Germany, 2018: 3-19.
[24] CAO Yun, XU Jiarui, LIN S, et al. GCNet: non-local networks meet squeeze-excitation networks and beyo- nd[C]//Proceedings of 2019 IEEE/CVF International Conference on Computer Vision Workshops. Seoul, South Korea, 2019: 1971-1980.
[25] FAN Zhibo, YU Jingang, LIANG Zhihao, et al. FGN: fully guided network for few-shot instance segmenta- tion[C]//Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle, United States, 2020: 9169-9178.
[26] LIU Dongnan, ZHANG Donghao, SONG Yang, et al. Cell R-CNN v3: a novel panoptic paradigm for instance segmentation in biomedical images[EB/OL]. (2020-02-15) [2020-07-21] https://arxiv.org/abs/2002.06345.
[27] FU Jun, LIU Jing, TIAN Haijie, et al. Dual attention network for scene segmentation[C]//Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach, United States, 2019: 3141-3149.
[28] CAO Junxu, CHEN Qi, GUO Jun, et al. Attention-guided context feature pyramid network for object detecti- on[EB/OL]. (2020-05-23)[2020-07-21] https://arxiv.org/abs/2005.11475.
Similar References:

Memo

-

Last Update: 1900-01-01

Copyright © CAAI Transactions on Intelligent Systems