[1]ZHOU Shiqi,WANG Yaonan,ZHONG Hang.Siamese network combined with visual saliency re-detection for UAV object tracking[J].CAAI Transactions on Intelligent Systems,2021,16(3):584-594.[doi:10.11992/tis.202101035]
Copy

Siamese network combined with visual saliency re-detection for UAV object tracking

CAAI Transactions on Intelligent Systems[ISSN 1673-4785/CN 23-1538/TP] Volume: 16 Number of periods: 2021 3 Page number: 584-594 Column: 人工智能院长论坛 Public date: 2021-05-05
Title:
Siamese network combined with visual saliency re-detection for UAV object tracking
Author(s):
ZHOU Shiqi1; 2;  WANG Yaonan1; 2;  ZHONG Hang1; 2
1. College of Electrical and Information Engineering, Hunan University, Changsha 410082, China;
2. National Engineering Laboratory for Robot Vision Perception and Control Technology, Hunan University, Changsha 410082, China
Keywords:
drone; computer vision; object tracking; MobileNetV2; siamese network; saliency detection; target occlusion; feature fusion
CLC:
TP242
DOI:
10.11992/tis.202101035
Abstract:
Considering the problems associated with rotorcraft trackings, such as target scale variation, fast motion, and viewpoint change, this paper proposes a siamese network-based target tracking algorithm using MobileNetV2, which can run in real-time on an onboard UAV processor. The algorithm consists of target score and scale estimation modules. Combined with the multifeature fusion strategy, the target position and target box IoU were accurately predicted. At the same time, by employing the IoU, the gradient ascent method was used to iteratively modify the target bounding box to further improve the prediction accuracy. In addition, to solve the problem of target loss caused by full occlusion, a re-detection algorithm based on visual saliency detection was developed, which efficiently predicted the saliency map of the image in real-time to guide the re-detection of the target and resume tracking. Finally, the feasibility of the algorithm was verified by comparing the standard UAV tracking dataset and the actual UAV tracking experiment.
References:
[1] 徐怀宇, 黄伟, 董明超, 等. 无人机目标跟踪综述[J]. 网络新媒体技术, 2019, 8(5):11-20
XU Huaiyu, HUANG Wei, DONG Mingchao, et al. Overview of UAV object tracking[J]. Journal of network new media, 2019, 8(5):11-20
[2] 刘芳, 孙亚楠, 王洪娟, 等. 基于残差学习的自适应无人机目标跟踪算法[J]. 北京航空航天大学学报, 2020, 46(10):1874-1882
LIU Fang, SUN Yanan, WANG Hongjuan, et al. Adaptive UAV target tracking algorithm based on residual learning[J]. Journal of Beijing University of Aeronautics and Astronautics, 2020, 46(10):1874-1882
[3] HENRIQUES J F, CASEIRO R, MARTINS P, et al. Exploiting the circulant structure of tracking-by-detection with kernels[C]//12th European Conference on Computer Vision. Florence, Italy, 2012:702-715.
[4] HENRIQUES J F, CASEIRO R, MARTINS P, et al. High-speed tracking with kernelized correlation filters[J]. IEEE transactions on pattern analysis and machine intelligence, 2015, 37(3):583-596.
[5] DANELLJAN M, ROBINSON A, KHAN F S, et al. Beyond correlation filters:learning continuous convolution operators for visual tracking[C]//14th European Conference on Computer Vision. Amsterdam, The Netherlands, 2016:472-488.
[6] DANELLJAN M, BHAT G, KHAN F S, et al. Eco:efficient convolution operators for tracking[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, USA, 2017:6638-6646.
[7] TAO Ren, GAVVES E, SMEULDERS A W M. Siamese instance search for tracking[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, USA, 2016:1420-1429.
[8] BERTINETTO L, VALMADRE J, HENRIQUES J F, et al. Fully-convolutional Siamese networks for object tracking[C]//European Conference on Computer Vision. Amsterdam, The Netherlands, 2016:850-865.
[9] VALMADRE J, BERTINETTO L, HENRIQUES J, et al. End-to-end representation learning for correlation filter based tracking[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, USA, 2017:2805-2813.
[10] LI Bo, YAN Junjie, WU Wei, et al. High performance visual tracking with Siamese region proposal network[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, USA, 2018:8971-8980.
[11] REN Shaoqing, HE Kaiming, GIRSHICK R, et al. Faster R-CNN:towards real-time object detection with region proposal networks[J]. IEEE transactions on pattern analysis and machine intelligence, 2015, 39(6):1137-1149.
[12] JIANG Borui, LUO Ruixuan, MAO Jiayuan, et al. Acquisition of localization confidence for accurate object detection[C]//Proceedings of the European Conference on Computer Vision (ECCV). Munich, Germany, 2018:816-832.
[13] SANDLER M, HOWARD A, ZHU Menglong, et al. MobileNetv2:inverted residuals and linear bottlenecks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake City, USA, 2018:4510-4520.
[14] LI Bo, WU Wei, WANG Qiang, et al. Siamrpn++:evolution of Siamese visual tracking with very deep networks[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach, USA, 2019:4277-4286.
[15] YU F, KOLTUN V. Multi-scale context aggregation by dilated convolutions[EB/OL]. (2020?12?29)[2021?01?10] https://arxiv.org/abs/1511.07122, 2016.
[16] HE Kaiming, ZHANG Xiangyu, REN Shaoqing, et al. Deep residual learning for image recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, USA, 2016:770-778.
[17] FAN Heng, LIN Liting, YANG Fan, et al. LaSOT:a high-quality benchmark for large-scale single object tracking[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach, USA, 2019:5369-5378.
[18] HUANG Lianghua, ZHAO Xin, HUANG Kaiqi. GOT-10k:a large high-diversity benchmark for generic object tracking in the wild[J]. IEEE transactions on pattern analysis and machine intelligence, 2021, 43(5):1562-1577.
[19] CORNIA M, BARALDI L, SERRA G, et al. Predicting human eye fixations via an LSTM-based saliency attentive model[J]. IEEE transactions on image processing, 2018, 27(10):5142-5154.
[20] KRONER A, SENDEN M, DRIESSENS K, et al. Contextual encoder-decoder network for visual saliency prediction[J]. Neural networks, 2020, 129:261-270.
[21] WANG Wenguan, SHEN Jianbing. Deep visual attention prediction[J]. IEEE transactions on image processing, 2018, 27(5):2368-2378.
[22] CORNIA M, BARALDI L, SERRA G, et al. A deep multi-level network for saliency prediction[C]//2016 23rd International Conference on Pattern Recognition (ICPR). Cancun, Mexico, 2016:3488-3493.
[23] SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[EB/OL]. (2020?12?19)[2021?01?10] https://arxiv.org/abs/1409.1556, 2014.
[24] LONG J, SHELHAMER E, DARRELL T. Fully convolutional networks for semantic segmentation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Boston, USA, 2015:3431-3440.
[25] JIANG Ming, HUANG Shengsheng, DUAN Juanyong, et al. SALICON:saliency in context[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Boston, USA, 2015:1072-1080.
[26] JETLEY S, MURRAY N, VIG E. End-to-end saliency mapping via probability distribution prediction[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, USA, 2016:5753-5761.
[27] KINGMA D P, BA J. Adam:a method for stochastic optimization[EB/OL]. (2020?12?29)[2021?01?10] https://arxiv.org/abs/1412.6980, 2014.
[28] MUELLER M, SMITH N, GHANEM B. A benchmark and simulator for UAV tracking[C]//14th European Conference on Computer Vision. Amsterdam, The Netherlands, 2016:445-461.
[29] LI Yang, ZHU Jianke. A scale adaptive kernel correlation filter tracker with feature integration[C]//European Conference on Computer Vision. Zurich, Switzerland, 2014:254-265.
[30] DANELLJAN M, H?GER G, KHAN F S, et al. Learning spatially regularized correlation filters for visual tracking[C]//Proceedings of the IEEE International Conference on Computer Vision. Santiago, Chile, 2015:4310-4318.
[31] HARE S, GOLODETZ S, SAFFARI A, et al. Struck:structured output tracking with kernels[J]. IEEE transactions on pattern analysis and machine intelligence, 2016, 38(10):2096-2109.
Similar References:

Memo

-

Last Update: 2021-06-25

Copyright © CAAI Transactions on Intelligent Systems