[1]周士琪,王耀南,钟杭.融合视觉显著性再检测的孪生网络无人机目标跟踪算法[J].智能系统学报,2021,16(3):584-594.[doi:10.11992/tis.202101035]
ZHOU Shiqi,WANG Yaonan,ZHONG Hang.Siamese network combined with visual saliency re-detection for UAV object tracking[J].CAAI Transactions on Intelligent Systems,2021,16(3):584-594.[doi:10.11992/tis.202101035]
点击复制
ZHOU Shiqi,WANG Yaonan,ZHONG Hang.Siamese network combined with visual saliency re-detection for UAV object tracking[J].CAAI Transactions on Intelligent Systems,2021,16(3):584-594.[doi:10.11992/tis.202101035]
融合视觉显著性再检测的孪生网络无人机目标跟踪算法
《智能系统学报》[ISSN 1673-4785/CN 23-1538/TP] 卷:
16
期数:
2021年第3期
页码:
584-594
栏目:
人工智能院长论坛
出版日期:
2021-05-05
- Title:
- Siamese network combined with visual saliency re-detection for UAV object tracking
- Keywords:
- drone; computer vision; object tracking; MobileNetV2; siamese network; saliency detection; target occlusion; feature fusion
- 分类号:
- TP242
- 摘要:
- 针对旋翼飞行器在跟踪过程中目标尺度变化、快速运动、视角变化等问题，本文提出了一种基于MobileNetV2的孪生网络目标跟踪算法，可在无人机机载处理器上实时运行。该算法主要包含目标得分估计模块与目标尺度估计模块两个部分。结合多特征融合的策略，可准确预测出目标位置与目标框IoU，同时以目标框IoU为指导，利用梯度上升法对目标框进行迭代修正，进一步提升预测精度。针对完全遮挡而导致的目标跟丢问题，本文设计了一个基于视觉显著性的目标再检测算法，该算法可实时高效地预测出图像的显著性区域，以指导对目标的再检测，进而恢复跟踪。最后，通过标准无人机跟踪数据集测试与实际无人机跟踪实验，验证了算法的可行性。
- Abstract:
- Considering the problems associated with rotorcraft trackings, such as target scale variation, fast motion, and viewpoint change, this paper proposes a siamese network-based target tracking algorithm using MobileNetV2, which can run in real-time on an onboard UAV processor. The algorithm consists of target score and scale estimation modules. Combined with the multifeature fusion strategy, the target position and target box IoU were accurately predicted. At the same time, by employing the IoU, the gradient ascent method was used to iteratively modify the target bounding box to further improve the prediction accuracy. In addition, to solve the problem of target loss caused by full occlusion, a re-detection algorithm based on visual saliency detection was developed, which efficiently predicted the saliency map of the image in real-time to guide the re-detection of the target and resume tracking. Finally, the feasibility of the algorithm was verified by comparing the standard UAV tracking dataset and the actual UAV tracking experiment.
- 参考文献/References:
-
[1] 徐怀宇, 黄伟, 董明超, 等. 无人机目标跟踪综述[J]. 网络新媒体技术, 2019, 8(5):11-20
XU Huaiyu, HUANG Wei, DONG Mingchao, et al. Overview of UAV object tracking[J]. Journal of network new media, 2019, 8(5):11-20
[2] 刘芳, 孙亚楠, 王洪娟, 等. 基于残差学习的自适应无人机目标跟踪算法[J]. 北京航空航天大学学报, 2020, 46(10):1874-1882
LIU Fang, SUN Yanan, WANG Hongjuan, et al. Adaptive UAV target tracking algorithm based on residual learning[J]. Journal of Beijing University of Aeronautics and Astronautics, 2020, 46(10):1874-1882
[3] HENRIQUES J F, CASEIRO R, MARTINS P, et al. Exploiting the circulant structure of tracking-by-detection with kernels[C]//12th European Conference on Computer Vision. Florence, Italy, 2012:702-715.
[4] HENRIQUES J F, CASEIRO R, MARTINS P, et al. High-speed tracking with kernelized correlation filters[J]. IEEE transactions on pattern analysis and machine intelligence, 2015, 37(3):583-596.
[5] DANELLJAN M, ROBINSON A, KHAN F S, et al. Beyond correlation filters:learning continuous convolution operators for visual tracking[C]//14th European Conference on Computer Vision. Amsterdam, The Netherlands, 2016:472-488.
[6] DANELLJAN M, BHAT G, KHAN F S, et al. Eco:efficient convolution operators for tracking[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, USA, 2017:6638-6646.
[7] TAO Ren, GAVVES E, SMEULDERS A W M. Siamese instance search for tracking[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, USA, 2016:1420-1429.
[8] BERTINETTO L, VALMADRE J, HENRIQUES J F, et al. Fully-convolutional Siamese networks for object tracking[C]//European Conference on Computer Vision. Amsterdam, The Netherlands, 2016:850-865.
[9] VALMADRE J, BERTINETTO L, HENRIQUES J, et al. End-to-end representation learning for correlation filter based tracking[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, USA, 2017:2805-2813.
[10] LI Bo, YAN Junjie, WU Wei, et al. High performance visual tracking with Siamese region proposal network[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, USA, 2018:8971-8980.
[11] REN Shaoqing, HE Kaiming, GIRSHICK R, et al. Faster R-CNN:towards real-time object detection with region proposal networks[J]. IEEE transactions on pattern analysis and machine intelligence, 2015, 39(6):1137-1149.
[12] JIANG Borui, LUO Ruixuan, MAO Jiayuan, et al. Acquisition of localization confidence for accurate object detection[C]//Proceedings of the European Conference on Computer Vision (ECCV). Munich, Germany, 2018:816-832.
[13] SANDLER M, HOWARD A, ZHU Menglong, et al. MobileNetv2:inverted residuals and linear bottlenecks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake City, USA, 2018:4510-4520.
[14] LI Bo, WU Wei, WANG Qiang, et al. Siamrpn++:evolution of Siamese visual tracking with very deep networks[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach, USA, 2019:4277-4286.
[15] YU F, KOLTUN V. Multi-scale context aggregation by dilated convolutions[EB/OL]. (2020?12?29)[2021?01?10] https://arxiv.org/abs/1511.07122, 2016.
[16] HE Kaiming, ZHANG Xiangyu, REN Shaoqing, et al. Deep residual learning for image recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, USA, 2016:770-778.
[17] FAN Heng, LIN Liting, YANG Fan, et al. LaSOT:a high-quality benchmark for large-scale single object tracking[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach, USA, 2019:5369-5378.
[18] HUANG Lianghua, ZHAO Xin, HUANG Kaiqi. GOT-10k:a large high-diversity benchmark for generic object tracking in the wild[J]. IEEE transactions on pattern analysis and machine intelligence, 2021, 43(5):1562-1577.
[19] CORNIA M, BARALDI L, SERRA G, et al. Predicting human eye fixations via an LSTM-based saliency attentive model[J]. IEEE transactions on image processing, 2018, 27(10):5142-5154.
[20] KRONER A, SENDEN M, DRIESSENS K, et al. Contextual encoder-decoder network for visual saliency prediction[J]. Neural networks, 2020, 129:261-270.
[21] WANG Wenguan, SHEN Jianbing. Deep visual attention prediction[J]. IEEE transactions on image processing, 2018, 27(5):2368-2378.
[22] CORNIA M, BARALDI L, SERRA G, et al. A deep multi-level network for saliency prediction[C]//2016 23rd International Conference on Pattern Recognition (ICPR). Cancun, Mexico, 2016:3488-3493.
[23] SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[EB/OL]. (2020?12?19)[2021?01?10] https://arxiv.org/abs/1409.1556, 2014.
[24] LONG J, SHELHAMER E, DARRELL T. Fully convolutional networks for semantic segmentation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Boston, USA, 2015:3431-3440.
[25] JIANG Ming, HUANG Shengsheng, DUAN Juanyong, et al. SALICON:saliency in context[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Boston, USA, 2015:1072-1080.
[26] JETLEY S, MURRAY N, VIG E. End-to-end saliency mapping via probability distribution prediction[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, USA, 2016:5753-5761.
[27] KINGMA D P, BA J. Adam:a method for stochastic optimization[EB/OL]. (2020?12?29)[2021?01?10] https://arxiv.org/abs/1412.6980, 2014.
[28] MUELLER M, SMITH N, GHANEM B. A benchmark and simulator for UAV tracking[C]//14th European Conference on Computer Vision. Amsterdam, The Netherlands, 2016:445-461.
[29] LI Yang, ZHU Jianke. A scale adaptive kernel correlation filter tracker with feature integration[C]//European Conference on Computer Vision. Zurich, Switzerland, 2014:254-265.
[30] DANELLJAN M, H?GER G, KHAN F S, et al. Learning spatially regularized correlation filters for visual tracking[C]//Proceedings of the IEEE International Conference on Computer Vision. Santiago, Chile, 2015:4310-4318.
[31] HARE S, GOLODETZ S, SAFFARI A, et al. Struck:structured output tracking with kernels[J]. IEEE transactions on pattern analysis and machine intelligence, 2016, 38(10):2096-2109.
- 相似文献/References:
-
[1]夏 凡,王 宏.基于局部异常行为检测的欺骗识别研究[J].智能系统学报,2007,2(5):12.
XIA Fan,WANG Hong.Methodologies for deception detection based on abnormal b ehavior[J].CAAI Transactions on Intelligent Systems,2007,2():12.
[2]秦世引,潘宇雄,苏善伟.小型无人机编队飞行的控制律设计与仿真[J].智能系统学报,2009,4(3):218.
QIN Shi-yin,PAN Yu-xiong,SU Shan-wei.Design and simulation of formation flight control laws for small unmanned aerial vehicles[J].CAAI Transactions on Intelligent Systems,2009,4():218.
[3]杨 戈,刘 宏.视觉跟踪算法综述[J].智能系统学报,2010,5(2):95.
YANG Ge,LIU Hong.Survey of visual tracking algorithms[J].CAAI Transactions on Intelligent Systems,2010,5():95.
[4]朱杰斌,秦世引.无人机编队飞行的分布式控制策略与控制器设计[J].智能系统学报,2010,5(5):392.[doi:10.3969/j.issn.1673-4785.2010.05.003]
ZHU Jie-bin,QIN Shi-yin.Distributed control strategy and controller design for UAV formation flight[J].CAAI Transactions on Intelligent Systems,2010,5():392.[doi:10.3969/j.issn.1673-4785.2010.05.003]
[5]刘宏,李哲媛,许超.视错觉现象的分类和研究进展[J].智能系统学报,2011,6(1):1.
LIU Hong,LI Zheyuan,XU Chao.The categories and research advances of visual illusions[J].CAAI Transactions on Intelligent Systems,2011,6():1.
[6]刘敏,邹杰,冯星,等.人工蜂群算法的无人机航路规划与平滑[J].智能系统学报,2011,6(4):344.
LIU Min,ZOU Jie,FENG Xing,et al.Smooth trajectory planning of an unmanned aerial vehicleusing an artificial bee colony algorithm[J].CAAI Transactions on Intelligent Systems,2011,6():344.
[7]叶果,程洪,赵洋.电影中吸烟活动识别[J].智能系统学报,2011,6(5):440.
YE Guo,CHENG Hong,ZHAO Yang.moking recognition in movies[J].CAAI Transactions on Intelligent Systems,2011,6():440.
[8]史晓鹏,何为,韩力群.采用Hough变换的道路边界检测算法[J].智能系统学报,2012,7(1):81.
SHI Xiaopeng,HE Wei,HAN Liqun.A road edge detection algorithm based on the Hough transform[J].CAAI Transactions on Intelligent Systems,2012,7():81.
[9]顾照鹏,刘宏.单目视觉同步定位与地图创建方法综述[J].智能系统学报,2015,10(4):499.[doi:10.3969/j.issn.1673-4785.201503003]
GU Zhaopeng,LIU Hong.A survey of monocular simultaneous localization and mapping[J].CAAI Transactions on Intelligent Systems,2015,10():499.[doi:10.3969/j.issn.1673-4785.201503003]
[10]赵军,於俊,汪增福.基于改进逆向运动学的人体运动跟踪[J].智能系统学报,2015,10(4):548.[doi:10.3969/j.issn.1673-4785.201403032]
ZHAO Jun,YU Jun,WANG Zengfu.Human motion tracking based on an improved inverse kinematics[J].CAAI Transactions on Intelligent Systems,2015,10():548.[doi:10.3969/j.issn.1673-4785.201403032]
[11]林淑彬,吴贵山,姚文勇,等.基于光照自适应动态一致性的无人机目标跟踪[J].智能系统学报,2022,17(6):1093.[doi:10.11992/tis.202110023]
LIN Shubin,WU Guishan,YAO Wenyong,et al.Unmanned aerial vehicles object tracking based on illumination adaptive dynamic consistency[J].CAAI Transactions on Intelligent Systems,2022,17():1093.[doi:10.11992/tis.202110023]
备注/Memo
收稿日期:2021-01-28。
基金项目:国家自然科学基金项目（61733004）；中国博士后科学基金项目（2020M682555）
作者简介:周士琪，硕士研究生，主要研究方向为计算机视觉、移动机器人;王耀南，中国工程院院士，教授，博士生导师，主要研究方向为智能控制、计算机视觉、机器人控制。获国家技术发明奖二等奖、国家科学技术进步奖二等奖、教育部科学技术进步奖一等奖等。获授权发明专利70余项，发表学术论文200余篇，出版著作8部。;钟杭，博士研究生，主要研究方向为飞行器建模与控制、机器人技术
通讯作者:钟杭.E-mail:zhonghang@hnu.edu.cn
更新日期/Last Update:
2021-06-25