[1]谢家阳,王行健,史治国,等.动态云台摄像机无人机检测与跟踪算法[J].智能系统学报,2021,16(5):858-869.[doi:10.11992/tis.202103032]
XIE Jiayang,WANG Xingjian,SHI Zhiguo,et al.Drone detection and tracking in dynamic pan-tilt-zoom cameras[J].CAAI Transactions on Intelligent Systems,2021,16(5):858-869.[doi:10.11992/tis.202103032]
点击复制
XIE Jiayang,WANG Xingjian,SHI Zhiguo,et al.Drone detection and tracking in dynamic pan-tilt-zoom cameras[J].CAAI Transactions on Intelligent Systems,2021,16(5):858-869.[doi:10.11992/tis.202103032]
动态云台摄像机无人机检测与跟踪算法
《智能系统学报》[ISSN 1673-4785/CN 23-1538/TP] 卷:
16
期数:
2021年第5期
页码:
858-869
栏目:
吴文俊人工智能技术发明奖一等奖
出版日期:
2021-09-05
- Title:
- Drone detection and tracking in dynamic pan-tilt-zoom cameras
- Keywords:
- anti-UAV; moving target detection; background motion compensation; object recognition; object tracking; deep learning; computer vision; convolutional neural network
- 分类号:
- TP391.4
- 摘要:
- 为应对小型无人机的黑飞、滥飞对个人隐私、公共安全造成的威胁，本文采用高清云台摄像机定点巡航的方式对近地动态复杂背景中的无人机进行检测与跟踪，并提出了一种适用于动态云台摄像机的闭环无人机检测与跟踪算法，包含检测与跟踪两种模式。在检测模式下，本文设计了一种基于运动背景补偿的运动目标检测算法来提取分类候选区域，然后利用基于神经网络结构搜索得到的轻量级卷积神经网络对候选区域进行分类识别，可在不缩小高清视频图像的条件下实现无人机检测；在跟踪模式下，本文提出了一种结合卡尔曼滤波的局部搜索区域重定位策略改进了核相关滤波跟踪算法，使之在高清云台伺服追踪过程中仍能对目标进行快速稳定的跟踪；为将检测模式与跟踪模式结合在闭环框架中，本文还提出了一种基于检测概率和跟踪响应图状态的自适应检测与跟踪切换机制。实验表明，本文算法可应用于定点巡航状态的高清云台摄像机，实现近地复杂动态背景中无人机的实时准确检测、识别与快速跟踪。
- Abstract:
- To cope with the threat to personal privacy and public safety caused by illegal uses of small unmanned aerial vehicles, we propose a closed-loop UAV detection and tracking algorithm suitable for dynamic PTZ camera, which uses the fixed-point cruise mode of high-definition PTZ camera to detect and track unmanned aerial vehicles in the near-earth dynamic complex background, including two modes of detection and tracking. In the detection mode, this paper designs a moving target detection algorithm based on moving background compensation to extract the classification candidate areas, and then uses the lightweight convolutional neural network based on neural network structure search to classify and identify the candidate areas, which can realize UAV detection without reducing the high-definition video image; In the tracking mode, this paper proposes a local search area relocation strategy combined with Kalman filter to improve the tracking algorithm of kernel correlation filter, so that it can still track the target quickly and stably in the servo tracking process of HD PTZ; In order to combine detection mode and tracking mode in a closed-loop framework, this paper also proposes an adaptive detection and tracking switching mechanism based on detection probability and tracking response graph state. Experiments show that the proposed algorithm can be applied to high-definition PTZ cameras in a fixed-point cruise state, and realize real-time accurate detection, recognition and fast tracking of UAV in complex dynamic background near the ground.
- 参考文献/References:
-
[1] SUN Yuyi, CHEN Jiming, HE Shibo, et al. High-confidence gateway planning and performance evaluation of a hybrid LoRa network[J]. IEEE internet of things journal, 2021, 8(2): 1071-1081.
[2] FLOREANO D, WOOD R J. Science, technology and the future of small autonomous drones[J]. Nature, 2015, 521(7553): 460-466.
[3] SHI Xiufang, YANG Chaoqun, XIE Weige, et al. Anti-drone system with multiple surveillance technologies: architecture, implementation, and challenges[J]. IEEE communications magazine, 2018, 56(4): 68-74.
[4] XIE Jiayang, YU Jin, WU Junfeng, et al. Adaptive switching spatial-temporal fusion detection for remote flying drones[J]. IEEE transactions on vehicular technology, 2020, 69(7): 6964-6976.
[5] DALAL N, TRIGGS B. Histograms of oriented gradients for human detection[C]//Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition. San Diego, USA, 2005: 886-893.
[6] REDMON J, FARHADI A. YOLOv3: an incremental improvement[J]. arXiv preprint arXiv: 1804.02767, 2018.
[7] LIU Wei, ANGUELOV D, ERHAN D, et al. SSD: single shot MultiBox detector[C]//Proceedings of the 14th European Conference on Computer Vision. Amsterdam, The Netherlands, 2016: 21-37.
[8] LAW H, DENG Jia. CornerNet: detecting objects as paired keypoints[C]//Proceedings of the 15th European Conference on Computer Vision. Munich, Germany, 2018: 765-781.
[9] DUAN Kaiwen, BAI Song, XIE Lingxi, et al. CenterNet: keypoint triplets for object detection[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. Seoul, Korea (South), 2019: 6568-6577.
[10] CARION N, MASSA F, SYNNAEVE G, et al. End-to-end object detection with transformers[C]//Proceedings of the 16th European Conference on Computer Vision. Glasgow, UK, 2020: 213-229.
[11] GIRSHICK R, DONAHUE J, DARRELL T, et al. Rich feature hierarchies for accurate object detection and semantic segmentation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Columbus, USA, 2014: 580-587.
[12] FELZENSZWALB P F, HUTTENLOCHER D P. Efficient graph-based image segmentation[J]. International journal of computer vision, 2004, 59(2): 167-181.
[13] UIJLINGS J R R, VAN DE SANDE K E A, GEVERS T, et al. Selective search for object recognition[J]. International journal of computer vision, 2013, 104(2): 154-171.
[14] ENDRES I, HOIEM D. Category independent object proposals[C]//Proceedings of the 11th European Conference on Computer Vision. Heraklion, Greece, 2010: 575-588.
[15] ZITNICK C L, DOLLáR P. Edge boxes: locating object proposals from edges[C]//Proceedings of the 13th European Conference on Computer Vision. Zurich, Switzerland, 2014: 391-405.
[16] REN Shaoqing, HE Kaiming, GIRSHICK R, et al. Faster R-CNN: towards real-time object detection with region proposal networks[C]//Proceedings of the 28th International Conference on Neural Information Processing Systems. Montreal, Quebec, Canada, 2015: 91-99.
[17] CANZIANI A, PASZKE A, CULURCIELLO E. An analysis of deep neural network models for practical applications[J]. arXiv preprint arXiv: 1605.07678, 2016.
[18] TAN Mingxing, LE Q V. EfficientNet: rethinking model scaling for convolutional neural networks[C]//Proceedings of the 36th International Conference on Machine Learning. Long Beach, USA, 2019: 6105-6114.
[19] ZHANG Lei, XU Ke, YU Shiqi, et al. An effective approach for active tracking with a PTZ camera[C]//Proceedings of IEEE International Conference on Robotics and Biomimetics. Tianjin, China, 2010: 1768-1773.
[20] BOLME D S, BEVERIDGE J R, DRAPER B A, et al. Visual object tracking using adaptive correlation filters[C]//Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. San Francisco, USA, 2010: 2544-2550.
[21] HENRIQUES J F, CASEIRO R, MARTINS P, et al. High-speed tracking with kernelized correlation filters[J]. IEEE transactions on pattern analysis and machine intelligence, 2015, 37(3): 583-596.
[22] DANELLJAN M, H?GER G, KHAN F S, et al. Discriminative scale space tracking[J]. IEEE transactions on pattern analysis and machine intelligence, 2017, 39(8): 1561-1575.
[23] DANELLJAN M, ROBINSON A, KHAN F S, et al. Beyond correlation filters: learning continuous convolution operators for visual tracking[C]//Proceedings of the 14th European Conference on Computer Vision. Amsterdam, The Netherlands, 2016: 472-488.
[24] DANELLJAN M, BHAT G, SHAHBAZ KHAN F, et al. ECO: efficient convolution operators for tracking[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, USA, 2017: 6931-6939.
[25] BERTINETTO L, VALMADRE J, HENRIQUES J F, et al. Fully-convolutional siamese networks for object tracking[C]//Proceedings of the European Conference on Computer Vision. Amsterdam, The Netherlands, 2016: 850-865.
[26] LI Bo, YAN Junjie, WU Wei, et al. High performance visual tracking with Siamese region proposal network[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, USA, 2018: 8971-8980.
[27] WANG Qiang, ZHANG Li, BERTINETTO L, et al. Fast online object tracking and segmentation: a unifying approach[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach, USA, 2019: 1328-1338.
[28] DéNIZ O, BUENO G, BERMEJO E, et al. Fast and accurate global motion compensation[J]. Pattern recognition, 2011, 44(12): 2887-2901.
[29] RUBLEE E, RABAUD V, KONOLIGE K, et al. ORB: an efficient alternative to SIFT or SURF[C]//Proceedings of International Conference on Computer Vision. Barcelona, Spain, 2011: 2564-2571.
[30] FISCHLER M A, BOLLES R C. Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography[J]. Communications of the ACM, 1981, 24(6): 381-395.
[31] BRUTZER S, H?FERLIN B, HEIDEMANN G. Evaluation of background subtraction techniques for video surveillance[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Colorado Springs, USA, 2011: 1937 -1944.
[32] CHENG Yuanhang, WANG Jing. A motion image detection method based on the inter-frame difference method[J]. Applied mechanics and materials, 2014, 490: 1283-1286.
[33] SERRA J, VINCENT L. An overview of morphological filtering[J]. Circuits, systems and signal processing, 1992, 11(1): 47-108.
[34] GRAHAM R L, YAO F F. Finding the convex hull of a simple polygon[J]. Journal of algorithms, 1983, 4(4): 324-331.
[35] HU Jie, SHEN Li, SUN Gang. Squeeze-and-excitation networks[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, USA, 2018: 7132-7141.
[36] GHIASI G, LIN T Y, LE Q V. DropBlock: a regularization method for convolutional networks[C]//Proceedings of 33nd Conference on Neural Information Processing Systems. Montréal, Canada, 2018: 10750-10760.
[37] FELZENSZWALB P F, GIRSHICK R B, MCALLESTER D, et al. Object detection with discriminatively trained part-based models[J]. IEEE transactions on pattern analysis and machine intelligence, 2010, 32(9): 1627-1645.
[38] HOERL A E, KANNARD R W, BALDWIN K F. Ridge regression: some simulations[J]. Communications in statistics, 1975, 4(2): 105-123.
[39] SCH?LKOPF B, SMOLA A J. Learning with kernels: support vector machines, regularization, optimization, and beyond[M]. Cambridge: MIT Press, 2002.
[40] GRAY R M. Toeplitz and circulant matrices: a review[J]. Foundations and trends? in communications and information theory, 2006, 2(3): 155-239.
[41] KIM Y, BANG H. Introduction to Kalman filter and its applications[M].GOVAERS F. Introduction and Implementations of the Kalman Filter. IntechOpen, 2019.
[42] GOYETTE N, JODOIN P M, PORIKLI F, et al. Changedetection.net: a new change detection benchmark dataset[C]//Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops. Providence, USA, 2012: 1-8.
[43] ALLEBOSCH G, DEBOEVERIE F, VEELAERT P, et al. EFIC: edge based foreground background segmentation and interior classification for dynamic camera viewpoints[C]//Proceedings of the 16th International Conference on Advanced Concepts for Intelligent Vision Systems. Catania, Italy, 2015: 130-141.
[44] JIANG Shengqin, LU Xiaobo. WeSamBE: a weight-sample-based method for background subtraction[J]. IEEE transactions on circuits and systems for video technology, 2018, 28(9): 2105-2115.
- 相似文献/References:
-
[1]周治平,李文慧.颜色和纹理特征的运动目标检测[J].智能系统学报,2015,10(5):729.[doi:10.11992/tis.201408034]
ZHOU Zhiping,LI Wenhui.Detection for moving targets based on color and texture features[J].CAAI Transactions on Intelligent Systems,2015,10():729.[doi:10.11992/tis.201408034]
备注/Memo
收稿日期:2021-03-23。
基金项目:国家自然科学基金项目(62088101，61772467)
作者简介:谢家阳，博士研究生，主要研究方向为计算机视觉、反无人机系统、深度学习、目标检测与跟踪;王行健，硕士研究生，主要研究方向为计算机视觉、深度学习、目标检测与跟踪;史治国，教授、博士生导师，主要研究方向为研究方向为信号处理及定位应用，物联网。IET Fellow、IEEE Network编委、IET Communications编委、Journal of The Franklin Institute编委，2020年获中国人工智能学会技术发明一等奖，2015年获教育部科技进步一等奖，2012年获浙江省科学技术二等奖。主持国家重点研发项目、国家自然科学基金项目、浙江省重点研发计划项目等多项，获授权发明专利80余项。发表学术论文100余篇
通讯作者:史治国.E-mail:@zju.edu.cn
更新日期/Last Update:
1900-01-01