[1]XIE Jiayang,WANG Xingjian,SHI Zhiguo,et al.Drone detection and tracking in dynamic pan-tilt-zoom cameras[J].CAAI Transactions on Intelligent Systems,2021,16(5):858-869.[doi:10.11992/tis.202103032]
Copy

Drone detection and tracking in dynamic pan-tilt-zoom cameras

CAAI Transactions on Intelligent Systems[ISSN 1673-4785/CN 23-1538/TP] Volume: 16 Number of periods: 2021 5 Page number: 858-869 Column: 吴文俊人工智能技术发明奖一等奖 Public date: 2021-09-05
Title:
Drone detection and tracking in dynamic pan-tilt-zoom cameras
Author(s):
XIE Jiayang1;  WANG Xingjian1;  SHI Zhiguo1;  WU Junfeng1;  CHEN Jiming1;  CHEN Qian2;  WANG Bin3
1. Faculty of Information Technology, Zhejiang University, Hangzhou 310027, China;
2. Shanghai Radio Equipment Research Institute, Shanghai 200090, China;
3. Hangzhou Hikvision Digital Technology Co., Ltd., Hangzhou 310052, China
Keywords:
anti-UAV; moving target detection; background motion compensation; object recognition; object tracking; deep learning; computer vision; convolutional neural network
CLC:
TP391.4
DOI:
10.11992/tis.202103032
Abstract:
To cope with the threat to personal privacy and public safety caused by illegal uses of small unmanned aerial vehicles, we propose a closed-loop UAV detection and tracking algorithm suitable for dynamic PTZ camera, which uses the fixed-point cruise mode of high-definition PTZ camera to detect and track unmanned aerial vehicles in the near-earth dynamic complex background, including two modes of detection and tracking. In the detection mode, this paper designs a moving target detection algorithm based on moving background compensation to extract the classification candidate areas, and then uses the lightweight convolutional neural network based on neural network structure search to classify and identify the candidate areas, which can realize UAV detection without reducing the high-definition video image; In the tracking mode, this paper proposes a local search area relocation strategy combined with Kalman filter to improve the tracking algorithm of kernel correlation filter, so that it can still track the target quickly and stably in the servo tracking process of HD PTZ; In order to combine detection mode and tracking mode in a closed-loop framework, this paper also proposes an adaptive detection and tracking switching mechanism based on detection probability and tracking response graph state. Experiments show that the proposed algorithm can be applied to high-definition PTZ cameras in a fixed-point cruise state, and realize real-time accurate detection, recognition and fast tracking of UAV in complex dynamic background near the ground.
References:
[1] SUN Yuyi, CHEN Jiming, HE Shibo, et al. High-confidence gateway planning and performance evaluation of a hybrid LoRa network[J]. IEEE internet of things journal, 2021, 8(2): 1071-1081.
[2] FLOREANO D, WOOD R J. Science, technology and the future of small autonomous drones[J]. Nature, 2015, 521(7553): 460-466.
[3] SHI Xiufang, YANG Chaoqun, XIE Weige, et al. Anti-drone system with multiple surveillance technologies: architecture, implementation, and challenges[J]. IEEE communications magazine, 2018, 56(4): 68-74.
[4] XIE Jiayang, YU Jin, WU Junfeng, et al. Adaptive switching spatial-temporal fusion detection for remote flying drones[J]. IEEE transactions on vehicular technology, 2020, 69(7): 6964-6976.
[5] DALAL N, TRIGGS B. Histograms of oriented gradients for human detection[C]//Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition. San Diego, USA, 2005: 886-893.
[6] REDMON J, FARHADI A. YOLOv3: an incremental improvement[J]. arXiv preprint arXiv: 1804.02767, 2018.
[7] LIU Wei, ANGUELOV D, ERHAN D, et al. SSD: single shot MultiBox detector[C]//Proceedings of the 14th European Conference on Computer Vision. Amsterdam, The Netherlands, 2016: 21-37.
[8] LAW H, DENG Jia. CornerNet: detecting objects as paired keypoints[C]//Proceedings of the 15th European Conference on Computer Vision. Munich, Germany, 2018: 765-781.
[9] DUAN Kaiwen, BAI Song, XIE Lingxi, et al. CenterNet: keypoint triplets for object detection[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. Seoul, Korea (South), 2019: 6568-6577.
[10] CARION N, MASSA F, SYNNAEVE G, et al. End-to-end object detection with transformers[C]//Proceedings of the 16th European Conference on Computer Vision. Glasgow, UK, 2020: 213-229.
[11] GIRSHICK R, DONAHUE J, DARRELL T, et al. Rich feature hierarchies for accurate object detection and semantic segmentation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Columbus, USA, 2014: 580-587.
[12] FELZENSZWALB P F, HUTTENLOCHER D P. Efficient graph-based image segmentation[J]. International journal of computer vision, 2004, 59(2): 167-181.
[13] UIJLINGS J R R, VAN DE SANDE K E A, GEVERS T, et al. Selective search for object recognition[J]. International journal of computer vision, 2013, 104(2): 154-171.
[14] ENDRES I, HOIEM D. Category independent object proposals[C]//Proceedings of the 11th European Conference on Computer Vision. Heraklion, Greece, 2010: 575-588.
[15] ZITNICK C L, DOLLáR P. Edge boxes: locating object proposals from edges[C]//Proceedings of the 13th European Conference on Computer Vision. Zurich, Switzerland, 2014: 391-405.
[16] REN Shaoqing, HE Kaiming, GIRSHICK R, et al. Faster R-CNN: towards real-time object detection with region proposal networks[C]//Proceedings of the 28th International Conference on Neural Information Processing Systems. Montreal, Quebec, Canada, 2015: 91-99.
[17] CANZIANI A, PASZKE A, CULURCIELLO E. An analysis of deep neural network models for practical applications[J]. arXiv preprint arXiv: 1605.07678, 2016.
[18] TAN Mingxing, LE Q V. EfficientNet: rethinking model scaling for convolutional neural networks[C]//Proceedings of the 36th International Conference on Machine Learning. Long Beach, USA, 2019: 6105-6114.
[19] ZHANG Lei, XU Ke, YU Shiqi, et al. An effective approach for active tracking with a PTZ camera[C]//Proceedings of IEEE International Conference on Robotics and Biomimetics. Tianjin, China, 2010: 1768-1773.
[20] BOLME D S, BEVERIDGE J R, DRAPER B A, et al. Visual object tracking using adaptive correlation filters[C]//Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. San Francisco, USA, 2010: 2544-2550.
[21] HENRIQUES J F, CASEIRO R, MARTINS P, et al. High-speed tracking with kernelized correlation filters[J]. IEEE transactions on pattern analysis and machine intelligence, 2015, 37(3): 583-596.
[22] DANELLJAN M, H?GER G, KHAN F S, et al. Discriminative scale space tracking[J]. IEEE transactions on pattern analysis and machine intelligence, 2017, 39(8): 1561-1575.
[23] DANELLJAN M, ROBINSON A, KHAN F S, et al. Beyond correlation filters: learning continuous convolution operators for visual tracking[C]//Proceedings of the 14th European Conference on Computer Vision. Amsterdam, The Netherlands, 2016: 472-488.
[24] DANELLJAN M, BHAT G, SHAHBAZ KHAN F, et al. ECO: efficient convolution operators for tracking[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, USA, 2017: 6931-6939.
[25] BERTINETTO L, VALMADRE J, HENRIQUES J F, et al. Fully-convolutional siamese networks for object tracking[C]//Proceedings of the European Conference on Computer Vision. Amsterdam, The Netherlands, 2016: 850-865.
[26] LI Bo, YAN Junjie, WU Wei, et al. High performance visual tracking with Siamese region proposal network[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, USA, 2018: 8971-8980.
[27] WANG Qiang, ZHANG Li, BERTINETTO L, et al. Fast online object tracking and segmentation: a unifying approach[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach, USA, 2019: 1328-1338.
[28] DéNIZ O, BUENO G, BERMEJO E, et al. Fast and accurate global motion compensation[J]. Pattern recognition, 2011, 44(12): 2887-2901.
[29] RUBLEE E, RABAUD V, KONOLIGE K, et al. ORB: an efficient alternative to SIFT or SURF[C]//Proceedings of International Conference on Computer Vision. Barcelona, Spain, 2011: 2564-2571.
[30] FISCHLER M A, BOLLES R C. Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography[J]. Communications of the ACM, 1981, 24(6): 381-395.
[31] BRUTZER S, H?FERLIN B, HEIDEMANN G. Evaluation of background subtraction techniques for video surveillance[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Colorado Springs, USA, 2011: 1937 -1944.
[32] CHENG Yuanhang, WANG Jing. A motion image detection method based on the inter-frame difference method[J]. Applied mechanics and materials, 2014, 490: 1283-1286.
[33] SERRA J, VINCENT L. An overview of morphological filtering[J]. Circuits, systems and signal processing, 1992, 11(1): 47-108.
[34] GRAHAM R L, YAO F F. Finding the convex hull of a simple polygon[J]. Journal of algorithms, 1983, 4(4): 324-331.
[35] HU Jie, SHEN Li, SUN Gang. Squeeze-and-excitation networks[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, USA, 2018: 7132-7141.
[36] GHIASI G, LIN T Y, LE Q V. DropBlock: a regularization method for convolutional networks[C]//Proceedings of 33nd Conference on Neural Information Processing Systems. Montréal, Canada, 2018: 10750-10760.
[37] FELZENSZWALB P F, GIRSHICK R B, MCALLESTER D, et al. Object detection with discriminatively trained part-based models[J]. IEEE transactions on pattern analysis and machine intelligence, 2010, 32(9): 1627-1645.
[38] HOERL A E, KANNARD R W, BALDWIN K F. Ridge regression: some simulations[J]. Communications in statistics, 1975, 4(2): 105-123.
[39] SCH?LKOPF B, SMOLA A J. Learning with kernels: support vector machines, regularization, optimization, and beyond[M]. Cambridge: MIT Press, 2002.
[40] GRAY R M. Toeplitz and circulant matrices: a review[J]. Foundations and trends? in communications and information theory, 2006, 2(3): 155-239.
[41] KIM Y, BANG H. Introduction to Kalman filter and its applications[M].GOVAERS F. Introduction and Implementations of the Kalman Filter. IntechOpen, 2019.
[42] GOYETTE N, JODOIN P M, PORIKLI F, et al. Changedetection.net: a new change detection benchmark dataset[C]//Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops. Providence, USA, 2012: 1-8.
[43] ALLEBOSCH G, DEBOEVERIE F, VEELAERT P, et al. EFIC: edge based foreground background segmentation and interior classification for dynamic camera viewpoints[C]//Proceedings of the 16th International Conference on Advanced Concepts for Intelligent Vision Systems. Catania, Italy, 2015: 130-141.
[44] JIANG Shengqin, LU Xiaobo. WeSamBE: a weight-sample-based method for background subtraction[J]. IEEE transactions on circuits and systems for video technology, 2018, 28(9): 2105-2115.
Similar References:

Memo

-

Last Update: 1900-01-01

Copyright © CAAI Transactions on Intelligent Systems