[1]HUANG Yucheng,XIAO Ziwang,WU Danfeng,et al.Spatiotemporal fusion and discriminative augmentation for improved Siamese tracking[J].CAAI Transactions on Intelligent Systems,2024,19(5):1218-1227.[doi:10.11992/tis.202306005]
Copy

Spatiotemporal fusion and discriminative augmentation for improved Siamese tracking

CAAI Transactions on Intelligent Systems[ISSN 1673-4785/CN 23-1538/TP] Volume: 19 Number of periods: 2024 5 Page number: 1218-1227 Column: 学术论文—智能系统 Public date: 2024-09-05
Title:
Spatiotemporal fusion and discriminative augmentation for improved Siamese tracking
Author(s):
HUANG Yucheng1;  XIAO Ziwang1;  WU Danfeng2;  HAMDULLA A1
1. School of Computer Science and Technology, Xinjiang University, Urumqi 830046, China;
2. College of Robotics, Beijing Union University, Beijing 100101, China
Keywords:
artificial intelligence; deep learning; computer vision; object tracking; neural network; Transformer; feature fusion; temporal modeling
CLC:
TP18
DOI:
10.11992/tis.202306005
Abstract:
The development of Siamese trackers has considerably enhanced the tracking performance. However, current trackers have difficulty accurately describing changes in the appearance of the target, which results in performance degradation under occlusion and scale changes. Cluttered backgrounds can interfere with the tracker response and mislead target localization. Therefore, two Transformer-based modules are introduced to improve the performance of Siamese trackers. Specifically, the spatiotemporal fusion module uses a cross attention mechanism for global feature association to iteratively accumulate historical clues for improving the robustness of the target appearance change. Meanwhile, the discriminative enhancement module associates semantic information between the target and the search area to enhance the target discrimination capability. In addition, adaptive weighted channel-spatial fusion is utilized to fully explore the spatiotemporal information of spatial distribution and semantic similarity. The proposed module can be embedded into mainstream Siamese trackers and exhibits superior performance on public datasets.
References:
[1] 韩瑞泽, 冯伟, 郭青, 等. 视频单目标跟踪研究进展综述[J]. 计算机学报, 2022, 45(9): 1877-1907.
HAN Ruize, FENG Wei, GUO Qing, et al. Single object tracking research: a survey[J]. Chinese journal of computers, 2022, 45(9): 1877-1907.
[2] 王梦亭, 杨文忠, 武雍智. 基于孪生网络的单目标跟踪算法综述[J]. 计算机应用, 2023, 43(3): 661-673.
WANG Mengting, YANG Wenzhong, WU Yongzhi. Survey of single target tracking algorithms based on Siamese network[J]. Journal of computer applications, 2023, 43(3): 661-673.
[3] 程旭, 刘丽华, 王莹莹, 等. 基于多帧一致性修正的自监督孪生网络目标跟踪方法[J]. 计算机学报, 2022, 45(12): 2544-2560.
CHENG Xu, LIU Lihua, WANG Yingying, et al. A multi-frame consistency correction based self-supervised Siamese network method for object tracking[J]. Chinese journal of computers, 2022, 45(12): 2544-2560.
[4] 周春月, 颜巧. 基于高分辨率孪生网络的单目标追踪算法[J]. 北京交通大学学报, 2020, 44(5): 104-110.
ZHOU Chunyue, YAN Qiao. Single object tracking algorithm based on high-resolution Siamese network[J]. Journal of Beijing Jiaotong University, 2020, 44(5): 104-110.
[5] BERTINETTO L, VALMADRE J, HENRIQUES J F, et al. Fully-convolutional Siamese networks for object tracking[C]//Lecture Notes in Computer Science. Cham: Springer, 2016: 850-865.
[6] LI Bo, YAN Junjie, WU Wei, et al. High performance visual tracking with Siamese region proposal network[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 8971-8980.
[7] GUO Qing, FENG Wei, ZHOU Ce, et al. Learning dynamic Siamese network for visual object tracking[C]//2017 IEEE International Conference on Computer Vision. Venice: IEEE, 2017: 1781-1789.
[8] ZHANG Yunhua, WANG Lijun, QI Jinqing, et al. Structured siamese network for real-time visual tracking[C]//European Conference on Computer Vision. Cham: Springer, 2018: 355-370.
[9] 程语嫣, 张九根, 杨圣伟. 多特征融合和尺度适应的相关滤波跟踪算法[J]. 计算机工程与设计, 2020, 41(12): 3444-3450.
CHENG Yuyan, ZHANG Jiugen, YANG Shengwei. Correlation filtering tracking algorithm based on multi-feature fusion and scale adaptation[J]. Computer engineering and design, 2020, 41(12): 3444-3450.
[10] 茅正冲, 沈雪松. 基于多特征融合的相关滤波跟踪算法[J]. 计算机与数字工程, 2020, 48(11): 2645-2648, 2782.
MAO Zhengchong, SHEN Xuesong. Correlation filter tracking algorithm based on multi-feature fusion[J]. Computer & digital engineering, 2020, 48(11): 2645-2648, 2782.
[11] 蒲磊, 冯新喜, 侯志强, 等. 基于自适应背景选择和多检测区域的相关滤波算法[J]. 电子与信息学报, 2020, 42(12): 3061-3067.
PU Lei, FENG Xinxi, HOU Zhiqiang, et al. Correlation filter algorithm based on adaptive context selection and multiple detection areas[J]. Journal of electronics & information technology, 2020, 42(12): 3061-3067.
[12] GIRSHICK R. Fast R-CNN[C]//2015 IEEE International Conference on Computer Vision. Santiago: IEEE, 2015: 1440-1448.
[13] ZHANG Zhipeng, PENG Houwen. Deeper and wider Siamese networks for real-time visual tracking[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach: IEEE, 2019: 4586-4595.
[14] LI Bo, WU Wei, WANG Qiang, et al. SiamRPN: evolution of Siamese visual tracking with very deep networks[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach: IEEE, 2019: 4277-4286.
[15] GUO Dongyan, WANG Jun, CUI Ying, et al. SiamCAR: Siamese fully convolutional classification and regression for visual tracking[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle: IEEE, 2020: 6268-6276.
[16] ZHU Zheng, WANG Qiang, LI Bo, et al. Distractor-aware siamese networks for visual object tracking[C]//European Conference on Computer Vision. Cham: Springer, 2018: 103–119.
[17] WANG Qiang, TENG Zhu, XING Junliang, et al. Learning attentions: residual attentional Siamese network for high performance online visual tracking[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 4854-4863.
[18] ZHANG Lichao, GONZALEZ-GARCIA A, VAN DE WEIJER J, et al. Learning the model update for Siamese trackers[C]//2019 IEEE/CVF International Conference on Computer Vision. Seoul: IEEE, 2019: 4009-4018.
[19] ZHU Zheng, WU Wei, ZOU Wei, et al. End-to-end flow correlation tracking with spatial-temporal attention[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 548-557.
[20] LI Feng, TIAN Cheng, ZUO Wangmeng, et al. Learning spatial-temporal regularized correlation filters for visual tracking[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 4904-4913.
[21] LUKE?IC A, VOJíR T, ZAJC L C, et al. Discriminative correlation filter with channel and spatial reliability[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE, 2017: 4847-4856.
[22] DANELLJAN M, H?GER G, KHAN F S, et al. Learning spatially regularized correlation filters for visual tracking[C]//2015 IEEE International Conference on Computer Vision. Santiago: IEEE, 2015: 4310-4318.
[23] VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems. Long Beach: ACM, 2017: 6000–6010.
[24] ZHANG Shifeng, CHI Cheng, YAO Yongqiang, et al. Bridging the gap between anchor-based and anchor-free detection via adaptive training sample selection[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle: IEEE, 2020: 9756-9765.
[25] TIAN Zhi, SHEN Chunhua, CHEN Hao, et al. FCOS: fully convolutional one-stage object detection[C]//2019 IEEE/CVF International Conference on Computer Vision. Seoul: IEEE, 2019: 9626-9635.
[26] HE Kaiming, ZHANG Xiangyu, REN Shaoqing, et al. Deep residual learning for image recognition[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016: 770-778.
[27] DENG Jia, DONG Wei, SOCHER R, et al. ImageNet: a large-scale hierarchical image database[C]//2009 IEEE Conference on Computer Vision and Pattern Recognition. Miami: IEEE, 2009: 248-255.
[28] FAN Heng, LIN Liting, YANG Fan, et al. LaSOT: a high-quality benchmark for large-scale single object tracking[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach: IEEE, 2019: 5369-5378.
[29] RUSSAKOVSKY O, DENG Jia, SU Hao, et al. ImageNet large scale visual recognition challenge[J]. International journal of computer vision, 2015, 115(3): 211-252.
[30] WU Yi, LIM J, YANG M H. Online object tracking: a benchmark[C]//2013 IEEE Conference on Computer Vision and Pattern Recognition. Portland: IEEE, 2013: 2411-2418.
[31] Kristan M , Leonardis A , Matas J , et al. The sixth visual object tracking VOT2018 challenge results[C]//European Conference on Computer Vision Workshops. Cham: Springer, 2018: 3–53.
[32] HUANG Lianghua, ZHAO Xin, HUANG Kaiqi. GOT-10k: a large high-diversity benchmark for generic object tracking in the wild[J]. IEEE transactions on pattern analysis and machine intelligence, 2021, 43(5): 1562-1577.
[33] LI Peixia, CHEN Boyu, OUYANG Wanli, et al. GradNet: gradient-guided network for visual object tracking[C]//2019 IEEE/CVF International Conference on Computer Vision. Seoul: IEEE, 2019: 6161-6170.
[34] HU Weiming, WANG Qiang, ZHANG Li, et al. SiamMask: a framework for fast online object tracking and segmentation[J]. IEEE transactions on pattern analysis and machine intelligence, 2023, 45(3): 3072-3089.
[35] YAN Bin, ZHAO Haojie, WANG Dong, et al. ‘skimming-perusal’ tracking: a framework for real-time and robust long-term tracking[C]//2019 IEEE/CVF International Conference on Computer Vision. Seoul: IEEE, 2019: 2385-2393.
[36] BHAT G, JOHNANDER J, DANELLJAN M, et al. Unveiling the power of deep tracking[C]//European Conference on Computer Vision. Cham: Springer, 2018: 493-509.
[37] DANELLJAN M, BHAT G, KHAN F S, et al. ECO: efficient convolution operators for tracking[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE, 2017: 6931-6939.
[38] DANELLJAN M, BHAT G, KHAN F S, et al. ATOM: accurate tracking by overlap maximization[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach: IEEE, 2019: 4655-4664.
Similar References:

Memo

-

Last Update: 2024-09-05

Copyright © CAAI Transactions on Intelligent Systems