[1]李想,张婷,刘兆英,等.基于显著性导引孪生网络的红外船目标跟踪[J].智能系统学报,2024,19(6):1428-1437.[doi:10.11992/tis.202306004]
LI Xiang,ZHANG Ting,LIU Zhaoying,et al.Infrared ship target tracking based on saliency guided siamese network[J].CAAI Transactions on Intelligent Systems,2024,19(6):1428-1437.[doi:10.11992/tis.202306004]
点击复制
LI Xiang,ZHANG Ting,LIU Zhaoying,et al.Infrared ship target tracking based on saliency guided siamese network[J].CAAI Transactions on Intelligent Systems,2024,19(6):1428-1437.[doi:10.11992/tis.202306004]
基于显著性导引孪生网络的红外船目标跟踪
《智能系统学报》[ISSN 1673-4785/CN 23-1538/TP] 卷:
19
期数:
2024年第6期
页码:
1428-1437
栏目:
学术论文—机器感知与模式识别
出版日期:
2024-12-05
- Title:
- Infrared ship target tracking based on saliency guided siamese network
- Keywords:
- infrared ship tracking; siamese network; salient object detection; feature fusion; shared correlation; multi-task learning; convolutional neural network; deep learning
- 分类号:
- TP391
- 摘要:
- 由于红外图像特征判别力低，现有方法很难从背景中分割目标。而受到红外成像机制的影响，红外目标通常具有较高的局部显著性，因此本文提出一种基于显著性导引孪生网络的跟踪方法，以目标的显著性信息为先验知识，引导跟踪模型准确地定位目标。本文提出显著性预测网络和显著性增强网络。显著性预测网络用于获得搜索区域的全局显著性图，并将其输入到显著性增强网络以增强目标，提高模型的判别能力；设计了一个共享互相关结构来计算模板图像特征与显著性增强后的搜索区域特征之间的相似度，通过分类和回归两个任务共享互相关特征图，同时提升模型的效率和性能；由于目前缺少公开的红外船跟踪数据集，本文构建了一个新的红外船目标跟踪数据集(infrared ship dataset, ISD)，共包括16种不同类型的船，7800幅带有标签的视频帧。在ISD上的实验结果显示，与其他18个常用跟踪模型相比，本模型达到了最高的准确率和最高的期望平均交并比。
- Abstract:
- Infrared images often have features with low discriminative power, making it difficult to segment targets from the background using existing methods. Owing to the nature of infrared imaging, targets usually exhibit high local saliency. To address this, we propose a method for tracking infrared ship targets using saliency-guided Siamese networks (SGSiam). This approach uses the target’s saliency as prior knowledge to guide the tracking model for precise target localization. First, this study presents a saliency prediction network and a saliency enhancement network. The saliency prediction network generates a global saliency map of the search area, which is input into the saliency enhancement network to strengthen the target and improve the discriminative ability of the model. Second, a shared cross-correlation architecture is designed to calculate the similarity between the template image features and the saliency-enhanced search region features, thus improving the model efficiency and performance through shared feature maps for classification and regression tasks. Finally, owing to the lack of publicly available infrared ship tracking data sets, we introduce a new infrared ship data set (ISD), which includes 16 different ship types and 7800 video frames with manual annotations. Experimental results on ISD show that our model outperforms 18 commonly used tracking models, achieving the highest accuracy and the highest expected average overlap score.
- 参考文献/References:
-
[1] YANG Xi, WANG Yan, WANG Nannan, et al. An enhanced SiamMask network for coastal ship tracking[J]. IEEE transactions on geoscience and remote sensing, 2022, 60: 5612011.
[2] LI Xin, LIU Qiao, FAN Nana, et al. Hierarchical spatial-aware Siamese network for thermal infrared object tracking[J]. Knowledge-based systems, 2019, 166: 71-81.
[3] CHEN Ruimin, LIU Shijian, MIAO Zhuang, et al. GFSNet: generalization-friendly Siamese network for thermal infrared object tracking[J]. Infrared physics and technology, 2022, 123: 104190.
[4] 刘万军, 孙虎, 姜文涛. 自适应特征选择的相关滤波跟踪算法[J]. 光学学报, 2019, 39(6): 242-255.
LIU Wanjun, SUN Hu, JIANG Wentao. Correlation filter tracking algorithm for adaptive feature selection[J]. Acta optica sinica, 2019, 39(6): 242-255.
[5] 姜文涛, 孟庆姣. 自适应时空正则化的相关滤波目标跟踪[J]. 智能系统学报, 2023, 18(4): 754-763.
JIANG Wentao,MENG Qingjiao. Correlation filter tracking for adaptive spatiotemporal regularization[J]. CAAI transactions on intelligent systems, 2023, 18(4): 754-763.
[6] BERTINETTO L, VALMADRE J, HENRIQUES J F, et al. Fully-convolutional Siamese networks for object tracking[C]//European Conference on Computer Vision. Cham: Springer, 2016: 850-865.
[7] HANG Zhipeng, PENG Houwen. Deeper and wider Siamese networks for real-time visual tracking[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach: IEEE, 2019: 4586-4595.
[8] WANG Qiang, ZHANG Li, BERTINETTO L, et al. Fast online object tracking and segmentation: a unifying approach[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach: IEEE, 2019: 1328-1338.
[9] LIN T Y, DOLLáR P, GIRSHICK R, et al. Feature pyramid networks for object detection[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE, 2017: 936-944.
[10] LI Bo, YAN Junjie, WU Wei, et al. High performance visual tracking with Siamese region proposal network[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 8971-8980.
[11] REN Shaoqing, HE Kaiming, GIRSHICK R, et al. Faster R-CNN: towards real-time object detection with region proposal networks[J]. IEEE transactions on pattern analysis and machine intelligence, 2017, 39(6): 1137-1149.
[12] KRIZHEVSKY A, SUTSKEVER I, HINTON G. Imagenet classification with deep convolutional neural networks[C]//Advances in Neural Information Processing Systems. Lake Tahoe: MIT, 2012: 1097-1105.
[13] LI Bo, WU Wei, WANG Qiang, et al. SiamRPN: evolution of Siamese visual tracking with very deep networks[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach: IEEE, 2019: 4277-4286.
[14] HE Kaiming, ZHANG Xiangyu, REN Shaoqing, et al. Deep residual learning for image recognition[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016: 770-778.
[15] ZHANG Lichao, GONZALEZ-GARCIA A, VAN DE WEIJER J, et al. Learning the model update for Siamese trackers[C]//2019 IEEE/CVF International Conference on Computer Vision. Seoul: IEEE, 2019: 4009-4018.
[16] ZHANG Zhipeng, PENG Houwen, FU Jianlong, et al. Ocean: object-aware anchor-free tracking[C]// European Conference on Computer Vision. Cham: Springer, 2020: 771-787.
[17] GUO Dongyan, SHAO Yanyan, CUI Ying, et al. Graph attention tracking[C]//2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Nashville: IEEE, 2021: 9538-9547.
[18] XU Yinda, WANG Zeyu, LI Zuoxin, et al. SiamFC++: towards robust and accurate visual tracking with target estimation guidelines[C]//Proceedings of the AAAI Conference on Artificial Intelligence. Palo Alto: AAAI Press, 2020: 12549-12556.
[19] LI Qiang, QIN Zekui, ZHANG Wenbo, et al. Siamese keypoint prediction network for visual object tracking[EB/OL].(2020-06-07)[2021-01-07]. https://arvix.org/abs/2006.04078.
[20] LI Weisheng, LYU Lanbing, ZHU Junye. Multigroup spatial shift models for thermal infrared tracking[J]. Knowledge-based systems, 2022, 255: 109705.
[21] LIU Qiao, LU Xiaohuan, HE Zhenyu, et al. Deep convolutional neural networks for thermal infrared object tracking[J]. Knowledge-based systems, 2017, 134: 189-198.
[22] LIU Qiao, LI Xin, HE Zhenyu, et al. Learning deep multi-level similarity for thermal infrared object tracking[J]. IEEE transactions on multimedia, 2021, 23: 2114-2126.
[23] LIU Qiao, YUAN Di, FAN Nana, et al. Learning dual-level deep representation for thermal infrared tracking[J]. IEEE transactions on multimedia, 2023, 25: 1269-1281.
[24] LIU Shu, QI Lu, QIN Haifang, et al. Path aggregation network for instance segmentation[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 8759-8768.
[25] LIAO Bingyan, WANG Chenye, WANG Yayun, et al. PG-net: pixel to global matching network for visual tracking[C]// European Conference on Computer Vision. Cham: Springer, 2020: 429-444.
[26] LIU Qiao, HE Zhenyu, LI Xin, et al. PTB-TIR: a thermal infrared pedestrian tracking benchmark[J]. IEEE transactions on multimedia, 2020, 22(3): 666-675.
[27] LIU Zhaoying, HE Junran, ZHANG Ting, et al. Infrared ship video target tracking based on cross-connection and spatial transformer network[C]//International Conference on Artificial Intelligence and Security. Cham: Springer, 2022: 100-114.
[28] CHEN Zedu, ZHONG Bineng, LI Guorong, et al. Siamese box adaptive network for visual tracking[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle: IEEE, 2020: 6667-6676.
[29] GUO Dongyan, WANG Jun, CUI Ying, et al. SiamCAR: Siamese fully convolutional classification and regression for visual tracking[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle: IEEE, 2020: 6268-6276.
[30] DONG Xingping, SHEN Jianbing, SHAO Ling, et al. CLNet: A compact latent network for fast adjusting Siamese trackers[C]//European Conference on Computer Vision. Cham: Springer, 2020: 378-395.
[31] CHENG Siyuan, ZHONG Bineng, LI Guorong, et al. Learning to filter: Siamese relation network for robust tracking[C]//2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Nashville: IEEE, 2021: 4419-4429.
[32] CAO Ziang, HUANG Ziyuan, PAN Liang, et al. TCTrack: temporal contexts for aerial tracking[C]//2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New Orleans: IEEE, 2022: 14778-14788.
[33] TANG Feng, LING Qiang. Ranking-based Siamese visual tracking[C]//2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New Orleans: IEEE, 2022: 8731-8740.
[34] DANELLJAN M, BHAT G, KHAN F S, et al. ATOM: accurate tracking by overlap maximization[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach: IEEE, 2019: 4655-4664.
[35] DANELLJAN M, BHAT G, KHAN F S, et al. ECO: efficient convolution operators for tracking[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE, 2017: 6931-6939.
[36] SONG Yibing, MA Chao, GONG Lijun, et al. CREST: convolutional residual learning for visual tracking[C]//2017 IEEE International Conference on Computer Vision. Venice: IEEE, 2017: 2574-2583.
[37] WANG Ning, SONG Yibing, MA Chao, et al. Unsupervised deep tracking[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach: IEEE, 2019: 1308-1317.
[38] QI Yuankai, ZHANG Shengping, QIN Lei, et al. Hedged deep tracking[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016: 4303-4311.
[39] MA Chao, HUANG Jiabin, YANG Xiaokang, et al. Hierarchical convolutional features for visual tracking[C]//2015 IEEE International Conference on Computer Vision. Santiago: IEEE, 2015: 3074-3082.
[40] VALMADRE J, BERTINETTO L, HENRIQUES J, et al. End-to-end representation learning for correlation filter based tracking[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE, 2017: 5000-5008.
- 相似文献/References:
-
[1]吴贵山,林淑彬,钟江华,等.区域损失函数的孪生网络目标跟踪[J].智能系统学报,2020,15(4):722.[doi:10.11992/tis.201910005]
WU Guishan,LIN Shubin,ZHONG Jianghua,et al.Regional loss function based siamese network for object tracking[J].CAAI Transactions on Intelligent Systems,2020,15():722.[doi:10.11992/tis.201910005]
[2]王德文,魏波涛.基于孪生变分自编码器的小样本图像分类方法[J].智能系统学报,2021,16(2):254.[doi:10.11992/tis.201906022]
WANG Dewen,WEI Botao.A small-sample image classification method based on a Siamese variational auto-encoder[J].CAAI Transactions on Intelligent Systems,2021,16():254.[doi:10.11992/tis.201906022]
[3]周士琪,王耀南,钟杭.融合视觉显著性再检测的孪生网络无人机目标跟踪算法[J].智能系统学报,2021,16(3):584.[doi:10.11992/tis.202101035]
ZHOU Shiqi,WANG Yaonan,ZHONG Hang.Siamese network combined with visual saliency re-detection for UAV object tracking[J].CAAI Transactions on Intelligent Systems,2021,16():584.[doi:10.11992/tis.202101035]
[4]雷涛,王洁,薛丁华,等.差异特征融合的无监督SAR图像变化检测[J].智能系统学报,2021,16(3):595.[doi:10.11992/tis.202103011]
LEI Tao,WANG Jie,XUE Dinghua,et al.Unsupervised SAR image change detection based on difference feature fusion[J].CAAI Transactions on Intelligent Systems,2021,16():595.[doi:10.11992/tis.202103011]
[5]于润羽,李雅文,李昂.融合领域特征的科技学术会议语义相似性计算方法[J].智能系统学报,2022,17(4):737.[doi:10.11992/tis.202203050]
YU Runyu,LI Yawen,LI Ang.Semantic similarity computing for scientific and technological conferences[J].CAAI Transactions on Intelligent Systems,2022,17():737.[doi:10.11992/tis.202203050]
[6]姜文涛,张大鹏.优化分类的弱目标孪生网络跟踪研究[J].智能系统学报,2023,18(5):984.[doi:10.11992/tis.202211043]
JIANG Wentao,ZHANG Dapeng.Research on weak object tracking based on Siamese network with optimized classification[J].CAAI Transactions on Intelligent Systems,2023,18():984.[doi:10.11992/tis.202211043]
备注/Memo
收稿日期:2023-6-1。
基金项目:国家自然科学基金区域创新发展联合基金项目（U23A20357）；北京市教育委员会科技计划一般项目（KM202110005028）；北京工业大学交叉科学研究院资助项目（2021020101）；北京工业大学国际科研合作种子基金项目（2021A01）.
作者简介:李想，硕士，主要研究方向为模式识别、深度学习和计算机视觉。E-mail：lixiang0123@emails.bjut.edu.cn;张婷，副教授，博士，主要研究方向为模式识别、深度学习和自然语言处理。E-mail：zhangting@bjut.edu.cn;刘兆英，副教授，博士，主要研究方向为图像处理、模式识别和深度学习。发表学术论文30余篇。E-mail：zhaoying.liu@bjut.edu.cn。
通讯作者:刘兆英. E-mail：zhaoying.liu@bjut.edu.cn
更新日期/Last Update:
2024-11-05