[1]许柏祥,刘丽,邱桃荣.面向近重复文本图像检索的三分支孪生网络[J].智能系统学报,2022,17(3):515-522.[doi:10.11992/tis.202105018]
XU Boxiang,LIU Li,QIU Taorong.Near-duplicate document image retrieval based on three-stream convolutional Siamese network[J].CAAI Transactions on Intelligent Systems,2022,17(3):515-522.[doi:10.11992/tis.202105018]
点击复制
XU Boxiang,LIU Li,QIU Taorong.Near-duplicate document image retrieval based on three-stream convolutional Siamese network[J].CAAI Transactions on Intelligent Systems,2022,17(3):515-522.[doi:10.11992/tis.202105018]
面向近重复文本图像检索的三分支孪生网络
《智能系统学报》[ISSN 1673-4785/CN 23-1538/TP] 卷:
17
期数:
2022年第3期
页码:
515-522
栏目:
学术论文—机器感知与模式识别
出版日期:
2022-05-05
- Title:
- Near-duplicate document image retrieval based on three-stream convolutional Siamese network
- Keywords:
- near-duplicate document image; image retrieval; three-stream convolutional Siamese network; triplet loss; image variations; triplet; feature extraction; robustness
- 分类号:
- TP391
- 摘要:
- 针对传统近重复文本图像检索方法需人工事先确定近重复文本图像之间存在的变换类型，易受到人主观性影响这一问题，提出一个面向近重复文本图像检索的三分支孪生网络，能自动学习图像之间存在的各种变换。该网络输入为三元组，包括查询图像、查询图像的近重复图像以及其非近重复图像，训练时采用三元损失使得查询图像和近重复图像之间的距离小于查询图像与非近重复图像之间的距离。提出的方法在两个数据集上的mAP (mean average precision)分别达到98.76%和96.50%，优于目前已有方法。
- Abstract:
- In the traditional near-duplicate document image retrieval methods, the variations among the near-duplicate document images had to be manually identified beforehand, which can be easily influenced by human subjectivity. To solve this problem, we propose a three-stream convolutional Siamese network orienting toward the near-duplicate text-image retrieval, which can automatically learn the variation types among the near-duplicate document images. The input to this network is a triplet, consisting of a query image, its near-duplicate image, and its non-near-duplicate image. Using the triplet loss, the distance between the query image and its near-duplicate image is guaranteed to be smaller than that between the query and its non-near-duplicate image. This approach achieves promising results with the mAP of 98.76% and 96.50% on two datasets, respectively, thereby greatly outperforming the state-of-the-art near-duplicate document image retrieval methods.
- 参考文献/References:
-
[1] 方涛, 陈志国, 傅毅. 神经网络多层特征信息融合的人脸识别方法[J]. 智能系统学报, 2021, 16(2): 279–285
FANG Tao, CHEN Zhiguo, FU Yi. Face recognition method based on neural network multi-layer feature information fusion[J]. CAAI transactions on intelligent systems, 2021, 16(2): 279–285
[2] WANG Zi, LI Chengcheng, WANG Xiangyang. Convolutional neural network pruning with structural redundancy reduction[C]//2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Nashville: IEEE, 2021: 14913–14922.
[3] SPITZ L. Duplicate document detection[C]//Proceedings of Document Recognition IV. San Jose, USA, 1997: 88–94.
[4] HULL J J. Document image matching and retrieval with multiple distortion-invariant descriptors[J].Proceedings of the international workshop on document analysis systems, 1994: 379–396.
[5] NAKAI T, KISE K, IWAMURA M. Real-time retrieval for images of documents in various languages using a web camera[C]//2009 10th International Conference on Document Analysis and Recognition. Barcelona: IEEE, 2009: 146–150.
[6] MORALEDA J. Large scalability in document image matching using text retrieval[J]. Pattern recognition letters, 2012, 33(7): 863–871.
[7] LIU Li, LU Yue, SUEN C Y, et al. Modeling local word spatial configurations for near duplicate document image retrieval[C]//2013 12th International Conference on Document Analysis and Recognition. Washington: IEEE, 2013: 235–239.
[8] LIU Li, LU Yue, SUEN C Y. Near-duplicate document image matching: a graphical perspective[J]. Pattern recognition, 2014, 47(4): 1653–1663.
[9] LIU Li, LU Yue, SUEN C Y. Document image matching using probabilistic graphical models[C]//Proceedings of the 21st International Conference on Pattern Recognition. Tsukuba, Japan, 2012: 637–640.
[10] BELLAVIA F, COLOMBO C. Is there anything new to say about SIFT matching?[J]. International journal of computer vision, 2020, 128(7): 1847–1866.
[11] VITALADEVUNI S, CHOI F, PRASAD R, et al. Detecting near-duplicate document images using interest point matching[C]//Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012). Tsukuba: IEEE, 2012: 347–350.
[12] ROYER E, BOUCHARA F. Guiding text image keypoints extraction through layout analysis[C]//2017 14th IAPR International Conference on Document Analysis and Recognition (ICDAR). Kyoto: IEEE, 2017: 9–14.
[13] SUN Yanan, XUE Bing, ZHANG Mengjie, et al. Automatically designing CNN architectures using the genetic algorithm for image classification[J]. IEEE transactions on cybernetics, 2020, 50(9): 3840–3854.
[14] JI Yuzhu, ZHANG Haijun, ZHANG Zhao, et al. CNN-based encoder-decoder networks for salient object detection: a comprehensive review and recent advances[J]. Information sciences, 2021, 546: 835–857.
[15] BABENKO A, SLESAREV A, CHIGORIN A, et al. Neural codes for image retrieval[C]//European Conference on Computer Vision. Cham:Springer, 2014: 584–599.
[16] BABENKO A, LEMPITSKY V. Aggregating deep convolutional features for image retrieval[C]//Proceedings of International Conference on Computer Vision. Santiago: IEEE, 2015: 1269–1277.
[17] MIN Weiqing, MEI Shuhuan, LI Zhou, et al. A two-stage triplet network training framework for image retrieval[J]. IEEE transactions on multimedia, 2020, 22(12): 3128–3138.
[18] HUSAIN S S, ONG E J, BOBER M. ACTNET: end-to-end learning of feature activations and multi-stream aggregation for effective instance image retrieval[J]. International journal of computer vision, 2021, 129(5): 1432–1450.
[19] GORDO A, ALMAZáN J, REVAUD J, et al. End-to-end learning of deep visual representations for image retrieval[J]. International journal of computer vision, 2017, 124(2): 237–254.
[20] TOLIAS G, SICRE R, JéGOU H. Particular object retrieval with integral max-pooling of CNN activations[EB/OL]//(2015–11–18)[2021–05–13]https://arxiv.org/abs/1511.05879.
[21] SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[EB/OL].(2014–09–4)[2021–05–13]https://arxiv.org/abs/1409.1556.
[22] ZHANG Shengdong, NEZHADARYA E, FASHANDI H, et al. Stochastic whitening batch normalization[C]//2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Nashville: IEEE, 2021: 10978–10987.
[23] LIU Guanghai, YANG Jingyu. Deep-seated features histogram: a novel image retrieval method[J]. Pattern recognition, 2021, 116: 107926.
[24] TAKEDA K, KISE K, IWAMURA M. Real-time document image retrieval on a smartphone[C]//2012 10th IAPR International Workshop on Document Analysis Systems. Gold Coast: IEEE, 2012: 225–229.
[25] HARLEY A W, UFKES A, DERPANIS K G. Evaluation of deep convolutional nets for document image classification and retrieval[C]//2015 13th International Conference on Document Analysis and Recognition (ICDAR). Tunis: IEEE, 2015: 991–995.
- 相似文献/References:
-
[1]辛雨璇,闫子飞.基于手绘草图的图像检索技术研究进展[J].智能系统学报,2015,10(2):167.[doi:10.3969/j.issn.1673-4785.201401045]
XIN Yuxuan,YAN Zifei.Research progress of image retrieval based on hand-drawn sketches[J].CAAI Transactions on Intelligent Systems,2015,10():167.[doi:10.3969/j.issn.1673-4785.201401045]
[2]陈阳,董肖莉,李卫军,等.基于仿生形象思维方法的图像检索算法的改进[J].智能系统学报,2015,10(2):209.[doi:10.3969/j.issn.1673-4785.201411022]
CHEN Yang,DONG Xiaoli,LI Weijun,et al.Improvement of an image retrieval algorithm based on biomimetic imaginal thinking[J].CAAI Transactions on Intelligent Systems,2015,10():209.[doi:10.3969/j.issn.1673-4785.201411022]
[3]龚震霆,陈光喜,任夏荔,等.基于卷积神经网络和哈希编码的图像检索方法[J].智能系统学报,2016,11(3):391.[doi:10.11992/tis.201603028]
GONG Zhenting,CHEN Guangxi,REN Xiali,et al.An image retrieval method based on a convolutional neural network and hash coding[J].CAAI Transactions on Intelligent Systems,2016,11():391.[doi:10.11992/tis.201603028]
[4]杨晓兰,强彦,赵涓涓,等.基于医学征象和卷积神经网络的肺结节CT图像哈希检索[J].智能系统学报,2017,12(6):857.[doi:10.11992/tis.201706035]
YANG Xiaolan,QIANG Yan,ZHAO Juanjuan,et al.Hashing retrieval for CT images of pulmonary nodules based on medical signs and convolutional neural networks[J].CAAI Transactions on Intelligent Systems,2017,12():857.[doi:10.11992/tis.201706035]
[5]阿丽亚·巴吐尔,努尔毕亚·亚地卡尔,吾尔尼沙·买买提,等.改进SURF特征的维吾尔文复杂文档图像匹配检索[J].智能系统学报,2019,14(2):296.[doi:10.11992/tis.201709014]
ALIYA Batur,NURBIYA Yadikar,HORNISA Mamat,et al.Complex Uyghur document image matching and retrieval based on modified SURF feature[J].CAAI Transactions on Intelligent Systems,2019,14():296.[doi:10.11992/tis.201709014]
备注/Memo
收稿日期:2021-05-13。
基金项目:国家自然科学基金青年项目(61603256).
作者简介:许柏祥，硕士研究生，主要研究方向为文本图像分析与识别、深度学习;刘丽，讲师，博士，主要研究方向为文本图像分析与识别、机器视觉、深度学习。主持完成国家自然科学基金项目1项。发表学术论文21篇;邱桃荣，教授，博士，主要研究方向为模式识别与人工智能、机器学习与脑电信号处理和应用。主持完成国家级项目2项、省级项目6项。发表学术论文39篇
通讯作者:刘丽.E-mail：liuli_033@163.com
更新日期/Last Update:
1900-01-01