[1]LIU Guanghui,ZHANG Yumin,MENG Yuebo,et al.Natural scene text detection based on double-branch cross-level feature fusion[J].CAAI Transactions on Intelligent Systems,2023,18(5):1079-1089.[doi:10.11992/tis.202303005]
Copy

Natural scene text detection based on double-branch cross-level feature fusion

CAAI Transactions on Intelligent Systems[ISSN 1673-4785/CN 23-1538/TP] Volume: 18 Number of periods: 2023 5 Page number: 1079-1089 Column: 学术论文—机器学习 Public date: 2023-09-05
Title:
Natural scene text detection based on double-branch cross-level feature fusion
Author(s):
LIU Guanghui;  ZHANG Yumin;  MENG Yuebo;  ZHAN Hua
School of Information and Control Engineering, Xi’an University of Architecture and Technology, Xi’an 710055, China
Keywords:
text detection; arbitrarily shaped; cross-level feature distribution enhancement; adaptive fusion; double branch; spatial dimension; channel dimension; differentiable binarization
CLC:
TP391
DOI:
10.11992/tis.202303005
Abstract:
Current scene text detection methods cause the inaccurate location of text regions and false detection of adjacent texts due to the influence of complex backgrounds in arbitrarily shaped texts. To solve this issue, a natural scene text detection method based on double-branch cross-level feature fusion is proposed. First, the initial features were extracted using Resnet50 as the backbone network, and then a cross-level feature distribution enhancement module was designed to improve the interaction of cross-level feature text information and the expression ability of features. Second, an adaptive fusion strategy was proposed to filter nontext or redundant features adaptively and reduce the false and missed detection rates using the double-branch structure to strengthen the relationship between different dimensional features and optimize the fusion process. Last, the differential binarization method was used to yield text detection results in the prediction phase. The proposed method was employed to perform ablation experiments on the ICDAR2015, ICDAR2017, Total-Text, and CTW1500 datasets. The findings revealed that this method can accurately locate the text area and overcome the impact of text miss and false detections.
References:
[1] 黄剑华, 唐降龙, 刘家锋, 等. 一种基于Homogeneity的文本检测新方法[J]. 智能系统学报, 2007, 2(1): 69-73
HUANG Jianhua, TANG Xianglong, LIU Jiafeng, et al. A new method for text detection based on Homogeneity[J]. CAAI Transactions on Intelligent Systems, 2007, 2(1): 69-73
[2] 吕国宁,高敏. 视觉感知式场景文字检测定位方法[J]. 智能系统学报, 2017, 12(4): 569
LYU Guoning, GAO Min. Scene text detection and localization scheme with visual perception mechanism[J]. CAAI transactions on intelligent systems, 2017, 12(4): 569
[3] TIAN Z, HUANG W, HE T, et al. Detecting text in natural image with connectionist text proposal network[C] //Computer Vision-ECCV 2016: 14th European Conferen -ce, Amsterdam, The Netherlands: Springer International Publishing, 2016: 56?72.
[4] SHI B, BAI X, BELONGIE S. Detecting oriented text in natural images by linking segments[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE, 2017: 2550?2558.
[5] WANG Y, XIE H, ZHA Z J, et al. ContourNet: taking a further step toward accurate arbitrary-shaped scene text detection[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2020: 11753?11762.
[6] WANG W, XIE E, LI X, et al. Shape robust text detection with progressive scale expansion network[C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach: IEEE, 2019: 9336?9345.
[7] WANG W, XIE E, SONG X, et al. Efficient and accurate arbitrary-shaped text detection with pixel aggregation network[C]//IEEE/CVF International Conference on Computer Vision. Long Beach: IEEE, 2019: 8439?8448.
[8] LIAO M, WAN Z, YAO C, et al. Real-time scene text detection with differentiable binarization[C]//Proceedings of the AAAI conference on artificial intelligence. Washington, D.C.: AAAI, 2020, 34(07): 11474?11481.
[9] WU Y, ZHANG W, WAN S. CE-text: a context-aware and embedded text detector in natural scene images[J]. Pattern recognition letters, 2022, 159: 77-83.
[10] 孟月波, 石德旺, 刘光辉, 等. 多维度卷积融合的密集不规则文本检测[J]. 光学精密工程, 2021, 29(9): 2210-2221
MENG Yuebo, SHI Dewang, LIU Guanghui, et al. Dense irregular text detection based on multi-dimensional convolution fusion[J]. Optics and precision engineering, 2021, 29(9): 2210-2221
[11] YANG C, CHEN M, YUAN Y, et al. BiP-net: bidirectional perspective strategy based arbitrary-shaped text detection network[C]//ICASSP 2022 IEEE International Conference on Acoustics, Speech and Signal Processing. New York: IEEE, 2022: 2255-2259.
[12] CHEN H, CHEN P, QIU Y, et al. FARNet: fragmented affinity reasoning network of text instances for arbitrary shape text detection[J]. IET image processing, 2023, 17(6): 1959-1977.
[13] ZHU Y, CHEN J, LIANG L, et al. Fourier contour embedding for arbitrary-shaped text detection[C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Nashville: IEEE, 2021: 3123-3131.
[14] YOU Y, LEI Y, ZHANG Z, et al. Arbitrary-shaped text detection with B-spline curve network[J]. Sensors, 2023, 23(5): 2418.
[15] ZHANG S X, ZHU X, CHEN L, et al. Arbitrary shape text detection via segmentation with probability maps[J]. IEEE transactions on pattern analysis and machine intelligence, 2022, 45(3): 2736-2750.
[16] TANG Q, FENG X, ZHANG X. A spatial feature adaptive network for text detection[J]. Multimedia tools and applications, 2022, 81(11): 15285-15302.
[17] 赵文清, 杨盼盼. 双向特征融合与注意力机制结合的目标检测[J]. 智能系统学报, 2021, 16(6): 1098-1105
ZHAO Wenqing, YANG Panpan. Target detection based on bidirectional feature fusion and an attention mechanism[J]. CAAI transactions on intelligent systems, 2021, 16(6): 1098-1105
[18] ZHOU X, YAO C, WEN H, et al. EAST: An efficient and accurate scene text detector[C]//IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE, 2017: 2642?2651.
[19] ZHANG C, LIANG B, HUANG Z, et al. Look more than once: an accurate detector for text of arbitrary shapes[C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach: IEEE, 2019: 10552-10561
[20] LONG S, RUAN J, ZHANG We, et al. TextSnake: A flexible representation for detecting text of arbitrary shapes[C]//Proceedings of the European Conference on Computer Vision. Munich: Springer, 2018: 20-36.
[21] XIE E, ZANG Y, SHAO S, et al. Scene text detection with supervised pyramid context network[C]// Proceedings of the AAAI conference on artificial intelligence. Honolulu: AAAI, 2019, 33(1): 9038-9045.
[22] BAEK Y, LEE B, HAN D, et al. Character region awareness for text detection[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach: IEEE, 2019: 9365-9374.
[23] WANG Hao, LU Pu, ZHANG Hui, et al. All You need is boundary: toward arbitrary-shaped text spotting[C]//Proceedings of the AAAI Conference on Artificial Intelligence. New York: AAAI, 2020, 34(7): 12160-12167.
[24] ZHANG S X, ZHU X, HOU J B, et al. Deep relational reasoning graph network for arbitrary shape text detection[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. New Orleans: IEEE, 2020: 9699?9708.
[25] LI J, LIN Y, LIU R, et al. RSCA: real-time segmentation-based context-aware scene text detection[C] //Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Nashville: IEEE, 2021: 2349-2358.
[26] WAN Q, JI H, SHEN L. Self-attention based text knowledge mining for text detection[C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognitio. Nashville: IEEE,?2021: 5983?5992.
[27] ZHU Y, DU J. TextMountain: accurate scene text detection via instance segmentation[J]. Pattern recognition, 2021, 110: 107336.
[28] WANG F, CHEN Y, WU F, et al. TextRay: contour-based geometric modeling for arbitrary-shaped scene text detection[C]//Proceedings of the 28th ACM International Conference on Multimedia. New York: ACM, 2020: 111-119.
[29] BAEK Y, SHIN S, BAEK J, et al. Character region attention for text spotting[C]//Computer Vision-ECCV 2020: 16th European Conference. Glasgow: Springer International Publishing, 2020: 504-521.
[30] LIU Y, CHEN H, SHEN C, et al. ABCNet: real-time scene text spotting with adaptive bezier-curve network[C] //Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. New Orleans: IEEE, 2020: 9809-9818.
[31] YE J, CHEN Z, LIU J, et al. TextFuseNet: Scene Text Detection with Richer Fused Features[C]//International Joint Conference on Artificial Intelligence. Yokohama: IJCAI, 2021: 512?518.
[32] BU?TA M, PATEL Y, MATAS J. E2E-MLT - an unconstrained end-to-end method for multi-language scene text[C]//Computer Vision-ACCV 2018 Workshops: 14th Asian Conference on Computer Vision. Perth: Springer International Publishing, 2019: 127-143.
[33] DENG D, LIU H, LI X, et al. PixelLink: detecting scene text via instance segmentation[C]// Proceedings of the AAAI conference on artificial intelligence. New Orleans: AAAI, 2018, 32(1): 6773-6780.
[34] LIU X, LIANG D, YAN S, et al. FOTS: fast oriented text spotting with a unified network[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 5676-5685.
[35] XING L, TIAN Z, HUANG W, et al. Convolutional character networks[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. Long Beach: IEEE, 2019: 9126-9136.
Similar References:

Memo

-

Last Update: 1900-01-01

Copyright © CAAI Transactions on Intelligent Systems