[1]刘光辉,张钰敏,孟月波,等.双分支跨级特征融合的自然场景文本检测[J].智能系统学报,2023,18(5):1079-1089.[doi:10.11992/tis.202303005]
LIU Guanghui,ZHANG Yumin,MENG Yuebo,et al.Natural scene text detection based on double-branch cross-level feature fusion[J].CAAI Transactions on Intelligent Systems,2023,18(5):1079-1089.[doi:10.11992/tis.202303005]
点击复制
LIU Guanghui,ZHANG Yumin,MENG Yuebo,et al.Natural scene text detection based on double-branch cross-level feature fusion[J].CAAI Transactions on Intelligent Systems,2023,18(5):1079-1089.[doi:10.11992/tis.202303005]
双分支跨级特征融合的自然场景文本检测
《智能系统学报》[ISSN 1673-4785/CN 23-1538/TP] 卷:
18
期数:
2023年第5期
页码:
1079-1089
栏目:
学术论文—机器学习
出版日期:
2023-09-05
- Title:
- Natural scene text detection based on double-branch cross-level feature fusion
- Keywords:
- text detection; arbitrarily shaped; cross-level feature distribution enhancement; adaptive fusion; double branch; spatial dimension; channel dimension; differentiable binarization
- 分类号:
- TP391
- 摘要:
- 现有的场景文本检测方法在处理任意形状文本时，由于复杂背景的影响会造成文本区域定位不准确、相邻文本漏检误检的问题，基于此提出一种双分支跨级特征融合的自然场景文本检测方法。首先，以Resnet50为主干网络提取初始特征，设计跨级特征分布增强模块（cross-level feature distribution enhancement module，CFDEM），增强跨级特征文本信息的交互性，提高特征的表达能力；然后，为自适应地选择过滤非文本或冗余特征，降低误检率和漏检率，提出自适应融合策略（adaptive fusion strategy，AFS），利用双分支结构加强不同维度特征之间的联系，优化融合过程；最后，预测阶段采用可微分二值化的方法来生成文本检测结果。所提方法在ICDAR2015、ICDAR2017、Total-Text、CTW1500数据集上进行消融实验，实验结果表明该方法能准确定位文本区域，克服文本漏检误检影响。
- Abstract:
- Current scene text detection methods cause the inaccurate location of text regions and false detection of adjacent texts due to the influence of complex backgrounds in arbitrarily shaped texts. To solve this issue, a natural scene text detection method based on double-branch cross-level feature fusion is proposed. First, the initial features were extracted using Resnet50 as the backbone network, and then a cross-level feature distribution enhancement module was designed to improve the interaction of cross-level feature text information and the expression ability of features. Second, an adaptive fusion strategy was proposed to filter nontext or redundant features adaptively and reduce the false and missed detection rates using the double-branch structure to strengthen the relationship between different dimensional features and optimize the fusion process. Last, the differential binarization method was used to yield text detection results in the prediction phase. The proposed method was employed to perform ablation experiments on the ICDAR2015, ICDAR2017, Total-Text, and CTW1500 datasets. The findings revealed that this method can accurately locate the text area and overcome the impact of text miss and false detections.
- 参考文献/References:
-
[1] 黄剑华, 唐降龙, 刘家锋, 等. 一种基于Homogeneity的文本检测新方法[J]. 智能系统学报, 2007, 2(1): 69-73
HUANG Jianhua, TANG Xianglong, LIU Jiafeng, et al. A new method for text detection based on Homogeneity[J]. CAAI Transactions on Intelligent Systems, 2007, 2(1): 69-73
[2] 吕国宁,高敏. 视觉感知式场景文字检测定位方法[J]. 智能系统学报, 2017, 12(4): 569
LYU Guoning, GAO Min. Scene text detection and localization scheme with visual perception mechanism[J]. CAAI transactions on intelligent systems, 2017, 12(4): 569
[3] TIAN Z, HUANG W, HE T, et al. Detecting text in natural image with connectionist text proposal network[C] //Computer Vision-ECCV 2016: 14th European Conferen -ce, Amsterdam, The Netherlands: Springer International Publishing, 2016: 56?72.
[4] SHI B, BAI X, BELONGIE S. Detecting oriented text in natural images by linking segments[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE, 2017: 2550?2558.
[5] WANG Y, XIE H, ZHA Z J, et al. ContourNet: taking a further step toward accurate arbitrary-shaped scene text detection[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2020: 11753?11762.
[6] WANG W, XIE E, LI X, et al. Shape robust text detection with progressive scale expansion network[C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach: IEEE, 2019: 9336?9345.
[7] WANG W, XIE E, SONG X, et al. Efficient and accurate arbitrary-shaped text detection with pixel aggregation network[C]//IEEE/CVF International Conference on Computer Vision. Long Beach: IEEE, 2019: 8439?8448.
[8] LIAO M, WAN Z, YAO C, et al. Real-time scene text detection with differentiable binarization[C]//Proceedings of the AAAI conference on artificial intelligence. Washington, D.C.: AAAI, 2020, 34(07): 11474?11481.
[9] WU Y, ZHANG W, WAN S. CE-text: a context-aware and embedded text detector in natural scene images[J]. Pattern recognition letters, 2022, 159: 77-83.
[10] 孟月波, 石德旺, 刘光辉, 等. 多维度卷积融合的密集不规则文本检测[J]. 光学精密工程, 2021, 29(9): 2210-2221
MENG Yuebo, SHI Dewang, LIU Guanghui, et al. Dense irregular text detection based on multi-dimensional convolution fusion[J]. Optics and precision engineering, 2021, 29(9): 2210-2221
[11] YANG C, CHEN M, YUAN Y, et al. BiP-net: bidirectional perspective strategy based arbitrary-shaped text detection network[C]//ICASSP 2022 IEEE International Conference on Acoustics, Speech and Signal Processing. New York: IEEE, 2022: 2255-2259.
[12] CHEN H, CHEN P, QIU Y, et al. FARNet: fragmented affinity reasoning network of text instances for arbitrary shape text detection[J]. IET image processing, 2023, 17(6): 1959-1977.
[13] ZHU Y, CHEN J, LIANG L, et al. Fourier contour embedding for arbitrary-shaped text detection[C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Nashville: IEEE, 2021: 3123-3131.
[14] YOU Y, LEI Y, ZHANG Z, et al. Arbitrary-shaped text detection with B-spline curve network[J]. Sensors, 2023, 23(5): 2418.
[15] ZHANG S X, ZHU X, CHEN L, et al. Arbitrary shape text detection via segmentation with probability maps[J]. IEEE transactions on pattern analysis and machine intelligence, 2022, 45(3): 2736-2750.
[16] TANG Q, FENG X, ZHANG X. A spatial feature adaptive network for text detection[J]. Multimedia tools and applications, 2022, 81(11): 15285-15302.
[17] 赵文清, 杨盼盼. 双向特征融合与注意力机制结合的目标检测[J]. 智能系统学报, 2021, 16(6): 1098-1105
ZHAO Wenqing, YANG Panpan. Target detection based on bidirectional feature fusion and an attention mechanism[J]. CAAI transactions on intelligent systems, 2021, 16(6): 1098-1105
[18] ZHOU X, YAO C, WEN H, et al. EAST: An efficient and accurate scene text detector[C]//IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE, 2017: 2642?2651.
[19] ZHANG C, LIANG B, HUANG Z, et al. Look more than once: an accurate detector for text of arbitrary shapes[C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach: IEEE, 2019: 10552-10561
[20] LONG S, RUAN J, ZHANG We, et al. TextSnake: A flexible representation for detecting text of arbitrary shapes[C]//Proceedings of the European Conference on Computer Vision. Munich: Springer, 2018: 20-36.
[21] XIE E, ZANG Y, SHAO S, et al. Scene text detection with supervised pyramid context network[C]// Proceedings of the AAAI conference on artificial intelligence. Honolulu: AAAI, 2019, 33(1): 9038-9045.
[22] BAEK Y, LEE B, HAN D, et al. Character region awareness for text detection[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach: IEEE, 2019: 9365-9374.
[23] WANG Hao, LU Pu, ZHANG Hui, et al. All You need is boundary: toward arbitrary-shaped text spotting[C]//Proceedings of the AAAI Conference on Artificial Intelligence. New York: AAAI, 2020, 34(7): 12160-12167.
[24] ZHANG S X, ZHU X, HOU J B, et al. Deep relational reasoning graph network for arbitrary shape text detection[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. New Orleans: IEEE, 2020: 9699?9708.
[25] LI J, LIN Y, LIU R, et al. RSCA: real-time segmentation-based context-aware scene text detection[C] //Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. Nashville: IEEE, 2021: 2349-2358.
[26] WAN Q, JI H, SHEN L. Self-attention based text knowledge mining for text detection[C]// Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognitio. Nashville: IEEE,?2021: 5983?5992.
[27] ZHU Y, DU J. TextMountain: accurate scene text detection via instance segmentation[J]. Pattern recognition, 2021, 110: 107336.
[28] WANG F, CHEN Y, WU F, et al. TextRay: contour-based geometric modeling for arbitrary-shaped scene text detection[C]//Proceedings of the 28th ACM International Conference on Multimedia. New York: ACM, 2020: 111-119.
[29] BAEK Y, SHIN S, BAEK J, et al. Character region attention for text spotting[C]//Computer Vision-ECCV 2020: 16th European Conference. Glasgow: Springer International Publishing, 2020: 504-521.
[30] LIU Y, CHEN H, SHEN C, et al. ABCNet: real-time scene text spotting with adaptive bezier-curve network[C] //Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. New Orleans: IEEE, 2020: 9809-9818.
[31] YE J, CHEN Z, LIU J, et al. TextFuseNet: Scene Text Detection with Richer Fused Features[C]//International Joint Conference on Artificial Intelligence. Yokohama: IJCAI, 2021: 512?518.
[32] BU?TA M, PATEL Y, MATAS J. E2E-MLT - an unconstrained end-to-end method for multi-language scene text[C]//Computer Vision-ACCV 2018 Workshops: 14th Asian Conference on Computer Vision. Perth: Springer International Publishing, 2019: 127-143.
[33] DENG D, LIU H, LI X, et al. PixelLink: detecting scene text via instance segmentation[C]// Proceedings of the AAAI conference on artificial intelligence. New Orleans: AAAI, 2018, 32(1): 6773-6780.
[34] LIU X, LIANG D, YAN S, et al. FOTS: fast oriented text spotting with a unified network[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 5676-5685.
[35] XING L, TIAN Z, HUANG W, et al. Convolutional character networks[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. Long Beach: IEEE, 2019: 9126-9136.
- 相似文献/References:
-
[1]黄剑华,唐降龙,刘家锋,等.一种基于Homogeneity的文本检测新方法[J].智能系统学报,2007,2(1):69.
HUANG Jian-hua,TANG Xiang-long,LIU Jia-feng,et al.A new method for text detection based on Homogeneity[J].CAAI Transactions on Intelligent Systems,2007,2():69.
[2]张铭泉,张泽恩,曹锦纲,等.结合Segformer与增强特征金字塔的文本检测方法[J].智能系统学报,2024,19(5):1111.[doi:10.11992/tis.202301013]
ZHANG Mingquan,ZHANG Zeen,CAO Jingang,et al.Text detection method combining Segformer with an enhanced feature pyramid[J].CAAI Transactions on Intelligent Systems,2024,19():1111.[doi:10.11992/tis.202301013]
备注/Memo
收稿日期:2023-3-2。
基金项目:国家自然科学基金项目（52278125）；陕西省重点研发计划（2021SF-429）.
作者简介:刘光辉，副教授，主要研究方向为计算机视觉理解、建筑智能化技术。近年来主持/参与多项国家自然科学基金项目、陕西省重点研发计划项目、陕西省基础研究项目，获陕西高等学校科学技术优秀成果奖;张钰敏，硕士研究生，主要研究方向为图像处理、场景文本检测与识别;孟月波，教授，博士生导师，博士，主要研究方向为机器视觉信息处理与分析、建筑智能化
通讯作者:刘光辉.E-mail：guanghuil@163.com
更新日期/Last Update:
1900-01-01