[1]彭阳,王德军,孟博,等.历史文档布局分析研究[J].智能系统学报,2026,21(3):727-738.[doi:10.11992/tis.202501011]
PENG Yang,WANG Dejun,MENG Bo,et al.Historical document layout analysis[J].CAAI Transactions on Intelligent Systems,2026,21(3):727-738.[doi:10.11992/tis.202501011]
点击复制
PENG Yang,WANG Dejun,MENG Bo,et al.Historical document layout analysis[J].CAAI Transactions on Intelligent Systems,2026,21(3):727-738.[doi:10.11992/tis.202501011]
历史文档布局分析研究
《智能系统学报》[ISSN 1673-4785/CN 23-1538/TP] 卷:
21
期数:
2026年第3期
页码:
727-738
栏目:
学术论文—智能系统
出版日期:
2026-05-05
- Title:
- Historical document layout analysis
- Keywords:
- document understanding; machine learning; historical documents; page structure; layout analysis; reading order; spatial features; semantic similarity
- 分类号:
- TP391
- 文献标志码:
- 2026-3-27
- 摘要:
- 文档布局分析是将扫描得到的页面图像转换为可搜索的全部文本，然而，该研究主要集中在结构化和半结构化文档领域，历史文档图像质量差、结构混乱，比常规结构化文档更具挑战性。为解决上述问题，本文在现有数据集上新增标注，构建融合文本内容、文档图像及空间特征的多模态网络；进一步地，通过结合语义相似度与空间邻接关系，设计关系预测网络来预测阅读顺序，获得最终的布局分析结果。本文方法在LA-READ和LA-FCR数据集上的mAP值分别达到92.4%和85.6%，总排序准确率(Acc)与现有方法的最好结果相比分别提升3.5%和2.7%，实验结果表明，所提方法有效提升了历史文档布局分析任务的效果。
- Abstract:
- Document layout analysis converts scanned page images into fully searchable text, yet existing research primarily focuses on structured and semi-structured documents. Compared with conventional structured documents, historical documents pose greater challenges due to poor image quality and irregular layouts. To address these issues, we augmented annotations on existing datasets and constructed a multimodal network that integrates textual content, document images, and spatial features. Furthermore, we designed a relation prediction network that combines semantic similarity with spatial adjacency to determine reading order, yielding the final layout analysis result. Our method achieves mean average precision (mAP) scores of 92.4% and 85.6% on the LA-READ and LA-FCR datasets, respectively, with total ordering accuracy (Acc) surpassing previous state-of-the-art results by 3.5% and 2.7%. Experimental results demonstrate the effectiveness of our approach for historical document layout analysis.
- 参考文献/References:
-
[1] 王军, 杨海峥, 刘石, 等. 系列笔谈之一: 智能时代古典文献学的机遇与挑战[J]. 数字人文, 2022(2): 108-132 WANG Jun, YANG Haizheng, LIU Shi, et al. The first of a series of pen talks: opportunities and challenges of classical philology in the intelligent age[J]. Digital humanities, 2022(2): 108-132
[2] 刘成林, 金连文, 白翔, 等. 文档智能分析与识别前沿: 回顾与展望[J]. 中国图象图形学报, 2023, 28(8): 2223-2252 LIU Chenglin, JIN Lianwen, BAI Xiang, et al. Frontiers of intelligent document analysis and recognition: review and prospects[J]. Journal of image and graphics, 2023, 28(8): 2223-2252
[3] TANG Y Y, LEE S W, SUEN C Y. Automatic document processing: a survey[J]. Pattern recognition, 1996, 29(12): 1931-1952
[4] GAO Liangcai, YI Xiaohan, JIANG Zhuoren, et al. ICDAR2017 competition on page object detection[C]//2017 14th IAPR International Conference on Document Analysis and Recognition. Kyoto: IEEE, 2017: 1417-1422.
[5] 王尚荣. 文档布局分析的多模态学习方法研究与实现[D]. 北京: 北京邮电大学, 2023. WANG Shangrong. Research and implementation of multimodal learning method for document layout analysis[D]. Beijing: Beijing University of Posts and Telecommunications, 2023.
[6] SASSIOUI A, BENOUINI R, EL OUARGUI Y, et al. Visually-rich document understanding: concepts, taxonomy and challenges[C]//2023 10th International Conference on Wireless Networks and Mobile Communications. Istanbul: IEEE, 2023: 1-7.
[7] AFZAL M Z, K?LSCH A, AHMED S, et al. Cutting the error by half: investigation of very deep CNN and advanced training strategies for document image classification[C]//2017 14th IAPR International Conference on Document Analysis and Recognition. Kyoto: IEEE, 2017: 883-888.
[8] DAVIS B, MORSE B, PRICE B, et al. End-to-end document recognition and understanding with dessurt[M]//Computer Vision–ECCV 2022 Workshops. Cham: Springer Nature Switzerland, 2023: 280-296.
[9] BAKKALI S, MING Zuheng, COUSTATY M, et al. VLCDoC: vision-language contrastive pre-training model for cross-modal document classification[J]. Pattern recognition, 2023, 139: 109419
[10] HONG T, KIM D, JI M, et al. BROS: a pre-trained language model focusing on text and layout for better key information extraction from documents[EB/OL]. (2021-09-10)[2024-01-01]. https://arxiv.org/abs/2108.04539.
[11] HUANG Yupan, LYUengchao, CUI Lei, et al. LayoutLMv3: pre-training for document AI with unified text and image masking[C]//Proceedings of the 30th ACM International Conference on Multimedia. Lisboa: ACM, 2022: 4083-4091.
[12] LI Xin, ZHENG Yan, HU Yiqing, et al. Relational representation learning in visually-rich documents[C]//Proceedings of the 30th ACM International Conference on Multimedia. Lisboa: ACM, 2022: 4614-4624.
[13] ZHANG Chong, TU Yi ZHAO Yixi, et al. Modeling layout reading order as ordering relations for visually-rich document Uunderstanding[EB/OL]. (2024-09-29)[2025-01-01]. https://arxiv.org/abs/2409.19672.
[14] AIELLO M, SMEULDERS A M W. Bidimensional relations for reading order detection[M]//EPRINTS-BOOK-TITLE. University of Groningen, Johann Bernoulli Institute for Mathematics and Computer Science, 2003.
[15] FERILLI S, GRIECO D, REDAVID D, et al. Abstract argumentation for reading order detection[C]//Proceedings of the 2014 ACM Symposium on Document Engineering. Fort Collins: ACM, 2014: 45-48.
[16] LI Liangcheng, GAO Feiyu, BU Jiajun, et al. An end-to-end OCR text re-organization sequence learning for rich-text detail image comprehension[M]//Computer Vision – ECCV 2020. Cham: Springer International Publishing, 2020: 85-100.
[17] QUIR?S L, VIDAL E. Reading order detection on handwritten documents[J]. Neural computing and applications, 2022, 34(12): 9593-9611
[18] 马伟洪. 面向古籍文档分析的文字检测识别与阅读顺序理解[D]. 广州: 华南理工大学, 2022. MA Weihong. Character detection, recognition and reading order comprehension for historical document analysis [D]. Guangzhou: South China University of Technology, 2022.
[19] ZHANG Chong, GUO Ya, TU Yi, et al. Reading order matters: Information extraction from visually-rich documents by token path prediction[EB/OL]. (2023-10-17)[2025-01-01]. https://arxiv.org/abs/2310.11016.
[20] QIAO Liang, LI Can, CHENG Zhanzhan, et al. Reading order detection in visually-rich documents with multi-modal layout-aware relation prediction[J]. Pattern recognition, 2024, 150: 110314
[21] 温绍杰, 吴瑞刚, 冯超文, 等. 基于Transformer的多模态级联文档布局分析网络[J]. 浙江大学学报(工学版), 2024, 58(2): 317-324, 369 WEN Shaojie, WU Ruigang, FENG Chaowen, et al. Multimodal cascaded document layout analysis network based on Transformer[J]. Journal of Zhejiang University (engineering science), 2024, 58(2): 317-324, 369
[22] WANG Jiawei, HU Kai, ZHONG Zhuoyao, et al. Detect-order-construct: a tree construction based approach for hierarchical document structure analysis[J]. Pattern recognit, 2017, 156: 110836
[23] REIMERS N, GUREVYCH I. SENTENCe-BERT: sentence embeddings using siamese BERT-Networks[EB/OL]. (2019-08-27)[2025-01-01]. https://arxiv.org/abs/1908.10084.
[24] XU Canhui, LI Yuteng, SHI Cao, et al. HiM: hierarchical multimodal network for document layout analysis[J]. Applied intelligence, 2023, 53(20): 24314-24326
[25] WANG Dongsheng, MA Zhiqiang, NOURBAKHSH A, et al. DocGraphLM: documental graph language model for information extraction[C]//Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval. Taipei: ACM, 2023: 1944-1948.
[26] 徐捷, 邵玉斌, 杜庆治, 等. 结合混合特征提取与深度学习的长文本语义相似度计算[J]. 计算机工程与科学, 2024, 46(8): 1513-1520 XU Jie, SHAO Yubin, DU Qingzhi, et al. Long text semantic similarity calculation combining hybrid feature extraction and deep learning[J]. Computer engineering & science, 2024, 46(8): 1513-1520
[27] PAPINENI K. BLEU: a method for automatic evaluation of MT[R]. New York: IBM, 2001.
[28] HE Kaiming, GKIOXARI G, DOLL?R P, et al. Mask R-CNN[C]//2017 IEEE International Conference on Computer Vision. Venice: IEEE, 2017: 2980-2988.
[29] HUANG Yupan, LYU ngchao, CUI Lei, et al. LayoutLMv3: pre-training for document AI with unified text and image masking[C]//Proceedings of the 30th ACM International Conference on Multimedia. Lisboa: ACM, 2022: 4083-4091.
[30] WANG Z, XU Y, CUI L, et al. Layoutreader: Pre-training of text and layout for reading order detection[EB/OL]. (2021-08-26)[2025-01-01]. https://arxiv.org/abs/2108.11591.
- 相似文献/References:
-
[1]叶志飞,文益民,吕宝粮.不平衡分类问题研究综述[J].智能系统学报,2009,4(2):148.
YE Zhi-fei,WEN Yi-min,LU Bao-liang.A survey of imbalanced pattern classification problems[J].CAAI Transactions on Intelligent Systems,2009,4():148.
[2]刘奕群,张 敏,马少平.基于非内容信息的网络关键资源有效定位[J].智能系统学报,2007,2(1):45.
LIU Yi-qun,ZHANG Min,MA Shao-ping.Web key resource page selection based on non-content inf o rmation[J].CAAI Transactions on Intelligent Systems,2007,2():45.
[3]马世龙,眭跃飞,许 可.优先归纳逻辑程序的极限行为[J].智能系统学报,2007,2(4):9.
MA Shi-long,SUI Yue-fei,XU Ke.Limit behavior of prioritized inductive logic programs[J].CAAI Transactions on Intelligent Systems,2007,2():9.
[4]姚伏天,钱沄涛.高斯过程及其在高光谱图像分类中的应用[J].智能系统学报,2011,6(5):396.
YAO Futian,QIAN Yuntao.Gaussian process and its applications in hyperspectral image classification[J].CAAI Transactions on Intelligent Systems,2011,6():396.
[5]文益民,强保华,范志刚.概念漂移数据流分类研究综述[J].智能系统学报,2013,8(2):95.[doi:10.3969/j.issn.1673-4785.201208012]
WEN Yimin,QIANG Baohua,FAN Zhigang.A survey of the classification of data streams with concept drift[J].CAAI Transactions on Intelligent Systems,2013,8():95.[doi:10.3969/j.issn.1673-4785.201208012]
[6]杨成东,邓廷权.综合属性选择和删除的属性约简方法[J].智能系统学报,2013,8(2):183.[doi:10.3969/j.issn.1673-4785.201209056]
YANG Chengdong,DENG Tingquan.An approach to attribute reduction combining attribute selection and deletion[J].CAAI Transactions on Intelligent Systems,2013,8():183.[doi:10.3969/j.issn.1673-4785.201209056]
[7]胡小生,钟勇.基于加权聚类质心的SVM不平衡分类方法[J].智能系统学报,2013,8(3):261.
HU Xiaosheng,ZHONG Yong.Support vector machine imbalanced data classification based on weighted clustering centroid[J].CAAI Transactions on Intelligent Systems,2013,8():261.
[8]丁科,谭营.GPU通用计算及其在计算智能领域的应用[J].智能系统学报,2015,10(1):1.[doi:10.3969/j.issn.1673-4785.201403072]
DING Ke,TAN Ying.A review on general purpose computing on GPUs and its applications in computational intelligence[J].CAAI Transactions on Intelligent Systems,2015,10():1.[doi:10.3969/j.issn.1673-4785.201403072]
[9]孔庆超,毛文吉,张育浩.社交网站中用户评论行为预测[J].智能系统学报,2015,10(3):349.[doi:10.3969/j.issn.1673-4785.201403019]
KONG Qingchao,MAO Wenji,ZHANG Yuhao.User comment behavior prediction in social networking sites[J].CAAI Transactions on Intelligent Systems,2015,10():349.[doi:10.3969/j.issn.1673-4785.201403019]
[10]姚霖,刘轶,李鑫鑫,等.词边界字向量的中文命名实体识别[J].智能系统学报,2016,11(1):37.[doi:10.11992/tis.201507065]
YAO Lin,LIU Yi,LI Xinxin,et al.Chinese named entity recognition via word boundarybased character embedding[J].CAAI Transactions on Intelligent Systems,2016,11():37.[doi:10.11992/tis.201507065]
备注/Memo
收稿日期:2025-1-8。
基金项目:国家重点研发计划项目(2020YFC1522900)；湖北省科技创新人才计划项目(2023DJC094)；民族语言智能分析与安全治理教育部重点实验室开放课题(ORP-2024-04).
作者简介:彭阳，硕士研究生，主要研究方向为目标检测、人工智能。E-mail：pengyang@mail.scuec.edu.cn。;王德军，副教授，博士，主要研究方向为人工智能、信息安全、大数据处理。获得湖北省科技进步二等奖2项。发表学术论文13篇。获国家发明授权专利 5项。E-mail：dejun@scuec.edu.cn。;孟博，教授，博士，主要研究方向为人工智能安全与隐私、网络与系统安全。主持和参与国家和省部级科研项目10余项，横向课题10余项；获湖北省科技进步二等奖和一等奖各1项。发表学术论文60余篇；在科学出版社出版专著3部；申请国家发明专利 40余项，其中获授权20余项；获软件著作权 20余项。E-mail：mengscuec@gmail.com。
通讯作者:王德军. E-mail：dejun@scuec.edu.cn
更新日期/Last Update:
1900-01-01