[1]赵振兵,王睿,王艺衡,等.联合结构-语义关系图知识推理的输电线路螺栓缺陷识别方法[J].智能系统学报,2024,19(6):1584-1592.[doi:10.11992/tis.202305050]
ZHAO Zhenbing,WANG Rui,WANG Yiheng,et al.Bolt defect recognition method for transmission line based on joint structure-semantic relationship graph knowledge reasoning[J].CAAI Transactions on Intelligent Systems,2024,19(6):1584-1592.[doi:10.11992/tis.202305050]
点击复制
ZHAO Zhenbing,WANG Rui,WANG Yiheng,et al.Bolt defect recognition method for transmission line based on joint structure-semantic relationship graph knowledge reasoning[J].CAAI Transactions on Intelligent Systems,2024,19(6):1584-1592.[doi:10.11992/tis.202305050]
联合结构-语义关系图知识推理的输电线路螺栓缺陷识别方法
《智能系统学报》[ISSN 1673-4785/CN 23-1538/TP] 卷:
19
期数:
2024年第6期
页码:
1584-1592
栏目:
人工智能院长论坛
出版日期:
2024-12-05
- Title:
- Bolt defect recognition method for transmission line based on joint structure-semantic relationship graph knowledge reasoning
- Keywords:
- transmission lines; bolts; defects recognition; knowledge representation; knowledge reasoning; graph neural networks; structural relationships; semantic relationships
- 分类号:
- TP18
- 摘要:
- 针对输电线路螺栓缺陷识别任务中存在的视觉不可分与语义歧义问题，提出联合结构-语义关系图知识推理的输电线路螺栓缺陷识别方法。通过语义关系图提取螺栓各属性有判别力的特征类映射，经结构关系图捕获螺栓上下文信息并建立跨不同尺度的空间关系，采用图卷积神经网络经协作学习，利用螺栓各属性之间的结构知识与语义知识实现语义关系图节点的更新，通过螺栓训练数据集统计的标签共现信息辅助图知识推理网络提高螺栓缺陷识别的准确率。在实验阶段，选取3类典型金具上的13类螺栓属性作为研究对象。对比实验结果表明，本文方法对螺栓缺陷的识别效果优于其他方法，较基线模型提升了8.12%的准确率。
- Abstract:
- To tackle the issues of visual inseparability and semantic ambiguity in identifying transmission line bolt defects, a new method using joint structure–semantic relationship graph knowledge reasoning is proposed. Initially, a semantic expression module extracts feature class mappings that highlight the discriminative attributes of each bolt. Subsequently, the structural relationship graph captures contextual bolt information and establishes spatial relationships across different scales. Utilizing a graph convolutional neural network and cooperative learning, the semantic relation graph nodes are updated with structural and semantic knowledge derived from the bolt attributes. Finally, the network employs label co-occurrence information from the bolt training data set to improve the accuracy of defect recognition. In the experimental stage, 13 types of bolt properties across 3 types of typical fittings were examined. Comparative experiments show that the method proposed in this study outperforms other methods in identifying bolt defects, boosting accuracy by 8.12% over the baseline model.
- 参考文献/References:
-
[1] 周远翔, 陈健宁, 张灵, 等. “双碳” 与“新基建” 背景下特高压输电技术的发展机遇[J]. 高电压技术, 2021, 47(7): 2396-2408.
ZHOU Yuanxiang, CHEN Jianning, ZHANG Ling, et al. Opportunity for developing ultra high voltage transmission technology under the emission peak, carbon neutrality and new infrastructure[J]. High voltage engineering, 2021, 47(7): 2396-2408.
[2] SUMAGAYAN M U, PREMACHANDRA C, MANGORSI R B, et al. Detecting power lines using point instance network for distribution line inspection[J]. IEEE access, 2021, 9: 107998-108008.
[3] 傅博, 姜勇, 王洪光, 等. 输电线路巡检图像智能诊断系统[J]. 智能系统学报, 2016, 11(1): 70-77.
FU Bo, JIANG Yong, WANG Hongguang, et al. Intelligent diagnosis system for patrol check images of power transmission lines[J]. CAAI transactions on intelligent systems, 2016, 11(1): 70-77.
[4] MAO Tianqi, HUANG Kai, ZENG Xianwu, et al. Development of power transmission line defects diagnosis system for UAV inspection based on binocular depth imaging technology[C]//2019 2nd International Conference on Electrical Materials and Power Equipment. Guangzhou: IEEE, 2019: 478-481.
[5] 赵振兵, 张帅, 蒋炜, 等. 基于DBSCAN-FPN的输电线路螺栓缺销检测方法[J]. 中国电力, 2021, 54(3): 45-54.
ZHAO Zhenbing, ZHANG Shuai, JIANG Wei, et al. Detection method for bolts with mission pins on transmission lines based on DBSCAN-FPN[J]. Electric power, 2021, 54(3): 45-54.
[6] ZHAO Zhenbing, QI Hongyu, QI Yincheng, et al. Detection method based on automatic visual shape clustering for pin-missing defect in transmission lines[J]. IEEE transactions on instrumentation and measurement, 2020, 69(9): 6080-6091.
[7] HE Hui, LI Yuchen, YANG Jing, et al. Pin-missing defect recognition based on feature fusion and spatial attention mechanism[J]. Energy reports, 2022, 8: 656-663.
[8] WANG Hongchao, SHAO Yunfeng, ZOU Suli, et al. Detection of cotter pins missing of connection fittings on transmission lines of power system[C]//2021 40th Chinese Control Conference. Shanghai: IEEE, 2021: 6873-6879.
[9] LIN Z, LIANG Y, JIANG Q. A bolt defect recognition algorithm based on attention model[C]//Proceedings of the 7th International Conference on Fuzzy Systems and Data Mining. Seoul: IOS, 2021: 86-93.
[10] 张珂, 何颖宣, 赵凯, 等. 可变形NTS-Net的螺栓属性多标签分类[J]. 中国图象图形学报, 2021, 26(11): 2582-2593.
ZHANG Ke, HE Yingxuan, ZHAO Kai, et al. Multi-label classification method of bolt attributes based on deformable NTS-Net[J]. Journal of image and graphics, 2021, 26(11): 2582-2593.
[11] ZHAO Zhenbing, WANG Rui, LI Yanxu, et al. A new multilabel recognition framework for transmission lines bolt defects based on the combination of semantic knowledge and structural knowledge[J]. IEEE transactions on instrumentation and measurement, 2022, 71: 5020211.
[12] 赵振兵, 王睿, 赵文清, 等. 基于图知识推理的输电线路缺销螺栓识别方法[J]. 智能系统学报, 2023, 18(2): 372-380.
ZHAO Zhenbing, WANG Rui, ZHAO Wenqing, et al. Pin-missing bolts recognition method for transmission lines based on graph knowledge reasoning[J]. CAAI transactions on intelligent systems, 2023, 18(2): 372-380.
[13] HE Kaiming, ZHANG Xiangyu, REN Shaoqing, et al. Deep residual learning for image recognition[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016: 770-778.
[14] 赵强, 左石. 输电线路金具理论与应用[M]. 北京: 中国电力出版社, 2013: 2-12.
[15] OGUZ YAZICI V, GONZALEZ-GARCIA A, RAMISA A, et al. Orderless recurrent models for multi-label classification[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle: IEEE, 2020: 13437-13446.
[16] KE Xiao, LIU Tongan, LI Zhenda. Human attribute recognition method based on pose estimation and multiple-feature fusion[J]. Signal, image and video processing, 2020, 14(7): 1441-1449.
[17] ZHOU Feng, HUANG Sheng, XING Yun. Deep semantic dictionary learning for multi-label image classification[EB/OL]. (2020-12-23)[2021-12-01]. https://arxiv.orglabs/2012.12509.
[18] CHEN Zhaomin, WEI Xiushen, WANG Peng, et al. Multi-label image recognition with graph convolutional networks[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach: IEEE, 2019: 5172-5181.
[19] GAO Binbin, ZHOU Hongyu. Learning to discover multi-class attentional regions for multi-label image recognition[J]. IEEE transactions on image processing: a publication of the IEEE signal processing society, 2021, 30: 5920-5932.
[20] NGUYEN H D, VU X S, LE D T. Modular graph transformer networks for multi-label image classification[C]// Proceedings of the AAAI Conference on Artificial Intelligence. Vancouver: Association for the Advancement of Artificial Intelligence, 2021: 9092-9100.
[21] FU Kui, LI Jia, MA Lin, et al. Intrinsic relationship reasoning for small object detection[EB/OL]. (2020-09-02)[2021-01-01]. https://arxiv.org/abs/2009.00833.
[22] ZHAO Jiawei, YAN Ke, ZHAO Yifan, et al. Transformer-based dual relation graph for multi-label image recognition[C]//2021 IEEE/CVF International Conference on Computer Vision. Montreal: IEEE, 2021: 163-172.
[23] YE Jin, HE Junjun, PENG Xiaojiang, et al. Attention-driven dynamic graph convolutional network for multi-label image recognition[C]//European Conference on Computer Vision. Cham: Springer, 2020: 649-665.
[24] LIU Yong, WANG Ruiping, SHAN Shiguang, et al. Structure inference net: object detection using scene-level context and instance-level relationships[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 6985-6994.
[25] 翟永杰, 杨旭, 赵振兵, 等. 融合共现推理的Faster R-CNN输电线路金具检测[J]. 智能系统学报, 2021, 16(2): 237-246.
ZHAI Yongjie, YANG Xu, ZHAO Zhenbing, et al. Integrating co-occurrence reasoning for Faster R-CNN transmission line fitting detection[J]. CAAI transactions on intelligent systems, 2021, 16(2): 237-246.
[26] KIPF T N, WELLING M. Semi-supervised classification with graph convolutional networks[EB/OL]. (2016-09-09)[2021-01-01]. https://arxiv.org/abs/1609.02907.
- 相似文献/References:
-
[1]岳湘,王洪光,姜勇,等.一种110kV输电线路巡检机器人机构研究[J].智能系统学报,2016,11(2):155.[doi:10.11992/tis.201511032]
YUE Xiang,WANG Hongguang,JIANG Yong,et al.Design approach for a 110 kV power transmission line inspection robot mechanism[J].CAAI Transactions on Intelligent Systems,2016,11():155.[doi:10.11992/tis.201511032]
[2]宋屹峰,王洪光,康文杰,等.DOI面向断股补修作业任务的电力机器人机构设计[J].智能系统学报,2017,12(2):150.[doi:10.11992/tis.201605004]
SONG Yifeng,WANG Hongguang,KANG Wenjie,et al.Optimizing electric power robot design for broken-strand repair tasks[J].CAAI Transactions on Intelligent Systems,2017,12():150.[doi:10.11992/tis.201605004]
[3]翟永杰,杨旭,赵振兵,等.融合共现推理的Faster R-CNN输电线路金具检测[J].智能系统学报,2021,16(2):237.[doi:10.11992/tis.202012023]
ZHAI Yongjie,YANG Xu,ZHAO Zhenbing,et al.Integrating co-occurrence reasoning for Faster R-CNN transmission line fitting detection[J].CAAI Transactions on Intelligent Systems,2021,16():237.[doi:10.11992/tis.202012023]
[4]翟永杰,王乾铭,杨旭,等.融合外部知识的输电线路多金具解耦检测方法[J].智能系统学报,2022,17(5):980.[doi:10.11992/tis.202107026]
ZHAI Yongjie,WANG Qianming,YANG Xu,et al.A multi-fitting decoupling detection method for transmission lines based on external knowledge[J].CAAI Transactions on Intelligent Systems,2022,17():980.[doi:10.11992/tis.202107026]
[5]戚银城,赵席彬,耿劭锋,等.基于遮挡关系推理的输电线路图像金具检测[J].智能系统学报,2022,17(6):1154.[doi:10.11992/tis.202108036]
QI Yincheng,ZHAO Xibin,GENG Shaofeng,et al.Fittings detection in transmission line images with occlusion relation inference[J].CAAI Transactions on Intelligent Systems,2022,17():1154.[doi:10.11992/tis.202108036]
[6]王巍,杨耀权,王乾铭,等.嵌入视觉关系掩码的多类别金具检测方法[J].智能系统学报,2023,18(3):440.[doi:10.11992/tis.202202021]
WANG Wei,YANG Yaoquan,WANG Qianming,et al.A multi-category fitting detection method with embedded visual relation masks[J].CAAI Transactions on Intelligent Systems,2023,18():440.[doi:10.11992/tis.202202021]
[7]赵振兵,郭广学,王艺衡,等.融合边缘感知与统计纹理知识的输电线路金具锈蚀检测[J].智能系统学报,2024,19(5):1228.[doi:10.11992/tis.202306009]
ZHAO Zhenbing,GUO Guangxue,WANG Yiheng,et al.Rust detection in transmission line fittings via fusion of edge perception and statistical texture knowledge[J].CAAI Transactions on Intelligent Systems,2024,19():1228.[doi:10.11992/tis.202306009]
[8]赵振兵,王睿,赵文清,等.基于图知识推理的输电线路缺销螺栓识别方法[J].智能系统学报,2023,18(2):372.[doi:10.11992/tis.202205004]
ZHAO Zhenbing,WANG Rui,ZHAO Wenqing,et al.Pin-missing bolts recognition method for transmission lines based on graph knowledge reasoning[J].CAAI Transactions on Intelligent Systems,2023,18():372.[doi:10.11992/tis.202205004]
[9]戚银城,耿劭锋,赵振兵,等.基于特征迁移的螺栓图像超分辨率处理方法[J].智能系统学报,2023,18(4):858.[doi:10.11992/tis.202201009]
QI Yincheng,GENG Shaofeng,ZHAO Zhenbing,et al.A method for super resolution processing of bolt image based on feature transfer[J].CAAI Transactions on Intelligent Systems,2023,18():858.[doi:10.11992/tis.202201009]
备注/Memo
收稿日期:2023-5-30。
基金项目:国家自然科学基金项目(61871182, U21A20486)；河北省自然科学基金项目(F2020502009, F2021502008, F2021502013).
作者简介:赵振兵，教授，博士，主要研究方向为电力视觉。主持国家自然科学基金项目等纵向课题10项，获省科技进步一等奖2项。以第一完成人获得国家专利授权16项；以第一作者出版专著2部，发表学术论文50余篇。E-mail：zhaozhenbing@ncepu.edu.cn;王睿，硕士研究生，主要研究方向为电力视觉与知识推理。E-mail：ncepuwr@163.com;苗思雨，博士研究生，主要研究方向为电力视觉技术。E-mail：siyumiao_hbu@163.com。
通讯作者:赵振兵. E-mail：zhaozhenbing@ncepu.edu.cn
更新日期/Last Update:
2024-11-05