[1]吴涛,董肖莉,孟伟,等.基于语义分割的简洁线条肖像画生成方法[J].智能系统学报,2021,16(1):134-141.[doi:10.11992/tis.202101003]
WU Tao,DONG Xiaoli,MENG Wei,et al.Concise line portrait generation method based on semantic segmentation[J].CAAI Transactions on Intelligent Systems,2021,16(1):134-141.[doi:10.11992/tis.202101003]
点击复制
WU Tao,DONG Xiaoli,MENG Wei,et al.Concise line portrait generation method based on semantic segmentation[J].CAAI Transactions on Intelligent Systems,2021,16(1):134-141.[doi:10.11992/tis.202101003]
基于语义分割的简洁线条肖像画生成方法
《智能系统学报》[ISSN 1673-4785/CN 23-1538/TP] 卷:
16
期数:
2021年第1期
页码:
134-141
栏目:
学术论文—人工智能基础
出版日期:
2021-01-05
- Title:
- Concise line portrait generation method based on semantic segmentation
- Keywords:
- semantic segmentation; region contour; edge tangent flow; concise line portrait; face line extraction; fdog; parameter adjustment
- 分类号:
- TP391
- 摘要:
- 针对目前主流的线条提取算法对于区域对比度不明显的边缘的检测能力较弱，且对于所有区域采用无差别、统一化的处理策略，所生成的线条画往往较复杂，非常不利于机器人机械臂绘图的问题，本文提出了一种基于语义分割的简洁线条肖像画生成方法(concise line portrait generation based on semantic segmentation, CLPG-SS)。首先，对人脸图像进行语义分割，将人脸划分为不同的区域，基于不同区域提取边缘轮廓与五官细节线条，进行边缘切向流优化，从而加强方向信息；在此基础上，利用线条图来生成调和图像，并利用优化后的边缘切向流、人脸语义分割结果以及调和图像，针对不同的分割区域调整线条提取方法的参数，实现对细节无关区域的线条过滤和细节重点区域的线条加强，生成简洁线条肖像画。实验结果表明：本文提出的CLPG-SS方法能够有效提取人脸主轮廓线条，并针对不同区域实现了对细节线条的针对性调节，提高了机器人机械臂的绘制效率。
- Abstract:
- Currently, mainstream line extraction algorithms have weak detection capabilities for edges with inconspicuous regional contrast, and they use an undifferentiated and unified processing strategy for all regions. The generated line drawings are often complex, which is very unfavorable for robot manipulator drawing. Given this situation, this paper proposes a concise line portrait generation based on the semantic segmentation (CLPG-SS) method. In this method, semantic segmentation is performed on the face image, and the face is divided into different regions. Edge contour and facial detail lines are extracted based on different regions, and edge tangent flow is optimized to enhance the direction information. On this basis, the line image is used to generate the harmonic image, and the optimized edge tangent flow, facial semantic segmentation results, and harmonic image are used to adjust the parameters of the line extraction method for different segmentation regions to realize the line filtering of the detail independent region and line enhancement of the detail focus region, generating a concise line portrait. The experimental results showed that the proposed CLPG-SS method could effectively extract the main contour lines of a human face, adjust the detail lines for different regions, and improve the rendering efficiency of a robot manipulator.
- 参考文献/References:
-
[1] 孟盼盼. 肖像绘制机器人技术研究[D]. 合肥:中国科学技术大学, 2011.
MENG Panpan. Research on auto-portrait robot technology[D]. Hefei:University of Science and Technology of China, 2011.
[2] 董肖莉. 基于人脸图像的机器肖像艺术技术研究[D]. 北京:中国科学院大学, 2018.
DONG Xiaoli. Research on technologies of machine artistic portrait based on facial images[D]. Beijing:University of Chinese Academy of Sciences, 2018.
[3] CANNY J. A computational approach to edge detection[J]. IEEE transactions on pattern analysis and machine intelligence, 1986, 8(6):679-698.
[4] GOOCH B, REINHARD E, GOOCH A. Human facial illustrations:creation and psychophysical evaluation[J]. ACM transactions on graphics, 2004, 23(1):27-44.
[5] MARR D, HILDRETH E. Theory of edge detection[J]. Proceedings of the royal society B:biological sciences, 1980, 207(1167):187-217.
[6] KANG H, LEE S, CHUI C K. Coherent line drawing[C]//Proceedings of the 5th International Symposium on Non-photorealistic Animation and Rendering. San Diego, CA, USA, 2007:43-50.
[7] WANG Shandong, MA Ziyang, LIU Xuehui, et al. Coherence-enhancing line drawing for color images[J]. Science China information sciences, 2013, 56(11):1-11.
[8] undefined王山东, 刘学慧, 陈彦云, 等. 基于特征流的抽象线条画绘制[J]. 计算机学报, 2014, 37(3):611-620.
WANG Shandong, LIU Xuehui, CHEN Yanyun, et al. Abstract line drawings from photographs using flow-based filters[J]. Chinese journal of computers, 2014, 37(3):611-620.
[9] WANG Shandong, WU Enhua, LIU Youquan, et al. Abstract line drawings from photographs using flow-based filters[J]. Computers and graphics, 2012, 36(4):224-231.
[10] 柳有权, 吴宗胜, 王山东, 等. 建筑物图像的线条画生成算法[J]. 软件学报, 2012, 23(S2):34-41
LIU Youquan, WU Zongsheng, WANG Shandong, et al. Line drawing technique for building images[J]. Journal of software, 2012, 23(S2):34-41
[11] KYPRIANIDIS J E, KANG H, D?LLNER J. Image and video abstraction by anisotropic Kuwahara filtering[J]. Computer graphics forum, 2009, 28(7):1955-1963.
[12] Winnem?ller H, Kyprianidis J E, Olsen S C. XDoG:an extended difference-of-Gaussians compendium including advanced image stylization[J]. Computers and graphics, 2012, 36(6):740-753.
[13] ZHANG Liliang, LIN Liang, WU Xian, et al. End-to-end photo-sketch generation via fully convolutional representation learning[C]//Proceedings of the 5th ACM on International Conference on Multimedia Retrieval. Shanghai, China, 2015:627-634.
[14] ZHU Junyan, PARK T, ISOLA P, et al. Unpaired image-to-image translation using cycle-consistent adversarial networks[C]//Proceedings of 2017 IEEE International Conference on Computer Vision. Venice, Italy, 2017:2223-2232.
[15] TAIGMAN Y, POLYAK A, WOLF L. Unsupervised cross-domain image generation[J]. arXiv:1611.02200, 2016.
[16] SASAKI K, IIZUKA S, SIMO SERRA E, et al. Learning to restore deteriorated line drawing[J]. The visual computer, 2018, 34(6/8):1077-1085.
[17] CHEN L C, PAPANDREOU G, KOKKINOS I, et al. Semantic image segmentation with deep convolutional nets and fully connected CRFs[C]//Proceedings of the 3rd International Conference on Learning Representations. San Diego, CA, USA, 2014.
[18] CHEN L C, PAPANDREOU G, KOKKINOS I, et al. DeepLab:Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs[J]. IEEE transactions on pattern analysis and machine intelligence, 2018, 40(4):834-848.
[19] CHEN L C, PAPANDREOU G, SCHROFF F, et al. Rethinking atrous convolution for semantic image segmentation[J]. arXiv preprint arXiv:1706.05587, 2017.
[20] CHEN L C, ZHU Yukun, PAPANDREOU G, et al. Encoder-decoder with atrous separable convolution for semantic image segmentation[C]//Proceedings of the 15th European Conference on Computer Vision (ECCV). Munich, Germany, 2018:801-818.
- 相似文献/References:
-
[1]王兴武,雷涛,王营博,等.基于多模态互补特征学习的遥感影像语义分割[J].智能系统学报,2022,17(6):1123.[doi:10.11992/tis.202201025]
WANG Xingwu,LEI Tao,WANG Yingbo,et al.Semantic segmentation of remote sensing image based on multimodal complementary feature learning[J].CAAI Transactions on Intelligent Systems,2022,17():1123.[doi:10.11992/tis.202201025]
[2]王潇棠,闫河,刘建骐,等.一种边缘梯度插值的双分支deeplabv3+语义分割模型[J].智能系统学报,2023,18(3):604.[doi:10.11992/tis.202111023]
WANG Xiaotang,YAN He,LIU Jianqi,et al.A new deeplabv3+ semantic segmentation model of edge gradient interpolation with double branch structure[J].CAAI Transactions on Intelligent Systems,2023,18():604.[doi:10.11992/tis.202111023]
[3]翟永杰,王家豪,张鑫,等.基于语义分割视觉伺服的种苗自动夹取系统设计[J].智能系统学报,2023,18(6):1259.[doi:10.11992/tis.202212026]
ZHAI Yongjie,WANG Jiahao,ZHANG Xin,et al.Design of automatic picking system for seedlings based on semantic segmentation visual servo[J].CAAI Transactions on Intelligent Systems,2023,18():1259.[doi:10.11992/tis.202212026]
[4]庞荣,杨燕,冷雄进,等.基于双分支点流语义先验的路面病害分割模型[J].智能系统学报,2024,19(1):153.[doi:10.11992/tis.202306037]
PANG Rong,YANG Yan,LENG Xiongjin,et al.Segmentation model of pavement diseases based on semantic priori of double-branched point flow[J].CAAI Transactions on Intelligent Systems,2024,19():153.[doi:10.11992/tis.202306037]
[5]邵凯,王明政,王光宇.基于Transformer的多尺度遥感语义分割网络[J].智能系统学报,2024,19(4):920.[doi:10.11992/tis.202304026]
SHAO Kai,WANG Mingzheng,WANG Guangyu.Transformer-based multiscale remote sensing semantic segmentation network[J].CAAI Transactions on Intelligent Systems,2024,19():920.[doi:10.11992/tis.202304026]
[6]赵振兵,郭广学,王艺衡,等.融合边缘感知与统计纹理知识的输电线路金具锈蚀检测[J].智能系统学报,2024,19(5):1228.[doi:10.11992/tis.202306009]
ZHAO Zhenbing,GUO Guangxue,WANG Yiheng,et al.Rust detection in transmission line fittings via fusion of edge perception and statistical texture knowledge[J].CAAI Transactions on Intelligent Systems,2024,19():1228.[doi:10.11992/tis.202306009]
备注/Memo
收稿日期:2021-01-14。
作者简介:吴涛，硕士研究生，主要研究方向为图像处理、机器学习;董肖莉，助理研究员，主要研究方向为图像处理、模式识别、机器视觉。参与国家重大科学仪器设备开发专项项目1项，授权发明专利8项、软件著作权11项。发表学术论文10余篇。;孟伟，副教授，主要研究方向为物联网技术、人工智能。参与或主持“十一五”国家科技支撑计划项目、校级创新团队项目和横向课题多项。发表学术论文10余篇。
通讯作者:孟伟. E-mail：mnancy@bjfu.edu.cn
更新日期/Last Update:
2021-02-25