[1]林琴,李卫军,董肖莉,等.基于双目视觉的人脸三维重建[J].智能系统学报,2018,13(4):534-542.[doi:10.11992/tis.201701020]
LIN Qin,LI Weijun,DONG Xiaoli,et al.Face reconstruction based on binocular stereo vision[J].CAAI Transactions on Intelligent Systems,2018,13(4):534-542.[doi:10.11992/tis.201701020]
点击复制
LIN Qin,LI Weijun,DONG Xiaoli,et al.Face reconstruction based on binocular stereo vision[J].CAAI Transactions on Intelligent Systems,2018,13(4):534-542.[doi:10.11992/tis.201701020]
基于双目视觉的人脸三维重建
《智能系统学报》[ISSN 1673-4785/CN 23-1538/TP] 卷:
13
期数:
2018年第4期
页码:
534-542
栏目:
学术论文—智能系统
出版日期:
2018-07-05
- Title:
- Face reconstruction based on binocular stereo vision
- Keywords:
- facial topological information; stereo matching; linear interpolation; dense correspondence
- 分类号:
- TP391
- 摘要:
- 基于双目立体匹配算法PatchMatch算法，提出了一种获取人脸三维点云的算法。该算法对局部立体匹配算法PatchMatch进行了优化。该方法既不需要昂贵的设备，也不需要通用的人脸三维模型，而是结合了人脸的拓扑结构信息以及立体视觉局部优化算法。此方法采用非接触式的双目视觉采集技术获取左右视角的人脸图像，利用回归树集合(ensemble of regression trees，ERT)算法对人脸图像进行关键点定位，恢复人脸稀疏的视差估计，运用线性插值方法初步估计脸部的稠密视差值，并结合局部立体匹配算法对得到的视差结果进行平滑处理，重建人脸的三维点云信息。实验结果表明，这种算法能够还原出光滑的稠密人脸三维点云信息，在人脸Bosphorus数据库上取得了更加准确的人脸重建结果。
- Abstract:
- In this paper, we propose a binocular stereo algorithm called PatchMatch for generating a 3D dense point cloud of the human face. The proposed algorithm optimizes a local stereo matching method, also known as PatchMatch, which combines topological information of the human face with a local optimization algorithm for stereo vision and requires neither expensive equipment nor generic face models. With this method, by applying a non-contact binocular vision selection technology, face images at both left and right visual angles are obtained. We use an ensemble of regression trees (ERT) algorithm to position key points of a face image and estimate the sparse disparity of facial landmarks. Then, we use a linear interpolation method to make a preliminarily estimation of the dense facial disparity, and by using the local stereo matching algorithm, we can smooth the obtained visual disparity results and use the three-dimensional point cloud information to rebuild the human face. The experimental results with the Bosphorus database show that the proposed algorithm can recover dense facial three-dimensional point cloud information and obtain more accurate face reconstruction results than other methods on Bosphorus database.
- 参考文献/References:
-
[1] 易唐唐, 董朝贤. 基于面部表情GEM和稀疏立方矩阵的三维人脸识别方法[J]. 重庆邮电大学学报:自然科学版, 2017, 29(2):257-264. YI Tangtang, DONG Chaoxian. 3D face recognition method based on facial expression generic elastic model and sparse cubic matrix[J]. Journal of Chongqing university of posts and telecommunications:natural science edition, 2017, 29(2):257-264.
[2] 赵军, 赵艳, 杨勇, 等. 基于降维的堆积降噪自动编码机的表情识别方法[J]. 重庆邮电大学学报:自然科学版, 2016, 28(6):844-848. ZHAO Jun, ZHAO Yan, YANG Yong, et al. Facial expression recognition method based on stacked denoising auto-encoders and feature reduction[J]. Journal of Chongqing university of posts and telecommunications:natural science edition, 2016, 28(6):844-848.
[3] ROY-CHOWDHURY A K, CHELLAPPA R. Statistical bias in 3-D reconstruction from a monocular video[J]. IEEE transactions on image processing, 2005, 14(8):1057-1062.
[4] KOO H S, LAM K M. Recovering the 3D shape and poses of face images based on the similarity transform[J]. Pattern recognition letters, 2008, 29(6):712-723.
[5] AISSAOUI A, AUGUSTE R, YAHIAOUI T, et al. Fast stereo matching method based on optimized correlation algorithm for face depth estimation[C]//Proceedings of International Conference on Image Processing. Orlando, Florida:IEEE Signal Processing Society, 2012:377-380.
[6] DA F, SUI Y. 3D reconstruction of human face based on an improved seeds-growing algorithm[J]. Machine vision and applications, 2011, 22(5):879-887.
[7] ZHANG R, TSAI P S, CRYER J E, et al. Shape-from-shading:a survey[J]. IEEE transactions on pattern analysis and machine intelligence, 1999, 21(8):690-706.
[8] HORN B K P. Obtaining shape from shading information[C]//Shape from shading. New York:MIT press, 1989:123-171.
[9] 王琨, 郑南宁. 基于SFM算法的三维人脸模型重建[J]. 计算机学报, 2005, 28(6):1048-1053. WANG Kun, ZHANG Nanning. 3D Face modeling based on SFM algorithm[J]. Chinese journal of computer, 2005, 28(6):1048-1053.
[10] SUN Z L, LAM K M, GAO Q W. Depth estimation of face images using the nonlinear least-squares model[J]. IEEE transactions on image processing, 2013, 22(1):17-30.
[11] SUN Z L, LAM K M. Depth estimation of face images based on the constrained ICA model[J]. IEEE Transactions on information forensics and security, 2011, 6(2):360-370.
[12] CHOI J, MEDIONI G, LIN Y, et al. 3D face reconstruction using a single or multiple views[C]//Proceedings of International Conference on Pattern Recognition 2010. Istanbul:IEEE Conference Publications, 2010:3959-3962.
[13] LIU F, ZENG D, ZHAO Q, et al. Joint face alignment and 3D face reconstruction[C]//Proceedings of European Conference on Computer Vision. Springer International Publishing, 2016:545-560.
[14] HUBER P, HU G, TENA R, et al. A multiresolution 3D Morphable Face Model and fitting framework[C]//Proceedings of the 11th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications.[S.l.], 2016:151–155.
[15] HIRSCHMULLER H. Improvements in real-time correlation-based stereo vision[C]//Proceedings of Stereo and Multi-Baseline Vision. Kauai, Hawaii:IEEE Computer Society, 2001:0141.
[16] HIRSCHMULLER H. Stereo processing by semiglobal matching and mutual information[J]. IEEE Transactions on pattern analysis and machine intelligence, 2008, 30(2):328-341.
[17] ZHANG C, LI Z, CHENG Y, et al. Meshstereo:lobal stereo model with mesh alignment regularization for view interpolation[C]//Proceedings of International Conference on Computer Vision. Santiago, Chile:the IEEE, 2015:2057-2065.
[18] AISSAOUI A, MARTINET J, DJERABA C. Rapid and accurate face depth estimation in passive stereo systems[J]. Multimedia tools and applications, 2014, 72(3):2413-2438.
[19] BLEYER M, RHEMANN C, ROTHER C. PatchMatch stereo-stereo matching with slanted support windows[C]//Proceedings of British Machine Vision Conference. Dundee:Warwick Print, 2011, 11:1-11.
[20] KAZEMI V, SULLIVAN J. One millisecond face alignment with an ensemble of regression trees[C]//Proceedings of Computer Vision and Pattern Recognition. Boston, Massachusetts:IEEE, 2014:1867-1874.
[21] ZABIH R, WOODFILL J. Non-parametric local transforms for computing visual correspondence[M]//Computer Vision-ECCV’94. Springer Berlin Heidelberg, 1994:151-158.
[22] SAVRAN A, ALYÜZ N, DIBEKLIO?LU H, et al. Bosphorus database for 3D faceanalysis[C]//Proceedings of European Workshop on Biometrics and Identity Management. Springer Berlin Heidelberg, 2008:47-56.
- 相似文献/References:
-
[1]贾贝贝,阮秋琦.双目立体视觉的三维人脸重建方法[J].智能系统学报,2009,4(6):513.[doi:10.3969/j.issn.1673-4785.2009.06.008]
JIA Bei-bei,RUAN Qiu-qi.3D face reconstruction using binocular stereo vision[J].CAAI Transactions on Intelligent Systems,2009,4():513.[doi:10.3969/j.issn.1673-4785.2009.06.008]
[2]尚倩,阮秋琦,李小利.双目立体视觉的目标识别与定位[J].智能系统学报,2011,6(4):303.
SHANG Qian,RUAN Qiuqi,LI Xiaoli.Target recognition and location based on binocular stereo vision[J].CAAI Transactions on Intelligent Systems,2011,6():303.
[3]何国豪,翟涌,龚建伟,等.车载双目视觉动态级联修正实时立体匹配网络[J].智能系统学报,2022,17(6):1145.[doi:10.11992/tis.202111013]
HE Guohao,ZHAI Yong,GONG Jianwei,et al.Real-time stereo matching network for vehicle binocular vision based on dynamic cascade correction[J].CAAI Transactions on Intelligent Systems,2022,17():1145.[doi:10.11992/tis.202111013]
备注/Memo
收稿日期:2017-01-22。
基金项目:国家自然科学基金项目（90920013）；国家公派留学基金项目（201404910237）.
作者简介:林琴,女,1992年生,硕士研究生,主要研究方向为图像处理、模式识别、计算机视觉;李卫军,男,1975年生,研究员,博士,主要研究方向为仿生图像处理技术、仿生模式识别理论与方法、近红外光谱定性分析技术、高维信息计算。发表学术论文30余篇;董肖莉,女,1985年生,助理研究员,主要研究方向为图像处理、模式识别及智能信息处理。
通讯作者:李卫军.E-mail:wjli@semi.ac.cn
更新日期/Last Update:
2018-08-25