[1]LIN Qin,LI Weijun,DONG Xiaoli,et al.Face reconstruction based on binocular stereo vision[J].CAAI Transactions on Intelligent Systems,2018,13(4):534-542.[doi:10.11992/tis.201701020]
Copy

Face reconstruction based on binocular stereo vision

CAAI Transactions on Intelligent Systems[ISSN 1673-4785/CN 23-1538/TP] Volume: 13 Number of periods: 2018 4 Page number: 534-542 Column: 学术论文—智能系统 Public date: 2018-07-05
Title:
Face reconstruction based on binocular stereo vision
Author(s):
LIN Qin1; 2; 3;  LI Weijun1; 2; 3;  DONG Xiaoli1; 2; 3;  NING Xin1; 2; 3;  CHEN Peng1; 2; 3
1. Laboratory of Artificial Neural Networks and High-peed Circuits, Institute of Semiconductors, Chinese Academy of Sciences, Beijing 100083, China;
2. School of Microelectronics, University of Chinese Academy of Sciences, Beijing 100029, China;
3. Cognitive Computing Technology Wei Fu Joint Lab, Beijing 100083, China
Keywords:
facial topological information; stereo matching; linear interpolation; dense correspondence
CLC:
TP391
DOI:
10.11992/tis.201701020
Abstract:
In this paper, we propose a binocular stereo algorithm called PatchMatch for generating a 3D dense point cloud of the human face. The proposed algorithm optimizes a local stereo matching method, also known as PatchMatch, which combines topological information of the human face with a local optimization algorithm for stereo vision and requires neither expensive equipment nor generic face models. With this method, by applying a non-contact binocular vision selection technology, face images at both left and right visual angles are obtained. We use an ensemble of regression trees (ERT) algorithm to position key points of a face image and estimate the sparse disparity of facial landmarks. Then, we use a linear interpolation method to make a preliminarily estimation of the dense facial disparity, and by using the local stereo matching algorithm, we can smooth the obtained visual disparity results and use the three-dimensional point cloud information to rebuild the human face. The experimental results with the Bosphorus database show that the proposed algorithm can recover dense facial three-dimensional point cloud information and obtain more accurate face reconstruction results than other methods on Bosphorus database.
References:
[1] 易唐唐, 董朝贤. 基于面部表情GEM和稀疏立方矩阵的三维人脸识别方法[J]. 重庆邮电大学学报:自然科学版, 2017, 29(2):257-264. YI Tangtang, DONG Chaoxian. 3D face recognition method based on facial expression generic elastic model and sparse cubic matrix[J]. Journal of Chongqing university of posts and telecommunications:natural science edition, 2017, 29(2):257-264.
[2] 赵军, 赵艳, 杨勇, 等. 基于降维的堆积降噪自动编码机的表情识别方法[J]. 重庆邮电大学学报:自然科学版, 2016, 28(6):844-848. ZHAO Jun, ZHAO Yan, YANG Yong, et al. Facial expression recognition method based on stacked denoising auto-encoders and feature reduction[J]. Journal of Chongqing university of posts and telecommunications:natural science edition, 2016, 28(6):844-848.
[3] ROY-CHOWDHURY A K, CHELLAPPA R. Statistical bias in 3-D reconstruction from a monocular video[J]. IEEE transactions on image processing, 2005, 14(8):1057-1062.
[4] KOO H S, LAM K M. Recovering the 3D shape and poses of face images based on the similarity transform[J]. Pattern recognition letters, 2008, 29(6):712-723.
[5] AISSAOUI A, AUGUSTE R, YAHIAOUI T, et al. Fast stereo matching method based on optimized correlation algorithm for face depth estimation[C]//Proceedings of International Conference on Image Processing. Orlando, Florida:IEEE Signal Processing Society, 2012:377-380.
[6] DA F, SUI Y. 3D reconstruction of human face based on an improved seeds-growing algorithm[J]. Machine vision and applications, 2011, 22(5):879-887.
[7] ZHANG R, TSAI P S, CRYER J E, et al. Shape-from-shading:a survey[J]. IEEE transactions on pattern analysis and machine intelligence, 1999, 21(8):690-706.
[8] HORN B K P. Obtaining shape from shading information[C]//Shape from shading. New York:MIT press, 1989:123-171.
[9] 王琨, 郑南宁. 基于SFM算法的三维人脸模型重建[J]. 计算机学报, 2005, 28(6):1048-1053. WANG Kun, ZHANG Nanning. 3D Face modeling based on SFM algorithm[J]. Chinese journal of computer, 2005, 28(6):1048-1053.
[10] SUN Z L, LAM K M, GAO Q W. Depth estimation of face images using the nonlinear least-squares model[J]. IEEE transactions on image processing, 2013, 22(1):17-30.
[11] SUN Z L, LAM K M. Depth estimation of face images based on the constrained ICA model[J]. IEEE Transactions on information forensics and security, 2011, 6(2):360-370.
[12] CHOI J, MEDIONI G, LIN Y, et al. 3D face reconstruction using a single or multiple views[C]//Proceedings of International Conference on Pattern Recognition 2010. Istanbul:IEEE Conference Publications, 2010:3959-3962.
[13] LIU F, ZENG D, ZHAO Q, et al. Joint face alignment and 3D face reconstruction[C]//Proceedings of European Conference on Computer Vision. Springer International Publishing, 2016:545-560.
[14] HUBER P, HU G, TENA R, et al. A multiresolution 3D Morphable Face Model and fitting framework[C]//Proceedings of the 11th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications.[S.l.], 2016:151–155.
[15] HIRSCHMULLER H. Improvements in real-time correlation-based stereo vision[C]//Proceedings of Stereo and Multi-Baseline Vision. Kauai, Hawaii:IEEE Computer Society, 2001:0141.
[16] HIRSCHMULLER H. Stereo processing by semiglobal matching and mutual information[J]. IEEE Transactions on pattern analysis and machine intelligence, 2008, 30(2):328-341.
[17] ZHANG C, LI Z, CHENG Y, et al. Meshstereo:lobal stereo model with mesh alignment regularization for view interpolation[C]//Proceedings of International Conference on Computer Vision. Santiago, Chile:the IEEE, 2015:2057-2065.
[18] AISSAOUI A, MARTINET J, DJERABA C. Rapid and accurate face depth estimation in passive stereo systems[J]. Multimedia tools and applications, 2014, 72(3):2413-2438.
[19] BLEYER M, RHEMANN C, ROTHER C. PatchMatch stereo-stereo matching with slanted support windows[C]//Proceedings of British Machine Vision Conference. Dundee:Warwick Print, 2011, 11:1-11.
[20] KAZEMI V, SULLIVAN J. One millisecond face alignment with an ensemble of regression trees[C]//Proceedings of Computer Vision and Pattern Recognition. Boston, Massachusetts:IEEE, 2014:1867-1874.
[21] ZABIH R, WOODFILL J. Non-parametric local transforms for computing visual correspondence[M]//Computer Vision-ECCV’94. Springer Berlin Heidelberg, 1994:151-158.
[22] SAVRAN A, ALYÜZ N, DIBEKLIO?LU H, et al. Bosphorus database for 3D faceanalysis[C]//Proceedings of European Workshop on Biometrics and Identity Management. Springer Berlin Heidelberg, 2008:47-56.
Similar References:

Memo

-

Last Update: 2018-08-25

Copyright © CAAI Transactions on Intelligent Systems