No Access

IMAGE BASED 3D FACE RECONSTRUCTION: A SURVEY

GEORGIOS STYLIANOU

Department of Computer Science, European University Cyprus, Cyprus

6 Diogenes str. P. O. Box 22006, 1516 Engomi, Nicosia, Cyprus.

Search for more papers by this author

and

ANDREAS LANITIS

Department of Multimedia and Graphic Arts, Cyprus University of Technology, Cyprus

P. O. Box 50329, 3036, Lemesos, Cyprus.

Search for more papers by this author

https://doi.org/10.1142/S0219467809003411Cited by:20 (Source: Crossref)

Abstract

The use of 3D data in face image processing applications has received considerable attention during the last few years. A major issue for the implementation of 3D face processing systems is the accurate and real time acquisition of 3D faces using low cost equipment. In this paper we provide a survey of 3D reconstruction methods used for generating the 3D appearance of a face using either a single or multiple 2D images captured with ordinary equipment such as digital cameras and camcorders. In this context we discuss various issues pertaining to the general problem of 3D face reconstruction such as the existence of suitable 3D face databases, correspondence of 3D faces, feature detection, deformable 3D models and typical assumptions used during the reconstruction process. Different approaches to the problem of 3D reconstruction are presented and for each category the most important advantages and disadvantages are outlined. In particular we describe example-based methods, stereo methods, video-based methods and silhouette-based methods. The issue of performance evaluation of 3D face reconstruction algorithms, the state of the art and future trends are also discussed.

Keywords:

Remember to check out the Check out our Most Cited Articles!
Check out these titles on Image Analysis

References

Face-Recognition, "Vendors," http://www.face-rec.org/vendors/, Last Accessed June 2008 . Google Scholar
W. Zhao, R. Chellappa, A. Rosenfeld and P. J. Phillips, "Face recognition: A literature survey," ACM Computing Surveys, pp. 399–458 (2003) . Google Scholar
W. Zhao and R. Chellapa, "Face processing — advanced modeling and methods," AP Academic Press (2006) . Google Scholar
K. W. Bowyer, K. Chang and P. Flynn, Computer Vision and Image Understanding 101(1), 1 (2006), DOI: 10.1016/j.cviu.2005.05.005. Crossref, Web of Science, Google Scholar
A. Scheenstra , A. Ruifrok and R. C. Veltkamp , A survey of 3D face recognition methods , Proceedings of the 5th International Conference of Audio- and Video-Based Biometric Person Authentication ( 2005 ) . Google Scholar
J. Kittler et al. , 3D assisted face recognition: A survey of 3D imaging, modeling and recognition approaches , Proceedings of the IEEE Workshop on Advanced 3D Imaging for Safety and Security ( 2005 ) . Google Scholar
Z. Wen and T. S. Huang, "3D face processing," Kluwer Academic Publishers (2004) . Google Scholar
V. Blanz and T. Vetter, Siggraph 187 (1999). Google Scholar
V. Blanz and T. Vetter, IEEE Transactions of Pattern Analysis and Machine Intelligence 25(9), 1063 (2003), DOI: 10.1109/TPAMI.2003.1227983. Crossref, Web of Science, Google Scholar
BJUT, "Bjut 3D face database," http://www.bjut.edu.cn/sci/multimedia/mul-lab/3dface/face_database.htm, Last Accessed December 2007 . Google Scholar
MPI, "Max Planck Institute face database," http://faces.kyb.tuebingen.mpg.de, Last Accessed June 2008 . Google Scholar
USF, "Human id database," http://www.cse.usf.edu/~sarkar/index_files/DataAnd Code.htm, Last Accessed June 2008 . Google Scholar
Cyberware, "Rapid 3D scanners," http://www.cyberware.com/, Last Accessed June 2008 . Google Scholar
Y. Huet al., Building Large Scale 3D Face Database for Face Analysis 4577, 343 (2007). Google Scholar
J. J. Atick, P. A. Griffin and N. A. Redlich, Neural Computation 8, 1321 (1996), DOI: 10.1162/neco.1996.8.6.1321. Crossref, Web of Science, Google Scholar
P. Besl and N. McKay, IEEE Transactions on Pattern Analysis and Machine Intelligence 14(2), 239 (1992), DOI: 10.1109/34.121791. Crossref, Web of Science, Google Scholar
T. J. Huttonet al., IEEE Transactions on Medical Imaging 22(6), (2003), DOI: 10.1109/TMI.2003.814784. Google Scholar
T. J. Hutton , B. F. Buxton and P. Hammond , Dense surface point distribution model of the human face , Proceedings of the CVPR Workshop on Mathematical Methods in Biomedical Image Analysis ( 2001 ) . Google Scholar
T. Akimoto, Y. Suenaga and R. S. Wallace, IEEE Computer Graphics and Applications (1993). Google Scholar
T. Goto, W.-S. Lee and N. Magnenat-Thalmann, Signal Processing: Image Communication (2001). Google Scholar
I. K. Park, H. Zhang and V. Vezhnevets, EURASIP Journal on Applied Signal Processing 13, 2072 (2005). Google Scholar
M. Brand, Morphable 3D Models from Video, Technical Report, Mitsubishi Electric Research Lab (2001) . Google Scholar
C. Bregler , A. Hertzmann and H. Biermann , Recovering non-rigid 3D shape from image streams , Proc. of IEEE Conf. on Computer Vision and Pattern Recognition ( 2000 ) . Google Scholar
S. Huq et al. , Stereo matching with energy minimizing snake grid for 3D face modeling , Defense and Security Symposium ( 2004 ) . Google Scholar
W. Y. Zhao and R. Chellappa, Computer Vision and Pattern Recognition (2000). Google Scholar
I. Shimshoni, Y. Mose and Y. Lindenbaum, Int. Journal of Computer Vision 39(2), 97 (2000), DOI: 10.1023/A:1008118909580. Crossref, Web of Science, Google Scholar
R. Dovgard and R. Basri , Statistical symmetric shape from shading for 3D structure recovery of faces , European Conference on Computer Vision ( 2004 ) . Google Scholar
D. Nandy and J. Ben-Arie, Shape from recognition and learning: Recovery of 3D face shapes, CVPR (1999) pp. 2–7. Google Scholar
M. Reiter et al. , Estimation of face depth maps from color textures using canonical correlation , Computer Vision Winter Workshop ( 2006 ) . Google Scholar
J. Ahlberg, Candide-3 — An Updated Parameterized Face — Techical Report, Tech. Rep. 2326, Dept. of Electrical Engineering, Linkoping University, Sweden (2001) . Google Scholar
M. Rydfalk, Candice, A Parameterized Face, Tech. Rep., Department of Electrical Engineering, Linkoping University (1987) . Google Scholar
M. Kirby and L. Sirovich, IEEE Transactions on Pattern Analysis and Machine Intelligence 12(1), 103 (1991), DOI: 10.1109/34.41390. Crossref, Web of Science, Google Scholar
M. Turk and A. Pentland, Journal of Cognitive Neuroscience 3(1), 71 (1991), DOI: 10.1162/jocn.1991.3.1.71. Crossref, Web of Science, Google Scholar
T. J. Hutton, Dense Surface Models of the Human Face, PhD thesis, University College London (2004) . Google Scholar
T. F. Cooteset al., Computer Vision Graphics and Image Understanding 61(1), 38 (1995), DOI: 10.1006/cviu.1995.1004. Crossref, Web of Science, Google Scholar
V. Blanz and T. Vetter, Informationstechnik und Technische Informatik (2002). Google Scholar
V. Blanz and T. Vetter, IEEE Trans. on Pattern Analysis and Machine Intelligence 25, 1063 (2003), DOI: 10.1109/TPAMI.2003.1227983. Crossref, Web of Science, Google Scholar
S. Baker et al. , Real time non-rigid driver head tracking for driver mental state estimation , 11th World Congress on Intelligent Transportation Systems ( 2004 ) . Google Scholar
V. Blanzet al., Computer Graphics Forum 22(3), 641 (2003), DOI: 10.1111/1467-8659.t01-1-00712. Crossref, Web of Science, Google Scholar
C. Basso and T. Vetter , Statistically motivated 3D faces reconstruction , Proceedings of the 2nd International Conference on Reconstruction of Soft Facial Parts 31 , BKA Research Series ( Luchterhand Publishers , 2005 ) . Google Scholar
C. Basso , P. Paysan and T. Vetter , Registration of expressions data using a 3D morphable model , Proceedings of the 7th IEEE International Conference Automatic Face and Gesture Recognition ( 2006 ) . Google Scholar
V. Blanz and T. Vetter, Reconstructing the Complete 3D Shape of Faces from Partial Information, Tech. Rep., University of Freiburg, 2002 . Google Scholar
A. J. O'Tooleet al., Perception 26, 719 (1997), DOI: 10.1068/p260719. Crossref, Web of Science, Google Scholar
T. Vetter and V. Blanz , Estimating colored 3D face models from single images: An example-based approach , Proceedings of the European Conference on Computer Vision ( 1998 ) . Google Scholar
S. Romdhani, Face Image Analysis using a Multiple Features Fitting Strategy, PhD thesis, University of Basel (2005) . Google Scholar
Y. Hu et al. , Automatic 3D reconstruction for face recognition , IEEE Int. Conf. on Automatic Face and Gesture Recognition ( 2004 ) . Google Scholar
J. Leeet al., Estimation of 3D faces and illumination from single photographs using a bilinear illumination model, Proc. of the Eurographics Symposium on Rendering Techniques (2005) pp. 73–82. Google Scholar
W. Lee , P. Kalra and N. M. Thalmann , Model-based face reconstruction for animation , Int. Conf. on Multimedia Modeling ( 1997 ) . Google Scholar
W. Lee , Y. Wu and N. Magnenat-Thalmann , Cloning and aging in a VR family , Proceedings of IEEE VR'99 (Virtual Reality) ( 1999 ) . Google Scholar
D. Onofrio et al. , 3D face reconstruction with a four camera acquisition system , Int. Workshop on Very Low Bit-Rate Video-Coding ( 2005 ) . Google Scholar
R. Lengagne , J. Tarel and O. Monga , From 2D images to 3D face geometry , Int. Conf. on Automatic Face and Gesture Recognition ( 1996 ) . Google Scholar
R. Lengagne, P. Fua and O. Monga, Image and Vision Computing 18(4), 337 (2000), DOI: 10.1016/S0262-8856(99)00058-X. Crossref, Web of Science, Google Scholar
P. Fua, Int. Journal of Computer Vision 38(2), 153 (2000), DOI: 10.1023/A:1008105802790. Crossref, Web of Science, Google Scholar
L. Xin et al. , Automatic 3D face modeling from video , IEEE Int. Conf. on Computer Vision ( 2005 ) . Google Scholar
W. H. Leunget al., Multimedia and Expo (2000). Google Scholar
Z. Liuet al., The Journal of Visualization and Computer Animation 12(4), 227 (2001), DOI: 10.1002/vis.260. Crossref, Web of Science, Google Scholar
A. R. K. Chowdhuryet al., 3D face reconstruction from video using a generic model, Proc of Int. Conf. on Multimedia (2002) pp. 449–452. Google Scholar
A. R. K. Chowdhury and R. Chellappa, Computer Vision and Image Understanding, Special Issue on Face Recognition 91(2), 188 (2003). Crossref, Web of Science, Google Scholar
B. Moghaddamet al., Analysis and Modeling of Faces and Gestures (2003). Google Scholar
S. Wang, L. Zhang and D. Samaras, Face reconstruction across different poses and arbitrary illumination conditions, Proc. of Audio- and Video-based Biometric Person Authentication (2005) pp. 91–101. Google Scholar
L. Gu and T. Kanade, "3D alignment of face in a single image," 1, 1305–1312 (2006) . Google Scholar
J. Xiao et al. , Real time combined 2d+3d active appearance models , Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition ( 2004 ) . Google Scholar
G. J. Edwards, C. J. Taylor and T. F. Cootes, Interpreting face images using active appearance models, Proceedings of the 5th European Conference on Computer Vision (1998) pp. 581–595. Google Scholar
T. F. Cootes, G. J. Edwards and C. J. Taylor, IEEE Transactions of Pattern Analysis and Machine Intelligence 23, 681 (2001), DOI: 10.1109/34.927467. Crossref, Web of Science, Google Scholar
J. Leeet al., Graphics Interface (2003). Google Scholar
W. S. Leeet al., Computer Animation 186 (1999). Google Scholar
MPEG, "Mpeg4 description," http://www.chiariglione.org/MPEG/standards/mpeg-4/mpeg-4.htm, Last Accessed June 2008 . Google Scholar
I. S. Pandzic and R. Forchheimer, "Mpeg-4 facial animation: The standard, implementation and applications," John Wiley & Sons (2002) . Google Scholar
G. Farin, Computer Aided Geometric Design 7(4), 281 (1990), DOI: 10.1016/0167-8396(90)90036-Q. Crossref, Web of Science, Google Scholar
F. Pighinet al., Synthesizing realistic facial expressions from photographs, Int. Conf. on Computer Graphics and Interactive Techniques (1998) pp. 75–84. Google Scholar
Q. Chen and G. Medioni, Journal of VSLI Signal Processing 127 (2001), DOI: 10.1023/A:1008131816432. Google Scholar
P. Leclercq et al. , Which stereo matching algorithm for accurate 3D face creation? , Int. Workshop on Combinatorial Image Analysis , Lecture Notes in Computer Science ( 2004 ) . Google Scholar
M. Chan , C. Chen and G. Barton , A strategy for 3D face analysis and synthesis , Proc. of Image and Vision Computing ( 2003 ) . Google Scholar
D. Scharstein and R. Szeliski, International Journal of Computer Vision 47(3), 7 (2002), DOI: 10.1023/A:1014573219977. Crossref, Web of Science, Google Scholar
R. Encisoet al., Synthesis of 3D faces, 1st USF International Workshop on Digital and Computational Video (DCV c99) (1999) pp. 8–15. Google Scholar
A. Yilmaz, O. Javed and M. Shah, "Object tracking: A survey," ACM Computing Surveys 38 (4) (2006) . Google Scholar
S. Srinivasan, International Journal of Computer Vision 37, 203 (2000), DOI: 10.1023/A:1008111923880. Crossref, Web of Science, Google Scholar
P. Fua and C. Miccio, From regular images to animated heads: A least squares approach, European Conf. on Computer Vision (1998) pp. 188–202. Google Scholar
P. Fua and C. Miccio, Computer Vision and Image Understanding 75(3), 247 (1999), DOI: 10.1006/cviu.1999.0778. Crossref, Web of Science, Google Scholar
D. Jianget al., Pattern Recognition, Special Issue on Image Understanding for Digital Photographs (2005). Google Scholar
Y. Wang and C. Chua, Image and Vision Computing 23(1), 1018 (2005), DOI: 10.1016/j.imavis.2005.07.005. Crossref, Web of Science, Google Scholar
V. Blanz et al. , A statistical method for robust 3D surface reconstruction from sparse data , Proc. of Int. Symposium on 3D Data Processing, Visualization and Transmission ( 2004 ) . Google Scholar
Georgy Gimel'farb , Binocular stereo by maximizing the likelihood ratio relative to a random terrain , Proceedings of International Workshop RobVis 2001 ( 2001 ) . Google Scholar
B. Tiddeman, M. Burt and D. Perrett, Computer Graphics and Applications 21(5), 42 (2001), DOI: 10.1109/38.946630. Crossref, Web of Science, Google Scholar
Y. Shan , Z. Liu and Z. Zhang , Image-based surface detail transfer , Proceedings of Computer Vision and Pattern Recognition ( 2001 ) . Google Scholar