EURASIP Journal on Image and Video Processing

Impact Factor 1.742

EURASIP Journal on Image and Video Processing Cover Image
Research Article
Open Access

Fusion of Appearance Image and Passive Stereo Depth Map for Face Recognition Based on the Bilateral 2DLDA

  • Jian-Gang Wang1Email author,
  • Hui Kong2,
  • Eric Sung2,
  • Wei-Yun Yau1 and
  • EamKhwang Teoh2
EURASIP Journal on Image and Video Processing20072007:038205

https://doi.org/10.1155/2007/38205

©  Jian-GangWang et al. 2007

Received: 27 April 2006

Accepted: 18 June 2007

Published: 28 August 2007

Abstract

This paper presents a novel approach for face recognition based on the fusion of the appearance and depth information at the match score level. We apply passive stereoscopy instead of active range scanning as popularly used by others. We show that present-day passive stereoscopy, though less robust and accurate, does make positive contribution to face recognition. By combining the appearance and disparity in a linear fashion, we verified experimentally that the combined results are noticeably better than those for each individual modality. We also propose an original learning method, the bilateral two-dimensional linear discriminant analysis (B2DLDA), to extract facial features of the appearance and disparity images. We compare B2DLDA with some existing 2DLDA methods on both XM2VTS database and our database. The results show that the B2DLDA can achieve better results than others.

[123456789101112131415161718192021222324252627282930313233343536373839]

Authors’ Affiliations

(1)
Institute for Infocomm Research
(2)
School of Electrical and Electronic Engineering, Nanyang Technological University

References

  1. Zhao W, Chellappa R, Phillips PJ, Rosenfeld A: Face recognition: a literature survey. ACM Computing Surveys 2003,35(4):399-458. 10.1145/954339.954342View ArticleGoogle Scholar
  2. Brunelli R, Falavigna D: Person identification using multiple cues. IEEE Transactions on Pattern Analysis and Machine Intelligence 1995,17(10):955-966. 10.1109/34.464560View ArticleGoogle Scholar
  3. Chang K, Bowyer K, Flynn P: Face recognition using 2D and 3D facial data. Proceedings of ACM Workshop on Multimodal User Authentication, December 2003, Santa Barbara, Calif, USA 25-32.Google Scholar
  4. Chang KI, Bowyer KW, Flynn PJ: An evaluation of multimodal 2D+3D face biometrics. IEEE Transactions on Pattern Analysis and Machine Intelligence 2005,27(4):619-624.View ArticleGoogle Scholar
  5. Tsalakanidou F, Tzovaras D, Strintzis MG: Use of depth and colour eigenfaces for face recognition. Pattern Recognition Letters 2003,24(9-10):1427-1435. 10.1016/S0167-8655(02)00383-5View ArticleMATHGoogle Scholar
  6. Wang J-G, Kong H, Venkateswarlu R: Improving face recognition performance by combining colour and depth fisherfaces. Proceedings of 6th Asian Conference on Computer Vision, January 2004, Jeju, Korea 126-131.Google Scholar
  7. Wang J-G, Toh K-A, Venkateswarlu R: Fusion of appearance and depth information for face recognition. Proceedings of the 5th International Conference on Audio- and Video-Based Biometric Person Authentication (AVBPA '05), July 2005, Rye Brook, NY, USA 919-928.View ArticleGoogle Scholar
  8. Bowyer KW, Chang K, Flynn P: A survey of approaches and challenges in 3D and multi-modal 3D + 2D face recognition. Computer Vision and Image Understanding 2006,101(1):1-15. 10.1016/j.cviu.2005.05.005View ArticleGoogle Scholar
  9. Mavridis N, Tsalakanidou F, Pantazis D, Malassiotis S, Strintzis MG: The HISCORE face recognition application: affordable desktop face recognition based on a novel 3D camera. Proceedings of International Conference on Augmented, Virtual Environments and Three Dimensional Imaging (ICAV3D '01), May-June 2001, Mykonos, Greece 157-160.Google Scholar
  10. Beumier C, Acheroy M: Automatic face authentication from 3D surface. Proceedings of British Machine Vision Conference (BMVC '98), September 1998, Southampton, UK 449-458.Google Scholar
  11. Gordon GG: Face recognition based on depth maps and surface curvature. Geometric Methods in Computer Vision, July 1991, San Diego, Calif, USA, Proceedings of SPIE 1570: 234-247.View ArticleGoogle Scholar
  12. Lu X, Jain AK: Deformation analysis for 3D face matching. Proceedings of the 7th IEEE Workshop on Applications of Computer Vision /IEEE Workshop on Motion and Video Computing (WACV/MOTION '05), January 2005, Breckenridge, Colo, USA 99-104.Google Scholar
  13. Philips PJ, Grother P, Micheals RJ, Blackburn DM, Tabassi E, Bone M: Face recognition vendor test 2002. In Tech. Rep. NIST IR 6965. National Institute of Standards and Technology, Gaithersburg, Md, USA; 2003.Google Scholar
  14. Rizvi SA, Phillips PJ, Moon H: The FERET verification testing protocol for face recognition algorithms. In Tech. Rep. NIST IR 6281. National Institute of Standards and Technology, Gaithersburg, Md, USA; 1998.Google Scholar
  15. Phillips PJ, Moon H, Rizvi SA, Rauss PJ: The FERET evaluation methodology for face-recognition algorithms. IEEE Transactions on Pattern Analysis and Machine Intelligence 2000,22(10):1090-1104. 10.1109/34.879790View ArticleGoogle Scholar
  16. Chang KI, Bowyer KW, Flynn PJ, Chen X: Multi-biometrics using facial appearance, shape and temperature. Proceedings of the 6th IEEE International Conference on Automatic Face and Gesture Recognition (FGR '04), May 2004, Seoul, Korea 43-48.Google Scholar
  17. Beumier C, Acheroy M: Face verification from 3D and grey level clues. Pattern Recognition Letters 2001,22(12):1321-1329. 10.1016/S0167-8655(01)00077-0View ArticleMATHGoogle Scholar
  18. Lee JC, Milios EE: Matching range images of human faces. Proceedings of the 3rd International Conference on Computer Vision (ICCV '90), December 1990, Osaka, Japan 722-726.Google Scholar
  19. Yacoob Y, Davis LS: Labeling of human face components from range data. CVGIP: Image Understanding 1994,60(2):168-178. 10.1006/ciun.1994.1045View ArticleGoogle Scholar
  20. Chua C-S, Han F, Ho YK: 3D human face recognition using point signature. Proceedings of the 4th IEEE International Conference on Automatic Face and Gesture Recognition (FG '00), March 2000, Grenoble, France 233-238.Google Scholar
  21. Blanz V, Vetter T: Face recognition based on fitting a 3D morphable model. IEEE Transactions on Pattern Analysis and Machine Intelligence 2003,25(9):1063-1074. 10.1109/TPAMI.2003.1227983View ArticleGoogle Scholar
  22. Blanz V, Vetter T: A morphable model for the synthesis of 3D faces. Proceedings of the 26th Annual Conference on Computer Graphics and Interactive Techniques(SIGGRAPH '99), August 1999, Los Angeles, Calif, USA 187-194.View ArticleGoogle Scholar
  23. Pan G, Wu Y, Wu Z: Investigating profile extracted from range data for 3D face recognition. Proceedings of the IEEE International Conference on Systems, Man and Cybernetics, October 2003, Washington, DC, USA 2: 1396-1399.Google Scholar
  24. Urquhart CW, McDonald JP, Siebert JP, Fryer RJ: Active animate stereo vision. In Proceedings of the 4th British Machine Vision Conference, September 1993, Guildford, UK. University of Surrey; 75-84.Google Scholar
  25. Liu K, Cheng Y-Q, Yang J-Y: Algebraic feature extraction for image recognition based on an optimal discriminant criterion. Pattern Recognition 1993,26(6):903-911. 10.1016/0031-3203(93)90056-3View ArticleGoogle Scholar
  26. Yang J, Zhang D, Frangi AF, Yang J-Y: Two-dimensional PCA: a new approach to appearance-based face representation and recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence 2004,26(1):131-137. 10.1109/TPAMI.2004.1261097View ArticleGoogle Scholar
  27. Ye J, Janardan R, Li Q: Two-dimensional linear discriminant analysis. Proceedings of Neural Information Processing Systems (NIPS '04), December 2004, Vancouver, British Columbia, Canada 1569-1576.Google Scholar
  28. Kong H, Wang L, Teoh EK, Wang J-G, Venkateswarlu R: A framework of 2D fisher discriminant analysis: application to face recognition with small number of training samples. Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR '05), June 2005, San Diego, Calif, USA 2: 1083-1088.Google Scholar
  29. Yang J, Zhang D, Yong X, Yang J-Y: Two-dimensional discriminant transform for face recognition. Pattern Recognition 2005,38(7):1125-1129. 10.1016/j.patcog.2004.11.019View ArticleMATHGoogle Scholar
  30. Visani M, Garcia C, Jolion J-M: Two-dimensional-oriented linear discriminant analysis for face recognition. Proceedings of the International Conference on Computer Vision and Graphics (ICCVG '04), September 2004, Warsaw, Poland 1008-1017.Google Scholar
  31. Videre Design : MEGA-D Megapixel Digital Stereo Head. http://users.rcn.com/mclaughl.dnai/sthmdcs.htm
  32. Messer K, Matas J, Kittler J, Luettin J, Maitre G: XM2VTSDB: the extended M2VTS database. Proceedings of International Conference on Audio- and Video-Based Biometric Person Authentication (AVBPA '99), March 1999, Washington, DC, USA 72-77.Google Scholar
  33. Catmull EE: A subdivision algorithm for computer display of curved surfaces, Ph.D. thesis. Department of Computer Science, University of Utah, Salt Lake City, Utah, USA; 1974.Google Scholar
  34. Wang J-G, Sung E: Frontal-view face detection and facial feature extraction using color and morphological operations. Pattern Recognition Letters 1999,20(10):1053-1068. 10.1016/S0167-8655(99)00072-0View ArticleGoogle Scholar
  35. Jenrich RI: Stepwise discriminant analysis. In Statistical Methods for Digital Computers. Edited by: Enslein K, Ralston A, Wilf HS. John Wiley & Sons, New York, NY, USA; 1977:76-95.Google Scholar
  36. Kittler J, Hatef M, Duin RPW, Matas J: On combining classifiers. IEEE Transactions on Pattern Analysis and Machine Intelligence 1998,20(3):226-239. 10.1109/34.667881View ArticleGoogle Scholar
  37. Choudhury T, Clarkson B, Jebara T, Pentland A: Multimodal person recognition using unconstrained audio and video. Proceedings of the 2nd International Conference on Audio- and Video-Based Person Authentication (AVBPA '99), March 1999, Washington, DC, USA 176-181.Google Scholar
  38. Socolinsky DA, Selinger A, Neuheisel JD: Face recognition with visible and thermal infrared imagery. Computer Vision and Image Understanding 2003,91(1-2):72-114. 10.1016/S1077-3142(03)00075-4View ArticleGoogle Scholar
  39. Belhumeur PN, Hespanha JP, Kriegman DJ: Eigenfaces vs. fisherfaces: recognition using class specific linear projection. IEEE Transactions on Pattern Analysis and Machine Intelligence 1997,19(7):711-720. 10.1109/34.598228View ArticleGoogle Scholar

Copyright

© Jian-GangWang et al. 2007

This article is published under license to BioMed Central Ltd. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.