Skip to main content
EURASIP Journal on Image and Video Processing
Download PDF

Fusion of Appearance Image and Passive Stereo Depth Map for Face Recognition Based on the Bilateral 2DLDA

EURASIP Journal on Image and Video Processing volume 2007, Article number: 038205 (2007) Cite this article

Abstract

This paper presents a novel approach for face recognition based on the fusion of the appearance and depth information at the match score level. We apply passive stereoscopy instead of active range scanning as popularly used by others. We show that present-day passive stereoscopy, though less robust and accurate, does make positive contribution to face recognition. By combining the appearance and disparity in a linear fashion, we verified experimentally that the combined results are noticeably better than those for each individual modality. We also propose an original learning method, the bilateral two-dimensional linear discriminant analysis (B2DLDA), to extract facial features of the appearance and disparity images. We compare B2DLDA with some existing 2DLDA methods on both XM2VTS database and our database. The results show that the B2DLDA can achieve better results than others.

[123456789101112131415161718192021222324252627282930313233343536373839]

References

  1. Zhao W, Chellappa R, Phillips PJ, Rosenfeld A: Face recognition: a literature survey. ACM Computing Surveys 2003,35(4):399-458. 10.1145/954339.954342

    Article  Google Scholar 

  2. Brunelli R, Falavigna D: Person identification using multiple cues. IEEE Transactions on Pattern Analysis and Machine Intelligence 1995,17(10):955-966. 10.1109/34.464560

    Article  Google Scholar 

  3. Chang K, Bowyer K, Flynn P: Face recognition using 2D and 3D facial data. Proceedings of ACM Workshop on Multimodal User Authentication, December 2003, Santa Barbara, Calif, USA 25-32.

    Google Scholar 

  4. Chang KI, Bowyer KW, Flynn PJ: An evaluation of multimodal 2D+3D face biometrics. IEEE Transactions on Pattern Analysis and Machine Intelligence 2005,27(4):619-624.

    Article  Google Scholar 

  5. Tsalakanidou F, Tzovaras D, Strintzis MG: Use of depth and colour eigenfaces for face recognition. Pattern Recognition Letters 2003,24(9-10):1427-1435. 10.1016/S0167-8655(02)00383-5

    Article  MATH  Google Scholar 

  6. Wang J-G, Kong H, Venkateswarlu R: Improving face recognition performance by combining colour and depth fisherfaces. Proceedings of 6th Asian Conference on Computer Vision, January 2004, Jeju, Korea 126-131.

    Google Scholar 

  7. Wang J-G, Toh K-A, Venkateswarlu R: Fusion of appearance and depth information for face recognition. Proceedings of the 5th International Conference on Audio- and Video-Based Biometric Person Authentication (AVBPA '05), July 2005, Rye Brook, NY, USA 919-928.

    Chapter  Google Scholar 

  8. Bowyer KW, Chang K, Flynn P: A survey of approaches and challenges in 3D and multi-modal 3D + 2D face recognition. Computer Vision and Image Understanding 2006,101(1):1-15. 10.1016/j.cviu.2005.05.005

    Article  Google Scholar 

  9. Mavridis N, Tsalakanidou F, Pantazis D, Malassiotis S, Strintzis MG: The HISCORE face recognition application: affordable desktop face recognition based on a novel 3D camera. Proceedings of International Conference on Augmented, Virtual Environments and Three Dimensional Imaging (ICAV3D '01), May-June 2001, Mykonos, Greece 157-160.

    Google Scholar 

  10. Beumier C, Acheroy M: Automatic face authentication from 3D surface. Proceedings of British Machine Vision Conference (BMVC '98), September 1998, Southampton, UK 449-458.

    Google Scholar 

  11. Gordon GG: Face recognition based on depth maps and surface curvature. Geometric Methods in Computer Vision, July 1991, San Diego, Calif, USA, Proceedings of SPIE 1570: 234-247.

    Article  Google Scholar 

  12. Lu X, Jain AK: Deformation analysis for 3D face matching. Proceedings of the 7th IEEE Workshop on Applications of Computer Vision /IEEE Workshop on Motion and Video Computing (WACV/MOTION '05), January 2005, Breckenridge, Colo, USA 99-104.

    Google Scholar 

  13. Philips PJ, Grother P, Micheals RJ, Blackburn DM, Tabassi E, Bone M: Face recognition vendor test 2002. In Tech. Rep. NIST IR 6965. National Institute of Standards and Technology, Gaithersburg, Md, USA; 2003.

    Google Scholar 

  14. Rizvi SA, Phillips PJ, Moon H: The FERET verification testing protocol for face recognition algorithms. In Tech. Rep. NIST IR 6281. National Institute of Standards and Technology, Gaithersburg, Md, USA; 1998.

    Google Scholar 

  15. Phillips PJ, Moon H, Rizvi SA, Rauss PJ: The FERET evaluation methodology for face-recognition algorithms. IEEE Transactions on Pattern Analysis and Machine Intelligence 2000,22(10):1090-1104. 10.1109/34.879790

    Article  Google Scholar 

  16. Chang KI, Bowyer KW, Flynn PJ, Chen X: Multi-biometrics using facial appearance, shape and temperature. Proceedings of the 6th IEEE International Conference on Automatic Face and Gesture Recognition (FGR '04), May 2004, Seoul, Korea 43-48.

    Google Scholar 

  17. Beumier C, Acheroy M: Face verification from 3D and grey level clues. Pattern Recognition Letters 2001,22(12):1321-1329. 10.1016/S0167-8655(01)00077-0

    Article  MATH  Google Scholar 

  18. Lee JC, Milios EE: Matching range images of human faces. Proceedings of the 3rd International Conference on Computer Vision (ICCV '90), December 1990, Osaka, Japan 722-726.

    Google Scholar 

  19. Yacoob Y, Davis LS: Labeling of human face components from range data. CVGIP: Image Understanding 1994,60(2):168-178. 10.1006/ciun.1994.1045

    Article  Google Scholar 

  20. Chua C-S, Han F, Ho YK: 3D human face recognition using point signature. Proceedings of the 4th IEEE International Conference on Automatic Face and Gesture Recognition (FG '00), March 2000, Grenoble, France 233-238.

    Google Scholar 

  21. Blanz V, Vetter T: Face recognition based on fitting a 3D morphable model. IEEE Transactions on Pattern Analysis and Machine Intelligence 2003,25(9):1063-1074. 10.1109/TPAMI.2003.1227983

    Article  Google Scholar 

  22. Blanz V, Vetter T: A morphable model for the synthesis of 3D faces. Proceedings of the 26th Annual Conference on Computer Graphics and Interactive Techniques(SIGGRAPH '99), August 1999, Los Angeles, Calif, USA 187-194.

    Chapter  Google Scholar 

  23. Pan G, Wu Y, Wu Z: Investigating profile extracted from range data for 3D face recognition. Proceedings of the IEEE International Conference on Systems, Man and Cybernetics, October 2003, Washington, DC, USA 2: 1396-1399.

    Google Scholar 

  24. Urquhart CW, McDonald JP, Siebert JP, Fryer RJ: Active animate stereo vision. In Proceedings of the 4th British Machine Vision Conference, September 1993, Guildford, UK. University of Surrey; 75-84.

    Google Scholar 

  25. Liu K, Cheng Y-Q, Yang J-Y: Algebraic feature extraction for image recognition based on an optimal discriminant criterion. Pattern Recognition 1993,26(6):903-911. 10.1016/0031-3203(93)90056-3

    Article  Google Scholar 

  26. Yang J, Zhang D, Frangi AF, Yang J-Y: Two-dimensional PCA: a new approach to appearance-based face representation and recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence 2004,26(1):131-137. 10.1109/TPAMI.2004.1261097

    Article  Google Scholar 

  27. Ye J, Janardan R, Li Q: Two-dimensional linear discriminant analysis. Proceedings of Neural Information Processing Systems (NIPS '04), December 2004, Vancouver, British Columbia, Canada 1569-1576.

    Google Scholar 

  28. Kong H, Wang L, Teoh EK, Wang J-G, Venkateswarlu R: A framework of 2D fisher discriminant analysis: application to face recognition with small number of training samples. Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR '05), June 2005, San Diego, Calif, USA 2: 1083-1088.

    Google Scholar 

  29. Yang J, Zhang D, Yong X, Yang J-Y: Two-dimensional discriminant transform for face recognition. Pattern Recognition 2005,38(7):1125-1129. 10.1016/j.patcog.2004.11.019

    Article  MATH  Google Scholar 

  30. Visani M, Garcia C, Jolion J-M: Two-dimensional-oriented linear discriminant analysis for face recognition. Proceedings of the International Conference on Computer Vision and Graphics (ICCVG '04), September 2004, Warsaw, Poland 1008-1017.

    Google Scholar 

  31. Videre Design : MEGA-D Megapixel Digital Stereo Head. http://users.rcn.com/mclaughl.dnai/sthmdcs.htm

  32. Messer K, Matas J, Kittler J, Luettin J, Maitre G: XM2VTSDB: the extended M2VTS database. Proceedings of International Conference on Audio- and Video-Based Biometric Person Authentication (AVBPA '99), March 1999, Washington, DC, USA 72-77.

    Google Scholar 

  33. Catmull EE: A subdivision algorithm for computer display of curved surfaces, Ph.D. thesis. Department of Computer Science, University of Utah, Salt Lake City, Utah, USA; 1974.

    Google Scholar 

  34. Wang J-G, Sung E: Frontal-view face detection and facial feature extraction using color and morphological operations. Pattern Recognition Letters 1999,20(10):1053-1068. 10.1016/S0167-8655(99)00072-0

    Article  Google Scholar 

  35. Jenrich RI: Stepwise discriminant analysis. In Statistical Methods for Digital Computers. Edited by: Enslein K, Ralston A, Wilf HS. John Wiley & Sons, New York, NY, USA; 1977:76-95.

    Google Scholar 

  36. Kittler J, Hatef M, Duin RPW, Matas J: On combining classifiers. IEEE Transactions on Pattern Analysis and Machine Intelligence 1998,20(3):226-239. 10.1109/34.667881

    Article  Google Scholar 

  37. Choudhury T, Clarkson B, Jebara T, Pentland A: Multimodal person recognition using unconstrained audio and video. Proceedings of the 2nd International Conference on Audio- and Video-Based Person Authentication (AVBPA '99), March 1999, Washington, DC, USA 176-181.

    Google Scholar 

  38. Socolinsky DA, Selinger A, Neuheisel JD: Face recognition with visible and thermal infrared imagery. Computer Vision and Image Understanding 2003,91(1-2):72-114. 10.1016/S1077-3142(03)00075-4

    Article  Google Scholar 

  39. Belhumeur PN, Hespanha JP, Kriegman DJ: Eigenfaces vs. fisherfaces: recognition using class specific linear projection. IEEE Transactions on Pattern Analysis and Machine Intelligence 1997,19(7):711-720. 10.1109/34.598228

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Institute for Infocomm Research, 21 Heng Mui Keng Terrace, 119613, Singapore

    Jian-Gang Wang & Wei-Yun Yau

  2. School of Electrical and Electronic Engineering, Nanyang Technological University, 50 Nanyang Avenue, 639798, Singapore

    Hui Kong, Eric Sung & EamKhwang Teoh

Authors
  1. Jian-Gang Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hui Kong
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Eric Sung
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Wei-Yun Yau
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. EamKhwang Teoh
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jian-Gang Wang.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 2.0 International License (https://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Reprints and Permissions

About this article

Cite this article

Wang, JG., Kong, H., Sung, E. et al. Fusion of Appearance Image and Passive Stereo Depth Map for Face Recognition Based on the Bilateral 2DLDA. J Image Video Proc 2007, 038205 (2007). https://doi.org/10.1155/2007/38205

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1155/2007/38205

Keywords

  • Face Recognition
  • Linear Discriminant Analysis
  • Facial Feature
  • Depth Information
  • Active Range