Skip to main content

Advertisement

EURASIP Journal on Image and Video Processing
EURASIP Journal on Image and Video Processing Cover Image

Fusion of Appearance Image and Passive Stereo Depth Map for Face Recognition Based on the Bilateral 2DLDA

EURASIP Journal on Image and Video Processing volume 2007, Article number: 038205 (2007) Cite this article

Article metrics

  • 1172 Accesses

  • 4 Citations

Abstract

This paper presents a novel approach for face recognition based on the fusion of the appearance and depth information at the match score level. We apply passive stereoscopy instead of active range scanning as popularly used by others. We show that present-day passive stereoscopy, though less robust and accurate, does make positive contribution to face recognition. By combining the appearance and disparity in a linear fashion, we verified experimentally that the combined results are noticeably better than those for each individual modality. We also propose an original learning method, the bilateral two-dimensional linear discriminant analysis (B2DLDA), to extract facial features of the appearance and disparity images. We compare B2DLDA with some existing 2DLDA methods on both XM2VTS database and our database. The results show that the B2DLDA can achieve better results than others.

[123456789101112131415161718192021222324252627282930313233343536373839]

References

  1. 1.

    Zhao W, Chellappa R, Phillips PJ, Rosenfeld A: Face recognition: a literature survey. ACM Computing Surveys 2003,35(4):399-458. 10.1145/954339.954342

  2. 2.

    Brunelli R, Falavigna D: Person identification using multiple cues. IEEE Transactions on Pattern Analysis and Machine Intelligence 1995,17(10):955-966. 10.1109/34.464560

  3. 3.

    Chang K, Bowyer K, Flynn P: Face recognition using 2D and 3D facial data. Proceedings of ACM Workshop on Multimodal User Authentication, December 2003, Santa Barbara, Calif, USA 25-32.

  4. 4.

    Chang KI, Bowyer KW, Flynn PJ: An evaluation of multimodal 2D+3D face biometrics. IEEE Transactions on Pattern Analysis and Machine Intelligence 2005,27(4):619-624.

  5. 5.

    Tsalakanidou F, Tzovaras D, Strintzis MG: Use of depth and colour eigenfaces for face recognition. Pattern Recognition Letters 2003,24(9-10):1427-1435. 10.1016/S0167-8655(02)00383-5

  6. 6.

    Wang J-G, Kong H, Venkateswarlu R: Improving face recognition performance by combining colour and depth fisherfaces. Proceedings of 6th Asian Conference on Computer Vision, January 2004, Jeju, Korea 126-131.

  7. 7.

    Wang J-G, Toh K-A, Venkateswarlu R: Fusion of appearance and depth information for face recognition. Proceedings of the 5th International Conference on Audio- and Video-Based Biometric Person Authentication (AVBPA '05), July 2005, Rye Brook, NY, USA 919-928.

  8. 8.

    Bowyer KW, Chang K, Flynn P: A survey of approaches and challenges in 3D and multi-modal 3D + 2D face recognition. Computer Vision and Image Understanding 2006,101(1):1-15. 10.1016/j.cviu.2005.05.005

  9. 9.

    Mavridis N, Tsalakanidou F, Pantazis D, Malassiotis S, Strintzis MG: The HISCORE face recognition application: affordable desktop face recognition based on a novel 3D camera. Proceedings of International Conference on Augmented, Virtual Environments and Three Dimensional Imaging (ICAV3D '01), May-June 2001, Mykonos, Greece 157-160.

  10. 10.

    Beumier C, Acheroy M: Automatic face authentication from 3D surface. Proceedings of British Machine Vision Conference (BMVC '98), September 1998, Southampton, UK 449-458.

  11. 11.

    Gordon GG: Face recognition based on depth maps and surface curvature. Geometric Methods in Computer Vision, July 1991, San Diego, Calif, USA, Proceedings of SPIE 1570: 234-247.

  12. 12.

    Lu X, Jain AK: Deformation analysis for 3D face matching. Proceedings of the 7th IEEE Workshop on Applications of Computer Vision /IEEE Workshop on Motion and Video Computing (WACV/MOTION '05), January 2005, Breckenridge, Colo, USA 99-104.

  13. 13.

    Philips PJ, Grother P, Micheals RJ, Blackburn DM, Tabassi E, Bone M: Face recognition vendor test 2002. In Tech. Rep. NIST IR 6965. National Institute of Standards and Technology, Gaithersburg, Md, USA; 2003.

  14. 14.

    Rizvi SA, Phillips PJ, Moon H: The FERET verification testing protocol for face recognition algorithms. In Tech. Rep. NIST IR 6281. National Institute of Standards and Technology, Gaithersburg, Md, USA; 1998.

  15. 15.

    Phillips PJ, Moon H, Rizvi SA, Rauss PJ: The FERET evaluation methodology for face-recognition algorithms. IEEE Transactions on Pattern Analysis and Machine Intelligence 2000,22(10):1090-1104. 10.1109/34.879790

  16. 16.

    Chang KI, Bowyer KW, Flynn PJ, Chen X: Multi-biometrics using facial appearance, shape and temperature. Proceedings of the 6th IEEE International Conference on Automatic Face and Gesture Recognition (FGR '04), May 2004, Seoul, Korea 43-48.

  17. 17.

    Beumier C, Acheroy M: Face verification from 3D and grey level clues. Pattern Recognition Letters 2001,22(12):1321-1329. 10.1016/S0167-8655(01)00077-0

  18. 18.

    Lee JC, Milios EE: Matching range images of human faces. Proceedings of the 3rd International Conference on Computer Vision (ICCV '90), December 1990, Osaka, Japan 722-726.

  19. 19.

    Yacoob Y, Davis LS: Labeling of human face components from range data. CVGIP: Image Understanding 1994,60(2):168-178. 10.1006/ciun.1994.1045

  20. 20.

    Chua C-S, Han F, Ho YK: 3D human face recognition using point signature. Proceedings of the 4th IEEE International Conference on Automatic Face and Gesture Recognition (FG '00), March 2000, Grenoble, France 233-238.

  21. 21.

    Blanz V, Vetter T: Face recognition based on fitting a 3D morphable model. IEEE Transactions on Pattern Analysis and Machine Intelligence 2003,25(9):1063-1074. 10.1109/TPAMI.2003.1227983

  22. 22.

    Blanz V, Vetter T: A morphable model for the synthesis of 3D faces. Proceedings of the 26th Annual Conference on Computer Graphics and Interactive Techniques(SIGGRAPH '99), August 1999, Los Angeles, Calif, USA 187-194.

  23. 23.

    Pan G, Wu Y, Wu Z: Investigating profile extracted from range data for 3D face recognition. Proceedings of the IEEE International Conference on Systems, Man and Cybernetics, October 2003, Washington, DC, USA 2: 1396-1399.

  24. 24.

    Urquhart CW, McDonald JP, Siebert JP, Fryer RJ: Active animate stereo vision. In Proceedings of the 4th British Machine Vision Conference, September 1993, Guildford, UK. University of Surrey; 75-84.

  25. 25.

    Liu K, Cheng Y-Q, Yang J-Y: Algebraic feature extraction for image recognition based on an optimal discriminant criterion. Pattern Recognition 1993,26(6):903-911. 10.1016/0031-3203(93)90056-3

  26. 26.

    Yang J, Zhang D, Frangi AF, Yang J-Y: Two-dimensional PCA: a new approach to appearance-based face representation and recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence 2004,26(1):131-137. 10.1109/TPAMI.2004.1261097

  27. 27.

    Ye J, Janardan R, Li Q: Two-dimensional linear discriminant analysis. Proceedings of Neural Information Processing Systems (NIPS '04), December 2004, Vancouver, British Columbia, Canada 1569-1576.

  28. 28.

    Kong H, Wang L, Teoh EK, Wang J-G, Venkateswarlu R: A framework of 2D fisher discriminant analysis: application to face recognition with small number of training samples. Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR '05), June 2005, San Diego, Calif, USA 2: 1083-1088.

  29. 29.

    Yang J, Zhang D, Yong X, Yang J-Y: Two-dimensional discriminant transform for face recognition. Pattern Recognition 2005,38(7):1125-1129. 10.1016/j.patcog.2004.11.019

  30. 30.

    Visani M, Garcia C, Jolion J-M: Two-dimensional-oriented linear discriminant analysis for face recognition. Proceedings of the International Conference on Computer Vision and Graphics (ICCVG '04), September 2004, Warsaw, Poland 1008-1017.

  31. 31.

    Videre Design : MEGA-D Megapixel Digital Stereo Head. http://users.rcn.com/mclaughl.dnai/sthmdcs.htm

  32. 32.

    Messer K, Matas J, Kittler J, Luettin J, Maitre G: XM2VTSDB: the extended M2VTS database. Proceedings of International Conference on Audio- and Video-Based Biometric Person Authentication (AVBPA '99), March 1999, Washington, DC, USA 72-77.

  33. 33.

    Catmull EE: A subdivision algorithm for computer display of curved surfaces, Ph.D. thesis. Department of Computer Science, University of Utah, Salt Lake City, Utah, USA; 1974.

  34. 34.

    Wang J-G, Sung E: Frontal-view face detection and facial feature extraction using color and morphological operations. Pattern Recognition Letters 1999,20(10):1053-1068. 10.1016/S0167-8655(99)00072-0

  35. 35.

    Jenrich RI: Stepwise discriminant analysis. In Statistical Methods for Digital Computers. Edited by: Enslein K, Ralston A, Wilf HS. John Wiley & Sons, New York, NY, USA; 1977:76-95.

  36. 36.

    Kittler J, Hatef M, Duin RPW, Matas J: On combining classifiers. IEEE Transactions on Pattern Analysis and Machine Intelligence 1998,20(3):226-239. 10.1109/34.667881

  37. 37.

    Choudhury T, Clarkson B, Jebara T, Pentland A: Multimodal person recognition using unconstrained audio and video. Proceedings of the 2nd International Conference on Audio- and Video-Based Person Authentication (AVBPA '99), March 1999, Washington, DC, USA 176-181.

  38. 38.

    Socolinsky DA, Selinger A, Neuheisel JD: Face recognition with visible and thermal infrared imagery. Computer Vision and Image Understanding 2003,91(1-2):72-114. 10.1016/S1077-3142(03)00075-4

  39. 39.

    Belhumeur PN, Hespanha JP, Kriegman DJ: Eigenfaces vs. fisherfaces: recognition using class specific linear projection. IEEE Transactions on Pattern Analysis and Machine Intelligence 1997,19(7):711-720. 10.1109/34.598228

Download references

Author information

Affiliations

  1. Institute for Infocomm Research, 21 Heng Mui Keng Terrace, 119613, Singapore

    • Jian-Gang Wang
    •  & Wei-Yun Yau

  2. School of Electrical and Electronic Engineering, Nanyang Technological University, 50 Nanyang Avenue, 639798, Singapore

    • Hui Kong
    • , Eric Sung
    •  & EamKhwang Teoh

Authors

  1. Search for Jian-Gang Wang in:

  2. Search for Hui Kong in:

  3. Search for Eric Sung in:

  4. Search for Wei-Yun Yau in:

  5. Search for EamKhwang Teoh in:

Corresponding author

Correspondence to Jian-Gang Wang.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 2.0 International License (https://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Reprints and Permissions

About this article

Cite this article

Wang, J., Kong, H., Sung, E. et al. Fusion of Appearance Image and Passive Stereo Depth Map for Face Recognition Based on the Bilateral 2DLDA. J Image Video Proc 2007, 038205 (2007) doi:10.1155/2007/38205

Download citation

Keywords

  • Face Recognition
  • Linear Discriminant Analysis
  • Facial Feature
  • Depth Information
  • Active Range