Table 2 An overview of the model-based face landmarking algorithms

Leung et al. [76],1995

Face image is Gaussian filtered at multiple orientations and scales. This process provides a set of candidate landmarks. Each possible configuration of candidates is validated through random graph matching.

The geometrical relationship between landmarks is expressed with a probabilistic model, which reduces the matching complexity and eliminates irrelevant points.

Eye centers and nose.

Wiskot et al. [9],1997

A labeled graph is constructed where links are the average distances between landmarks and where nodes represent 40-dimensional Gabor jets at candidate locations. The face graph is elastically deformed toward the query face.

Multiple face graphs capture head rotations and bunch graphs capture the various appearances.

An example graph:

Cootes et al. [79],1998

AAM, a generalization of ASM, jointly models the shape and texture variation of the fiducial points. The main goal is to find the appropriate model parameters that minimize the difference between the query and the model face.

PCA models of both texture and shape.

An example of fitting:

Cristinacce et al. [80, 81], 2003

Multiple landmark detectors are run on the face and locate the initial landmarks. Then, two steps are repeated until convergence: First, estimated locations are improved by boosted regression; second, shape model is fitted to the updated landmark locations.

Configurational constraints are applied to eliminate false positives as well as to recover missing landmarks.

17 landmarks: eye, eyebrow, nose, mouth and chin.

Cristinacce et al.[83], 2008

Local templates per each landmark type are combined into a geometrical configuration. The estimated locations are updated by a shape-driven search.

Learned global shape model to avoid non-plausible face shapes.

22 landmarks.

Milborrow andNicolls [31], 2008

Enhancements on ASM such as stacking of two ASMs for better initialization, 2D profile search for individual landmarks etc.

Learned profile models for the individual landmarks and learned global shape model via PCA

76 landmarks.

Belhumeur et al.[106], 2011

A local detector collects SIFT features and landmark-specific SVMs output landmark likelihoods. A Bayesian framework unifies the local evidences into a global shape.

Anatomical and geometrical constraints on facial landmarks derived implicitly from the exemplars.

29 features.

Zhu & Ramanan[99], 2012

Local and global information merged from beginning via tree-connected patches covering the landmarkable zones of the face. Patches represent HOG features while global shape is imposed via quadratic springs between them. The maximum likelihood setting of the tree is searched.

Linearly-parameterized, tree-structured pictorial structure of the landmark rich parts of the face.

68 landmarks forfrontal and 39landmarks for profile faces.