Adaptive deformable model for mouth boundary detection

A new generalized algorithm is proposed to automatically extract a mouth boundary model from human face images. Such an algorithm can contribute to human face recognition and lip-reading-assisted speech recognition systems, in particular, and multimodal human computer interaction systems, in general. The new model is an iterative algorithm based on a hierarchical model adaptation scheme using deformable templates, as a generalization of some of the previous works. The role of prior knowledge is essential for perceptual organization in the algorithm. The prior knowledge about the mouth shape is used to define and initialize a primary deformable model. Each primary boundary curve of a mouth is formed on three control points, including two mouth corners, whose locations are optimized using a primary energy functional. This energy functional essentially captures the knowledge of the mouth shape to perceptually organize image information. The primary model is finely tuned in the second stage of optimization algorithm using a generalized secondary energy functional. Basically each boundary curve is finely tuned using more control points. The primary model is replaced by an adapted model if there is an increase in the secondary energy functional. The results indicate that the new model adaptation technique satisfactorily generalizes the mouth boundary model extraction in an automated fashion.