Analysis and synthesis of facial expressions using decomposable nonlinear generative models

This paper presents a new framework that models facial expressions in multiple people with different expressions and synthesize new stylized subtle facial expressions using the generative models. As facial expressions pass through nonlinear shape deformations during facial expressions, we model the facial expression in nonlinear mapping space based on low dimensional embedding and kernel mapping. Characteristics of different type of expressions and variances in different people are decomposed by analyzing the nonlinear mapping between the Euclidean space of facial motion and a low dimensional embedding of these expressions. Using high resolution tracking of densely sampled 3D data, the generative model can control subtle facial expression characteristics of different person in different expression by low dimensional person dependent style factor and expression type dependent expression factor. The temporal characteristics of the motion can also be controlled by the trajectory sampling on the low dimensional embedding manifold which is independent of person style and expression type. Our experimental results are shown for subtle differences in different smile expressions in different people from dense 3D tracking.