Automatic Face Re-Enactment in Real-World Portrait Videos to Manipulate Emotional Expression

The face is one of the main channels for humans to communicate. Accordingly, we are able to recognize facial expressions and emotions efficiently and detect wrongly expressed or fake emotions. Therefore, tools that artificially generate facial expressions must pay great attention to the representation of emotions.

Current facial manipulation techniques are already able to achieve photo-realistic and temporally-consistent results. Although the technical possibilities are rapidly progressing, current approaches mainly focus on achieving fast, perceptually plausible results, disregarding further effects derived from altering the original facial expressivity of the recorded individual.

Thus, in this work we propose a pipeline that builds up on a state-of the-art technique to reenact video portraits of a person with different expressions gathered from motion capture data.

To examine the efficiency of the presented technique, we conduct two perceptual studies with more than $800$ participants in total. The results indicate that the generated videos are, indeed, able to convey the intended emotions. Furthermore, this thesis also provides empirical evidence that the effect of using the proposed reenacted facial expressions only concerns the conveyed meaning, and has no further impact on the way people judge the personality of the manipulated subject. Thus, is a safe and promising technique to alter the expressions on a video portrait without undesired effects, e.g., substantially changing the perceived inner qualities of the person.