Visual Perception of Emotional and Conversational Facial Expressions

One of the defining attributes of the human species is sophisticated communication, for which facial expressions are crucial. Traditional research has so far mainly investigated a minority of 6 basic emotional expressions displayed as pictures. Despite the important insights of this approach, its ecological validity is limited: facial movements express more than emotions, and facial expressions are more than just pictures. The objective of the present thesis is therefore to improve the understanding of facial expression recognition by investigating the internal representations of a large range of facial expressions, displayed both as static pictures and as dynamic videos.

To this end, it was necessary to develop and validate a new facial expression database which includes 20.000 stimuli of 55 expressions (study 1). Perceptual representations of the six basic emotional expressions were found previously to rely on evaluation of valence and arousal; study 2 showed that this evaluation generalises to many more expressions, particularly when displayed as videos. While it is widely accepted that knowledge influences perception, how these are linked is largely unknown; study 3 investigated this question by asking how knowledge about facial expressions, instantiated as conceptual representations, relates to perceptual representations of these expressions. A strong link was found which changed with the kind of expressions and the type of display.

In probably the most extensive behavioural studies (with regards to the number of facial expressions used) to date, this thesis suggests that there are commonalities but also differences in processing of emotional and of other types of facial expressions. Thus, to understand facial expression processing, one needs to consider more than the 6 basic emotional expressions. These findings outline first steps towards a new domain in facial expression research, which has implications for a number of research and application fields where facial expressions play a role, ranging from social, developmental, and clinical psychology to computer vision and affective computing research.