Affective Visual Perception Using Machine Pareidolia of Facial Expressions

Abstract

This article presents a computer vision approach that can detect and classify abstract face-like patterns, including subliminal faces within a scene. This can be regarded as a way of simulating the phenomenon of pareidolia, that is, the tendency of humans to ‘see faces’ in random structures such as clouds or rocks. The paper describes the system consisting of a component-based face detector and an expression classifier. The face detector creates a number of component images from the original image at different resolutions. A component image is a binary edge image where the edges are segmented into components using a labelling method with a border-following technique. The component images are then overlaid to produce a component height map where large and notable components across all resolutions have high values, while specular and noisy components have low values. The method retains three-shape components, representing two eyes and a mouth, that have height map values that are larger than the noise cut-off value. Support vector machines using scale-invariant feature vectors are applied for ranking these three-shape components by their geometry and size, and their shape semblance to human faces in the training data. The outcome is a facial expression analysis system that uses face components, with the potential to estimate an emotional expression value for a scene by producing an array of emotion scores corresponding to Ekman’s seven Universal Facial Expressions of Emotion. An advantage of this technique, when compared to a holistic method, is that the face components are explicitly isolated. This supports a process of abstraction that can facilitate the detection of distorted and minimal face-like patterns.