Uncertain facing

Abstract

Uncertain Facing is a data-driven, interactive audiovisual installation that aims to represent the uncertainty of data points of which their positions in 3D space are estimated by machine learning techniques. It also tries to raise concerns about the possibility of the unintended use of machine learning with synthetic/fake data. Uncertain Facing visualizes the realtime clustering of fake faces in 3D space through t-SNE, a non-linear dimensionality reduction technique, with face embeddings of the faces. This clustering reveals what faces are similar to each other based on the assumption of a probability distribution over data points. However, unlike the original purpose of t-SNE that is meant to be used in an objective data exploration in machine learning, it represents data points as metaballs, in which two or more face images become a merged face when they are close enough, to reflect the uncertain and probabilistic nature of data locations the t-SNE algorithm yields. As a result, metaball rendering is used as a means of an abstract, probabilistic representation of data as opposed to exactness that we expect from the use of scientific visualizations. Along with the t-SNE and metaball-based visualization, Uncertain Facing sonifies the change of the overall data distribution in 3D space based on a granular sound synthesis technique. Uncertain Facing also reflects error values, which t-SNE measures at each iteration between a distribution in original high dimensions and a deduced low-dimensional distribution, to represent the uncertainty of data as jittery motion and inharmonic sound. As an interactive installation, Uncertain Facing allows the audience to see the relationship between their face and the fake faces, implying an aspect that machine learning could be misused in an unintended way as face recognition technology does not distinguish between real and fake faces.