The DataScope: A Mixed-Initiative Architecture for Data Labeling

Abstract

Machine learning promises many advantages, but achieving these promises requires methods that smooth the interaction between humans and machine learning. Machine learning systems require meticulous training on large, labeled datasets. Labeling data is a tedious expensive process, and many times requires complex human-judgment skills. Mixed-initiative designs divide the labor between the artificial agent and the human to make the task at hand more efficient and effective. This paper proposes a mixed-initiative method for efficient coding of video and image data in general and emotion data in particular. We integrate an unsupervised dimensionality reduction algorithm and the R Shiny platform to develop an interactive method that leverages human expertise to label the data more efficiently and effectively. The method, through the interactive web tool, allows the user to explore the data interactively, examine similarities and dissimilarities in the data, and label clusters of many images and video frames at once. The combination of the unsupervised learning algorithm and the R Shiny platform enables interactive exploration and annotation of high-dimensional, complicated data. This method can be used to annotate large data sets faster and can advance research in machine vision.