Transformation of an Uncertain Video Search Pipeline to a Sketch-Based Visual Analytics Loop

Philip A Legg,Iwan W Griffiths,David H S Chung,Rhodri Bown,Mark W Jones,Matthew L Parry,Min Chen Min Chen

doi:10.1109/tvcg.2013.207

Philip A Legg, Iwan W Griffiths + Show 5 more

Open Access

https://doi.org/10.1109/tvcg.2013.207

Copy DOI

Abstract

Traditional sketch-based image or video search systems rely on machine learning concepts as their core technology. However, in many applications, machine learning alone is impractical since videos may not be semantically annotated sufficiently, there may be a lack of suitable training data, and the search requirements of the user may frequently change for different tasks. In this work, we develop a visual analytics systems that overcomes the shortcomings of the traditional approach. We make use of a sketch-based interface to enable users to specify search requirement in a flexible manner without depending on semantic annotation. We employ active machine learning to train different analytical models for different types of search requirements. We use visualization to facilitate knowledge discovery at the different stages of visual analytics. This includes visualizing the parameter space of the trained model, visualizing the search space to support interactive browsing, visualizing candidature search results to support rapid interaction for active learning while minimizing watching videos, and visualizing aggregated information of the search results. We demonstrate the system for searching spatiotemporal attributes from sports video to identify key instances of the team and player performance.

Highlights

With the advances in video capture and storage, many application areas have come to rely on collecting vast amounts of video data for the purpose of archive and review [16]
We pursue the idea of visual analytics to present the search results to the user and allow further interaction to tune the system through the acceptance or rejection of results
The user should be presented with an overview visualization that depicts the play, along with additional interactive analysis tools that would help inform their decisions. This should include a search space visualization that shows the search similarity in conjunction with match event data recorded using notational analysis, and a model visualization that depicts how each video frame compares against the similarity metrics that the system employs

Summary

Introduction

With the advances in video capture and storage, many application areas have come to rely on collecting vast amounts of video data for the purpose of archive and review [16]. Analysts will collect video data from multiple cameras for every match of a season They may be collecting and reviewing match videos from opposition teams to analyse traits in their performance that can be exploited in their own strategy. Often the analysts will need to collect together a selection of example video clips that illustrate team strategy, to analyse whether it resulted in positive or negative outcome for the team. Identifying such clips from the video soon becomes a tedious and laborious task for the analysts to perform manually

Results

Discussion

Conclusion