Active Visual Recognition Research Articles

Collaborative Active Visual Recognition from Crowds: A Distributed Ensemble Approach.

Active learning is an effective way of engaging users to interactively train models for visual recognition more efficiently. The vast majority of previous works focused on active learning with a single human oracle. The problem of active learning with multiple oracles in a collaborative setting has not been well explored. We present a collaborative computational model for active learning with multiple human oracles, the input from whom may possess different levels of noises. It leads to not only an ensemble kernel machine that is robust to label noises, but also a principled label quality measure to online detect irresponsible labelers. Instead of running independent active learning processes for each individual human oracle, our model captures the inherent correlations among the labelers through shared data among them. Our experiments with both simulated and real crowd-sourced noisy labels demonstrate the efficacy of our model.

A Joint Gaussian Process Model for Active Visual Recognition with Expertise Estimation in Crowdsourcing.

We present a noise resilient probabilistic model for active learning of a Gaussian process classifier from crowds, i.e., a set of noisy labelers. It explicitly models both the overall label noise and the expertise level of each individual labeler with two levels of flip models. Expectation propagation is adopted for efficient approximate Bayesian inference of our probabilistic model for classification, based on which, a generalized EM algorithm is derived to estimate both the global label noise and the expertise of each individual labeler. The probabilistic nature of our model immediately allows the adoption of the prediction entropy for active selection of data samples to be labeled, and active selection of high quality labelers based on their estimated expertise to label the data. We apply the proposed model for four visual recognition tasks, i.e., object category recognition, multi-modal activity recognition, gender recognition, and fine-grained classification, on four datasets with real crowd-sourced labels from the Amazon Mechanical Turk. The experiments clearly demonstrate the efficacy of the proposed model. In addition, we extend the proposed model with the Predictive Active Set Selection Method to speed up the active learning system, whose efficacy is verified by conducting experiments on the first three datasets. The results show our extended model can not only preserve a higher accuracy, but also achieve a higher efficiency.

Role of external stimulation in shaping evoked activity in a macroscopic model of cortex

The activity of the brain exhibits stereotypical patterns at various temporal and spatial scales. Novel techniques such as functional magnetic resonance imaging (fMRI) allow the simultaneous recording of the macroscopic activity from all brain regions while subjects perform various tasks, such as passive hearing and active visual recognition. How neural activity is related to cognitive tasks is a long-standing question for the neuroscience community. This study examines an interesting aspect of this broad question, namely the comparison between the resting state (when the subjects is idle) and evoked activity during a task. Neural activity patterns consist of the correlations of firing activity between cortical regions - or functional connectivity - at a slow time scale (hundreds of ms), which can be related to data obtained from fMRI. The cortex has an intricate architecture of connections, both local and long-range. The latter have been mapped for the whole human cortex using diffusion techniques [1]. A recent model has demonstrated that this broad-scale structural connectivity is crucial to reproduce resting-state patterns of cortical activity [2]. Here we consider the same model and simulate tasks by exciting specific brain regions. We rely on mathematical and numerical analysis in order to uncover the respective roles of the long-range projections and external inputs in shaping the evoked activity. Our primary aim is to understand how the stimulation of specific brain regions can dramatically modify the functional connectivity, as compared to the resting state. We also examine the influence of long-range excitatory projections onto inhibitory populations, a novel dynamical ingredient compared to the previous model, in decorrelating brain regions.

A discrete event framework for autonomous observation under uncertainty

In this work we establish a framework for the general problem of observation, which may be applied to different kinds of visual tasks. We construct ‘intelligent’ high-level control mechanisms for active visual recognition of different processes within a hybrid dynamic system. We address the problem of observing a manipulation process in order to illustrate the ideas and motive behind our framework. We use a discrete event dynamic system as a high-level structuring technique to model the manipulation system. The formulation utilizes the knowledge about the system and the different actions in order to solve the observer problem in an efficient, stable and practical manner. The model uses different tracking mechanisms so that the observer can ‘see’ the workspace of the manipulating robot. An automaton is developed for the hand/object interaction over time and a stabilizing observer is constructed. Low-level modules are developed for recognizing the visual ‘events’ that causes state transitions within the dynamic manipulation system in real time. A coarse quantization of the manipulation actions is used in order to attain an active, adaptive and goaldirected sensing mechanism. The formulation provides high-level symbolic interpretations of the scene under observation. The discrete event framework is augmented with mechanisms for recovering the continuous parametric evolution of the scene under observation and for asserting the state of the manipulation agent. This work examines closely the possibilities for errors, mistakes and uncertainties in the manipulation system, observer construction process and event identification mechanisms. We identify and suggest techniques for modeling these uncertainties. Ambiguities are allowed to develop and are resolved after finite time. Error recovery mechanisms are also devised. The computed uncertainties are utilized for navigating the observer automaton state space, asserting state transitions and developing a suitable tracking mechanism.