Multi-Task Transfer Methods to Improve One-Shot Learning for Multimedia Event Detection

Abstract

This paper proposes a new multi-task learning method with implicit intertask relevance estimation, and applies it to complex Internet video event detection, which is a challenging and important problem in practice, yet seldom has been addressed. In this paper, “detection” means to detect videos corresponding to the event of interest from a (large) video dataset, not to localize the event spatially or temporally in a video. In the problem definition, one positive and plenty of negative samples of one event are given as training data, and the goal is to return the videos of the same event from a large video dataset. In addition, we assume samples of other events are available. Fig. 1 shows an overview of the proposed methods. The widths of the lines between the one-exemplar event and others represent the inter-event relevance, which is unknown a priori in our problem settings. However, the proposed method can implicity infer the relevance and utilize the most relevant event(s) more in multi-task learning, where the shared information from the relevant events helps to build a better model from the one exemplar. The proposed method does not assume the relevance between other events, as indicated by the red line. Although the learning algorithm outputs models of all input events, only that of the one-exemplar event is applied to detect videos of the event of interest from the video set. Our method builds on the approach of graph-guided multi-task learning [1], which is described first. The training set {(xti,yti)∈R×{−1,+1}, t = 1,2, . . . ,T, i = 1,2, . . . ,Nt} is grouped into T related tasks, which are further organized as a graph G = . The tasks correspond to the elements in the vertex set V , and the pairwise relevance between Task t and k are represented by the weight rtk on edges etk ∈ E. The more relevant the two tasks are, the larger the edge weight is. The graph guided multi-task learning algorithm learns the corresponding T models jointly, by solving the optimization problem