Real Video Datasets Research Articles

Event Analysis Based on Multiple Interactive Motion Trajectories

Motion information is regarded as one of the most important cues for developing semantics in video data. Yet it is extremely challenging to build semantics of video clips particularly when it involves interactive motion of multiple objects. Most of the existing research has focused on capturing and modelling the motion of each object individually thus loosing interaction information. Such approaches yield low precision-recall ratios and limited indexing and retrieval performances. This paper presents a novel framework for compact representation of multi-object motion trajectories. Three efficient multi-trajectory indexing and retrieval algorithms based on multilinear algebraic representations are proposed. These include: (i) geometrical multiple-trajectory indexing and retrieval (GMIR), (ii) unfolded multiple-trajectory indexing and retrieval (UMIR), and (iii) concentrated multiple-trajectory indexing and retrieval (CMIR). The proposed tensor-based representations not only remarkably reduce the dimensionality of the indexing space but also enable the realization of fast retrieval systems. The proposed representations and algorithms can be robustly applied to both full and partial (segmented) multiple motion trajectories with varying number of objects, trajectory lengths, and sampling rates. The proposed algorithms have been implemented and evaluated using real video datasets. Simulation results demonstrate that the CMIR algorithm provides superior precision-recall metrics, and smaller query processing time compared to the other approaches.

Batch Nearest Neighbor Search for Video Retrieval

To retrieve similar videos to a query clip from a large database, each video is often represented by a sequence of high- dimensional feature vectors. Typically, given a query video containing m feature vectors, an independent nearest neighbor (NN) search for each feature vector is often first performed. After completing all the NN searches, an overall similarity is then computed, i.e., a single content-based video retrieval usually involves m individual NN searches. Since normally nearby feature vectors in a video are similar, a large number of expensive random disk accesses are expected to repeatedly occur, which crucially affects the overall query performance. Batch nearest neighbor (BNN) search is stated as a batch operation that performs a number of individual NN searches. This paper presents a novel approach towards efficient high-dimensional BNN search called dynamic query ordering (DQO) for advanced optimizations of both I/O and CPU costs. Observing the overlapped candidates (or search space) of a pervious query may help to further reduce the candidate sets of subsequent queries, DQO aims at progressively finding a query order such that the common candidates among queries are fully utilized to maximally reduce the total number of candidates. Modelling the candidate set relationship of queries by a candidate overlapping graph (COG), DQO iteratively selects the next query to be executed based on its estimated pruning power to the rest of queries with the dynamically updated COG. Extensive experiments are conducted on real video datasets and show the significance of our BNN query processing strategy.

Hand Motion Oscillatory Gestures and Multimodal Discourse Analysis

Gesture and speech are part of a single human language system. They are coexpressive and complementary channels in the act of speaking. Whereas speech carries the major load of symbolic presentation, gesture provides the imagistic content. Proceeding from the established cotemporality of gesture and speech, our work on oscillatory gestures and multimodal discourse is discussed. Our new techniques of analyzing hand gestures in the frequency domain are described. By tracking an individual's hands during a speech, hand motion trajectory signals are extracted from real video datasets. Our wavelet-based approach in gestural oscillation extraction is presented as frequency ridges in a frequency-time space. Wavelet ridges are extracted from responses of wavelet analysis. These wavelet ridges are employed to characterize frequency properties of hand motion trajectory signals. Hand motion oscillatory gestures can be extracted from these frequency properties. The potential of such computational cross-modal language analysis is motivated by performing a microanalysis of 2 video datasets. In the first dataset, a participant describes her living space to an interlocutor. In the second, a participant describes her action plan to an interlocutor. The ability of our algorithm to extract gestural oscillations is demonstrated, and the way that oscillatory gestures reveal portions of the discourse structure is shown.

Hierarchical Indexing Structure for Efficient Similarity Search in Video Retrieval

With the rapid increase in both centralized video archives and distributed WWW video resources, content-based video retrieval is gaining its importance. To support such applications efficiently, content-based video indexing must be addressed. Typically, each video is represented by a sequence of frames. Due to the high dimensionality of frame representation and the large number of frames, video indexing introduces an additional degree of complexity. In this paper, we address the problem of content-based video indexing and propose an efficient solution, called the ordered VA-file (OVA-file) based on the VA-file. OVA-file is a hierarchical structure and has two novel features: 1) partitioning the whole file into slices such that only a small number of slices are accessed and checked during k nearest neighbor (kNN) search and 2) efficient handling of insertions of new vectors into the OVA-file, such that the average distance between the new vectors and those approximations near that position is minimized. To facilitate a search, we present an efficient approximate kNN algorithm named ordered VA-LOW (OVA-LOW) based on the proposed OVA-file. OVA-LOW first chooses possible OVA-slices by ranking the distances between their corresponding centers and the query vector, and then visits all approximations in the selected OVA-slices to work out approximate kNN. The number of possible OVA-slices is controlled by a user-defined parameter delta. By adjusting delta, OVA-LOW provides a trade-off between the query cost and the result quality. Query by video clip consisting of multiple frames is also discussed. Extensive experimental studies using real video data sets were conducted and the results showed that our methods can yield a significant speed-up over an existing VA-file-based method and (distance with high query result quality. Furthermore, by incorporating temporal correlation of video content, our methods achieved much more efficient performance

Tri-focal tensor-based multiple video synchronization with subframe optimization

In this paper, we present a novel method for synchronizing multiple (more than two) uncalibrated video sequences recording the same event by free-moving full-perspective cameras. Unlike previous synchronization methods, our method takes advantage of tri-view geometry constraints instead of the commonly used two-view one for their better performance in measuring geometric alignment when video frames are synchronized. In particular, the tri-ocular geometric constraint of point/line features, which is evaluated by tri-focal transfer, is enforced when building the timeline maps for sequences to be synchronized. A hierarchical approach is used to reduce the computational complexity. To achieve subframe synchronization accuracy, the Levenberg-Marquardt method-based optimization is performed. The experimental results on several synthetic and real video datasets demonstrate the effectiveness and robustness of our method over previous methods in synchronizing full-perspective videos.