Interpretation Of Video Sequences Research Articles

On the use of semantic technologies for video analytics

The rapid proliferation of smart devices, surveillance cameras, infrastructures and buildings enhanced with the Internet of Things (IoT) technologies has led to a huge explosion of contents, especially in the video domain, determining an ever increasing interest towards the development of methods and tools for automatic analysis and interpretation of video sequences. Through the years, the availability of contextual knowledge has proven to improve video analysis performances in several ways, although the formal representation of semantic content in a shareable and fusion oriented manner is still an open problem, also considering the wide diffusion of Fog and Edge computing architectures for video analytics lately. In this context, an interesting answer has come from Semantic Web (SW) technologies, that opened a new perspective for the so-called Knowledge Based Computer Vision (KBCV), adding novel analytics opportunities, improving accuracy, and facilitating data exchange between video analysis systems in an open extensible manner. In this work, we propose a survey of the papers from the last eighteen years, back when first applications of semantic technologies to video analytics have appeared. The papers, analyzed under different perspectives to give a comprehensive overview of the technologies involved, reveal an interesting trend towards the adoption of SW technologies for video analytics scopes. As a result of our work, some insights about future challenges are also provided.

Video Sequence Compared to Conventional Freeze Image Documentation: A Way to Improve the Sonographic Assessment of Breast Lesions?

To compare video sequence and conventional freeze image documentation of breast ultrasound findings with respect to lesion assessment and the diagnostic power of established ultrasound characteristics. Digitally stored freeze images and corresponding video sequences of 50 breast lesions were randomly arranged and interpreted by eight investigators with breast ultrasound training and experience. Established ultrasound criteria were documented on a standardized classification form for every lesion. The investigators were blinded to the clinical and radiological findings, patient characteristics including age, and lesion histology. Statistical analysis compared both groups and correlated the results with the lesion histology. A receiver operating characteristics (ROC) analysis was performed to evaluate the diagnostic performance of ultrasound criteria in lesions documented by video sequences compared to freeze images. Breast lesions were assessed almost identically in video sequences and freeze images. Only the features echogenic halo, orientation, and margin varied among both groups. The dynamic features compressibility and mobility were highly statistically significant correlated with the lesion histology in the video sequence interpretation (p < 0.0001). ROC analysis revealed almost identical diagnostic accuracy in both groups (area under the curve 0.719 for video sequences and 0.762 for freeze images). Video sequences are an appropriate tool to document lesions in breast ultrasound. In contrast to our hypothesis, however, this tool did not improve the diagnostic power of established ultrasound characteristics compared to freeze image documentation.

Temporal reasoning for scenario recognition in video-surveillance using Bayesian networks

The authors propose a high-level scenario recognition algorithm for video sequence interpretation. The recognition of scenarios is based on a Bayesian networks approach. The model of a scenario contains two main layers. The first one allows events from the observed visual features to be highlighted and the second layer is focused on the temporal reasoning stage. The temporal layer uses specific nodes permitting an event-based approach. These nodes focus on the lifetime of events highlighted from the results of the first layer. The temporal layer then estimates the qualitative and quantitative relations between the different temporal events helpful for the recognition task. The global recognition algorithm is illustrated over real indoor image sequences of an abandoned baggage scenario.

Towards the automatic analysis of complex human body motions

Abstract The classification of human body motion is an integral component for the automatic interpretation of video sequences. In a first part we present an effective approach that uses mixed discrete/continuous states to couple perception with classification. A spline contour is used to track the outline of the person. We show that for a quasi-periodic human body motion, an autoregressive process (ARP) is a suitable model for the contour dynamics. A collection of ARP can then be used as a dynamical model for mixed state Condensation filtering, switching automatically between different motion classes. Subsequently this method is applied to automatically segment sequences which contain different motions into subsequences, which contain only one type of motion. Tracking the contour of moving people is, however, difficult. This is why we propose to classify the type of motion directly from the spatio-temporal features of the image sequence. Representing the image data as a spatio-temporal or XYT cube and taking the ‘epipolar slices’ [Workshop on Computer Vision, Representation and Control, Shanty Creek, MI, October (1985) 168] of the cube reveals that different motions, such as running and walking, have characteristic patterns. A new method, which effectively compresses these motion patterns into a low-dimensional feature vector is introduced. The convincing performance of this new feature extraction method is demonstrated for both the classification and automatic segmentation of video sequences for a diverse set of motions.

Content-based video sequence interpretation

Video sequence interpretation attempts to make sense of events that occur when they have been captured on video. When such scenario understanding can be employed in real-time or near-instantaneously, it can be applied to situations where constant watch would otherwise be needed. The potential for consumer applications is wide-ranging. Examples include safety monitoring of unattended elderly persons, children, pets and security surveillance. In this paper, we present a content-based video sequence interpretation algorithm for confined or semi-confined spaces. The algorithm comprises spatio-temporal video object (VO) segmentation, feature extraction, and scene analysis. Application of the proposed algorithm is illustrated in the context of security monitoring. Our system activates an alarm to alert the user of a suspicious event virtually instantaneously.