Digital Video Analysis Research Articles

The relationship between right hemisphere damage and gesture in spontaneous discourse

Background: The assessment and rehabilitation of acquired neurogenic communication disorders rarely involves a systematic analysis of gesture use. The right cerebral hemisphere has been identified as a possible locus of control for gesture. McNeill's (McNeill & Duncan, 2000) growth point theory posits a structure for the organisation of processes from both cerebral hemispheres which serves to support the integration of gestural and verbal messages that emerge from a non‐modality specific cognitive growth point or idea unit. Aims: The first aim of this research was to describe male non‐brain‐damaged (NBD) speakers' gesture use in the context of spontaneous discourse. The second aim was to compare the gesture production patterns of five individual males with right cerebral hemisphere damage (RHD) with the NBD group's pattern of performance. Methods & Procedures: Gesture rates and variation of fundamental frequency were analysed across four speaking conditions; a personal narrative, two procedural narratives, two emotional narratives, and three comic book descriptions. The discourse stimuli were selected to elicit highly emotional versus neutral content. Gesture use was classified according to the system described by McNeill (1992) for digital video analysis. Discourse samples were segmented into intonational phrases (Pierrehumbert & Hirschberg, 1990). Gesture rates and variation in fundamental frequency were calculated within intonational phrases. Outcomes & Results: The NBD group demonstrated higher rates of body beats and head movements in discourse samples with high emotional content. Procedural narratives were accompanied by higher rates of representational gestures than in the other conditions. The RHD participants showed variability in their gesture use across the discourse genres. The majority of RHD participants used lower non‐body‐focused gesture rates but significantly fewer gestures in the discourse samples with high emotional content. Differences in visuo‐spatial ability, variations in fundamental frequency of speech, and body‐focused gesture failed to reveal systematic patterns in the RHD participants. Conclusions: McNeill's (1992) growth point theory provides a useful platform for interpreting the observed reduction in overall frequency of gesture use evident in the RHD participants, but cannot account for the observed interaction involving the emotional narrative. While the impact of emotion on gesture in the non‐brain‐damaged population was unsurprising, the reversal of this pattern in people with right hemisphere damage poses a challenge for theoretical work in this area. The results of this study indicate that the analysis of gesture use is important for a deeper understanding of expressive communication impairments associated with acquired neurogenic impairment.

Mucociliary clearance – a critical upper airway host defense mechanism and methods of assessment

Mucociliary clearance is a critical host defense mechanism of the airways. Effective mucociliary clearance requires appropriate mucus production and coordinated ciliary activity. The important role of these two components is best demonstrated in disorders such as primary ciliary dyskinesia and cystic fibrosis, both of which lead to lifelong recurrent respiratory tract infections. We review the methods used to analyze mucociliary clearance. Utilization of microdialysis probes has improved temporal resolution of mucociliary clearance in murine airways, availing many genetic mouse models to critical mucociliary clearance analysis, while improved fixation technique for transmission electron microscopy has allowed for detailed resolution of the airway surface liquid. High-speed digital video analysis has improved quantification of ciliary beat frequency while advancements in air-liquid interface culturing techniques have generated in-vitro models to investigate mucociliary clearance. Advancements in techniques for analysis of mucociliary clearance have improved our understanding of the interaction between the respiratory epithelium and the airway surface liquid, resulting in the ability to study pathologic processes involving mucociliary clearance in great detail.

Techniques used and open challenges to the analysis, indexing and retrieval of digital video

Video in digital format is now commonplace and widespread in both professional use, and in domestic consumer products from camcorders to mobile phones. Video content is growing in volume and while we can capture, compress, store, transmit and display video with great facility, editing videos and manipulating them based on their content is still a non-trivial activity. In this paper, we give a brief review of the state of the art of video analysis, indexing and retrieval and we point to research directions which we think are promising and could make searching and browsing of video archives based on video content, as easy as searching and browsing (text) web pages. We conclude the paper with a list of grand challenges for researchers working in the area.

Novel method for high-throughput phenotyping of sleep in mice

Assessment of sleep in mice currently requires initial implantation of chronic electrodes for assessment of electroencephalogram (EEG) and electromyogram (EMG) followed by time to recover from surgery. Hence, it is not ideal for high-throughput screening. To address this deficiency, a method of assessment of sleep and wakefulness in mice has been developed based on assessment of activity/inactivity either by digital video analysis or by breaking infrared beams in the mouse cage. It is based on the algorithm that any episode of continuous inactivity of > or =40 s is predicted to be sleep. The method gives excellent agreement in C57BL/6J male mice with simultaneous assessment of sleep by EEG/EMG recording. The average agreement over 8,640 10-s epochs in 24 h is 92% (n = 7 mice) with agreement in individual mice being 88-94%. Average EEG/EMG determined sleep per 2-h interval across the day was 59.4 min. The estimated mean difference (bias) per 2-h interval between inactivity-defined sleep and EEG/EMG-defined sleep was only 1.0 min (95% confidence interval for mean bias -0.06 to +2.6 min). The standard deviation of differences (precision) was 7.5 min per 2-h interval with 95% limits of agreement ranging from -13.7 to +15.7 min. Although bias significantly varied by time of day (P = 0.0007), the magnitude of time-of-day differences was not large (average bias during lights on and lights off was +5.0 and -3.0 min per 2-h interval, respectively). This method has applications in chemical mutagenesis and for studies of molecular changes in brain with sleep/wakefulness.

Framework for Automatic Condition Assessment of Culverts

A framework for real-time and automated monitoring of the condition of culverts based on identifying internal defects via video inspection is developed. Manual inspection using closed-circuit television has several drawbacks, such as inconsistency and subjectivity due to human evaluation. Analysis of digital video, consisting of thousands of megabytes even at lower resolutions, can be laborious and not suited for real-time implementation. An innovative approach is to extract image frames judiciously from the video and analyze frames to locate and categorize major defects. Rather than analyze all extracted frames, one can skip consecutive frames at a minimal loss of accuracy and bring considerable savings in memory and system requirements. Each frame is preprocessed to enhance contrast, through use of an adaptive scheme, and to reduce dimensionality in pixel-space by implemention of region-based processing. Preprocessing is followed by a two-step image segmentation process, which implements a background elimination procedure in the first step and shape detection in the second step. Fuzzy clustering is used as the underlying segmentation model. Defect shape and depth information after post-processing are used as input to an assessment methodology for automated condition state. A simple formulation based on both the damage area and depth is then used to assess the condition of culverts based on a four-point condition assessment scale. The proposed framework is demonstrated with a test example. Future research would entail consolidating the concept by extensive testing and integration for real-time application.

Framework for Automatic Condition Assessment of Culverts

A framework for real-time and automated monitoring of the condition of culverts based on identifying internal defects via video inspection is developed. Manual inspection using closed-circuit television has several drawbacks, such as inconsistency and subjectivity due to human evaluation. Analysis of digital video, consisting of thousands of megabytes even at lower resolutions, can be laborious and not suited for real-time implementation. An innovative approach is to extract image frames judiciously from the video and analyze frames to locate and categorize major defects. Rather than analyze all extracted frames, one can skip consecutive frames at a minimal loss of accuracy and bring considerable savings in memory and system requirements. Each frame is preprocessed to enhance contrast, through use of an adaptive scheme, and to reduce dimensionality in pixel-space by implemention of region-based processing. Preprocessing is followed by a two-step image segmentation process, which implements a background elimination procedure in the first step and shape detection in the second step. Fuzzy clustering is used as the underlying segmentation model. Defect shape and depth information after post-processing are used as input to an assessment methodology for automated condition state. A simple formulation based on both the damage area and depth is then used to assess the condition of culverts based on a four-point condition assessment scale. The proposed framework is demonstrated with a test example. Future research would entail consolidating the concept by extensive testing and integration for real-time application.

Quantification of Ciliary Beat Frequency in Sinonasal Epithelial Cells Using Differential Interference Contrast Microscopy and High-Speed Digital Video Imaging

Mucociliary clearance is a critical upper airway host defense mechanism. Ciliated epithelium in the mammalian airway continually beat at a baseline frequency. Importantly, during times of stress such as exercise or infection, the cilia beat faster to increase clearance. Nasal epithelial ciliary beat frequency (CBF) has been analyzed previously in ex vivo specimens using a variety of methods including photodiode detectors and conventional video recording. Recent studies performed using lower airway ciliated mucosa have shown poor correlation between CBF quantified by photodiode/conventional video and those using high-speed digital video capture at temperatures close to physiological temperatures. Thus, to more rigorously interrogate sinonasal CBF at physiological conditions, we have incorporated a high-speed digital video camera to our CBF analysis system. This is the first report of sinonasal epithelial CBF analysis performed using high-speed video digital analysis. Ex vivo samples of sinonasal epithelium were placed in lactated Ringer's in a temperature-controlled microscope stage chamber. An edge of tissue containing beating cilia was observed at a magnification of 630x using differential interference contrast microscopy. The images were captured using a high-speed digital camera with a sampling rate of 250 frames per second. CBF was determined using computerized data analysis. The mean nasal CBF was calculated from a minimum of five regions for each sample. Temperature curves were generated from tissue obtained from chronically infected subjects. Analysis of high-speed digital video capture of sinonasal CBF observed under differential interference contrast microscopy is a powerful method to investigate environmental as well as host influences on mucociliary clearance within the upper airways.

A study of video scenes clustering based on shot key frames

In digital video analysis, browse, retrieval and query, shot is incapable of meeting needs. Scene is a cluster of a series of shots, which partially meets above demands. In this paper, an algorithm of video scenes clustering based on shot key frame sets is proposed. We useX 2 histogram match and twin histogram comparison for shot detection. A method is presented for key frame set extraction based on distance of non adjacent frames, further more, the minimum distance of key frame sets as distance of shots is computed, eventually scenes are clustered according to the distance of shots. Experiments of this algorithm show satisfactory performance in correctness and computing speed.

Video perspectivity meets wild and crazy teens: a design ethnography

This article is a digital video design ethnography describing the first phase of introducing a perspectivity meme into a classroom. A meme is an idea that spreads throughout a system. A perspectivity meme is the idea that people who share their viewpoints and interpretations will gradually affect role changes in the learning environment. They will not only ‘see’ each other's points of view, but also share roles and viewpoints. Digital video technology is the medium that enables the perspectivity meme to spread throughout this learning culture. Educators and junior high school students use digital cameras and computers to record, reflect, present, discuss, and debate. In this description, I present a detailed narrative of a day in the life of learners, a teacher, and three researchers. I describe how a perspectivity meme was planted in a culture. I describe how the community began to think more deeply and personally about the curriculum. In closing, I discuss the importance of sharing roles in the learning and teaching process, thereby activating incremental changes to the learning environment. ORION™, an experimental tool for digital video analysis—previously known as WebConstellations and Constellations—is used to present interactive video data to readers of this article, on the web. ORION supports online collaborative organization, analysis, and presentation of video segments and clusters and can be used by researchers, teachers, and/or learners. Readers are invited to participate online at http://orion.njit.edu in the Burnsview Galaxy.

A new method for the assessment of spatial orientation and spatial anxiety in mice

The implication of integrated functional sensory relations of the body to space in anxiety disorders is a very important issue which encourages the development of animal models, in particular, for pharmacological perspectives and for the functional assessment of the deficits induced by genetic manipulation in the mouse or the rat. A new experimental device is presented here: It is comprised of a rotating tunnel and a rotating-beam controlled by computer which can be used for multiple visuo-idiothetic and kinesthetic sensory conflict situations during active locomotor behaviour by mice. The system is linked to a digital video system, Video-Track™, designed to track and record the movements of the animals. Anxious BALB/cByJ mice were compared to non-anxious C57BL/6J mice and were seen to display highly disturbed locomotor behaviour in a sensory conflict situation. The model highlights the advantages of video-digital analysis for animal behavioural sciences.

Electroreception in neonatal bonnethead sharks, Sphyrna tiburo

The ability of sharks to orient to weak electric fields is well documented, but a detailed analysis of orientation pathways is lacking. Digital video analysis was used to quantify the behavioral response of naive neonatal bonnethead sharks, Sphyrna tiburo, to prey-simulating weak electric fields. Sharks less than 24 h post-parturition failed to demonstrate a positive feeding response to the electrodes whereas vigorous biting at the electrodes was observed in all sharks greater than 32 h post-parturition. Orientation behaviors were classified as one of five types: "straight" approach, "single turn," "overshoot," "spiral tracking," and "orient without biting." One-third of all orientations were elicited at stimulus intensities of less than 20 nV cm−1. The median electric stimulus threshold for initiation of orientation was 47 nV cm−1 and the minimum was less than 1 nV cm−1. Most orientations to the dipole were from a distance of less than 10 cm with a maximum orientation distance of 22 cm. The innate feeding response to electric stimuli is demonstrated for the first time in a chondrichthyan fish.

An Automatic Video Classification System Based on a Combination of HMM and Video Summarization

As digital libraries and video databases grow, we need methods to assist us in the synthesis and analysis of digital video. Since the information in video databases can be measured in thousands of gigabytes of uncompressed data, tools for efficient summarizing and indexing of video sequences are indispensable. In this paper, we present a method for effective classification of different types of videos that makes use of a video summarization that is in the form of a storyboard of keyframes. To produce the summarization, we first generate a universal basis on which to project a video frame that effectively reduces any video to the same lighting conditions. Each frame is represented by a compressed chromaticity signature. We then set out a multi-stage hierarchical clustering method to efficiently summarize a video. Finally we classify TV videos using a trained Hidden Markov Model that utilizes the compressed chromaticity signatures as well as temporal features of videos derived from their keyframe summaries.

Measuring reliability of effort in functional capacity evaluations using digital video analysis

Functional capacity evaluations (FCE) are utilized to evaluate the physical abilities of workers to perform the specific demands of a job. FCEs are being used increasingly by insurers, employers, health care providers and attorneys as the basis to make decisions regarding workplace supports, vocational rehabilitation, disability determination, and litigation. These factors require that FCEs must have objectivity, reliability, validity and defensibility. One of the key elements in the usefulness of an FCE is a measure of the level of effort given by the individual being tested. This study investigated determinants of an FCE that identified corresponding level of effort. Thirty-five healthy adults participated in a study to investigate whether computerized video movement analysis was an effective tool to determine consistency of effort during lifting (floor to waist and waist to overhead) and squatting activities of an FCE. Video data were examined using computerized analysis of each movement. The association between six factors identified from the video analysis and the distinction between real and intentionally sub-maximal efforts was analyzed. Terminal acceleration, mean velocity, and peak velocity were consistently, significantly correlated with consistency of effort in all three FCE activities. Computerized analyses of video taped FCE activities can identify factors associated with consistency of effort.

Development and Risk-validation of a Digital Video Analysis System for the Assessment of Low Back Pain Risk due to Trunk Kinematic Variables

Development and Risk-validation of a Digital Video Analysis System for the Assessment of Low Back Pain Risk due to Trunk Kinematic Variables

Analysis of ciliary beat pattern and beat frequency using digital high speed imaging: comparison with the photomultiplier and photodiode methods

BACKGROUNDThe aim of this study was to determine the relationship of the power and recovery stroke of respiratory cilia using digital high speed video imaging. Beat frequency measurements made using...

Large-scale evaluation of plant growth in Arabidopsis thaliana by non-invasive image analysis

Analysis of plant growth is a necessary step for the evaluation of plant performance and productivity. For the study of plant performance in Arabidopsis thaliana, a number of mutant collections is available, but a comprehensive methodology to monitor their growth behaviour is lacking. We demonstrate that a novel system, comprising of digital video and image analysis can determine plant size non-destructively by measuring plant leaf area of A. thaliana ecotypes, Columbia 0 and Landsberg erecta, under short and long day conditions. This system provides a non-intrusive method for the monitoring and quantification of plant growth and the evaluation of growth rates. It is applicable for large-scale approaches, such as screening for alteration in growth (rate) in mutant collections. This new system provides the tool for a very accurate determination of individual plant growth rate, even during early developmental stages, requiring as little time as 1 week.

Content-based retrieval from digital video

There is already a huge demand for efficient image indexing and content-based retrieval. With TV going digital, advances in real-time video decompression, easy access to the Internet and the availability of cheap mass storage and fast graphics adaptor cards, digital video will become the next `big' media. Unfortunately, automatic indexing and feature extraction from digital video is even harder than still-image analysis. Presently, automatic analysis of digital video is mostly restricted to automatic detection of scene changes. In this paper we present a framework suitable to immediately explore the consequences of content-based video retrieval with a high granularity of video content. The framework employs Semantic networks to represent video contents on a high level of abstraction and uses time-varying sensitive regions to link objects in a video to the knowledge base. A prototype was implemented under NEXTSTEP, exploiting the rich user-interface capabilities of this platform to feature drag and drop queries and authoring of the video retrieval system.

Multimedia Content Analysis and Indexing for Filtering and Retrieval Applications

Introduction Data compression coupled with the availability of high-bandwidth networks and storage capacity have created the overwhelming production of multimedia content. In addition, the introduction of digital video will completely change the landscape of the entire video value chain. For content producers, advertisers, and consumers, there will be increased availability and increased challenges to manage the data. Users in the consumer and corporate domains will be given an overwhelming and confusing number of traditional and Internet media viewing choices and will look for ways to help them manage these choices [2,14,15,24,26,34,49,92]. Image and video archives in broadcast studios, corporate archives of multimedia collaborative sessions, video conferencing sessions, and educational content require tools for providing quick overview and transparent access. From a content production point of view, a broadcast studio archive will produce video at a rate of 19.2 Mb/s which translates into 207GB storage per day, assuming that only new content is broadcast. At that rate, the studio archive will require 75TB per year which means that in 14 years digital broadcast studios will have to cope with data in the petabyte range. The sheer size of the stored video data will pose serious issues for content owners to find and reuse some of the archived material. Content management for studio archives is just one of the many applications that incorporate tools for content analysis and retrieval of multimedia data. Content management tools will aid in applications that will facilitate effective access, interaction, browsing and display of complex and inhomogeneous information consisting of images, video and audio. Such tools are important in various cases of professional and consumer applications such as education, digital libraries, entertainment, content authoring tools, geographical information systems, bio-medical systems, investigation services, surveillance and many others [64]. In this paper, we survey the techniques for content-based analysis, retrieval and filtering of digital images, audio and video. We will focus on basic methods for extracting features that will enable indexing and search applications. The deployment of a variety of these methods will enable powerful tools for both professionals and consumers to cope with multimedia data. Although the goal of these methods is content understanding which stems from computer vision systems, the methods surveyed here would be more similar to database methods for indexing. This is because a gap exists between these two aspects of the retrieval problem: databases do not provide content analysis and segmentation, and vision systems do not provide database query capabilities. The data acquisition in traditional databases relies primarily on the user to type in the data. Similarly, in the past image and video databases provided keyword descriptions of the visual descriptions of the visual data. However, the annotation based description is being augmented or replaced by automatic methods for feature extraction, indexing and content understanding [11,17]. This paper is organized as follows. Section 2 provides a survey of the methods for image analysis and retrieval. Section 3 describes techniques for video analysis, retrieval and filtering. Section 4 presents methods for audio based analysis and retrieval. In section 5 we provide a high level survey of systems and standardization efforts in content description and retrieval. Methods for Image Analysis and Retrieval Image analysis is concerned with extraction of features for content representation. Often, extraction of low-level features such as color, texture and shape are used for this purpose. On the other hand, the intent of most image retrieval systems is to give the user tools to search for images using higher-level semantic descriptions. For example, the user may want to find all the city pictures or the beach pictures from the last vacations [26,29,54]. …

Using digital video analysis in introductory mechanics projects

During the first semester of our calculus-based introductory physics course sequence, students work in small groups on mechanics projects. Many teams choose digital video analysis for projects involving mechanics. We discuss the management of collaborative projects and schemes for critiquing collaborative papers and assigning individual grades. We also describe some of the more interesting projects. Finally, we offer some advice on how to find the information needed to create good digital video movies.

Digital video analysis and recognition for content-based access

Digital video analysis and recognition for content-based access