Static Video Research Articles

Video Summarization using Keyframe Extraction Methods

Video summarization plays an important role in too many fields, such as video indexing, video browsing, video compression, video analyzing and so on. One of the fundamental units in the video structure analysis is the keyframe extraction, Keyframe provides meaningful frames from the video. The keyframe consists of the meaningful frame from the videos which help for video summarization. In this proposed model, we presented an approach that is based on Convolutional Neural Network, keyframe extraction from videos and static video summarization. First, the video should be converted to frames. Then we perform redundancy elimination techniques to reduce the redundancy from frames. Then extract the keyframes from video by using the Convolutional Neural Network(CNN) model. From the extracted keyframe, we form a video summarization.

The effectiveness of a marketing virtual reality learning simulation: A quantitative survey with psychophysiological measures

AbstractVirtual reality (VR) technology has been shown to be a promising teaching method in STEM subjects. Extending these findings, the current study is the first to develop and examine the feasibility of using a VR simulation in marketing (ie, a non‐STEM subject). Specifically, the levels of immersion and three learning outcomes (learning attitude, enjoyment and performance) were compared between the marketing students who learned the same content through a VR simulation prototype versus a traditional static video presentation. Learning enjoyment was measured using facial electromyography, which served as a more objective measures for the feeling of enjoyment. Compared to the video condition, the results showed that VR resulted in a higher experience of immersion, learning attitude and learning enjoyment. Furthermore, immersion was found to fully mediate the positive effects of the VR simulation on learning attitude, but not enjoyment. Surprisingly, students in the video condition performed better on the knowledge‐based test than those in the VR condition. The current study suggests that the current prototype of the VR simulations should be used as supplementary resource to increase the learning attitude and enjoyment, but not as a main teaching material to enhance knowledge‐based performance in the marketing discipline.

Visual crowding in driving

Visual crowding—the deleterious influence of nearby objects on object recognition—is considered to be a major bottleneck for object recognition in cluttered environments. Although crowding has been studied for decades with static and artificial stimuli, it is still unclear how crowding operates when viewing natural dynamic scenes in real-life situations. For example, driving is a frequent and potentially fatal real-life situation where crowding may play a critical role. In order to investigate the role of crowding in this kind of situation, we presented observers with naturalistic driving videos and recorded their eye movements while they performed a simulated driving task. We found that the saccade localization on pedestrians was impacted by visual clutter, in a manner consistent with the diagnostic criteria of crowding (Bouma's rule of thumb, flanker similarity tuning, and the radial-tangential anisotropy). In order to further confirm that altered saccadic localization is a behavioral consequence of crowding, we also showed that crowding occurs in the recognition of cluttered pedestrians in a more conventional crowding paradigm. We asked participants to discriminate the gender of pedestrians in static video frames and found that the altered saccadic localization correlated with the degree of crowding of the saccade targets. Taken together, our results provide strong evidence that crowding impacts both recognition and goal-directed actions in natural driving situations.

Experiment of Structural Geometric Morphology Monitoring for Bridges Using Holographic Visual Sensor

To further improve the precision and efficiency of structural health monitoring technology and the theory of large-scale structures, full-field non-contact structural geometry morphology monitoring is expected to be a breakthrough technology in structural safety state monitoring and digital twins, owing to its economic, credible, high frequency, and holographic advantages. This study validates a proposed holographic visual sensor and algorithms in a computer-vision-based full-field non-contact displacement and vibration measurement. Using an automatic camera patrol experimental device, original segmental dynamic and static video monitoring data of a model bridge under various damage/activities were collected. According to the temporal and spatial characteristics of the series data, the holographic geometric morphology tracking algorithm was introduced. Additionally, the feature points set of the structural holography geometry and the holography feature contours were established. Experimental results show that the holographic visual sensor and the proposed algorithms can extract an accurate holographic full-field displacement signal, and factually and sensitively accomplish vibration measurement, while accurately reflecting the real change in structural properties under various damage/action conditions. The proposed method can serve as a foundation for further research on digital twins for large-scale structures, structural condition assessment, and intelligent damage identification.

Comparative characteristics of modern methods of recognition of people on the basis of computer processing of their image

The problem of automated recognition of faces and gestures is relatively new and has not yet been fully resolved. In recent years, a number of different methods and algorithms have been proposed for processing, localizing and recognizing faces and gestures in static images, such as «own faces» (Principal Component Analysis, PCA), neural networks, evolutionary algorithms, AdaBoost algorithm, support vector method, etc. However, these approaches for recognizing objects have insufficient accuracy, reliability, and speed in a complex real environment characterized by the presence of noise in images and video sequences. Nowadays CCTV systems have become very widespread. There are so many companies that release their equipment for this purpose. These are video capture cards with special software. But there are problems, of course, this is software that is made for a certain range of tasks, in most cases it is very disturbing. For example, in order for the surveillance system at the checkpoint to calculate the number of passing people, it is necessary to pay a considerable amount of money, since it requires special software processing of the video stream in order to realize it in order to programmatically recognize the images and to calculate them. It should be noted that there are a number of factors that complicate the recognition of objects in static images and video sequences. These include: changing lighting conditions during the shooting process, poor quality of video images, the difficulty of separating the object from the background, the presence of many objects in the video frame, etc. This article discusses the task of detecting objects in an image, as well as methods for processing and analyzing data. The recognition methods, which are one of the first practical tasks, are studied, which has become an incentive for the development of the theory of object recognition. The issues of recognition of faces and gestures, which finds application in various fields of human activity, are analyzed. In this paper, face recognition algorithms are considered.

Dynamic Spatial Predicted Background.

We present a novel method for online background modeling for static video cameras - Dynamic Spatial Predicted Background (DSPB). Our unique method employs a small subset of image pixels to predict the whole scene by exploiting pixel correlations (distant and close). DSPB acts as a hybrid model combining successful elements taken from two major approaches: local-adaptive that propose to fit a distribution pixelwise, and global-linear that reconstruct the background by finding a lowrank version of the scene. To our knowledge, this is the first attempt to combine these approaches in a unified system. DSPB models the scene as a superposition of illumination effects and predicts each pixel's value by a linear estimator comprised of only 5 pixels of the scene and can initialize the background starting from the 5th frame. By doing so, we keep the computational load low, allowing our method to be used in many real-time applications using simple hardware. The suggested prediction model of scene appearance is novel, and the scheme is very accurate and efficient computationally. We show the method merits on an application for video FG-BG separation, and how some of the main existing approaches may be challenged and how their drawbacks are less dominant in our model. Experimental results validate our findings, by computation speed and mean F-measure values on several public datasets. We also examine how results may improve by analyzing each video individually according to its content. DSPB can be successfully incorporated in other image processing tasks like change detection, video compression and video inpainting.

Multi-Frame Based Homography Estimation for Video Stitching in Static Camera Environments.

In this paper, a multi-frame based homography estimation method is proposed for video stitching in static camera environments. A homography that is robust against spatio-temporally induced noise can be estimated by intervals, using feature points extracted during a predetermined time interval. The feature point with the largest blob response in each quantized location bin, a representative feature point, is used for matching a pair of video sequences. After matching representative feature points from each camera, the homography for the interval is estimated by random sample consensus (RANSAC) on the matched representative feature points, with their chances of being sampled proportional to their numbers of occurrences in the interval. The performance of the proposed method is compared with that of the per-frame method by investigating alignment distortion and stitching scores for daytime and noisy video sequence pairs. It is shown that alignment distortion in overlapping regions is reduced and the stitching score is improved by the proposed method. The proposed method can be used for panoramic video stitching with static video cameras and for panoramic image stitching with less alignment distortion.

RotateView: A Video Composition System for Interactive Product Display

Product display in e-commerce commonly uses static images or videos. In this paper, we design a novel video composition system “RotateView” which displays real-world products videos on the mobile phone in an interactive and 3D-like way. The system estimates the rotation direction of products using optical flow, segments products using an improved version of online adaptive video object segmentation, and adjusts motion and color for composition. The rotation direction estimation methods are evaluated on a video dataset of 1500 rotated videos that are uploaded by sellers, and on our collected real-world video object segmentation (RVOS) dataset. On our RVOS dataset, experimental results show an improvement in segmentation over the state of the art. Moreover, the proposed algorithms for motion and color adjustment are evaluated using extensive user studies. The RVOS dataset can be downloaded via https://realvos.github.io/ for further studies and code will be made available.

Autonomous Vehicular Surveillance using License Plate Recognition over Cloud Computing Architecture

License Plate Recognition (LPR) is the extraction and identification of licence plate numbers from license plates. The extraction process requires ample image pre-processing using normalization, gray scaling and edge removal techniques. These extracted plates can then be identified using image processing techniques such as neural networks and support vector machines. These license plates are captured using stationary video cameras, which extracts images from their feed as inputs into the image extraction algorithm. For the purposes of vehicular surveillance, these cameras are inefficient, as a lot of them will be required to monitor vehicles effectively. Hence there is a need for a larger scale model to carry out effective vehicular surveillance. For this purpose, the cameras embedded in self driving cars are utilised as replacements to stationary video cameras. These cameras have to advantage of being constantly mobile, hence being able to carry out a larger scale of surveillance. These cameras capture meaningful images of license plates from their video feed, and upload these images to the cloud using a Vehicular Cloud Computing (VCC) architecture. This centralized cloud carries out the image extraction and image processing tasks. The identified license plates can be used to monitor the cars they belong to. The cloud compares them to a database of license plates that are flagged by law enforcement. If the license plate is found to be flagged, then the respective law enforcement authorities are notified of the location of the car. If the plate belongs to a car with a history of misbehavior, the car capturing the plate is informed of thus, making it easier to safely navigate around the problematic driver.

Domain independent redundancy elimination based on flow vectors for static video summarization

Video summarization aims to find a compact representation of input videos. The method finds out interesting parts of the video by discarding the remaining parts of the video. The abstracts thus generated enhances browsing and retrieval of video data. The quality of summaries generated by video summarization algorithms can be improved if the redundant frames in the input video are taken care of before summarization. This paper presents a novel domain-independent method for redundancy elimination from input videos before summarization maintaining keyframes in the original video. The frames of input video are first presampled by selecting two frames in one second. The flow vectors between consecutive frames are computed using SIFT Flow algorithm. The magnitude of flow vectors at each pixel position of the frame are summed up to find the displacement magnitude between the consecutive frames. The redundant frames are filtered out based on local averaging of the displacement values. The evaluation of the method is done using two standard datasets namely VSUMM and OVP. The results demonstrate that an average reduction rate of 97.64% is achieved consistently on videos of all categories. The method also gives superior results compared to other state-of-the-art redundancy elimination methods for video summarization

A hybrid egocentric video summarization method to improve the healthcare for Alzheimer patients

Alzheimer patients face difficulty to remember the identity of persons and performing daily life activities. This paper presents a hybrid method to generate the egocentric video summary of important people, objects and medicines to facilitate the Alzheimer patients to recall their deserted memories. Lifelogging video data analysis is used to recall the human memory; however, the massive amount of lifelogging data makes it a challenging task to select the most relevant content to educate the Alzheimer’s patient. To address the challenges associated with massive lifelogging content, static video summarization approach is applied to select the key-frames that are more relevant in the context of recalling the deserted memories of the Alzheimer patients. This paper consists of three main modules that are face, object, and medicine recognition. Histogram of oriented gradient features are used to train the multi-class SVM for face recognition. SURF descriptors are employed to extract the features from the input video frames that are then used to find the corresponding points between the objects in the input video and the reference objects stored in the database. Morphological operators are applied followed by the optical character recognition to recognize and tag the medicines for Alzheimer patients. The performance of the proposed system is evaluated on 18 real-world homemade videos. Experimental results signify the effectiveness of the proposed system in terms of providing the most relevant content to enhance the memory of Alzheimer patients.

Teacher's visual attention when scaffolding collaborative mathematical problem solving

Teachers' role in scaffolding students' problem-solving process is crucial. New technology provides researchers with possibilities to explore this aspect of teaching from the viewpoint of teacher attention. The aim of this mixed-method case study was to investigate the relation between a teacher's scaffolding intentions and his gaze behavior. The data was collected during a mathematics lesson using mobile gaze tracking devices, stationary video cameras, and interview. The results show that the teacher's scaffolding intentions affected his gaze targets significantly and that mobile gaze tracking can provide novel insight to situational processes of teacher-student interaction.

Static and Dynamic Video Summarization

Video represented by a large number of frames synchronized with audio making video saving requires more storage, it's delivery slower, and computation cost expensive. Video summarization provides entire video information in minimum amount of time. This paper proposes static and dynamic video summarizationmethods. The proposed static video summarization method includes several steps which are extracting frames from video, keyframes selection, feature extraction and description, and matching feature descriptor with bag of visual words, and finally save frames when features matched. The proposed dynamic video summarizationmethod includes in general extracting audio from video, calculating audio features using the average of samples in windows and find the highest average which reflects portion of video with loudest sound. The experimental results for the proposed static video summarization show that there is no redundancy between selected representative keyframes and the subjective evaluation results ensure the importance of the selected keyframes. While the experimental results for the proposed static video summarization show that all the segments of goals have been extracted to provide video summary. Static and dynamic video summarization methods done to football or soccer video type.

Stationary video monitoring reveals habitat use of stingrays in mangroves

Mangroves are highly productive habitats offering many elasmobranch species abundant food resources and physical refuge from predators, which can be important for survival of juveniles. However, habitat use and behavioural patterns of stingrays are poorly understood within mangrove areas, and the drivers of direct use of mangroves remain unclear. Stationary video monitoring was used to investigate fine-scale spatial use by juvenile mangrove whiprays Urogymnus granulatus and cowtail stingrays Pastinachus ater in a mangrove-fringed intertidal area in Pioneer Bay, Orpheus Island, northeast Australia. Abundance, residence time and behaviour were compared among 3 different habitats (inside mangroves, mangrove edge and sandflat) between summer and winter. Abundance of U granulatus was not significantly different between seasons, while P. ater was significantly more abundant in summer than in winter, suggesting a seasonal habitat shift. U granulatus exhibited tide-associated movement, with individuals occupying mangroves during flood tides and moving to sandflats at ebb tides. Juvenile U. granulatus were frequently observed feeding within mangrove habitats. P. ater predominantly occurred on sandflats regardless of tidal height and were observed feeding only on sandflats, confirming the possible importance of sandflats for feeding by this species. Potential predators rarely occurred in these mangrove habitats, suggesting that U. granulatus reduced predation risk by residing in flooded mangrove habitats. Mangroves can form a critical habitat for early life stages of many species, and their loss may have significant negative effects on populations of these species.

Detection of frame deletion in HEVC-Coded video in the compressed domain

In this paper, we propose an algorithm for detecting frame deletion in HEVC-coded video in the compressed domain. Specifically, we focus on the frame type changes occurring upon frame deletion, which cause slight differences between the coding patterns in original and forged video. Then, we identify discriminating coding patterns for use as features, which are classified by machine learning classifiers. Furthermore, we propose video sequence integrity detection on a group-of-picture basis, which is computationally efficient and robust even when applied to static videos and videos with genuine scene changes. The experimental results show that the proposed technique can classify HEVC-coded videos more accurately than previous methods. In addition, the results demonstrate that the selected features work harmoniously in discrimination and that the learning-based classifiers are more robust and reliable than model-based classifiers.

Is video communication technology building bridges between the primary health care and the emergency department in the Region of Southern Denmark?

Background: In 2017, 2523 patients above the age of 64 were admitted from the community of Aabenraa to the Emergency Department (ED). Around 54% of them discharged from the ED within 48 hours. The aim of the CoLab project is to explore the information barriers in the transition of elder patients between primary health care (PHC) and the ED and back. Furthermore, to investigate whether a standardized communication tool combined with video communication technology improves the cross-sectoral communication and collaboration between nurses in the ED and PHC.
 Methods: This was an exploratory and qualitative study. Data was generated from semi-structured group interviews with seven PHC nurses and two ED nurses. The standardized communication tool was developed from both a similar tool constructed by the Danish society for patient safety: ISBAR MED SALSA and the results from the interviews. Video communication via iPad was used between the acute care nurses in PHC and ED and a stationary video communication tool was installed between the ED and the two call centers in PHC.
 Results: The interim analysis of the study so far suggests that is it not possible to establish a scheduled time for communication between ED and the PHC because of the established fast track patient pathways. Using a communication tool as guide for information sharing led to a higher quality exchange of information between PHC and ED. Both the PHC and ED nurses saw improvement in exchanging information by combining the standardized communication tool with video conferencing. This has raised the understanding of each other’s work areas and the need for sharing information about the transition of the elder patients between the PHC and ED.
 Conclusion: The project is still in progress. However, the midterm analysis has shown evidence that a standardized communication tool combined with video communication would improve the quality of cross-sectoral communication and collaboration between nurses in the ED and PHC. We believe that communication barriers should be reduced or eliminated in order to improve cross-sectoral collaboration and the continuity of care for the elder patient during the transition between PHC and ED.

Anfis Classifier Based Moving Object Detection and Segmentation in Indoor and Outdoor Environments

Background: Moving object detection in dynamic environment video is more complex than the static environment videos. In this paper, moving objects in video sequences are detected and segmented using feature extraction based Adaptive Neuro-Fuzzy Inference System (ANFIS) classifier approach. The proposed moving object detection methodology is tested on different video sequences in both indoor and outdoor environments. Methods: This proposed methodology consists of background subtraction and classification modules. The absolute difference image is constructed in background subtraction module. The features are extracted from this difference image and these extracted features are trained and classified using ANFIS classification module. Results: The proposed moving object detection methodology is analyzed in terms of Accuracy, Recall, Average Accuracy, Precision and F-measure. The proposed moving object segmentation methodology is executed on different Central Processing Unit (CPU) processor as 1.8 GHz and 2.4 GHz for evaluating the performance during moving object segmentation. At present, some moving object detection systems used 1.8 GHz CPU processor. Recently, many systems for moving object detection are using 2.4 GHz CPU processor. Hence, CPU processors 1.8 GHz and 2.4 GHz are used in this paper for detecting the moving objects in video sequences. Table 1 shows the performance evaluation of proposed moving object detection on CPU processor 1.8 GHz (100 sequence). Table 2 shows the performance evaluation of proposed moving object detection on CPU processor 2.8 GHz (100 sequence). The average moving object detection time on CPU processor 1.8 GHz for fountain sequence is 62.5 seconds, for airport sequence is 64.7 seconds, for meeting room sequence is 71.6 seconds and for Lobby sequence is 73.5 seconds, respectively, as depicted in Table 3. The average elapsed time for moving object detection on 100 sequences is 68.07 seconds. The average moving object detection time on CPU processor 2.4 GHz for fountain sequence is 56.5 seconds, for airport sequence is 54.7 seconds, for meeting room sequence is 65.8 seconds and for Lobby sequence is 67.5 seconds, respectively, as depicted in Table 4. The average elapsed time for moving object detection on 100 sequences is 61.12 seconds. It is very clear from Table 3 and Table 4; the moving object detection time is reduced when the frequency of the CPU processor increases. Conclusion: In this paper, moving object is detected and segmented using ANFIS classifier. The proposed method initially segments the background image and then features are extracted from the threshold image. These features are trained and classified using ANFIS classification method. The proposed moving object detection method is tested on different video sequences which are obtained from different indoor and outdoor environments. The performance of the proposed moving object detection and segmentation methodology is analyzed in terms of Accuracy, Recall, Average Accuracy, Precision and F-measure.

Reconstruction of road accident using video recording

Presented was the procedure of reconstructing a motorcycle-motor car collision with both vehicles moving in the same direction and performing simultaneous manoeuvres: a left turn and overtaking. Because of the conflicting reports of witnesses, the procedure also involved recordings from a stationary video camera. These data allowed us to retrace the movement and manoeuvres of vehicles as a function of time and then to identity the perpetrator of the accident.

Method of Moving Region Detection for Static Camera

The moving object detection from a stationary video sequence is a primary task in various computer vision applications. In this proposed system; three processing levels are suppose to perform: detects moving objects region from the background image; reduce noise from the pixels of detected region and extract meaningful objects and their features (area of object, center point of area etc.). In this paper; background subtraction techniques is used for segments moving objects from the background image, which is capable for pixel level processing. Morphology operation (Erosion and dilation) are used to remove pixel to pixel noise. In last level, CCL algorithm is used for sorts out foregrounds pixels are grouped into meaningful connected regions and their features.

Human consistency evaluation of static video summaries

Automatic video summarization aims to provide brief representation of videos. Its evaluation is quite challenging, usually relying on comparison with user summaries. This study views it in a different perspective in terms of verifying the consistency of user summaries, as the outcome of video summarization is usually judged based on them. We focus on human consistency evaluation of static video summaries in which the user summaries are evaluated among themselves using the consistency modelling method we proposed recently. The purpose of such consistency evaluation is to check whether the users agree among themselves. The evaluation is performed on different publicly available datasets. Another contribution lies in the creation of static video summaries from the available video skims of the SumMe datatset. The results show that the level of agreement varies significantly between the users for the selection of key frames, which denotes the hidden challenge in automatic video summary evaluation. Moreover, the maximum agreement level of the users for a certain dataset, may indicate the best performance that the automatic video summarization techniques can achieve using that dataset.