Segmentation Of Video Sequences Research Articles

Video Sequence Segmentation Based on K-Means in Air-Gap Data Transmission for a Cluttered Environment

An air gap is a technique that increases the security of information systems. The use of unconventional communication channels allows for obtaining communication that is of interest to the attacker as well as to cybersecurity engineers. One of the very dangerous forms of attack is the use of computer screen brightness modulation, which is not visible to the user but can be observed from a distance by the attacker. Once infected, the computer can transmit data over long distances. Even in the absence of direct screen visibility, transmission can be realized by analyzing the modulated reflection of the monitor's afterglow. The paper presents a new method for the automatic segmentation of video sequences to retrieve the transmitted data that does not have the drawbacks of the heretofore known method of growth (filling) based on an analysis of adjacent pixels. A fast camera operating at 380 fps was used for image acquisition. The method uses the characteristics of the amplitude spectrum for individual pixels, which is specific to the light sources in the room, and clustering with the k-means algorithm to group pixels into larger areas. Then, using the averaging of values for individual areas, it is possible to recover the 2-PAM (pulse-amplitude modulation) signal even at a 1000 times greater level of interference in the area to the transmitted signal, as shown in the experiments. The method does not require high-quality lenses.

A visual object segmentation algorithm with spatial and temporal coherence inspired by the architecture of the visual cortex.

Scene analysis in video sequences is a complex task for a computer vision system. Several schemes have been addressed in this analysis, such as deep learning networks or traditional image processing methods. However, these methods require thorough training or manual adjustment of parameters to achieve accurate results. Therefore, it is necessary to develop novel methods to analyze the scenario information in video sequences. For this reason, this paper proposes a method for object segmentation in video sequences inspired by the structural layers of the visual cortex. The method is called Neuro-Inspired Object Segmentation, SegNI. SegNI has a hierarchical architecture that analyzes object features such as edges, color, and motion to generate regions that represent the objects in the scenario. The results obtained with the Video Segmentation Benchmark VSB100 dataset demonstrate that SegNI can adapt automatically to videos with scenarios that have different nature, composition, and different types of objects. Also, SegNI adapts its processing to new scenario conditions without training, which is a significant advantage over deep learning networks.

Correction to: Video sequence feature extraction and segmentation using likelihood regression model

Correction to: Video sequence feature extraction and segmentation using likelihood regression model

Video sequence feature extraction and segmentation using likelihood regression model

The development of digital technology is utilized by people to capture and share video frames. At present, rather than capturing images, people are interested in recording video footage for exploring information. Here, retrieval of video from large databases is challenging due to the continuous frame count. To overcome these challenges associated with the retrieval of video from available databases, this research proposed a likelihood-based regression approach for video processing. To improve the retrieval accuracy of video sequences, the proposed method utilizes a likelihood estimation technique integrated with a regression model. The likelihood estimate measures the pixel level roughly for estimating the pixel range, after which the regression approach measures the pixel level for transforming certainly blurred and unwanted pixels. In the proposed likelihood regression approach, the video is converted into a video frame and stored in a database. Query frames are taken into account by the generated database depending on the features which are used for a given video to be retrieved. The significant video retrieval performance obtained from the simulation results for the proposed likelihood-based regression model shows that the proposed model performs well over the other state-of-the-art techniques.

FastTTPS: fast approach for video transcoding time prediction and scheduling for HTTP adaptive streaming videos

HTTP adaptive streaming of video content becomes an integrated part of the Internet and dominates other streaming protocols and solutions. The duration of creating video content for adaptive streaming ranges from seconds or up to several hours or days, due to the plethora of video transcoding parameters and video source types. Although, the computing resources of different transcoding platforms and services constantly increase, accurate and fast transcoding time prediction and scheduling is still crucial. We propose in this paper a novel method called fast video transcoding time prediction and scheduling (FastTTPS) of x264 encoded videos based on three phases: (i) transcoding data engineering, (ii) transcoding time prediction, and (iii) transcoding scheduling. The first phase is responsible for video sequence selection, segmentation and feature data collection required for predicting the transcoding time. The second phase develops an artificial neural network (ANN) model for segment transcoding time prediction based on transcoding parameters and derived video complexity features. The third phase compares a number of parallel schedulers to map the predicted transcoding segments on the underlying high-performance computing resources. Experimental results show that our predictive ANN model minimizes the transcoding mean absolute error (MAE) and mean square error (MSE) by up to 1.7 and 26.8, respectively. In terms of scheduling, our method reduces the transcoding time by up to 38% using a Max–Min algorithm compared to the actual transcoding time without prediction information.

Online Mutual Foreground Segmentation for Multispectral Stereo Videos

The segmentation of video sequences into foreground and background regions is a low-level process commonly used in video content analysis and smart surveillance applications. Using a multispectral camera setup can improve this process by providing more diverse data to help identify objects despite adverse imaging conditions. The registration of several data sources is however not trivial if the appearance of objects produced by each sensor differs substantially. This problem is further complicated when parallax effects cannot be ignored when using close-range stereo pairs. In this work, we present a new method to simultaneously tackle multispectral segmentation and stereo registration. Using an iterative procedure, we estimate the labeling result for one problem using the provisional result of the other. Our approach is based on the alternating minimization of two energy functions that are linked through the use of dynamic priors. We rely on the integration of shape and appearance cues to find proper multispectral correspondences, and to properly segment objects in low contrast regions. We also formulate our model as a frame processing pipeline using higher order terms to improve the temporal coherence of our results. Our method is evaluated under different configurations on multiple multispectral datasets, and our implementation is available online.

PMHI: Proposals From Motion History Images for Temporal Segmentation of Long Uncut Videos

This letter proposes a method for the generation of temporal action proposals for the segmentation of long uncut video sequences. The presence of consecutive multiple actions in video sequences makes the temporal segmentation a challenging problem due to the unconstrained nature of actions in space and time. To address this issue, we exploit the nonaction segments present between the actual human actions in uncut videos. From the long uncut video, we compute the energy of consecutive nonoverlapping motion history images (MHIs), which provides spatiotemporal information of motion. Our proposals from MHIs (PMHI) are based on clustering the MHIs into actions and nonaction segments by detecting minima from the energy of MHIs. PMHI efficiently segments the long uncut videos into a small number of nonoverlapping temporal action proposals. The strength of PMHI is that it is unsupervised, which alleviates the requirement for any training data. Our temporal action proposal method outperforms the existing proposal methods on the Multi-view Human Action video (MuHAVi)-uncut and Computer Vision and Pattern recognition (CVPR) 2012 Change Detection datasets with an average recall rate of 86.1% and 86.0%, respectively.

Cast and Self Shadow Segmentation in Video Sequences using Interval based Eigen Value Representation

Tracking of motion objects in the surveillance videos is useful for the monitoring and analysis. The performance of the surveillance system will deteriorate when shadows are detected as moving objects. Therefore, shadow detection and elimination usually benefits the next stages. To overcome this issue, a method for detection and elimination of shadows is proposed. This paper presents a method for segmenting moving objects in video sequences based on determining the Euclidian distance between two pixels considering neighborhood values in temporal domain. Further, a method that segments cast and self shadows in video sequences by computing the Eigen values for the neighborhood of each pixel is proposed. The dual-map for cast and self shadow pixels is represented based on the interval of Eigen values. The proposed methods are tested on the benchmark IEEE CHANGE DETECTION 2014 dataset.

From pixels to gestures: learning visual representations for human analysis in color and depth data sequences

The visual analysis of humans from images is an important topic of interest due to its relevance to many computer vision applications likepedestrian detection, monitoring and surveillance, human-computer interaction, e-health or content-based image retrieval, among others.In this dissertation we are interested in learning different visual representations of the human body that are helpful for the visual analysis of humans in images and video sequences. To that end, we analyze both RGB and depth image modalities and address the problem from three different research lines, at different levels of abstraction; from pixels to gestures: human segmentation, human pose estimation and gesture recognition.First, we show how binary segmentation (object vs. background) of the human body in image sequences is helpful to remove all the background clutter present in the scene. The presented method, based on Graph cuts optimization, enforces spatio-temporal consistency of the produced segmentation masks among consecutive frames. Secondly, we present a framework for multi-label segmentation for obtaining much more detailed segmentation masks: instead of just obtaining a binary representation separating the human body from the background, finer segmentation masks can be obtained separating the different body parts.At a higher level of abstraction, we aim for a simpler yet descriptive representation of the human body. Human pose estimation methods usually rely on skeletal models of the human body, formed by segments (or rectangles) that represent the body limbs, appropriately connected following the kinematic constraints of the human body. In practice, such skeletal models must fulfill some constraints in order to allow for efficient inference, while actually limiting the expressiveness of the model. In order to cope with this, we introduce a top-down approach for predicting the position of the body parts in the model, using a mid-level part representation based on Poselets.Finally, we propose a framework for gesture recognition based on the bag of visual words framework. We leverage the benefits of RGB and depth image modalities by combining modality-specific visual vocabularies in a late fusion fashion. A new rotation-variant depth descriptor is presented, yielding better results than other state-of-the-art descriptors. Moreover, spatio-temporal pyramids are used to encode rough spatial and temporal structure. In addition, we present a probabilistic reformulation of Dynamic Time Warping for gesture segmentation in video sequences. A Gaussian-based probabilistic model of a gesture is learnt, implicitly encoding possible deformations in both spatial and time domains.

A novel robust approach for handling illumination changes in video segmentation

In this work we propose a novel algorithm for foreground segmentation in video sequences that achieves better accuracy, while maintaining low complexity and allows real time execution. The proposed algorithm combines selected features from a number of well-established and state of the art algorithms, such as the Gaussian mixture models, the Self Organizing Maps and the illumination sensitive method. The presented methodology is capable of efficiently handling sudden light changes, both from natural and multiple artificial light sources, which may have caused erroneous segmentation for other algorithms. Comparative results are presented utilizing user-defined ground truth segmentation for two different types of indoor video sequences, one of which was obtained by a hemispheric omnidirectional dioptric (fish-eye) camera, with and without illumination changes and the second by a plain camera. The behavior of the algorithm with respect to its controlling parameters is investigated and its computational burden is studied.

A Saliency-Based Method for Early Smoke Detection in Video Sequences

Video-based smoke detection requires suspected smoke regions to be segmented from the complex background in the initial stage of detection. This segmentation is also important to the subsequent processes of detection. This paper proposes a novel method of segmenting a smoke region in smoke pixel classification based on saliency detection. A salient smoke detection model based on color and motion features is used. First, smoke regions are identified by enhancing the smoke color nonlinearly. The enhanced map and motion map are then used to measure saliency. Finally, the motion energy and saliency map are used to estimate the suspected smoke regions. The estimation result is regarded as our final smoke pixel segmentation result. The performance of the proposed algorithm is verified on a set of videos containing smoke. In the experiments, the method achieves average smoke segmentation precision of 93.0%, and the precision is as high as 99.0% for forest fires. The results are compared with those of three other methods used in the literature, revealing the proposed method to have both a better segmentation result and better precision. We also present encouraging results of smoke segmentation in video sequences obtained using the proposed saliency detection method. Furthermore, the proposed smoke segmentation method can be used for real-time fire detection in color video sequences.

Color video segmentation by lateral inhibition in accumulative computation

The lateral inhibition in accumulative computation (LIAC) algorithm has proved to be an efficient method for moving object segmentation in gray-level video sequences. This paper reviews the main steps and features of the LIAC algorithm, and assesses the suitability of applying the LIAC algorithm to the segmentation of color videos. Two widely used color spaces, namely \(RGB\) and \(HLS\), are used for validating the LIAC algorithm, and a comparison is provided after performance evaluation of the algorithm in both color spaces.

A System for Movement Detecting Congestion.

In this paper, we present a system for tracking and provide early information of hazardous locationsin huge gatherings. It is based on optic flow estimations and detects sequences of crowd motion that are characteristic for devastating congestions. For optic flow computation, Lucas- Kanade method is employed to determine the optical flow vectors for the gathered video. Segmentation of video sequences is done and optic flow is determined for respective segments. A threshold optic flow is chosen in such a way that the tracking of congested area in video is easilydoneby comparing it with respective segment’s determined optic flow values. Finally, we present the location of crowd congestion which helps in taking further protective measures to handle unusual events. Â

Probability-based Dynamic Time Warping and Bag-of-Visual-and-Depth-Words for Human Gesture Recognition in RGB-D

We present a methodology to address the problem of human gesture segmentation and recognition in video and depth image sequences. A Bag-of-Visual-and-Depth-Words (BoVDW) model is introduced as an extension of the Bag-of-Visual-Words (BoVW) model. State-of-the-art RGB and depth features, including a newly proposed depth descriptor, are analysed and combined in a late fusion form. The method is integrated in a Human Gesture Recognition pipeline, together with a novel probability-based Dynamic Time Warping (PDTW) algorithm which is used to perform prior segmentation of idle gestures. The proposed DTW variant uses samples of the same gesture category to build a Gaussian Mixture Model driven probabilistic model of that gesture class. Results of the whole Human Gesture Recognition pipeline in a public data set show better performance in comparison to both standard BoVW model and DTW approach.

Joint Random Field Model for All-Weather Moving Vehicle Detection

This paper proposes a joint random field (JRF) model for moving vehicle detection in video sequences. The JRF model extends the conditional random field (CRF) by introducing auxiliary latent variables to characterize the structure and evolution of visual scene. Hence, detection labels (e.g., vehicle/roadway) and hidden variables (e.g., pixel intensity under shadow) are jointly estimated to enhance vehicle segmentation in video sequences. Data-dependent contextual constraints among both detection labels and latent variables are integrated during the detection process. The proposed method handles both moving cast shadows/lights and various weather conditions. Computationally efficient algorithm has been developed for real-time vehicle detection in video streams. Experimental results show that the approach effectively deals with various illumination conditions and robustly detects moving vehicles even in grayscale video.

Convergence analysis of active contours

Active contours are very useful tools in image segmentation and object tracking in video sequences. The practical implementations are built with an iterative algorithm based on a second order system defined in the spatial domain, where the elasticity and rigidity are the static parameters for its characterization and mass and damping are the dynamic parameters. In the process, the contour is influenced by external and internal forces varying its shape adaptively. The number of iterations required by the contour to delineate the objects is determined by these forces, by its initialization and by the coefficients of the second order system. This paper analyzes the convergence of active contours using the frequency based formulation and shows that the convergence depends on the dynamic parameters of the second order system and the distance between nodes of the contour attracted by the external forces.

Joint moving cast shadows segmentation and light source detection in video sequences

This paper proposes a new method which allows a joint estimation of the light source projection on the image plane and the segmentation of moving cast shadows in natural video sequences. It allows improving the segmentation of moving objects by separating clearly cast shadows from moving objects. The method is based on a shadow model which mainly assumes that the cast shadows are projected on plane and Lambertian surfaces, and that the light source is unique. The moving cast shadows, including the penumbra, are detected using a segmentation method based on a comparison between a reference image and the original one. The light source position is estimated using geometrical relations linking the light source, the object and its cast shadow on the 2-D image plane. This is obtained using a robust temporal filtering method. For each image using the current estimation of the light source position and the video object contours, a cast shadow search area is defined. This reduces the risk of false detections during the segmentation process, and thus allows increasing the detection rate and reducing the false alarm one. Experimental results show that good shadow and object contours and light source locations are obtained with the proposed method even if the theoretical assumptions are not fully valid.

Spatiotemporal video segmentation based on graphical models

This paper proposes a probabilistic framework for spatiotemporal segmentation of video sequences. Motion information, boundary information from intensity segmentation, and spatial connectivity of segmentation are unified in the video segmentation process by means of graphical models. A Bayesian network is presented to model interactions among the motion vector field, the intensity segmentation field, and the video segmentation field. The notion of the Markov random field is used to encourage the formation of continuous regions. Given consecutive frames, the conditional joint probability density of the three fields is maximized in an iterative way. To effectively utilize boundary information from the intensity segmentation, distance transformation is employed in local objective functions. Experimental results show that the method is robust and generates spatiotemporally coherent segmentation results. Moreover, the proposed video segmentation approach can be viewed as the compromise of previous motion based approaches and region merging approaches.

A New Real Time Object Segmentation and Tracking Algorithm and its Parallel Hardware Architecture

Most of the emerging content-based multimedia technologies are based on efficient methods to solve machine early vision tasks. Among other tasks, object segmentation is perhaps the most important problem in single image processing. The solution of this problem is the key technology of the development of the majority of leading-edge interactive video communication technology and telepresence systems. The aim of this paper is to present a robust framework for real-time object segmentation and tracking in video sequences taken simultaneously from different perspectives. The other contribution of the paper is to present a new dedicated parallel hardware architecture. It's composed of a mixture of Digital Signal Processing (DSP) and Field Programmable Gate Array (FPGA) technologies and uses the Content Addressable Memory (CAM) as a main processing unit. Experimental results indicate that small amount of hardware can deliver real-time performance and high accuracy. This is an improvement over previous systems, where execution time of the second-order using a greater amount of hardware has been proposed.

Video segmentation using genetic algorithms with automatic parameter adaptation

A novel genetic algorithm for video sequence segmentation is presented. The novelty of the approach is that the mating rates such as crossover rate and mutation rate are not constant, but spatio-temporally varying. Experimental results demonstrate the effectiveness of the proposed method.