Real Video Sequences Research Articles

Network Coding for Video Distortion Reduction in Device-to-Device Communications

In this paper, we study the problem of distributing a real-time video sequence to a group of partially connected cooperative wireless devices using instantly decodable network coding (IDNC). In such a scenario, the coding conflicts occur to service multiple devices with an immediately decodable packet, and the transmission conflicts occur from simultaneous transmissions of multiple devices. To avoid these conflicts, we introduce a novel IDNC graph that represents all feasible coding and transmission conflict-free decisions in one unified framework. Moreover, a real-time video sequence has a hard deadline and unequal importance of video packets. Using these video characteristics and the new IDNC graph, we formulate the problem of minimizing the mean video distortion before the deadline as a finite horizon Markov decision process (MDP) problem. However, the backward induction algorithm that finds the optimal policy of the MDP formulation has high modeling and computational complexities. To reduce these complexities, we further design a two-stage maximal independent set selection algorithm, which can efficiently reduce the mean video distortion before the deadline. Simulation results over a real video sequence show that our proposed IDNC algorithms improve the received video quality compared with the existing IDNC algorithms.

High accuracy block-matching sub-pixel motion estimation through detection of error surface minima

The present paper focuses on high-accuracy block-based sub-pixel motion estimation utilizing a straightforward error minimization approach. In particular, the mathematics of bilinear interpolation are utilized for the selection of the candidate motion vectors that minimize the error criterion, by estimating local minima in the error surface with arbitrary accuracy. The implemented approach favors optimum accuracy over computational load demands, making it ideal as a benchmark for faster methods to compare against; however, it is not best suited to real-time critical applications (i.e. video compression). Other video processing needs relying on motion vectors and requiring high-resolution/accuracy can also take advantage of the proposed solution (and its simplified nature in terms of underlying theoretical complexity), such as motion-compensation filtering for super resolution image enhancement, motion analysis in sensitive areas (e.g. high-speed video monitoring, medical imaging, motion analysis in sport science, big-data visual surveillance, etc.). The proposed method is thoroughly evaluated using both real video and synthetic motion sequences from still images, adopting well-tested block-based motion estimation evaluation procedures. Assessment includes comparisons to a number of existing block-based methods with respect to PSNR and SSIM metrics over ground-truth samples. The conducted evaluation takes into consideration both the original (arbitrary-accuracy) and the truncated motion vectors (after rounding them to the nearest half, quarter, or eighth of a pixel), where superior performance with more accurate motion vector estimation is revealed. In this context, the degree to which sub-pixel motion estimation methods actually produce sub-pixel motion vectors is investigated, and the implications thereof are discussed.

Mobile System of Decision-Making on Road Threats

Detecting and recognizing brake lights is essential to avoid collisions and accidents caused by a vehicle driving behind another vehicle. This refers to traffic risk, among other things, to man-controlled vehicles as well as autonomous cars. There are many systems supporting drivers, which are also applied in autonomous cars. This paper presents a mobile decision-making system warning against traffic risks related to breaking a preceding vehicle. The new approach is two-stage and real-time one. Unlike other approaches, the presented solution, thanks to proper limitation of a searched area, works effectively on a mobile platform with restricted system resources. The results of experiments conducted on real video sequences on roads in various weather conditions showed high effectiveness of the proposed approach.

Aerial vehicles detection and recognition for UAV vision system

This article focuses on aerial vehicle detection and recognition by a wide field of view monocular vision system that can be installed on UAVs (unmanned aerial vehicles). The objects are mostly observed on the background of clouds under regular daylight conditions. The main idea is to create a multi-step approach based on a preliminary detection, regions of interest (ROI) selection, contour segmentation, object matching and localization. The described algorithm is able to detect small targets, but unlike many other approaches is designed to work with large-scale objects as well. The suggested algorithm is also intended to recognize and track the aerial vehicles of specific kind using a set of reference objects defined by their 3D models. For that purpose a computationally efficient contour descriptor for the models and the test objects is calculated. An experimental research on real video sequences is performed. The video database contains different types of aerial vehicles: airplanes, helicopters, and UAVs. The proposed approach shows good accuracy in all case studies and can be implemented in onboard vision systems.

Pothole detection using spatio‐temporal saliency

This study proposes a simple and fast pothole detection method of a video using spatio-temporal saliency, where a pothole is a type of failure on an asphalt pavement. In monocular vision, potholes typically have two visual properties. First, potholes have low-intensity areas that are darker than nearby pavement because of shadows. Second, the texture inside the potholes is coarser than the nearby pavement. However, these two visual properties are not sufficient for accurately detecting potholes because of shadows on the pavement and lack of the texture. In this study, the above challenges are addressed by detecting slower objects that are coming closer to a moving vehicle. Pothole is a stationary object ahead on the road. In terms of the relative speed, pothole is a slower object than a moving vehicle. In the video sequence acquired from a dash-cam, different image characteristics appear depending on the relative speed of objects such as potholes and cars ahead. The proposed method detects outliers using spatio-temporal saliency in the yt image: a slower object is represented as diagonal outliers. Experiments with several dash-cam videos show the effectiveness of the proposed method for real video sequences that contain potholes, cars, road lanes, and pavement markings.

An Automatic Facial Localization Tracking Identification Biometric System through PCA and Wavelet Distribution in 3D Mesh Environment

Background/Objectives: In this paper, the process of automatically facial region localization and tracking in video frames through 3D mesh model is estimated. Methods/Statistical Analysis: The morphological regions of the face are modeled into 18 geometry based regions based on the various shape and expressions. The feature is estimated based on the covariance matrix in high region space. Then, it undergoes to PCA to estimate facial deformations. Findings: The patterns feature is extracted and it mapped with the multi geometry mapping. Then, the corresponding wavelet transforms extracted region into various dimension for geometry matching for classification. The proposed method achieves robustness based on its accurate transformation through wavelet analysis. The XM2VTSDB multi-modal face database is used to compare the real video sequence images. Application/Improvements: The experimental evaluation shows favorable results and yields 99% detection rate over 15000 video frame images. The frames in the facial tracking data are calculated through various parameters and compared with the ground truth against standard Euler angles of the muti-modal database.

Visual attention based on a joint perceptual space of color and brightness for improved video tracking

This paper proposes a new visual attention model based on a joint perceptual space of both color and brightness, and shows that this model is able to extract more discriminant visual features, especially when dealing with objects that are very similar visually. That joint color and brightness space is based on a biologically inspired theoretical perceptual model originally proposed by Izmailov and Sokolov in the scope of psychophysics. The present paper proposes a computational model that allows the application of Izmailov and Sokolov's theoretical model to digital images, since the original model can only be applied to perceptual data directly drawn from psychophysical experiments. Experimental results with real video sequences show that the proposed visual attention model yields significantly more accurate results in the particular application scope of video tracking than well-known visual attention models that process color and brightness separately.

Robust deformable shape reconstruction from monocular video with manifold forests

Existing approaches to recover structure of 3D deformable objects and camera motion parameters from an uncalibrated images assume the object’s shape could be modelled well by a linear subspace. These methods have been proven effective and well suited when the deformations are relatively small, but fail to reconstruct the objects with relatively large deformations. This paper describes a novel approach for 3D non-rigid shape reconstruction, based on manifold decision forest technique. The use of this technique can be justified by noting that a specific type of shape variations might be governed by only a small number of parameters, and therefore can be well represented in a low-dimensional manifold. The key contributions of this work are the use of random decision forests for the shape manifold learning and robust metric for calculation of the re-projection error. The learned manifold defines constraints imposed on the reconstructed shapes. Due to a nonlinear structure of the learned manifold, this approach is more suitable to deal with large and complex object deformations when compared to the linear constraints. The robust metric is applied to reduce the effect of measurement outliers on the quality of the reconstruction. In many practical applications outliers cannot be completely removed and therefore the use of robust techniques is of particular practical interest. The proposed method is validated on 2D points sequences projected from the 3D motion capture data for ground truth comparison and also on real 2D video sequences. Experiments show that the newly proposed method provides better performance compared to previously proposed ones, including the robustness with respect to measurement noise, missing measurements and outliers present in the data.

The Algorithm for Compensating Distortion of Images of Type “Blur”

The paper considers the model of linear blurring with a uniform rectilinear motion of an object when its speed is constant, and its direction is parallel to the lines of the formed image. An algorithm for linear blurring compensation based on the account of the physical processes of its formation when obtaining the images is proposed. A system of equations for the restoration of the original video signal samples undistorted image of the object is obtained. Processed the frame of a real video sequence with a horizontal blur and presented its results. Considered the comparison of the results of the restoration of distorted images by the proposed algorithm with known methods.

Probabilistic choice between symmetric disparities in motion stereo matching for a lateral navigation system

Two consecutive frames of a lateral navigation camera video sequence can be considered as an appropriate approximation to epipolar stereo. To overcome edge-aware inaccuracy caused by occlusion, we propose a model that matches the current frame to the next and to the previous ones. The positive disparity of matching to the previous frame has its symmetric negative disparity to the next frame. The proposed algorithm performs probabilistic choice for each matched pixel between the positive disparity and its symmetric disparity cost. A disparity map obtained by optimization over the cost volume composed of the proposed probabilistic choice is more accurate than the traditional left-to-right and right-to-left disparity maps cross-check. Also, our algorithm needs two times less computational operations per pixel than the cross-check technique. The effectiveness of our approach is demonstrated on synthetic data and real video sequences, with ground-

A Video Super-Resolution Framework Using SCoBeP

Super-resolution as an exciting application in image processing was studied widely in the literature. This paper presents new approaches to video super-resolution based on sparse coding and belief propagation. First, find candidate match pixels on multiple frames using sparse coding and belief propagation. Second, incorporate information from these candidate pixels with weights computed using the nonlocal-means method in the first approach or using the sparse coding and belief propagation method in the second approach. The effectiveness of the proposed methods is demonstrated for both synthetic and real video sequences in the experiment section. In addition, the experimental results show that our models are naturally robust in handling super-resolution on video sequences affected by scene motions and/or small camera motions.

Blind Super Resolution of Real-Life Video Sequences.

Super resolution (SR) for real-life video sequences is a challenging problem due to complex nature of the motion fields. In this paper, a novel blind SR method is proposed to improve the spatial resolution of video sequences, while the overall point spread function of the imaging system, motion fields, and noise statistics are unknown. To estimate the blur(s), first, a nonuniform interpolation SR method is utilized to upsample the frames, and then, the blur(s) is(are) estimated through a multi-scale process. The blur estimation process is initially performed on a few emphasized edges and gradually on more edges as the iterations continue. Also for faster convergence, the blur is estimated in the filter domain rather than the pixel domain. The high-resolution frames are estimated using a cost function that has the fidelity and regularization terms of type Huber-Markov random field to preserve edges and fine details. The fidelity term is adaptively weighted at each iteration using a masking operation to suppress artifacts due to inaccurate motions. Very promising results are obtained for real-life videos containing detailed structures, complex motions, fast-moving objects, deformable regions, or severe brightness changes. The proposed method outperforms the state of the art in all performed experiments through both subjective and objective evaluations. The results are available online at http://lyle.smu.edu/~rajand/Video_SR/.

Efficient Background Modeling Based on Sparse Representation and Outlier Iterative Removal

Background modeling is a critical component for various vision-based applications. Most traditional methods tend to be inefficient when solving large-scale problems. In this paper, we introduce sparse representation into the task of large-scale stable-background modeling, and reduce the video size by exploring its discriminative frames. A cyclic iteration process is then proposed to extract the background from the discriminative frame set. The two parts combine to form our sparse outlier iterative removal (SOIR) algorithm. The algorithm operates in tensor space to obey the natural data structure of videos. Experimental results show that a few discriminative frames determine the performance of the background extraction. Furthermore, SOIR can achieve high accuracy and high speed simultaneously when dealing with real video sequences. Thus, SOIR has an advantage in solving large-scale tasks.

Moving Target Tracking in Infrared Image Sequences Based on Differential Kernel Covariance Descriptor

Forward looking infrared (FLIR) imaging has been used in many areas of research and everyday life, but it has been mostly employed in military and security domains. In these fields, remote infrared target tracking is a crucial element for surveillance. However, long-range captured IR image sequences generally have poor contrast, variable illumination, and high background clutter. These challenges make target tracking difficult. This paper suggests a technique for target tracking in different ranges in challenging FLIR image sequences, based on Differential Kernel Covariance Descriptor (DKCD). This new method diminishes rotation and illumination variation effects. The proposed technique calculates the differential kernel matrix of reference target by using various statistical and spatial features such as first and second derivatives, location information, and the intensity value of pixels. Later, the differential covariance matrix is constructed by using different pixel features and applying the appropriate kernel function to the matrix. Thanks to the kernel functions, the algorithm redefines the target's differential spatial features in Hilbert space. This process makes the descriptor non-linear. The predicted position of the target is calculated with the nearest neighbor algorithm in the candidate regions in the sub-frame. The performance of the suggested single target tracking system is then tested on challenging real-life video sequences.

Scene Text Deblurring in Non-stationary Video Sequences

Text detection in natural scenes burdened by imperfect shooting conditions and blurring artifacts is the subject of the present paper. The text as a linguistic component provides a significant amount of information for scene understanding, scene categorization, image retrieval, and many other challenging problems. Usually real video sequences suffer from the superposition of the complicated impacts that are often analyzed separately. The main attention focuses on the text detection with geometric and blurring distortions under blurring and camera shooting artifacts. The original methodology based on the analysis of the gradient sharp profiles includes the automatic text detection in fully or partially blurred frames of a non-stationary video sequence. Also, the blind technique of a blurred text restoration is discussed. Additionally some results of the text detection are mentioned. The detection results for corrupted text fragments from test dataset ICDAR 2015 achieve 76–83% and prevail the detection results of the non-processed by deblurring procedure text fragments upon 40–52%.

Target tracking with dynamically adaptive correlation

A reliable algorithm for target tracking based on dynamically adaptive correlation filtering is presented. The algorithm is capable of tracking with high accuracy the location of a target in an input video sequence without using an offline training process. The target is selected at the beginning of the algorithm. Afterwards, a composite correlation filter optimized for distortion tolerant pattern recognition is designed to recognize the target in the next frame. The filter is dynamically adapted to each frame using information of current and past scene observations. Results obtained with the proposed algorithm in synthetic and real-life video sequences, are analyzed and compared with those obtained with recent state-of-the-art tracking algorithms in terms of objective metrics.

3D-Model-Based Video Analysis for Computer Generated Faces Identification

Modern computer graphics technologies brought realism in computer-generated characters, making them achieve truly natural appearance. Besides traditional virtual reality applications such as avatars, games, or cinema, these synthetic characters may be used to generate realistic fakes, which may lead to improper use of the technology. This fact raises the demand for advanced tools able to discriminate real and artificial human faces in digital media. In this paper, we propose a method to distinguish between computer generated and natural faces by modeling and evaluating their dynamic behavior. Because of a 3D-model-based video analysis, the proposed technique allows identifying synthetic characters by detecting their more limited variability over time. Experimental results demonstrate the effectiveness of the proposed approach also on very challenging and realistic video sequences.

The probability relationship between video’s instantaneous and average aggregate rates

The growing demand for video applications and services has contributed substantially to the increase of video traffic on the Internet. Measurement-based admission control was proposed with the primary aim of eliminating or reducing the need of flow state information; also to control overhead for admission decision and maximize utilization at the potential cost of QoS degradation. Some of the admission algorithms depend on the instantaneous rate for its operation. On the other hand, the average aggregate rate has been proposed to better suit variable rate such as video traffic. In this paper, we investigate the probability relationship between the instantaneous and average aggregate rates for video traffic. A mathematical model has been developed to quantify the probability relationship between both rates and validated through extensive simulations using real video sequences. The average rate was found to be lower than instantaneous for a small number of flows, however there was no pronounced difference for a large number of flows. Furthermore, the difference between both rates increases for fast moving content such as sport or longer measurement time window.

Assessing the performance of a motion tracking system based on optical joint transform correlation

We present an optimized system specially designed for the tracking and recognition of moving subjects in a confined environment (such as an elderly remaining at home). In the first step of our study, we use a VanderLugt correlator (VLC) with an adapted pre-processing treatment of the input plane and a postprocessing of the correlation plane via a nonlinear function allowing us to make a robust decision. The second step is based on an optical joint transform correlation (JTC)-based system (NZ-NL-correlation JTC) for achieving improved detection and tracking of moving persons in a confined space. The proposed system has been found to have significantly superior discrimination and robustness capabilities allowing to detect an unknown target in an input scene and to determine the target’s trajectory when this target is in motion. This system offers robust tracking performance of a moving target in several scenarios, such as rotational variation of input faces. Test results obtained using various real life video sequences show that the proposed system is particularly suitable for real-time detection and tracking of moving objects.

Hybrid Video Stabilization for Mobile Vehicle Detection on SURF in Aerial Surveillance

Detection of moving vehicles in aerial video sequences is of great importance with many promising applications in surveillance, intelligence transportation, or public service applications such as emergency evacuation and policy security. However, vehicle detection is a challenging task due to global camera motion, low resolution of vehicles, and low contrast between vehicles and background. In this paper, we present a hybrid method to efficiently detect moving vehicle in aerial videos. Firstly, local feature extraction and matching were performed to estimate the global motion. It was demonstrated that the Speeded Up Robust Feature (SURF) key points were more suitable for the stabilization task. Then, a list of dynamic pixels was obtained and grouped for different moving vehicles by comparing the different optical flow normal. To enhance the precision of detection, some preprocessing methods were applied to the surveillance system, such as road extraction and other features. A quantitative evaluation on real video sequences indicated that the proposed method improved the detection performance significantly.