Motion Vector Field Research Articles

Moving object detection in the H.264/AVC compressed domain for video surveillance applications

In this paper a novel method is presented to detect moving objects in H.264/AVC [T. Wiegand, G. Sullivan, G. Bjontegaard, G. Luthra, Overview of the H.264/AVC video coding standard, IEEE Transactions on Circuits and Systems for Video Technology, 13 (7) (2003) 560–576] compressed video surveillance sequences. Related work, within the H.264/AVC compressed domain, analyses the motion vector field to find moving objects. However, motion vectors are created from a coding perspective and additional complexity is needed to clean the noisy field. Hence, an alternative approach is presented here, based on the size (in bits) of the blocks and transform coefficients used within the video stream. The system is restricted to the syntax level and achieves high execution speeds, up to 20 times faster than the related work. To show the good detection results, a detailed comparison with related work is presented for different challenging video sequences. Finally, the influence of different encoder settings is investigated to show the robustness of our system.

Region-Level Motion-Based Foreground Segmentation Under a Bayesian Network

This paper presents a probabilistic approach for automatically segmenting foreground objects from a video sequence. In order to save computation time and be robust to noise effects, a region detection algorithm incorporating edge information is first proposed to identify the regions of interest, within which the spatial relationships are represented by a region adjacency graph. Next, we consider the motion of the foreground objects and, hence, utilize the temporal coherence property in the regions detected. Thus, the foreground segmentation problem is formulated as follows. Given two consecutive image frames and the segmentation result priorly obtained, we simultaneously estimate the motion vector field and the foreground segmentation mask in a mutually supporting manner by maximizing the conditional joint probability density function of these two elements. To represent the conditional joint probability density function in a compact form, a Bayesian network is adopted, which is derived to model the interdependency of these two elements. Experimental results for several video sequences are provided to demonstrate the effectiveness of the proposed approach.

3D Convective Storm Identification, Tracking, and Forecasting—An Enhanced TITAN Algorithm

Abstract Storm identification, tracking, and forecasting make up an essential part of weather radar and severe weather surveillance operations. Existing nowcasting algorithms using radar data can be generally classified into two categories: centroid and cross-correlation tracking. Thunderstorm Identification, Tracking, and Nowcasting (TITAN) is a widely used centroid-type nowcasting algorithm based on this paradigm. The TITAN algorithm can effectively identify, track, and forecast individual convective storm cells, but TITAN tends to provide incorrect identification, tracking, and forecasting in cases where there are dense cells whose shape changes rapidly or where clusters of storm cells occur frequently. Aiming to improve the performance of TITAN in such scenarios, an enhanced TITAN (ETITAN) algorithm is presented. The ETITAN algorithm provides enhancements to the original TITAN algorithm in three aspects. First, in order to handle the false merger problem when two storm cells are adjacent, and to isolate individual storm cells from a cluster of storms, ETITAN uses a multithreshold identification method based on mathematical morphology. Second, in the tracking phase, ETITAN proposes a dynamic constraint-based combinatorial optimization method to track storms. Finally, ETITAN uses the motion vector field calculated by the cross-correlation method to forecast the position of the individual isolated storm cells. Thus, ETITAN combines aspects of the two general classes of nowcasting algorithms, that is, cross-correlation and centroid-type methods, to improve nowcasting performance. Results of experiments presented in this paper show the performance improvements of the ETITAN algorithm.

Optical flow approximation based motion object extraction for MPEG-2 video stream

This paper presents a compressed-domain motion object extraction algorithm based on optical flow approximation for MPEG-2 video stream. The discrete cosine transform (DCT) coefficients of P and B frames are estimated to reconstruct DC + 2AC image using their motion vectors and the DCT coefficients in I frames, which can be directly extracted from MPEG-2 compressed domain. Initial optical flow is estimated with Black’s optical flow estimation framework, in which DC image is substituted by DC + 2AC image to provide more intensity information. A high confidence measure is exploited to generate dense and accurate motion vector field by removing noisy and false motion vectors. Global motion estimation and iterative rejection are further utilized to separate foreground and background motion vectors. Region growing with automatic seed selection is performed to extract accurate object boundary by motion consistency model. The object boundary is further refined by partially decoding the boundary blocks to improve the accuracy. Experimental results on several test sequences demonstrate that the proposed approach can achieve compressed-domain video object extraction for MPEG-2 video stream in CIF format with real-time performance.

An Efficient Intermode Decision Algorithm Based on Motion Homogeneity for H.264/AVC

The latest video coding standard H.264/AVC significantly outperforms previous standards in terms of coding efficiency. H.264/AVC adopts variable block sizes ranging from 4 times 4 to 16 times 16 in inter frame coding, and achieves significant gain in coding efficiency compared to coding a macroblock (MB) using regular block size. However, this new feature causes extremely high computation complexity when rate-distortion optimization (RDO) is performed using the scheme of full mode decision. This paper presents an efficient intermode decision algorithm based on motion homogeneity evaluated on a normalized motion vector (MV) field, which is generated using MVs from motion estimation on the block size of 4 times 4. Three directional motion homogeneity measures derived from the normalized MV field are exploited to determine a subset of candidate intermodes for each MB, and unnecessary RDO calculations on other intermodes can be skipped. Experimental results demonstrate that our algorithm can reduce the entire encoding time about 40% on average, without any noticeable loss of coding efficiency.

An Efficient Framework for Configurable Video Encoder

The implementation of real-time video coding systems with limited resources such as memory volume and CPU speed is a challenging task. In this paper, an efficient embedded multi-modal MPEG-4 video encoder (EMVC) is proposed and implemented to meet the requirement of encoding different modals (colors, resolutions, and bandwidths) video in size and power-restricted scenario. The hardware framework of EMVC is based on FPGA and DSP, which takes full advantage of the strongpoint of DSP and FPGA—powerful digital signal processing ability of DSP is utilized to complete most of the complex video coding, and flexible configuration, the parallelism and high speed of FPGA cooperates to accomplish logical control and auxiliary functions in video coding. The hardware framework is designed with strong processing ability, flexible configuration, and extensive coverage. In software design, we first improve some key algorithms in MPEG-4 standard that have great impact on coding efficiency, a fast motion estimation algorithm—smooth motion vector field adaptive search technique (SMVFAST) and a DSP-based efficient quantization computation method—EQ4DSP are put forward respectively; second, according to the characters of DSP's hardware resource, the encoder software is optimized at system and code levels. At the system level, EDMA (Enhanced Direct Memory Access) channels in DSP are used to parallel the data transfer, and the software structure of the video encoders is changed from frame-based to GMB-based (Group of Macro-Block) to make use of the parallel processing capacity of DSP. At code level, the codes of the frequently called modules in the encoder are the critical optimized part, and the main code optimization methods used in EMVC include: using an intrinsic function to replace the complicated C/C++ code, using packed data processing to maximize the data throughput, unrolling the loop to increase software pipelines, translating C code to Linear Assembly, and so on. By making use of the reserved user-data field in the MPEG-4 standard syntax, a multi-modal function of EMVC is realized, namely, the system can switch among 720 × 576 and 352 × 288 resolutions, grey and color video, 2 M and 256 K bandwidth. Moreover, the mode switch can be achieved in real time according to the input control command. Experimental results show that EMVC not only runs efficiently but also has good flexibility and generality. Furthermore, by realizing diverse video or image coding algorithms, it can also satisfy the requirements of various embedded video applications, especially those with strict restriction in size, power consumption, and control flexibility.

Biomimicking the Brain for Biomedical Image Analysis

Event Abstract Back to Event Biomimicking the Brain for Biomedical Image Analysis Bart-ter Haar-Romeny1* 1 Eindhoven University of Technology, Netherlands The production of images in medical applications and life sciences research is overwhelming. Mathematical paradigms for quantitative image analysis can be inspired by apparent models of early stages of visual perception. We discuss models of cortical neurons as multi-scale derivative operators for shape analysis, segmentation and retrieval, and adaptive feedback loops to the LGN for edge preserving smoothing. Voltage sensitive dye measurements of cortical columns inspire to multi-orientation contextual filters. The resulting algorithms lead to robust feature extraction, enhancement of dim lines and tensor valued images, retrieval of sub-scenes, and the extraction of dense motion vector fields. Many illustrative examples will be presented. Conference: Neuroinformatics 2009, Pilsen, Czechia, 6 Sep - 8 Sep, 2009. Presentation Type: Oral Presentation Topic: Workshop 1- Advances in the automatic analysis of multi-dimensional data Citation: Haar-Romeny B (2019). Biomimicking the Brain for Biomedical Image Analysis. Front. Neuroinform. Conference Abstract: Neuroinformatics 2009. doi: 10.3389/conf.neuro.11.2009.08.129 Copyright: The abstracts in this collection have not been subject to any Frontiers peer review or checks, and are not endorsed by Frontiers. They are made available through the Frontiers publishing platform as a service to conference organizers and presenters. The copyright in the individual abstracts is owned by the author of each abstract or his/her employer unless otherwise stated. Each abstract, as well as the collection of abstracts, are published under a Creative Commons CC-BY 4.0 (attribution) licence (https://creativecommons.org/licenses/by/4.0/) and may thus be reproduced, translated, adapted and be the subject of derivative works provided the authors and Frontiers are attributed. For Frontiers’ terms and conditions please see https://www.frontiersin.org/legal/terms-and-conditions. Received: 10 Jun 2009; Published Online: 09 May 2019. * Correspondence: Bart-ter Haar-Romeny, Eindhoven University of Technology, Eindhoven, Netherlands, B.M.terHaarRomeny@tue.nl Login Required This action requires you to be registered with Frontiers and logged in. To register or login click here. Abstract Info Abstract The Authors in Frontiers Bart-ter Haar-Romeny Google Bart-ter Haar-Romeny Google Scholar Bart-ter Haar-Romeny PubMed Bart-ter Haar-Romeny Related Article in Frontiers Google Scholar PubMed Abstract Close Back to top Javascript is disabled. Please enable Javascript in your browser settings in order to see all the content on this page.

Efficient Frame Error Concealment Using Bilateral Motion Estimation for Low Bit-Rate Video Transmission

In low bit-rate video transmission, the payload of a single packet can often contain a whole coded frame due to the high compression ratio in both spatial and temporal domains. Thus, the loss of a single packet can lead to the loss of a whole video frame. In this paper, we propose a novel error concealment algorithm that can effectively reconstruct the lost frame and protect the quality of video streams from the degradation caused by propagation errors. The proposed algorithm employs a bilateral motion estimation scheme where the weighted sum of the received motion vectors (MVs) in the neighboring frames is utilized to construct the MV field for the concealed frame. Unlike the conventional algorithms, the proposed scheme does not produce any overlapped pixel and hole region in the reconstructed frame. The proposed algorithm can be applied not only to the case of single frame loss but also adaptively extended to the case of multiframe loss. Experimental results show that the proposed algorithm outperforms other conventional techniques in terms of both peak signal-to-noise ratio (PSNR) performance and subjective visual quality.

Improvement of cardiac CT reconstruction using local motion vector fields

The motion of the heart is a major challenge for cardiac imaging using CT. A novel approach to decrease motion blur and to improve the signal to noise ratio is motion compensated reconstruction which takes motion vector fields into account in order to correct motion. The presented work deals with the determination of local motion vector fields from high contrast objects and their utilization within motion compensated filtered back projection reconstruction. Image registration is applied during the quiescent cardiac phases. Temporal interpolation in parameter space is used in order to estimate motion during strong motion phases. The resulting motion vector fields are during image reconstruction. The method is assessed using a software phantom and several clinical cases for calcium scoring. As a criterion for reconstruction quality, calcium volume scores were derived from both, gated cardiac reconstruction and motion compensated reconstruction throughout the cardiac phases using low pitch helical cone beam CT acquisitions. The presented technique is a robust method to determine and utilize local motion vector fields. Motion compensated reconstruction using the derived motion vector fields leads to superior image quality compared to gated reconstruction. As a result, the gating window can be enlarged significantly, resulting in increased SNR, while reliable Hounsfield units are achieved due to the reduced level of motion artefacts. The enlargement of the gating window can be translated into reduced dose requirements.

Automatic generation of time resolved motion vector fields of coronary arteries and 4D surface extraction using rotational x-ray angiography

Rotational coronary angiography provides a multitude of x-ray projections of the contrast agent enhanced coronary arteries along a given trajectory with parallel ECG recording. These data can be used to derive motion information of the coronary arteries including vessel displacement and pulsation. In this paper, a fully automated algorithm to generate 4D motion vector fields for coronary arteries from multi-phase 3D centerline data is presented. The algorithm computes similarity measures of centerline segments at different cardiac phases and defines corresponding centerline segments as those with highest similarity. In order to achieve an excellent matching accuracy, an increasing number of bifurcations is included as reference points in an iterative manner. Based on the motion data, time-dependent vessel surface extraction is performed on the projections without the need of prior reconstruction. The algorithm accuracy is evaluated quantitatively on phantom data. The magnitude of longitudinal errors (parallel to the centerline) reaches approx. 0.50 mm and is thus more than twice as large as the transversal 3D extraction errors of the underlying multi-phase 3D centerline data. It is shown that the algorithm can extract asymmetric stenoses accurately. The feasibility on clinical data is demonstrated on five different cases. The ability of the algorithm to extract time-dependent surface data, e.g. for quantification of pulsating stenosis is demonstrated.

Motion-compensated iterative cone-beam CT image reconstruction with adapted blobs as basis functions

This paper presents a three-dimensional method to reconstruct moving objects from cone-beam X-ray projections using an iterative reconstruction algorithm and a given motion vector field. For the image representation, adapted blobs are used, which can be implemented efficiently as basis functions. Iterative reconstruction requires the calculation of line integrals (forward projections) through the image volume, which are compared with the actual measurements to update the image volume. In the existence of a divergent motion vector field, a change in the volumes of the blobs has to be taken into account in the forward and backprojections. An efficient method to calculate the line integral through the adapted blobs is proposed. It solves the problem, how to compensate for the divergence in the motion vector field on a grid of basis functions. The method is evaluated on two phantoms, which are subject to three different known motions. Moreover, a motion-compensated filtered back-projection reconstruction method is used, and the reconstructed images are compared. Using the correct motion vector field with the iterative motion-compensated reconstruction, sharp images are obtained, with a quality that is significantly better than gated reconstructions.

Multiframe-based bilateral motion estimation with emphasis on stationary caption processing for frame rate up-conversion

In this paper, we present a new motion compensated frame rate up-conversion algorithm that uses multiframes to enhance the accuracy of motion estimation. We also develop adaptive motion vector smoothing to correct outliers in a motion vector field. In addition, stationary caption processing is carried out for removing block artifacts from stationary captions. Finally, we propose adaptive-weighted bidirectional motion compensated interpolation to reduce object blurring. In experiments using benchmark test sequences, the proposed algorithm improves the average PSNR of interpolated frames by up to 5.34 dB compared to the conventional algorithm.

RD Optimized Coding for Motion Vector Predictor Selection

The H.264/MPEG4-AVC video coding standard has achieved a higher coding efficiency compared to its predecessors. The significant bitrate reduction is mainly obtained by efficient motion compensation tools, as variable block sizes, multiple reference frames, 1/4-pel motion accuracy and powerful prediction modes (e.g., SKIP and DIRECT). These tools have contributed to an increased proportion of the motion information in the total bit- stream. To achieve the performance required by the future ITU-T challenge, namely to provide a codec with 50% bitrate reduction compared to the current H.264, the reduction of this motion information cost is essential. This paper proposes a competing framework for better motion vector coding and SKIP mode. The predictors for the SKIP mode and the motion vector predictors are optimally selected by a rate-distortion criterion. These methods take advantage from the use of the spatial and the temporal redundancies in the motion vector fields, where the simple spatial median usually fails. An adaptation of the temporal predictors according to the temporal distances between motion vector fields is also described for multiple reference frames and B-slices options. These two combined schemes lead to a systematic bitrate saving on Baseline and High profile, compared to an H.264/MPEG4-AVC standard codec, which reaches up to 45%.

Simplified Motion-Refined Scheme for Fine-Granularity Scalability

In this paper, we introduce a low-complexity fine-granularity scalable (FGS) video encoder that refines both residue and motion information in the quality layers. The current scalable video coding (SVC) draft shows that significant gains can be achieved when each enhancement layer undergoes the motion estimation/motion compensation (ME/MC) process with its own motion vector field (MVF). However, given the high computational cost of ME/MC, a motion-refined FGS scheme can be expensive to implement. The proposed scheme controls the macroblock (MB) mode allowed in the base layer and channels computational resources to refine motion in enhancement layers. Through a proper selection of Lagrangian factor for the generation of the first MVF, it is possible to design a low-complexity FGS encoder that has good overall coding performance. A simplified motion-refinement scheme is also adopted for selected MBs in enhancement layers by exploiting the correlation of MB-type information between successive layers to further reduce the complexity of the FGS encoder. Meanwhile, a framework of rate-distortion-complexity optimization is proposed for the FGS by considering the interpolation complexity during the ME/MC in each layer. The FGS decoder can be simplified through the reduction in the number of interpolations.

Enhanced motion parameters estimation for an active vision system

In this paper, we present an enhanced approach for estimating 3D motion parameters from 2D motion vector fields. The proposed method achieves valuable reduction in computational time and shows high robustness against noise in the input data. The output of the algorithm is part in a multiobject segmentation approach implemented in an active vision system. Hence, the improvement in the motion parameters estimation process leads to speed-up in the overall segmentation process.

Camera Motion Estimation Using a Novel Online Vector Field Model in Particle Filters

In this paper, a novel algorithm for parametric camera motion estimation is introduced. More particularly, a novel stochastic vector field model is proposed, which can handle smooth motion patterns derived from long periods of stable camera motion and can also cope with rapid camera motion changes and periods when the camera remains still. The stochastic vector field model is established from a set of noisy measurements, such as motion vectors derived, e.g., from block matching techniques, in order to provide an estimation of the subsequent camera motion in the form of a motion vector field. A set of rules for a robust and online update of the camera motion model parameters is also proposed, based on the expectation maximization algorithm. The proposed model is embedded in a particle filters framework in order to predict the future camera motion based on current and prior observations. We estimate the subsequent camera motion by finding the optimum affine transform parameters so that, when applied to the current video frame, the resulting motion vector field to approximate the one estimated by the stochastic model. Extensive experimental results verify the usefulness of the proposed scheme in camera motion pattern classification and in the accurate estimation of the 2D affine camera transform motion parameters. Moreover, the camera motion estimation has been incorporated into an object tracker in order to investigate if the new schema improves its tracking efficiency, when camera motion and tracked object motion are combined.

Adaptive frame rate up-conversion based on motion classification

In this paper, a new technique on video frame rate up-conversion (FRUC) is presented by combining the adaptive motion classification (AMC) for image sequences and the mixed motion estimation (ME). In the proposed FRUC scheme, the AMC classifies sequences into global and local motion sequences (G/LMS) whose advantages are fully utilized. The mixture of bidirectional ME (BDME) and unidirectional ME (UDME) is proposed to avoid hole and “background replace object” phenomenon in interpolated frames. Moreover, the motion vectors (MVs) refinement technique is employed to smooth MVs field. For LMS, the motion detection is used to deal with static background and occlusions are located by using forward prediction of motion regions. Correctional sum of absolute difference (CSAD) which gives more precise physical MVs field than conventional SAD is applied during all of the ME process. Simulation results demonstrate the effectiveness of the proposed FRUC algorithm.

Motion Estimation for Content Adaptive Video Compression

Motion estimation is a multistep process that encompasses techniques such as motion vector prediction, determination of search range and search patterns, and identification of termination criteria. Each of these techniques has several diversions that may suit a particular set of video characteristics. It would be hard to conceive a universal algorithm that can perform well for all kinds of video contents. However, if important characteristics of a video sequence can be identified and utilized for adjusting various steps of motion estimation, one can design an adjustable algorithm that can tune its parameters to suit the video at hand. A multistage motion estimation algorithm that includes a pre-stage for analyzing the motion characteristics of a video sequence online is proposed. This stage predicts the motion vector field (MVF) from the previous coded frame, and clusters macroblocks into background and foreground regions based on the predicted MVF. The information from the pre-stage are passed on to the next stage, which includes a mathematical model for block distortion surface to estimate the distance from the current search point to the global optimal position. This allows the motion estimation stage to adjust its search strategies accordingly. The search is performed on a precise area according to the statistical properties of the motion vector prediction error, separately for foreground and background regions. The proposed algorithm, including the pre-stage, is fast and, therefore, is suitable for online and real-time encoding. Extensive simulation results obtained for several video sequences affirm the effectiveness of the proposed algorithm. The self-tuning property enables the algorithm to perform well for several types of benchmark sequences, yielding better video quality without exceeding the time complexity as compared to the other predictive motion estimation algorithms.

A motion-compensated scheme for helical cone-beam reconstruction in cardiac CT angiography

Since coronary heart disease is one of the main causes of death all over the world, cardiac computed tomography (CT) imaging is an application of very high interest in order to verify indications timely. Due to the cardiac motion, electrocardiogram (ECG) gating has to be implemented into the reconstruction of the measured projection data. However, the temporal and spatial resolution is limited due to the mechanical movement of the gantry and due to the fact that a finite angular span of projections has to be acquired for the reconstruction of each voxel. In this article, a motion-compensated reconstruction method for cardiac CT is described, which can be used to increase the signal-to-noise ratio or to suppress motion blurring. Alternatively, it can be translated into an improvement of the temporal and spatial resolution. It can be applied to the entire heart in common and to high contrast objects moving with the heart in particular, such as calcified plaques or devices like stents. The method is based on three subsequent steps: As a first step, the projection data acquired in low pitch helical acquisition mode together with the ECG are reconstructed at multiple phase points. As a second step, the motion-vector field is calculated from the reconstructed images in relation to the image in a reference phase. Finally, a motion-compensated reconstruction is carried out for the reference phase using those projections, which cover the cardiac phases for which the motion-vector field has been determined.

Motion Vector Sharing and Bitrate Allocation for 3D Video-Plus-Depth Coding

The video-plus-depth data representation uses a regular texture video enriched with the so-called depth map, providing the depth distance for each pixel. The compression efficiency is usually higher for smooth, gray level data representing the depth map than for classical video texture. However, improvements of the coding efficiency are still possible, taking into account the fact that the video and the depth map sequences are strongly correlated. Classically, the correlation between the texture motion vectors and the depth map motion vectors is not exploited in the coding process. The aim of this paper is to reduce the amount of information for describing the motion of the texture video and of the depth map sequences by sharing one common motion vector field. Furthermore, in the literature, the bitrate control scheme generally fixes for the depth map sequence a percentage of 20% of the texture stream bitrate. However, this fixed percentage can affect the depth coding efficiency, and it should also depend on the content of each sequence. We propose a new bitrate allocation strategy between the texture and its associated per-pixel depth information. We provide comparative analysis to measure the quality of the resulting sequences.