Optimal Motion Vector Research Articles

Multi-level rate-constrained successive elimination algorithm tailored to suboptimal motion estimation in HEVC

In the context of motion estimation for video coding, successive elimination algorithms (SEAs) significantly reduce the number of candidates evaluated during motion estimation without altering the resulting optimal motion vector. Nevertheless, SEA is often only used in conjunction with exhaustive search algorithms (e.g., full search). In this paper, we combine the multi-level successive elimination algorithm (ML-SEA) and the rate-constrained successive elimination algorithm (RCSEA) and show that they can be advantageously applied to suboptimal search algorithms. We demonstrate that the savings brought about by the new multi-level RCSEA (ML-RCSEA) outweigh the pre-computational costs of this approach for the Test Zonal (TZ) Search algorithm found in the HM reference encoder. We propose a novel multi-level composition pattern for performing RCSEA on an asymmetric partitioning. We introduce a double-check mechanism for RCSEA, and show that on average, it avoids computing 71% of motion vector (MV) costs. We also apply the proposed ML-RCSEA to bi-predictive refinement search and leverage a cost-based search ordering to remove 56% of error metric computations, on average. When compared to the HM reference encoder, our experiments show that the proposed solution reduces the TZ Search time by approximately 45%, contributing to an average encoding time reduction of about 7%, without increasing the Bjøntegaard delta rate (BD-Rate).

Entropy-Based Video Steganalysis of Motion Vectors.

In this paper, a new method is proposed for motion vector steganalysis using the entropy value and its combination with the features of the optimized motion vector. In this method, the entropy of blocks is calculated to determine their texture and the precision of their motion vectors. Then, by using a fuzzy cluster, the blocks are clustered into the blocks with high and low texture, while the membership function of each block to a high texture class indicates the texture of that block. These membership functions are used to weight the effective features that are extracted by reconstructing the motion estimation equations. Characteristics of the results indicate that the use of entropy and the irregularity of each block increases the precision of the final video classification into cover and stego classes.

A nowcasting technique based on application of the particle filter blending algorithm

To improve the accuracy of nowcasting, a new extrapolation technique called particle filter blending was configured in this study and applied to experimental nowcasting. Radar echo extrapolation was performed by using the radar mosaic at an altitude of 2.5 km obtained from the radar images of 12 S-band radars in Guangdong Province, China. The first bilateral filter was applied in the quality control of the radar data; an optical flow method based on the Lucas–Kanade algorithm and the Harris corner detection algorithm were used to track radar echoes and retrieve the echo motion vectors; then, the motion vectors were blended with the particle filter blending algorithm to estimate the optimal motion vector of the true echo motions; finally, semi-Lagrangian extrapolation was used for radar echo extrapolation based on the obtained motion vector field. A comparative study of the extrapolated forecasts of four precipitation events in 2016 in Guangdong was conducted. The results indicate that the particle filter blending algorithm could realistically reproduce the spatial pattern, echo intensity, and echo location at 30- and 60-min forecast lead times. The forecasts agreed well with observations, and the results were of operational significance. Quantitative evaluation of the forecasts indicates that the particle filter blending algorithm performed better than the cross-correlation method and the optical flow method. Therefore, the particle filter blending method is proved to be superior to the traditional forecasting methods and it can be used to enhance the ability of nowcasting in operational weather forecasts.

Fast Motion Estimation Algorithm using Hybrid Search Patterns for Video Streaming Application

The objective of the paper is to develop new block matching Motion Estimation (ME) algorithm using hybrid search patterns along the direction of best match. The search efficiency for sequences with fast motions and high resolutions is improved by proposing New Cross Diagonal-Hexagon Search (NCDHS) algorithm which involves a novel multi half-hexagon grid global search pattern and a cross diagonal-hexagon local search pattern. The new search pattern enables the proposed algorithm to perform better search using 9.068 search points on an average, to obtain optimal motion vector with slight improvement in quality. This inturn reduces ME Time upto 50.11%, 47.12%, 32.99% and 43.28% on average when compared to the existing Diamond Search (DS), Hexagon Search (HS), New Cross Hexagon Search (NHEXS) and Enhanced Diamond Search (EDS) algorithms respectively. The novelty of the algorithm is further achieved by applying the algorithm proposed for live streaming application. The NCDHS algorithm is run on two MATLAB sessions on the same computer by establishing the connection using Transmission Control Protocol (TCP) /Internet Protocol (IP) network. The ME Time obtained is 14.5986 seconds for a block size 16x16, is less when compared to existing algorithms and that makes the NCDHS algorithm suitable for real time streaming application.

Semi-autonomous mobility assistance for power wheelchair users navigating crowded environments.

Power wheelchair users suffering from cognitive or physical impairment often face difficulties in maneuvering their wheelchairs through crowded environments. Currently, users need to be continuously aware of all traffic around them to actively avoid all collisions. This is an especially difficult task since many wheelchair users are unable to accurately view or perceive their surroundings. Additionally, imprecise joystick control, slowed reaction time, or imperfect interpretation of the environment can lead to unintended collisions with objects in the environment. This work looks to augment user's input with data gathered from an ultrasonic sensor ring to prevent accidental collisions. Using data gathered from the sensors, we detect objects within a certain radius of the chair. This sensor information is combined with the user input from a joystick to generate a potential field description for the intended motion of the wheelchair. An optimal motion vector is calculated which works to avoid collision with obstacles. Ultimately, this control method reduces the cognitive load on the user and enables them to navigate complex environments by providing simple and/or imprecise input to the system.

Multidirectional parabolic prediction-based interpolation-free sub-pixel motion estimation

Motion estimation is one of the most effective key components in video codecs to eliminate temporal redundancies between successive frames. Sub-pixel motion estimation, including sub-pixel interpolation and sub-pixel motion vector prediction, has been employed in many recent international video coding standards to further improve the rate-distortion performance. Although sub-pixel motion estimation plays an important role in improving coding efficiency, it simultaneously brings significant computational complexity, especially in sub-pixel interpolation and motion vector prediction. In this paper, we propose a multidirectional parabolic prediction-based interpolation-free sub-pixel motion estimation scheme. First, four different directional parabolic prediction patterns are developed to estimate the candidate optimal sub-pixel positions, making full use of the available rate-distortion costs of neighboring positions in the integer-pixel motion estimation. Second, an improvement for each directional parabolic prediction pattern is realized by projection operations to generate a more precise candidate optimal sub-pixel position. Third, an optimal sub-pixel position decision method is presented to select the most likely predictive position from the four candidate sub-pixel positions derived in the previous steps. Finally, by fully reusing the information of the powerful advanced motion vector prediction (AMVP) in HEVC, a prediction enhancement mechanism is presented in which the final optimal sub-pixel motion vector (MV) is derived between the MV corresponding to the most likely predictive position and the predictive motion vector (PMV), without requiring extra interpolation operations. Experimental results show that the proposed algorithm significantly reduces the computational complexity of sub-pixel motion estimation and outperforms state-of-the-art interpolation-free sub-pixel motion estimation algorithms with much improved encoding performance and comparable computational complexity reduction.

Context-adaptive based CU processing for 3D-HEVC.

The 3D High Efficiency Video Coding (3D-HEVC) standard aims to code 3D videos that usually contain multi-view texture videos and its corresponding depth information. It inherits the same quadtree prediction structure of HEVC to code both texture videos and depth maps. Each coding unit (CU) allows recursively splitting into four equal sub-CUs. At each CU depth level, it enables 10 types of inter modes and 35 types of intra modes in inter frames. Furthermore, the inter-view prediction tools are applied to each view in the test model of 3D-HEVC (HTM), which uses variable size disparity-compensated prediction to exploit inter-view correlation within neighbor views. It also exploits redundancies between a texture video and its associated depth using inter-component coding tools. These achieve the highest coding efficiency to code 3D videos but require a very high computational complexity. In this paper, we propose a context-adaptive based fast CU processing algorithm to jointly optimize the most complex components of HTM including CU depth level decision, mode decision, motion estimation (ME) and disparity estimation (DE) processes. It is based on the hypothesis that the optimal CU depth level, prediction mode and motion vector of a CU are correlated with those from spatiotemporal, inter-view and inter-component neighboring CUs. We analyze the video content based on coding information from neighboring CUs and early predict each CU into one of five categories i.e., DE-omitted CU, ME-DE-omitted CU, SPLIT CU, Non-SPLIT CU and normal CU, and then each type of CU adaptively adopts different processing strategies. Experimental results show that the proposed algorithm saves 70% encoder runtime on average with only a 0.1% BD-rate increase on coded views and 0.8% BD-rate increase on synthesized views. Our algorithm outperforms the state-of-the-art algorithms in terms of coding time saving or with better RD performance.

Motion vector recovery for video error concealment based on the plane fitting

In a wireless transmission, the compressed video stream can result in lower quality of image reconstruction with the motion vector mismatch. A motion vector recovery for video error concealment based on the plane fitting algorithm is proposed. According to the characteristics of the H.264 video standard, the adjacent macro blocks and the multi reference frame have the correlation. The motion vector which surrounds the lost macro-block in each divided block is projected by one plane which adopts one point to present the MV changing tendency in small regions. This algorithm reconstructs the motion vector of the damaged macro-block by the fitting plane algorithm based on the improved boundary matching algorithm, and then the optimal motion vector is selected to reconstruct the damaged images. This algorithm not only avoids the block effects that caused by the original algorithm, but also in different probability of RTP packet losing, the peak signal to noise ratio of the algorithm has improved 0.3 ~ 2.5 dB compared to the adaptive intra-frame error concealment algorithm .

Fast Motion Estimation Based on Content Property for Low-Complexity H.265/HEVC Encoder

The high definition (HD) and ultra HD videos can be widely applied in broadcasting applications. However, with the increased resolution of video, the volume of the raw HD visual information data increases significantly, which becomes a challenge for storage, processing, and transmitting the HD visual data. The state-of-the-art video compression standard-H.265/High Efficiency Video Coding (HEVC) compresses the raw HD visual data efficiently, while the high compression rate comes at the cost of heavy computation load. Hence, reducing the encoding complexity becomes vital for the H.265/HEVC encoder to be used in broadcasting applications. In this paper, based on the best motion vector selection correlation among the different size prediction modes, we propose a fast motion estimation (ME) method to reduce the encoding complexity of the H.265/HEVC encoder. First, according to the prediction unit (PU) partition type, all PUs are classified into two classes, parent PU and children PUs, respectively. Then, based on the best motion vector selection correlation between the parent PU and children PUs, the block matching search process of the children PUs is adaptively skipped if their parent PU chooses the initial search point as its final optimal motion vector in the ME process. Experimental results show that the proposed method achieves an average of 20% ME time saving as compared with the original HM-TZSearch. Meanwhile, the rate distortion performance degradation is negligible.

Adaptive motion search range adjustment algorithm for HEVC inter coding

The next generation international standard High Efficiency Video Coding (HEVC) can achieve higher compression performance than all previous video coding standards. In the inter mode decision process of HEVC, motion estimation (ME) adopts a fixed search range and using all the possible treeblock sizes and prediction modes to find the one with the least rate distortion (RD) cost using Lagrange multiplier. This technique achieves the highest possible coding efficiency, but it brings extremely high computational complexity which limits HEVC from practical applications. In this paper, an adaptive motion search range adjustment algorithm based on spatially and temporally correlations is proposed to reduce 3D-HEVC computational complexity. Since the optimal motion vectors (MVs) prediction is highly content-dependent, it is not efficient to use all MVs in a fixed search range. We can adjust MVs search range and skip some specific search ranges rarely used in the spatially and temporally neighboring treeblocks. Experimental results show that the proposed algorithm can reduce 48% computational complexity while maintaining almost the same rate-distortion (RD) performance in comparison with the original HEVC encoder.

Motion estimation using the firefly algorithm in ultrasonic image sequence of soft tissue.

Ultrasonic image sequence of the soft tissue is widely used in disease diagnosis; however, the speckle noises usually influenced the image quality. These images usually have a low signal-to-noise ratio presentation. The phenomenon gives rise to traditional motion estimation algorithms that are not suitable to measure the motion vectors. In this paper, a new motion estimation algorithm is developed for assessing the velocity field of soft tissue in a sequence of ultrasonic B-mode images. The proposed iterative firefly algorithm (IFA) searches for few candidate points to obtain the optimal motion vector, and then compares it to the traditional iterative full search algorithm (IFSA) via a series of experiments of in vivo ultrasonic image sequences. The experimental results show that the IFA can assess the vector with better efficiency and almost equal estimation quality compared to the traditional IFSA method.

Texture aided depth frame interpolation

Recent development of depth acquiring technique has accelerated the progress of 3D video in the market. Utilizing the acquired depth, arbitrary view frames can be generated based on depth image based rendering (DIBR) technique in free viewpoint video system. Different from texture video, depth sequence is mainly utilized for virtual view generation rather than viewing. Inspired by this, a depth frame interpolation scheme using texture information is proposed in this paper. The proposed scheme consists of a texture aided motion estimation (TAME) and texture aided motion compensation (TAMC) to fully explore the correlation between depth and the accompanying textures. The optimal motion vectors in TAME and the best interpolation weights in TAMC are respectively selected taking the geometric mapping relationship between depth and the accompanying texture frames into consideration. The proposed scheme is able to not only maintain the temporal consistency among interpolated depth sequence but also improve the quality of virtual frames generated by interpolated depth. Besides, it can be easily applied to arbitrary motion compensation based frame interpolation scheme. Experimental results demonstrate that the proposed depth frame interpolation scheme is able to improve the quality of virtual view texture frames in both subjective and objective criterions compared with existing schemes.

The efficient optimal and suboptimal motion estimation algorithms

The new fast full-search motion estimation algorithm for optimal motion estimation is proposed in this paper. This algorithm presents fast computing method that calculates the tighter boundaries faster by exploiting the computational redundancy. The proposed algorithm, first, determines the possible motion vectors (PMVs) that are not rejected by the first two tighter boundaries to facilitate the prediction of the best initial motion vector (IMV) for the follow-up search. Thereafter, an optimal motion vector (OMV) will be traced out in the PMV set. The IMV greatly helps in the early rejection of impossible candidate blocks while tracing the OMV. Experimental results show that the proposed algorithm outperforms the other previous optimal motion estimation algorithms in reducing the number of computations on several video sequences. The proposed algorithm achieves about 156–456 speed-up gain over full search on several video sequences. But, the state-of-the-art algorithms such as adaptive MSEA and Winner Update algorithm with Integral image algorithms can achieve only about 72–382 speed-up gain over full search on the same video sequences. Finally, the proposed new fast full-search motion estimation algorithm is modified to suboptimal motion estimation algorithm, resulting only in a trivial average peak signal-to-noise ratio drop of about 0.2 dB, but it achieves a very fast computational speed.

A low-power VLSI implementation for fast full-search variable block size motion estimation

Variable block size motion estimation (VBSME) is becoming the new coding technique in H.264/AVC. This article presents a low-power VLSI implementation for VBSME, which employs a fast full-search block-matching algorithm to reduce power consumption, while preserving the optimal motion vectors (MVs). The fast full-search algorithm is based on the comparison of the current minimum sum of absolute difference (SAD) to a conservative lower bound so that unnecessary SAD calculations can be eliminated. We first experimentally determine the specific conservative lower bound of SAD and then implement the fast full-search algorithm in FPGA and 0.18 µm CMOS technology. To the best of our knowledge, this is the first time that a fast full-search block-matching algorithm is explored to reduce power consumption in the context of VBSME and implemented in hardware. Experiment results show that the proposed design can save power consumption by 45% compared to conventional VBSME designs that give optimal MV based on the full-search algorithms.

Modified TZ Search Method for Motion Estimation in Multi-View Video Coding

Motion estimation (ME) plays an important role in digital video compression. But it requires huge complexity to find an optimal motion vector. Multi-view video is obtained by capturing one three-dimensional scene with many cameras at different positions. The computational complexity for motion estimation in multi view video coding increases in proportion to the number of cameras. To reduce computational complexity and maintain the image quality, a modified TZ search method for motion estimation in multi-view video coding is proposed in this paper. The proposed search method exploits the characteristics of the distribution of motion vectors to place the search points. Experiment results show that the speedup improvement of the proposed method over Pel Block Search and TZ search method (JMVC) can be up to more than 77 times and 1.8 ~4.5 times faster respectively by reducing the computational complexity and the image quality degradation is about to 0.05 ~ 1(dB) and 0.010.24 (dB) respectively.

다시점 비디오 부호화를 위한 저 복잡도 움직임 추정 탐색 기법

Although Motion estimation (ME) plays an important role in digital video compression, it requires a complicated search procedure to find an optimal motion vector. Multi-view video is obtained by capturing one three-dimensional scene with many cameras at different positions. The computational complexity of motion estimation for Multi-view video coding increases in proportion to the number of cameras. To reduce computational complexity and maintain the image quality, a low complexity motion estimation search method is proposed in this paper. The proposed search method consists of four-grid diamond search patten, two-gird diamond search pattern and TZ 2 Point search pattern. These search patterns exploit the characteristics of the distribution of motion vectors to place the search points. Experiment results show that the speedup improvement of the proposed method over TZ search method (JMVC) can be up to 1.8~4.5 times faster by reducing the computational complexity and the image quality degradation is about to 0.01~0.24 (dB).

Motion Estimation Without Integer-Pel Search

The typical motion estimation (ME) consists of three main steps, including spatial-temporal prediction, integer-pel search, and fractional-pel search. The integer-pel search, which seeks the best matched integer-pel position within a search window, is considered to be crucial for video encoding. It occupies over 50% of the overall encoding time (when adopting the full search scheme) for software encoders, and introduces remarkable area cost, memory traffic, and power consumption to hardware encoders. In this paper, we find that video sequences (especially high-resolution videos) can often be encoded effectively and efficiently even without integer-pel search. Such counter-intuitive phenomenon is not only because that spatial-temporal prediction and fractional-pel search are accurate enough for the ME of many blocks. In fact, we observe that when the predicted motion vector is biased from the optimal motion vector (mainly for boundary blocks of irregularly moving objects), it is also hard for integer-pel search to reduce the final rate-distortion cost: the deviation of reference position could be alleviated with the fractional-pel interpolation and rate-distortion optimization techniques (e.g., adaptive macroblock mode). Considering the decreasing proportion of boundary blocks caused by the increasing resolution of videos, integer-pel search may be rather cost-ineffective in the era of high-resolution. Experimental results on 36 typical sequences of different resolutions encoded with x264, which is a widely-used video encoder, comply with our analysis well. For 1080p sequences, removing the integer-pel search saves 57.9% of the overall H.264 encoding time on average (compared to the original x264 with full integer-pel search using default parameters), while the resultant performance loss is negligible: the bit-rate is increased by only 0.18%, while the peak signal-to-noise ratio is decreased by only 0.01 dB per frame averagely.

Error detection scheme for the H.264/AVC using the RD optimized motion vector constraints

The corruption of compressed video bitstreams during their transmission over error-prone communication channels leads to degradation in the visual quality at the decoder. In order to effectively conceal the corrupted video data an exact error detection scheme is of utmost importance. In this paper, we present a new error detection scheme for the H.264/AVC using the RD optimized motion vector constraints to improve the error detection rate. The simulation results show that the detection rate of the proposed algorithm is about 86%, although the proposed scheme brings about a small amount of bitrate increment and causes a slight decrement in the PSNR during the encoding.

에러 강인성을 위한 모순 검증 기반 움직임 벡터 부-복호화

This papers proposes a motion vector coding method which selects the optimal predictive motion vector after excluding some candidate predictive motion vectors in the minimum rate-distortion sense, and prunes the candidate predictive motion vectors by a contradiction testing that operates under a predefined criterion at both encoder and decoder for reducing the signaling information. Experimental results show that the proposed method gains in BDBR by up to 1.8% compared to the H.264/AVC.

Adaptive Search Range Decision for Fast Intra- and Inter-prediction Mode Decision in H.264/AVC

H.264/AVC, the latest video coding standard, adopts rate-distortion optimization (RDO) technique to obtain the best intra- and inter-prediction, while maximizing visual quality and minimizing the required bitrate. However, the full RD cost calculation for all intra-prediction modes, the exhaustive searches for finding optimal motion vectors for all block sizes, and the multiple references frame procedure considerably increase its computational complexity with the allowed number of prediction modes. The authors have previously proposed an approach for both inter- and intra-mode decisions that takes into account the several effective parameters, image content type, quantization parameter, correlation of motion vectors and motion details of video objects. In order to reduce the complexity, here we propose a new two-step adaptive search range decision by developing a combination of previous algorithm. The first step of our adaptive search range decision determines the search range for current MB according to the motion vectors of the previous MBs. In the second step, the motion vector obtained in the current frame is used to adaptively searching the window size of the previous reference frame for multi-frame motion estimation. As such, only a subset of inter- and intra-modes is chosen for RDO calculation. Experimental results show that the proposed algorithm reduces the total encoding cost with negligible loss in PSNR and a slightly increase in the required bitrate, when compared to our previous algorithm and other algorithms reported in the literature.