Global Motion Vector Research Articles

Pre-Processing, Quality Assurance, and Use of Global Atmospheric Motion Vector Observations in CRA

Pre-Processing, Quality Assurance, and Use of Global Atmospheric Motion Vector Observations in CRA

Mechanical and Electronic Video Stabilization Strategy of Mortars with Trajectory Correction Fuze Based on Infrared Image Sensor.

For a higher attack accuracy of projectiles, a novel mechanical and electronic video stabilization strategy is proposed for trajectory correction fuze. In this design, the complexity of sensors and actuators were reduced. To cope with complex combat environments, an infrared image sensor was used to provide video output. Following the introduction of the fuze’s workflow, the limitation of sensors for mechanical video stabilization on fuze was proposed. Particularly, the parameters of the infrared image sensor that strapdown with fuze were calculated. Then, the transformation relation between the projectile’s motion and the shaky video was investigated so that the electronic video stabilization method could be determined. Correspondingly, a novel method of dividing sub-blocks by adaptive global gray threshold was proposed for the image pre-processing. In addition, the gray projection algorithm was used to estimate the global motion vector by calculating the correlation between the curves of the adjacent frames. An example simulation and experiment were implemented to verify the effectiveness of this strategy. The results illustrated that the proposed algorithm significantly reduced the computational cost without affecting the accuracy of the motion estimation. This research provides theoretical and experimental basis for the intelligent application of sensor systems on fuze.

Intelligent software defined network based digital video stabilization system using frame transparency threshold pattern stabilization method

Digital Video Stabilization (DVS) allows the acquisition of video sequences without disturbing jerkiness and removing unwanted camera motion. Many researchers attempt to improve the performance of Digital Video Stabilization but all of them not get perfect results. Therefore in this research work we introduce different DVS strategy that proposes to transfer camera veins to a Frame Transparency Threshold Pattern Stabilization (FTTPS) algorithm. The proposed FTTPS algorithm consists of four steps: (i) Frame Representation; (ii) Motion Estimation; (iii) Filtering and (iv) Compensation. The proposed FTTPS method also utilizes an adaptive criterion for filtering and validation of motion vectors moreover to register a Global Motion Vector (GMV) for each frame, the proposed DVS method satisfactorily assesses the think camera motion by moving an unwanted motion of characteristics. The proposed Frame Transparency Threshold Pattern Stabilization method is basically customized in Matlab programming language, the simulation of the proposed system validated through Inter-Frame Transformation Fidelity (ITF) and Peak to Signal Noise Ratio (PSNR) Values and which is more pragmatic and takes less chip asset and general 93.12% efficiency accomplished utilizing proposed Frame Transparency Threshold Pattern Stabilization system. The suggested digital video stabilization system can be combined with sensible and robot vision, and visual monitoring structures for various low cameras display quality updates.

Winds of Change for Future Operational AMV at EUMETSAT

EUMETSAT, the European Organization for the Exploitation of Meteorological Satellites, is one of the key contributors to global atmospheric motion vector (AMV) production around the world. Its current contribution includes geostationary satellites at 0.0 and 41.5 degrees east, and several products extracted from the Metop low-orbit satellites. These last ones mainly cover high-latitude regions completing the observations from the geostationary ring. In the upcoming years, EUMETSAT will launch a new generation of geostationary and low-orbit satellites. The imager instruments Flexible Combined Imager (FCI) and METImage will take over the nominal AMV production at EUMETSAT around 2022 and 2024. The enhanced characteristics of these new-generation instruments are expected to increase AMV production and to improve the quality of the products. This paper presents an overview of the current EUMETSAT AMV operational production, together with a roadmap of the preparation activities for the new generation of satellites. The characteristics of the upcoming AMV products are described and compared to the current operational AMV products. This paper also presents a recent investigation into AMV extraction using the Sentinel-3 Sea and Land Surface Temperature Radiometer (SLSTR) instrument, as well as the retrieval of wind profiles from infrared sounders.

Visual tracking in unstabilized real time videos using SURF

Auto target detection and tracking has always been a challenging task for the researchers. The problem becomes worse, if the input video to tracking algorithm is itself unstabilized. This complex problem of auto target detection and tracking in an unstabilized video is not well addressed and solved in the literature. An algorithm is developed and presented in this paper to auto-stabilize the video, auto detect the target of interest followed by tracking. Global and local motion vectors are computed in the same algorithm simultaneously for frame stabilization and target tracking respectively. Speeded Up Robust Features (SURF) are used innovatively to compute Global Motion Vector (GMV) and Local Motion Vector (LMV) separately. The algorithm auto-detects the unstabilized parameters if any and stabilizes the video by moving the frame in opposite GMV direction with anti rotation. First four frames are used for detecting and training the target of interest, then only two very recent frames are used for auto-detection of target, hence algorithm is very fast and suitable for online real time applications. The algorithm has been experimentally verified and outperforms with bench marked visual tracking algorithms.

Background Modeling and Referencing for Moving Cameras-Captured Surveillance Video Coding in HEVC

Surveillance video coding is crucial for improving compression efficiency in intelligent video surveillance systems and applications. Plenty of work has been done, which can be roughly divided into two categories: the former mainly focuses on low-complexity background modeling to obtain the clear background, while the latter focuses on an appropriate coding strategy to generate the high-quality background reference picture for effective background prediction. However, almost all existing works focus only on stationary camera scenes, while moving cameras-captured surveillance video coding is left untouched and is still an open problem. In this paper, a background modeling and referencing scheme for moving cameras-captured surveillance video coding in high-efficiency video coding (HEVC) is proposed. First, this paper proposes a low-complexity motion background modeling algorithm for surveillance video coding using the running average based on a global-motion-compensation method. To obtain the global motion vector, we propose a global motion detection method based on character blocks by establishing a low-rank singular value decomposition model for clustering and estimating motion vectors of background character blocks in the cameras movement circumstance. Second, we propose a background referencing coding strategy, in which the motion background coding tree units (MBCTUs) would be selected by anchoring the input video frame on the modeling background frame and coded with the optimized quantization parameter. Then, the reconstructed MBCTU will be used to update the previous coding tree unit in the global compensation location of the background reference picture. Extensive experimental results show that the proposed scheme can achieve significant bit savings of up to 26.6% and, on average, 6.7% with similar subjective quality and negligible encoding complexity, compared to HM12.0. Besides, the proposed scheme consistently outperforms two state-of-the-art surveillance video coding schemes with remarkable bitrate savings.

Real-time rolling shutter compensation for a complementary metal-oxide semiconductor image sensor

Complementary metal-oxide semiconductor image sensors suffer from undesirable geometric image distortion due to the rolling shutter effect. However, conventional algorithms cannot be used for strict real-time use in mobile devices. An extremely low computational method to compensate rolling shutter effect for the mobile devices is proposed. The proposed method efficiently predicts the rotational camera motion in three-dimensional space from the global motion vectors predicted in two-dimensional image space. Hence, the computational complexity is extremely reduced. To further improve the video quality, the video stabilization to reduce the high-frequency jitter of the camera motion without increasing the complexity is introduced. The results of simulation demonstrate that while the computational complexity is extremely low, the proposed algorithm provides acceptable performance.

Robust global motion estimation for video security based on improved k-means clustering

The global motion vectors estimation is the most critical step for eliminating undesirable disturbances in unsafe video. In this paper, we proposed a novel global motion estimation approach based on improved K-means clustering algorithm to acquire trustworthy sequences. Firstly, the speeded up robust feature algorithm is employed to match feature points between two adjacent frames, and then we calculate the motion vectors of these matching points. Secondly, to remove the local motion vectors and reduce redundancy from the motion vectors, an improved K-means clustering algorithm is proposed. Thirdly, by using matching points from richest cluster, global motion vectors are calculated by homography transformation. The experimental simulation results demonstrate that the proposed method can obtain significantly higher computational efficiency and superior video security performance than traditional approaches.

Digital Image Stabilization Method Based on Variational Mode Decomposition and Relative Entropy

Cameras mounted on vehicles frequently suffer from image shake due to the vehicles’ motions. To remove jitter motions and preserve intentional motions, a hybrid digital image stabilization method is proposed that uses variational mode decomposition (VMD) and relative entropy (RE). In this paper, the global motion vector (GMV) is initially decomposed into several narrow-banded modes by VMD. REs, which exhibit the difference of probability distribution between two modes, are then calculated to identify the intentional and jitter motion modes. Finally, the summation of the jitter motion modes constitutes jitter motions, whereas the subtraction of the resulting sum from the GMV represents the intentional motions. The proposed stabilization method is compared with several known methods, namely, medium filter (MF), Kalman filter (KF), wavelet decomposition (MD) method, empirical mode decomposition (EMD)-based method, and enhanced EMD-based method, to evaluate stabilization performance. Experimental results show that the proposed method outperforms the other stabilization methods.

A study on surgical robot image stabilization

This study aims to investigate and analyze various image stabilization methods used in surgical robotics. An in-vitro phantom experiment was conducted using a master slave tele-manipulated system. An articulable image probe on a tool (eye-on-tool) was installed on an open-source surgical robot, RAVEN. The 2D non-stereo camera image was used to validate the image stabilization methods, which were evaluated individually for each processing step: preprocessing, motion estimation, and motion compensation. During the preprocessing procedure, the performance of three different filters was tested for effective noise suppression. Various algorithms were compared to estimate the global motion vectors (GMVs) in the motion estimation step. Finally, three filters were analyzed to estimate the compensation motion vector (CMV) during the motion estimation procedure.

A Hybrid System for Measuring Solid and Liquid Contaminants in Steam Generators

This paper presents a hybrid device for real-time measurement and imaging of solid and liquid contaminants that may occur in steam generators. The device uses a dedicated near infrared device to determine the type of contaminants (i.e., water droplets and iron oxide particles) and a THz imaging system, which measures the amount of contaminants as well as its flow rate. The near infrared response (NIR) device can also determine the concentration of contaminants at sub-mg accuracy when its value is relatively low using spectrometry technique combined with principal component analysis (PCA). Three principal components (PC1, PC2, and PC3) were enough for this purpose. The PCA classification was performed using the least square support vector machine method. In case of relatively high concentration, the THz imaging system which uses block-based motion estimation algorithm can determine the velocity of individual contaminant particles to compute the global motion vector, the intensity and direction of which represents the overall flow rate and flow regime of the contaminants. The usage of image processing techniques together with an NIR spectrometry constitutes a new promising step in flow metering. This is demonstrated by the extensive experiments which have been conducted for different scenario where the NIR subsystem could determine the concentration of water droplets and solid contaminants with a maximum uncertainty of +/– 1.45% and +/– 1.16%, respectively. With the NIR subsystem, pixel-level accuracy of motion vector was achieved, whereas the concentration of solid contaminants showed consisted proportionality with the average pixel intensity.

Motion-Compensated Temporal Frame Interpolation Algorithm Based on Global Entirety Unidirectional Motion Estimation and Local Fast Bidirectional Motion Estimation

To solve the uncertainty of motion for objects displayed on liquid crystal display (LCD), the motion estimation (ME) algorithm on the basis of global entirety unidirectional motion and local fast bidirectional motion (GEU-LFB) is proposed for motion-compensated temporal frame interpolation (MCTFI). Firstly, by observing the moving objects in the scene, we will obtain a set of global motion vectors (MVs). Secondly, the fast local search based on the bidirectional ME is executed. Additionally, to make up for the defect of the bidirectional ME, a method to generate the exposure mask and the occlusion mask is proposed by cleverly using the matching criterion of absolute difference and the set of global MVs. Next, the optimal MV field is smoothed by the vector median filter. Finally, the temporal interpolation frame obtained through the weighted filtering compensation method is further improved by using masks. Overall, the experimental results show that the proposed algorithm has better performance than the existing methods in terms of objective and subjective criteria. Moreover, it can solve problems of exposure and occlusion, and get better frame interpolation for the video sequence with fast moving targets.

Using satellite-derived Atmospheric Motion Vector (AMV) observations in the ensemble data assimilation system

The use of global Atmospheric Motion Vectors (AMV) satellite observations in the meteorological data assimilation system based on Local Ensemble Transform Kalman Filter (LETKF) algorithm is considered. The height assignment is the most crucial error source for AMV observations. To reduce its impact, the AMV height reassignment method is implemented; it is based on the consistency coefficient bet ween the observed and the background winds. The other way to improve the analysis quality is a more accurate specification of AMV observation errors. This necessitates the use of the nondiagonal observation-error covariance matrix R in the data assimilation scheme. The first results of these studies are presented. It is demonstrated that the use of AMV observations in the data assimilation system reduces the errors of forecasts computed from the initial data of this system.

Digital image stabilization in mountain areas using complete ensemble empirical mode decomposition with adaptive noise and structural similarity

Cameras mounted on scouting vehicles frequently suffer from image shake because of unintentional motions. Image shake is a main source of inaccuracies that lead to bad scouting results, particularly in mountain areas with complicated terrains. To overcome this disadvantage, this study proposes a digital image stabilization method based on complete ensemble empirical mode decomposition with adaptive noise (CEEMDAN) and structural similarity (SSIM) to generate a stable scouting video sequence. The proposed method first calculates the global motion vector (GMV) from a scouting video sequence using the bit-plane matching algorithm. To separate jitter motion from intentional motion, we decompose GMV into several modes using CEEMDAN. Then according to different structural characteristics, SSIM is used to draw a boundary among modes to separate jitter motion from intentional motion. To evaluate stabilization performance in complicated situations, several known methods and the proposed stabilization method are compared. Experimental results show that CEEMDAN outperforms the other stabilization methods under mountain area conditions.

Improved global motion estimation via motion vector clustering for video stabilization

Video stabilization technique is often used in handheld multimedia devices, whereas the difficulties in the accurate extraction aspect of global motion vectors restrict its development. This paper proposes a novel video stabilization approach that is based on the shortest spanning path clustering algorithm for effective and reliable estimation of the global motion vectors. As demonstrated in our experimental results, the proposed approach achieves superior stabilized effectiveness compared with the other state-of-the-art approaches based on both qualitative and quantitative measurements.

A Two-Stage Algorithm of High Resolution Image Alignment for Mobile Applications

Global motion estimation (GME) algorithms are widely applied to computer vision and video processing. In the previous works, the image resolutions are usually low for the real-time requirement (e.g. video stabilization). However, in some mobile devices applications (e.g. image sequence panoramic stitching), the high resolution is necessary to obtain satisfactory quality of panoramic image. However, the computational cost will become too expensive to be suitable for the low power consumption requirement of mobile device. The full search algorithm can obtain the global minimum with extremely computational cost, while the typical fast algorithms may suffer from the local minimum problem. This paper proposed a fast algorithm to deal with 2560 × 1920 high-resolution (HR) image sequences. The proposed method estimates the motion vector by a two-level coarse-to-fine scheme which only exploits sparse reference blocks (25 blocks in this paper) in each level to determine the global motion vector, thus the computational costs are significantly decreased. In order to increase the effective search range and robustness, the predictive motion vector (PMV) technique is used in this work. By the comparisons of computational complexity, the proposed algorithm costs less addition operations than the typical Three-Step Search algorithm (TSS) for estimating the global motion of the HR images without the local minimum problem. The quantitative evaluations show that our method is comparable to the full search algorithm (FSA) which is considered to be the golden baseline.

Very large‐scale integration architecture for video stabilisation and implementation on a field programmable gate array‐based autonomous vehicle

Autonomous vehicles engaged in terrain exploration are typically equipped with a camera. The camera is subjected to vibration as the vehicle moves so that the videos captured require stabilisation to facilitate accurate interpretation by remote operators. Dedicated architectures for video stabilisation that offer high performance while consuming low area and power are desirable for this application. This study presents a pipelined very large-scale integration architecture. It is based on exploiting the separability property of the two-dimensional (2-D) Sobel matrix and the 2-D Gaussian filtering matrix to obtain an efficient corner point detection architecture. It also employs the coordinate rotation digital computer architecture for global motion vector calculation. The proposed architecture has been coded in Verilog and synthesised for a field programmable gate array (FPGA), which offers massive parallelism at fairly low power. The proposed architecture is shown to be highly area efficient. An FPGA-based autonomous vehicle has been fabricated, and experiments with a camera mounted on the vehicle are presented and analysed.

1 kHz 2D Visual Motion Sensor Using 20<formula formulatype="inline"><tex Notation="TeX">$\,\times\,$</tex></formula>20 Silicon Retina Optical Sensor and DSP Microcontroller

Optical flow sensors have been a long running theme in neuromorphic vision sensors which include circuits that implement the local background intensity adaptation mechanism seen in biological retinas. This paper reports a bio-inspired optical motion sensor aimed towards miniature robotic and aerial platforms. It combines a 20 × 20 continuous-time CMOS silicon retina vision sensor with a DSP microcontroller. The retina sensor has pixels that have local gain control and adapt to background lighting. The system allows the user to validate various motion algorithms without building dedicated custom solutions. Measurements are presented to show that the system can compute global 2D translational motion from complex natural scenes using one particular algorithm: the image interpolation algorithm (I2A). With this algorithm, the system can compute global translational motion vectors at a sample rate of 1 kHz, for speeds up to ±1000 pixels/s, using less than 5 k instruction cycles (12 instructions per pixel) per frame. At 1 kHz sample rate the DSP is 12% occupied with motion computation. The sensor is implemented as a 6 g PCB consuming 170 mW of power.

Local Binary Pattern Based Fast Digital Image Stabilization

In this letter, a fast digital image stabilization method based on local binary pattern (LBP) for real time applications is presented. The LBP approach utilized in this work enables efficient representation of the image frames in 1-bit depth resolution. A simple Boolean exclusive-OR (XOR) based matching criterion is employed to decide global motion vector between consecutive image frames. The constant velocity model based Kalman filtering is executed on the global motion vectors to obtain smoothed frame positions. Experiments show that the proposed approach provides comparable or better performance against the methods at the same category by requiring lower computational complexity.

Fast-rolling shutter compensation based on piecewise quadratic approximation of a camera trajectory

Rolling shutter effect commonly exists in a video camera or a mobile phone equipped with a complementary metal-oxide semiconductor sensor, caused by a row-by-row exposure mechanism. As video resolution in both spatial and temporal domains increases dramatically, removing rolling shutter effect fast and effectively becomes a challenging problem, especially for devices with limited hardware resources. We propose a fast method to compensate rolling shutter effect, which uses a piecewise quadratic function to approximate a camera trajectory. The duration of a quadratic function in each segment is equal to one frame (or half-frame), and each quadratic function is described by an initial velocity and a constant acceleration. The velocity and acceleration of each segment are estimated using only a few global (or semiglobal) motion vectors, which can be simply predicted from fast motion estimation algorithms. Then geometric image distortion at each scanline is inferred from the predicted camera trajectory for compensation. Experimental results on mobile phones with full-HD video demonstrate that our method can not only be implemented in real time, but also achieve satisfactory visual quality.