Digital Video Stabilization Research Articles

Digital Video Stabilization Method Based on Periodic Jitters of Airborne Vision of Large Flapping Wing Robots

Digital Video Stabilization Method Based on Periodic Jitters of Airborne Vision of Large Flapping Wing Robots

Lightweight digital video stabilization for small-size robot

Lightweight digital video stabilization for small-size robot

Rethinking two-dimensional camera motion estimation assessment for digital video stabilization: A camera motion field-based metric

Digital video stabilization aims to remove camera motion jitters through software implementation. The first step of the classical video stabilization methodology is called camera motion estimation, which is usually performed using only RGB frames of the unstable video. Despite recent advances in camera motion estimation strategies, methods classified as two-dimensional are still not properly evaluated, even though it is well known that motion estimation is a crucial step for classical approaches to video stabilization. The main purpose of this work is to draw attention to two-dimensional camera motion estimation assessment and reinforce its importance on video stabilization progress. We proposed a new approach to perform this evaluation using camera motion fields in a pixel-by-pixel comparison and demonstrated through experimental results that our metrics are reliable for diverse scenarios comparing them to image similarity metrics. In addition, we showed and analyzed the results of our metrics for a global and a local method of camera motion estimation. We believe that our assessment and study presented in this work is an important starting point for a more rigorous analysis of this task. In addition, this can be a foundation for coming 2D camera motion estimation methods based on deep learning.

Real-Time Digital Video Stabilization of Bioinspired Robotic Fish Using Estimation-and-Prediction Framework

The rhythmic movement of bioinspired robotic fish brings about undesirable visual jitter. Note that the unstable camera path of this kind of robot is characterized by obvious regularity and predictability, of which traditional stabilization methods have not made full advantage. This article proposes a novel estimation-and-prediction framework for real-time digital video stabilization of bioinspired robotic fish. First, based on the attitude information of an inertial measurement unit (IMU), a camera–IMU model is established, where the homography transformation with eight degrees of freedom (DOFs) is reduced to translation transformation with two DOFs. Second, traditional optical flow and gray projection methods as well as a novel translation estimation network are employed to estimate the translations between consecutive frames. Third, a lightweight long short-term memory (LSTM) network is constructed, allowing remarkable prediction and smoothing of the camera path. Finally, aquatic experiments under various scenarios are conducted on a manta-inspired robot, demonstrating the effectiveness of the proposed method. Specifically, compared with the state-of-the-art commercial offline stabilization software, our online stabilization algorithm achieves approximate visual stability and remarkably faster stabilization speed. The obtained results shed light on visual sensing and control applications of bioinspired underwater vehicles.

Survey on Digital Video Stabilization: Concepts, Methods, and Challenges

Digital video stabilization is a challenging task that aims to transform a potentially shaky video into a pleasant one by smoothing the camera trajectory. Despite the various works found in the literature addressing this task, their organization and analysis have not yet received much attention. In this work, we present a thorough review of the literature for video stabilization, organized according to a proposed taxonomy. A formal definition for the problem is introduced, along with a brief interpretation in physical terms. We also provide a comprehensive discussion on the main challenges and future trends for this active area.

Digital Video Stabilization Verification Based on Genetic Algorithm Template Matching

Digital Video Stabilization Verification Based on Genetic Algorithm Template Matching

A novel DNN tracking algorithm for structural system identification

In the field of structural health monitoring (SHM), cameras record videos and tracking methods can be applied to calculate the structural displacement. Commercial and unmanned aerial vehicle (UAV) cameras are promising non-contact sensors owning to their high availability and easy installation. However, effective tracking methods need to be developed. In this study, we firstly propose an end-to-end vision measuring framework with a novel deep neural network (DNN) tracker, named Siamese Single Decoder Network (SiamSDN). The system requires no target installation and uses cellphone cameras. For SiamSDN, the position and scale of bounding box are formulated through statistical parameter estimation. Unlike generative trackers, SiamSDN does not require manually extracted features or pre-defined motion areas. The tracking object is solely identified in the first frame. A shaking table test of a five-storey structure is carried out to demonstrate the efficiency. Besides, a UAV is used to simulate the field test. To minimize the error caused by the vibrations of UAV, digital video stabilization (DVS) is proposed to eliminate the drifts. Videos taken by both the commercial and UAV cameras are analyzed to calculate the displacements. Comparing our DNN tracker with feature point matching approach, SiamSDN improves the displacement measuring accuracy by 66.16% and 57.54%, respectively, and the frequency characteristics are obtained precisely.

Evaluation and Selection of Video Stabilization Techniques for UAV-Based Active Infrared Thermography Application.

Unmanned Aerial Vehicles (UAVs) that can fly around an aircraft carrying several sensors, e.g., thermal and optical cameras, to inspect the parts of interest without removing them can have significant impact in reducing inspection time and cost. One of the main challenges in the UAV based active InfraRed Thermography (IRT) inspection is the UAV’s unexpected motions. Since active thermography is mainly concerned with the analysis of thermal sequences, unexpected motions can disturb the thermal profiling and cause data misinterpretation especially for providing an automated process pipeline of such inspections. Additionally, in the scenarios where post-analysis is intended to be applied by an inspector, the UAV’s unexpected motions can increase the risk of human error, data misinterpretation, and incorrect characterization of possible defects. Therefore, post-processing is required to minimize/eliminate such undesired motions using digital video stabilization techniques. There are number of video stabilization algorithms that are readily available; however, selecting the best suited one is also challenging. Therefore, this paper evaluates video stabilization algorithms to minimize/mitigate undesired UAV motion and proposes a simple method to find the best suited stabilization algorithm as a fundamental first step towards a fully operational UAV-IRT inspection system.

Feature based video stabilization based on boosted HAAR Cascade and representative point matching algorithm

The success of handheld video capturing devices has further fueled the need of improved video stabilization. The videos often contain many foreground facial features like eyes, nose etc. These foreground features can be considered as feature points and may be used to stabilize videos. This paper proposes an innovative and effective digital video stabilization technique, which utilizes foreground features present in the video to produce consistent and stabilized output. It uses successive stages of Boosted HAAR cascade and representative point matching digital motion stabilization algorithm to identify and stabilize the video. The feature based tracking of object improves motion estimation accuracy between two frames thereby increasing the correlation calculation and compensation motion vector. This work achieves a significantly smoother sequence after the motion compensation. It also improves the robustness, precision and quality of the video when compared to traditional digital stabilization algorithms. The simulation results compared with pre-existing techniques reflect distinct improvements in Inter-Frame Transformation Fidelity values and Structural Similarity Index along with lesser standard deviation between image frames.

Intelligent software defined network based digital video stabilization system using frame transparency threshold pattern stabilization method

Digital Video Stabilization (DVS) allows the acquisition of video sequences without disturbing jerkiness and removing unwanted camera motion. Many researchers attempt to improve the performance of Digital Video Stabilization but all of them not get perfect results. Therefore in this research work we introduce different DVS strategy that proposes to transfer camera veins to a Frame Transparency Threshold Pattern Stabilization (FTTPS) algorithm. The proposed FTTPS algorithm consists of four steps: (i) Frame Representation; (ii) Motion Estimation; (iii) Filtering and (iv) Compensation. The proposed FTTPS method also utilizes an adaptive criterion for filtering and validation of motion vectors moreover to register a Global Motion Vector (GMV) for each frame, the proposed DVS method satisfactorily assesses the think camera motion by moving an unwanted motion of characteristics. The proposed Frame Transparency Threshold Pattern Stabilization method is basically customized in Matlab programming language, the simulation of the proposed system validated through Inter-Frame Transformation Fidelity (ITF) and Peak to Signal Noise Ratio (PSNR) Values and which is more pragmatic and takes less chip asset and general 93.12% efficiency accomplished utilizing proposed Frame Transparency Threshold Pattern Stabilization system. The suggested digital video stabilization system can be combined with sensible and robot vision, and visual monitoring structures for various low cameras display quality updates.

Real-time high-resolution video stabilization using high-frame-rate jitter sensing

In this study, the novel approach of real-time video stabilization system using a high-frame-rate (HFR) jitter sensing device is demonstrated to realize the computationally efficient technique of digital video stabilization for high-resolution image sequences. This system consists of a high-speed camera to extract and track feature points in gray-level 512times 496 image sequences at 1000 fps and a high-resolution CMOS camera to capture 2048times 2048 image sequences considering their hybridization to achieve real-time stabilization. The high-speed camera functions as a real-time HFR jitter sensing device to measure an apparent jitter movement of the system by considering two ways of computational acceleration; (1) feature point extraction with a parallel processing circuit module of the Harris corner detection and (2) corresponding hundreds of feature points at the current frame to those in the neighbor ranges at the previous frame on the assumption of small frame-to-frame displacement in high-speed vision. The proposed hybrid-camera system can digitally stabilize the 2048times 2048 images captured with the high-resolution CMOS camera by compensating the sensed jitter-displacement in real time for displaying to human eyes on a computer display. The experiments were conducted to demonstrate the effectiveness of hybrid-camera-based digital video stabilization such as (a) verification when the hybrid-camera system in the pan direction in front of a checkered pattern, (b) stabilization in video shooting a photographic pattern when the system moved with a mixed-displacement motion of jitter and constant low-velocity in the pan direction, and (c) stabilization in video shooting a real-world outdoor scene when an operator holding hand-held hybrid-camera module while walking on the stairs.

Video stabilization performance enhancement for low-texture videos

Digital video stabilization (DVS) aims to remove irregular global motion effects from an image sequence. This work aims at developing a real-time video stabilization algorithm for rectifying high-frequency jitter in marine surveillance applications. A DVS system consists of a global motion estimation system and motion correction system. The development of global motion estimation system resistant to failures in low texture videos is the primary goal. Due to the computational advantage and inherent properties, the phase correlation method is adopted as the basic global motion estimation algorithm. The basic algorithm is then modified to adapt to the varying texture content of the video sequences under consideration. An adaptive phase correlation-based global motion estimation is suggested and verified on the videos of varying textures.

Full-Reference Stability Assessment of Digital Video Stabilization Based on Riemannian Metric.

Assessing the quality of the motion stability is important to evaluating the performance of video stabilization algorithms. This paper presents a novel quality assessment scheme for the video motion stability in a full-reference (FR) manner. Given ideally stable videos and their corresponding shaky videos, our method measures the geodesic distance between motion paths of the stable and the stabilized videos. Due to the use of the Riemannian metric defined on the manifold of spatial transformations, our method enables the intrinsic and faithful measurement on pairwise motion disparities. To facilitate the FR assessment, a data set of stable and shaky videos is constructed by directly capturing realistic stable/shaky videos with a customized device. Then, digital video stabilization algorithms can be run on shaky videos to obtain the stabilized sequence of frames, whereupon their performances are evaluated by using our stability assessment. The experiments demonstrate that our stability assessment gains good concordance with the subjective assessment.

Digital video stabilization based on multilayer gray projection

Video stabilization is an important video optimization technology which is aimed at removing unconscious shakiness from videos. In this paper, we present a multilayer gray projection method to estimate image motion. While classical gray projection works only on the videos which just have translation. To achieve this, a multilayer gray projection algorithm is proposed to estimate translation, rotation and scaling motion between target and reference images. First, differential gray projection is applied to the test and target images to estimate relative image translation, the best projection results can help us find corresponding projection center of ring-projection and circular projection in the next two steps. We then calculate the scaling and rotation between test and target images by doing ring-projection and circular projection. The performance of our method is tested by robust experiments, estimation accuracy analysis and experiments over several videos.

Digital video stabilization based on adaptive camera trajectory smoothing

The development of multimedia equipments has allowed a significant growth in the production of videos through professional and amateur cameras, smartphones, and other mobile devices. Examples of applications involving video processing and analysis include surveillance and security, telemedicine, entertainment, teaching, and robotics. Video stabilization refers to the set of techniques required to detect and correct glitches or instabilities caused during the video acquisition process due to vibrations and undesired motion when handling the camera. In this work, we propose and evaluate a novel approach to video stabilization based on an adaptive Gaussian filter to smooth the camera trajectories. Experiments conducted on several video sequences demonstrate the effectiveness of the method, which generates videos with adequate trade-off between stabilization rate and amount of frame pixels. Our results were compared to YouTube’s state-of-the-art method, achieving competitive results.

Combination of local feature detection methods for digital video stabilization

Technological advances in compact and portable cameras have enabled the generation of large volumes of video sequences. However, videos captured by amateurs are subject to unwanted vibrations due to camera shaking. To overcome such problem, video stabilization aims to remove undesired motion from videos to enhance visual quality, improving applications such as detection and tracking of objects. In this work, we develop and analyze a consensual method for combining a set of local feature techniques for camera motion estimation. Several video sequences are used to evaluate the proposed methodology. Experimental results demonstrate the effectiveness of the combination method over individual local feature approaches.

Real-Time Feature-Based Video Stabilization on FPGA

Digital video stabilization is an important video enhancement technology that aims to remove unwanted camera vibrations from video sequences. Trading off between stabilization performance and real-time hardware implementation feasibility, this paper presents a feature-based full-frame video stabilization method and a novel complete fully pipelined architectural design to implement it on field-programmable gate array (FPGA). In the proposed method, feature points are first extracted with the oriented features from accelerated segment test and rotated binary robust independent elementary features algorithm and matched between consecutive frames. Next, the matched point pairs are fitted to the affine transformation model using a random-sample consensus-based approach to estimate inter-frame motion robustly. Then, the estimated results are accumulated to compute the cumulative motion parameters between the current and reference frames, and the translational components are smoothed by a Kalman filter representing intentional camera movement. Finally, a mosaicked image is constructed based on cumulative motion parameters using an image mosaicking technique, and then a display window is created with the desired frame size according to the computed intentional camera movement to obtain a full motion-compensated frame. Using pipelining and parallel processing strategies, the whole process has been designed using a novel complete fully pipelined architecture and implemented on Altera’s Cyclone III FPGA to build a real-time stabilization system. The experimental results have shown that the proposed system can deal with standard PAL video input including arbitrate translation and rotation and can produce full-frame stabilized output providing a better viewing experience at 22.37 ms/frame, thus achieving real-time processing performance.

Structure-based improvement of scene warped locally in digital video stabilization

Local warping is often used in applications of computer vision for stabilization, image resizing, panorama creation, and image editing. In current research, the improvement of the stabilized scene after global and local warping is achieved using stable key points’ trajectories and the correspondences of line segments and arcs. Local warping may cause distortion artifacts that are highly noticeable, especially in structural elements of a scene with lines, arcs, and planes. In the case of the complex scene with multi-level motion, it is reasonable to classify the key points into multi-level sets in order to find the correspondences in the restricted surrounding of the current motion level. First, the proposed method detects the line segments and rarely the arcs in a key frame and groups them in the motion levels. Second, the stable key points’ trajectories are classified into the motion levels. Third, the end points of segments are matched in the current stabilized frame relative to the nearest key frame. The line segments are simply replaced, while the arcs require a special tuning in a view of interpolation or arc samples. The experiments demonstrate the sufficient improvement of visibility in scenes warped locally.

Стабилизация видеоизображения в режиме реального времени с использованием MEMS-датчиков

This article describes our ongoing research on real-time digital video stabilization using MEMS-sensors. The authors propose to use the described method for stabilizing the video that is transmitted to the mobile robot operator who controls the vehicle remotely, as well as increasing the precision of video-based navigation for subminiature autonomous models. The article describes the general mathematical models needed to implement the video stabilization module based on the MEMS sensors readings. These models includes the camera motion model, frame transformation model and rolling-shutter model. The existing approaches to stabilization using sensors data were analyzed and considered from the point of view of the application in a real-time mode. This article considers the main problems that came up during the experiments that were not resolved in the previous research papers. Such problems include: calibration of the camera and sensors, synchronization of the camera and sensors, increasing the accuracy of determining the camera position from sensors data. The authors offer possible solutions to these problems that would help improve quality of the work of existing algorithms, such as a system for parallel synchronized recording of video and sensor data based on the Android operating system. As the main result, the authors represent a framework for implementing video stabilization algorithms based on MEMS sensors readings.

Gyroscope and visual fusion solution for digital video stabilization

Gyroscope and visual fusion solution for digital video stabilization