Image Stabilization Technique Research Articles

All‐Optically Controlled Retinomorphic Memristor for Image Processing and Stabilization

AbstractImage stabilization is a crucial field in machine vision, aiming to eliminate image blurring or distortion caused by the camera or object jitter. However, traditional image stabilization techniques often suffer from the drawbacks of requiring complex equipment or extensive computing resources, resulting in inefficiencies. In contrast, the human retina performs a highly efficient all‐in‐one system, encompassing the detection and processing of light stimuli. In this study, an all‐optically controlled retinomorphic memristor based on the CsxFAyMA1‐x‐yPb(IzBr1‐z)3 is proposed, which integrates perception, storage, and processing functions. This memristor exhibits significant advantages in image stabilization. It is capable of positively and negatively modulating its conductance using specific intensities (11.8 and 0.9 mW cm−2, respectively) of red light (630 nm). To demonstrate the effectiveness of the proposed approach, handwritten digit recognition simulations are conducted. The application of specific light stimuli effectively highlights the characteristics of blurred images. The processed images are then fed into a conductance‐mapped neural network for rapid recognition. Remarkably, the recognition rates of the processed images reach 83.5% after 19 000 iterations, surpassing the performance of blurred images (only 56.2% after 19 000 iterations). These results highlight the immense potential of retinomorphic memristors as the hardware foundation for next‐generation image stabilization systems.

Design of Sanda Action Reconstruction Model Based on 3D Images

In order to improve the training level of Sanda movement, this article uses an image analysis method to reconstruct the detailed characteristics of the movement and apply them to the actual training process. Since the traditional wavelet reconstruction method is affected by the accuracy of the decomposition scale, this paper proposes an improved method of Sanda action based on 3D image reconstruction. First, the method relies on frame adjacent phase compensation and digital image stabilization techniques to perform digital frame operations on the image. Then, scanning and corner detection are used for image reconstruction, where adjacent phase compensation methods are used to match feature points and gray pixels. Image extraction is performed by extracting key feature points of the action 3D image, and a fast frame detection method is used to stabilize the image of the digital image, thereby improving the image quality of image reconstruction. The experimental results show that the method has good image output effect and has a high application value in Sanda guidance and optimization.

Modal Investigation of Vortex Ring Shedding from an Oscillating Disc

Vortex shedding from an oscillating disc with a chamfered tip is investigated at a Reynolds number of 8715 and a Keulegan–Carpenter number of 0.6. The experiments are performed using a high-speed Particle Image Velocimetry (PIV) system with 160 Hz frame acquisition frequency. Six full cycles are recorded with a total of 1999 recordings, i.e. 333 velocity measurements resolving a single cycle. The 175-mm-diameter disc oscillation frequency was 0.48 Hz with a 20 mm total displacement amplitude in water. Dynamic image masking is performed to remove the disc shape from the PIV raw images, using rigid object tracking and image stabilization techniques. This implies a coordinate transformation and allows the investigation of flow field results with respect to the disc, as if it was stationary. This allows the use of statistical analyses and phase-locked averaging. The results indicate that the vortex formation and shedding from-cycle-to-cycle is very stable and predictable. The chamfered disc tip geometry has great influence in the vortex dynamics: When the motion is towards the chamfered side, a big, attached trailing vortex is present on the flat side; and when the motion is reversed towards the flat side the big trailing vortex is shed off outwards from the center, forming a shear layer with a 45° orientation. These findings have been supported by modal analyses using Proper Orthogonal Decomposition (POD) and Oscillating Pattern Decomposition (OPD). The static and dynamic modes of POD and OPD successfully describe the underlying flow physics of vortex ring shedding from an oscillating disc and reveal intricate details about the flow field which cannot be captured by statistical analysis or phase locked averaging.

RAVEN Eyes Around the Instrument from Modular Axis Sharing

One of the current limitations in MIS robotic surgery is the limited field of view due to the use of a sole rigid 3D–imaging camera. This study proposes a modular 2D flexible imaging probe that can be integrated to the teleoperated surgical robot, RAVEN, in order to provide visual access in real time to occluded surgical targets. The probe proposed in this work guides a state–of–the–art scanning fiber endoscope SFE with minimal actuation. It is based on a modular axis–shared conception that avoids interruptions in the surgical flow to connect or exchange the device. The viewpoint from the probe is adjusted by a surgeon from the robot master console with independent actuation, sensing and control from the robot system. A 2–Hz oscillating mock–up model of the abdominal cavity containing several points of interest occluded for the central camera is used to test the capabilities of the probe. Results show how the totality of these interest points are visualized using the flexible probe to improve the information at the surgeon site. Motion compensation with image stabilization techniques are incorporated to improve unfocused and blurred images as well as to maintain the field of view for the target during the manipulation. The experimental results in this case were compared to those of one of the conventional software solutions, and provided 45.9% less maximum error than the conventional software during the simple surgical procedure.

Advanced digital image stabilization using similarity-constrained optimization

As many people have portable video devices such as cameras on cell phones and camcorders, image stabilization technique is a crucial and challenging task in computer vision applications, and many image stabilization techniques have been researched over many years. We propose a digital image stabilization method that only uses a software algorithm without additional hardware devices. Furthermore, a novel digital image stabilization method composed of three steps that use similarity-constrained nonlinear optimizer is introduced and applied to many unstabilized videos. First, a feature detection technique called moment-based speeded-up robust features (MSURF) is utilized to obtain the transformation matrix. Second, the k-means clustering algorithm is used to detect and remove some of the outliers that cause residual errors during feature matching. Third, the transformation matrix is optimized using nonlinear optimization algorithms to maintain the similarity of the transformation matrix. The experimental results prove that the proposed algorithm provides accurate image stabilization performance.

Adaptive bulk motion exclusion for improved robustness of abdominal magnetic resonance imaging.

Non‐Cartesian magnetic resonance imaging (MRI) sequences have shown great promise for abdominal examination during free breathing, but break down in the presence of bulk patient motion (i.e. voluntary or involuntary patient movement resulting in translation, rotation or elastic deformations of the body). This work describes a data‐consistency‐driven image stabilization technique that detects and excludes bulk movements during data acquisition. Bulk motion is identified from changes in the signal intensity distribution across different elements of a multi‐channel receive coil array. A short free induction decay signal is acquired after excitation and used as a measure to determine alterations in the load distribution. The technique has been implemented on a clinical MR scanner and evaluated in the abdomen. Six volunteers were scanned and two radiologists scored the reconstructions. To show the applicability to other body areas, additional neck and knee images were acquired. Data corrupted by bulk motion were successfully detected and excluded from image reconstruction. An overall increase in image sharpness and reduction of streaking and shine‐through artifacts were seen in the volunteer study, as well as in the neck and knee scans. The proposed technique enables automatic real‐time detection and exclusion of bulk motion during MR examinations without user interaction. It may help to improve the reliability of pediatric MRI examinations without the use of sedation.

A Comprehensive Motion Estimation Technique for the Improvement of EIS Methods Based on the SURF Algorithm and Kalman Filter.

Video stabilization is an important technology for removing undesired motion in videos. This paper presents a comprehensive motion estimation method for electronic image stabilization techniques, integrating the speeded up robust features (SURF) algorithm, modified random sample consensus (RANSAC), and the Kalman filter, and also taking camera scaling and conventional camera translation and rotation into full consideration. Using SURF in sub-pixel space, feature points were located and then matched. The false matched points were removed by modified RANSAC. Global motion was estimated by using the feature points and modified cascading parameters, which reduced the accumulated errors in a series of frames and improved the peak signal to noise ratio (PSNR) by 8.2 dB. A specific Kalman filter model was established by considering the movement and scaling of scenes. Finally, video stabilization was achieved with filtered motion parameters using the modified adjacent frame compensation. The experimental results proved that the target images were stabilized even when the vibrating amplitudes of the video become increasingly large.

Real-time digital image stabilization based on regional field image gray projection

Digital image stabilization technique plays important roles in video surveillance and object acquisition. Many useful electronic image stabilization algorithms have been studied. A real-time algorithm is proposed based on field image gray projection which enables the regional odd and even field image to be projected into x and y directions and thus to get the regional gray projection curves in x and y directions, respectively. For the odd field image channel, motion parameters can be estimated via iterative minimum absolute difference based on two successive field image regional gray projection curves. Then motion compensations can be obtained after using the Kalman filter method. Finally, the odd field image is adjusted according to the compensations. In the mean time, motion compensation is applied to the even field image channel with the odd field image gray projection curves of the current frame. By minimizing absolute difference between odd and even field image gray projection curves of the current frame, the inter-field motion parameters can be estimated. Therefore, the even field image can be adjusted by combining the inter-field motion parameters and the odd field compensations. Finally, the stabilized image sequence can be obtained by synthesizing the adjusted odd and even field images. Experimental results show that the proposed algorithm can run in real-time and have a good stabilization performance. In addition, image blurring can be improved.

Strategy for quantum image stabilization

Image stabilization is a process to smooth the unstable motion vector of video sequences to achieve its stabilization. Even though the classical image stabilization techniques seem already very mature so far, similar advances have not been extended to the quantum computing domain. In this study, we explore a novel quantum video framework and make a modest attempt to perform the image stabilization based on it by utilizing the quantum comparator and quantum image translation operations. The proposed method is capable of estimating the camera motion during exposure and compensating for the video jitter caused by the motion. In addition, the quantum properties, i.e., entanglement and parallelism, ensure that the quantum image stabilization is feasible and effective. Finally, a simple experiment to stabilize a four-frame jittered quantum video is implemented using Matlab based on linear algebra with complex vectors as quantum states and unitary matrices as unitary transforms to show the feasibility and merits of this proposal.

January 2015: This Month in JoVE - Introducing JoVE Developmental Biology

Here's a look at what's coming up in the January 2015 issue of JoVE: The Journal of Visualized Experiments. JoVE has been revolutionizing scientific publishing since 2006, when we released our first video articles in JoVE Biology. We've grown over the years-adding sections in Neuroscience, Immunology & Infection, Clinical & Translational Medicine, Bioengineering, Applied Physics, Behavior, Chemistry, and Environment. We are now pleased to introduce a new addition to the JoVE family: JoVE Development, which covers the entire field of developmental biology from the underlying genetic and epigenetic mechanisms to the growth and differentiation of single cells into organs and whole organisms. In many species, developmental processes offer clues about evolution. For example, the European lancelet (Branchiostoma lanceolatum) has many features of modern fish, but it doesn't have a backbone-so it's emerging as a useful model for studying the divergence of vertebrates from invertebrate ancestors. Hirsinger et al. have developed a method for visualizing embryonic development in the European lancelet. They inject oocytes with mRNAs that encode fluorescent proteins, and following fertilization, developmental processes can be visualized in vivo. Development also explores interactions between different cell types, and these interactions are fundamental for repairing and regenerating damaged or diseased tissues, like muscle. To study these interactions, Agley et al. take human skeletal muscle biopsies then purify and culture different cell types. They also characterize the cells using immunocytochemical methods that can be adapted to other cell types. One of the most exciting topics in developmental biology is the engineering of stem cells for experimental and clinical applications. This month we feature two stem cell protocols: Zielins et al. describe the isolation and purification of human adipose-derived stromal cells for bone engineering, and Lei et al. demonstrate how to differentiate embryonic stem cells into embryoid bodies, and then derive cardiac progenitor cells that further differentiate into cardiomyocytes and smooth muscle cells. Above all, developmental processes are fascinating to visualize, and JoVE Development features techniques for optimizing real-time imaging, such as this protocol for live-imaging of the Drosophila pupal eye. Using image-stabilization techniques, Hellerman et al. compensate for tissue movement and uneven topology to enhance the visualization of the developing Drosophila eye. You've just had a sneak peek of JoVE's new Development section in the January 2015 issue. Visit the website to see the full-length articles plus our other scientific sections in JoVE: The Journal of Visualized Experiments.

디지털 카메라용 이미지 안정화 시스템 제어

As high quality image is required for digital cameras in recent use, the image stabilization technique has drawn much attention to prevent image degradation from shaky disturbance by users. In this paper, the optical image stabilization (OIS) system for DSLR (Digital Single Lens Reflex) camera is considered. First, the analytic model of an OIS system is presented to demonstrate the mechanism of image destabilization due to unknown disturbance that causes blurry images on CCD sensor. Then, to enhance the stabilization performance, a sliding mode control based on the min-max nonlinear control is introduced. Through the experiments and simulations, the effectiveness of the proposed method will be verified.

Dual stage and an image processing-based method for sight stabilization

This paper presents a combined dual stage-based mechanical and image-based stabilization scheme for a three-axis image-tracking sight system. To improve the stabilization and tracking accuracy, a secondary stage actuated by a pair of electro-magnets is mounted on a conventional elevation gimbal. For the remaining roll axis stabilization, an electronic digital- image stabilization technique is introduced to estimate and correct roll motions. Experimental results are given to demonstrate the effectiveness of the proposed stabilization system and the image-stabilization scheme.

A Recursive Fuzzy System for Efficient Digital Image Stabilization

A novel digital image stabilization technique is proposed in this paper. It is based on a fuzzy Kalman compensation of the global motion vector (GMV), which is estimated in the log-polar plane. The GMV is extracted using four local motion vectors (LMVs) computed on respective subimages in the logpolar plane. The fuzzy Kalman system consists of a fuzzy system with the Kalman filter's discrete time-invariant definition. Due to this inherited recursiveness, the output results into smoothed image sequences. The proposed stabilization system aims to compensate any oscillations of the frame absolute positions, based on the motion estimation in the log-polar domain, filtered by the fuzzy Kalman system, and thus the advantages of both the fuzzy Kalman system and the log-polar transformation are exploited. The described technique produces optimal results in terms of the output quality and the level of compensation.

SUNRISE: High resolution UV/VIS observations of the sun from the stratosphere

SUNRISE is an international project for the development, construction and operation of a balloon-borne solar telescope with an aperture of 1 m, working in the UV/VIS spectral domain. The main scientific goal of SUNRISE is to understand the structure and dynamics of the magnetic field in the atmosphere of the Sun. SUNRISE will provide near diffraction-limited images of the photosphere and chromosphere with an unprecedented resolution down to 35 km on the solar surface at wavelengths around 220 nm. Active in-flight alignment and image stabilization techniques are used. The focal-plane instrumentation consists of a polarization sensitive spectrograph, a Fabry–Perot filter magnetograph and a phase-diverse filter imager working in the near UV. The first stratospheric long-duration balloon flight of SUNRISE is planned in summer 2009 from the Swedish ESRANGE station. SUNRISE is a joint project of the German Max-Planck-Institut für Sonnensystemforschung (MPS), Katlenburg-Lindau, with the Kiepenheuer-Institut für Sonnenphysik (KIS), Freiburg, Germany, the High-Altitude Observatory (HAO), Boulder, USA, the Lockheed-Martin Solar and Astrophysics Laboratory (LMSAL), Palo Alto, USA, and the Spanish IMaX consortium. This paper will give an overview about the mission and a description of its scientific and technological aspects.

CMOS Digital Image Stabilization

In this paper, we present a CMOS digital image stabilization algorithm based on the characteristics of a rolling shutter camera. Due to the rolling shuttering mechanism of a CMOS sensor, a CMOS video frame shows CMOS distortions which are not observed in a CCD video frame, and previous video stabilization techniques cannot handle these distortions properly even though they can make a visually stable CMOS video sequence. In our proposed algorithm, we first suggest a CMOS distortion model. This model is based on a rolling shutter mechanism which provides a solution to solve the CMOS distortion problem. Next, we estimate the global image motion and the CMOS distortion transformation directly from the homography between CMOS frames. Using the two transformations, we remove CMOS distortions as well as jittering motions in a CMOS video. In the experimental results, we demonstrate that our proposed algorithm can handle the CMOS distortion problem more effectively as well as the jittering problem in a CMOS video compared to previous CCD-based digital image stabilization techniques.

A Robust In-Car Digital Image Stabilization Technique

Machine vision is a key technology used in an intelligent transportation system (ITS) to augment human drivers' visual capabilities. For the in-car applications, additional motion components are usually induced by disturbances such as the bumpy ride of the vehicle or the steering effect, and they will affect the image interpretation processes that is required by the motion field (motion vector) detection in the image. In this paper, a novel robust in-car digital image stabilization (DIS) technique is proposed to stably remove the unwanted shaking phenomena in the image sequences captured by in-car video cameras without the influence caused by moving object (front vehicles) in the image or intentional motion of the car, etc. In the motion estimation process, the representative point matching (RPM) module combined with the inverse triangle method is used to determine and extract reliable motion vectors in plain images that lack features or contain a large low-contrast area to increase the robustness in different imaging conditions, since most of the images captured by in-car video cameras include large low-contrast sky areas. An adaptive background evaluation model is developed to deal with irregular images that contain large moving objects (front vehicles) or a low-contrast area above the skyline. In the motion compensation processing, a compensating motion vector (CMV) estimation method with an inner feedback-loop integrator is proposed to stably remove the unwanted shaking phenomena in the images without losing the effective area of the images with a constant motion condition. The proposed DIS technique was applied to the on-road captured video sequences with various irregular conditions for performance demonstrations

Digital Image Stabilization Based on Circular Block Matching

In this paper, a novel digital image stabilization algorithm based on circular block matching is proposed to estimate the global scaling, rotational and translational motion parameters between each two consecutive frames in jitter videos. In the algorithm, the rotation-invariant-features based circular block matching is proposed for the first time to estimate the local true motion vectors for the centers of all considered circular blocks. With these local motion vectors and their corresponding block positions, a linear system is constructed and solved by repeated least-squares to generate the global motion parameters (GMP) which are then refined by a gradient descent based iteration scheme. The experimental results demonstrate that the proposed DIS technique can generate more precise GMP and deal with larger motions compared with the ever presented optical flow based digital image stabilization technique.

A robust digital image stabilization technique based on inverse triangle method and background detection

In this paper, a novel robust digital image stabilization (DlS) technique is proposed to remove the unwanted shaking phenomena in the image sequences captured by hand-held camcorders without affecting the moving objects in the image sequence and intentional motion of panning condition, etc. It consists of a motion estimation unit and a motion compensation unit. To increase the robustness in the adverse image conditions, an inverse triangle method is proposed to extract reliable motion vectors in plain images which are lack of features or contain large low-contrast area, etc., and a background evaluation model is developed to deal with irregular images which contain large moving objects, etc. In the motion compensation unit, a CMV estimation method with an inner feedback-loop integrator is proposed to remove the unwanted shaking phenomena without losing the effective area of the image in panning condition. We also propose a smoothness index (Sl) to quantitatively evaluate the performances of different image stabilization methods. The experimental results are on-line available to demonstrate the remarkable performance of the proposed DIS technique.

The role of eye movements and masking in monocular rivalry

Using an image stabilization technique, we have examined the role of eye movements on the phenomenon of monocular rivalry. Cross-correlation analysis confirms that perceptual onset of the vertical grating coincides with perceptual disappearance of the horizontal (and vice versa). Vertical and horizontal retinal image motion can modulate, selectively, the visibility of the horizontal and vertical gratings. However, a stable percept was observed when constant retinal image motion was matched for both vertical and horizontal components. These results support an eye-movement and afterimage explanation for the phenomenon of monocular rivalry (Georgeson, 1984). However, in addition to enhancing the visibility of a horizontal grating, a vertical “saccadic-like” retinal image motion decreased the visibility of the vertical component. This result is consistent with rivalry. That is, the visibility of both gratings are not independent, but are inversely related to each other. Our results are inconsistent with a hypothesis for monocular rivalry based upon inherently unstable neural interactions, but they also show that unstable fixational eye movements and afterimages are not the only factors determining the perceptual fluctuations.