Image Motion Compensation Research Articles

Technology of Image Motion Compensation for Different Velocity of Aerial Camera Based on Integration Level and Frequency Division Control

When an aircraft is flying sideways, the aerial camera will be in a squint state, and there will be multiple image motions of different sizes on the imaging target at the same time. In this paper, the principle of different velocity image motion in squint imaging of an aircraft is analysed, and the compensation strategy of different velocity image motion is proposed. Then, three methods are used to analyse the adjustment strategy of different velocity image motion compensation, namely, adjusting the integration level of (Time Delayed and Integration) TDICCD, grouping and frequency division control of TDICCD on the image surface, and optimizing the control based on the combination of integration level and frequency division control. Through the simulation analysis, it is concluded that the adjustment of the integration level of TDICCD and the slice grouping control is the optimal method. This method not only ensures the required image SNR of the optical system but also reduces the complex problems of electronic circuit and control system caused by the presence of too many segments and groups. In the actual aerial camera system imaging, this method can be used as an effective method for the compensation for the image motion of different velocity when the aerial camera tilts. Then, the design of the driving circuit and timing generator are described. The validity of the method is verified by indoor simulations and a flight test.

Conceptual Design and Image Motion Compensation Rate Analysis of Two-Axis Fast Steering Mirror for Dynamic Scan and Stare Imaging System.

In order to enable the aerial photoelectric equipment to realize wide-area reconnaissance and target surveillance at the same time, a dual-band dynamic scan and stare imaging system is proposed in this paper. The imaging system performs scanning and pointing through a two-axis gimbal, compensating the image motion caused by the aircraft and gimbal angular velocity and the aircraft liner velocity using two two-axis fast steering mirrors (FSMs). The composition and working principle of the dynamic scan and stare imaging system, the detailed scheme of the two-axis FSM and the image motion compensation (IMC) algorithm are introduced. Both the structure and the mirror of the FSM adopt aluminum alloys, and the flexible support structure is designed based on four cross-axis flexural hinges. The Root-Mean-Square (RMS) error of the mirror reaches 15.8 nm and the total weight of the FSM assembly is 510 g. The IMC rate equations of the two-axis FSM are established based on the coordinate transformation method. The effectiveness of the FSM and IMC algorithm is verified by the dynamic imaging test in the laboratory and flight test.

Multi‐actuator control with modal switching and different disturbance for scanning imaging motion compensation

Scanning imaging can be used to obtain wide field-of-view, high-resolution remote-sensing images with a telescope lens. However, the motion between the image and the sensor causes blurring during scanning, which reduces the image quality. In this paper, a motion compensation method based on multi-actuator control with modal switching is proposed. In this method, the tracking controller realizes scanning and compensation; the cooperative controller reduces the relative motion that causes motion blur, and modal switching solves the travel limitation of the linear actuator. Further, to improve the control performance, an adaptive sliding mode controller is proposed and combined with a disturbance observer. The stability theorems for multi-actuator and modal switching are studied. Experimental results show that the accuracy, robustness, and frame rate of the proposed method improved compared with those of traditional methods. Compared with the non-cooperative structure using a PD controller, the root mean square error is reduced by 76.61 % .

Imaging Attitude Control and Image Motion Compensation Residual Analysis Based on a Three-Axis Inertially Stabilized Platform

The airborne area camera has received broad application in aerial reconnaissance, land resource surveying, environmental monitoring, photogrammetry mapping, and natural disaster information acquisition. A three-axis, inertially stabilized platform with a large rotation range for the roll axis is designed, which is based on the cantilever structure, in order to realize a large-angle sweep imaging function for airborne area cameras. An image attitude control algorithm in the inertial space is proposed, which can regulate the line of sight (LOS) as well as the image orientation. The area camera image motion calculation model and image motion compensation residual computing method are proposed, utilizing space position and velocity vector transformation mathematics and derivations. The variation of linear velocity of the image motion in the sensor frame is analyzed, and the changing laws of the maximum deviation of image motion with the image attitude are studied. Flight tests imply that the vertical imaging technique correctly regulates the LOS along the local geodetic vertical. The along-flight overlap rate is greater than 65%, which meets the stereo mapping requirement. The sweep imaging technique considerably enlarges the cross-flight angle of view. The LOS and image orientation during sweep imaging are correctly controlled, and gap-free coverage of the survey area is maintained. The image’s azimuth or roll deviation is less than 0.1°, and the image pitch deviation is less than 0.35°. The quality of the test images is superior. Black and white line pairs for evaluation can be clearly distinguished. The image’s motion is well compensated, and the image motion compensation residual is well constrained. These verify the validity of the proposed imaging technique and the image motion analysis model.

Image Motion of Remote Sensing Camera With Wide Field of View Over the Antarctic and Arctic

Imaging by the remote sensing camera with a wide field of view (WFV) is of great significance in the issue of polar environment. However, the instability of the image motion velocity poses a huge challenge to the observation task since the direction of the earth's rotation velocity keeps changing in the polar regions. The edge blur of polar image by the camera with the WFV is usually ignored. Therefore, a specific theoretical model that the camera images over the polar region is supposed to describe the instability distinctly. Mathematical expressions of the image motion velocity field are obtained with a novel method of velocity projection and coordinate transformation. The quantitative analysis of simulation reveals that the increasing anisotropy of the instantaneous image motion velocity field gets most significant near the poles. The value of modulation transfer function at the edge of the field of view decreased by 0.33%, which results in a discrepancy of sharpness in an image. The model is capable of explaining the edge blur, of the image from BNU-1, which provides a theoretical basis for the image motion compensation strategy of cameras with the WFV.

Compensation method for the image motion of a rotary-scan space camera

Space-borne observation using optical detection requires wide-area and high-resolution imaging. We propose and demonstrate a novel rotary-scan space camera for remote sensing. The camera has a 2-m resolution and it is designed with an extra-wide swath of 1000 km and a wide image motion range. The motion range is achieved with the inclination and rotation of the optical axis. To avoid image blurring, an opto-mechanical method using a 2D fast steering mirror and a timely rotating detector is proposed to compensate for the rotary-scan image motion, which is over 200 pixels and spatial-variant. Finally, a physical simulation was conducted where the image motion was reduced from 20 pixels to less than 1 pixel with more than 95% compensation, validating the efficiency of the image motion compensation scheme and the feasibility of the rotary-scan imaging mode.

Line of Sight and Image Motion Compensation for Step and Stare Imaging System

In recent years, applications such as marine search and rescue, border patrol, etc. require electro-optical equipment to have both high resolution and precise geographic positioning abilities. The step and stare working based on a composite control system is a preferred solution. This paper proposed a step and stare system composed of two single-axis fast steering mirrors and a two-axis gimbal. The fast steering mirrors (FSMs) realize image motion compensation and the gimbal completes pointing control. The working principle and the working mode of the system are described first. According to the imaging optical path, the algorithm and control flow of the line of sight (LOS) and image motion compensation are developed. The proposed method is verified through ground imaging and flight tests. Under the condition of flight, the pointing accuracy of the target can be controlled within 15 m. The proposed algorithm can achieve effective motion compensation and get high-resolution images. This achieves high resolution and accurate LOS simultaneously.

Advanced Pointing Imaging Camera (APIC) for planetary science and mission opportunities

The Advanced Pointing Imaging Camera (APIC) is designed to obtain high-resolution imaging data to measure a target’s geophysical and geodetic properties. The development of APIC originates from NASA’s Homesteader program of technology development for candidate New Frontiers missions. The unique science enabled by APIC derives from its ability to simultaneously take images of the target and star field, allowing high-precision camera pointing knowledge with each high-resolution target image. APIC is small (28 ​cm ​× ​18 ​cm ​× ​24 ​cm encompassing volume), light-weight (6 ​kg total), and moderate in power (13 ​W maximum) while being high performance and robust to long missions in deep space. APIC incorporates two imagers, one narrow-angle camera (NAC) and one wide-angle camera (WAC) that can operate simultaneously. Both cameras utilize the CMOS-based Mars 2020 Engineering Camera technology with an option of either clear or Red-Green-Blue colors and have wide apertures to enable short exposures and thus perform at a wide range of targets. The NAC has a pixel resolution of 18 ​μrad and 4° field of view and the WAC has a pixel resolution of 82 ​μrad and 18° field of view. APIC also has two gimbals, allowing rapid camera pointing updates without the need to change the spacecraft attitude; thus, not interfering with other onboard sensors or spacecraft operations. Both gimbals are capable of compensating for relative spacecraft-target motion (i.e., image motion compensation) with an angular speed of up to 30°/s (i.e., 0.5 ​rad/s). Many of APIC components are commercial-off-the-shelf (COTS), or adapted from other NASA flight programs, which makes APIC very competitive in cost and gives it a high technical maturity. APIC’s high-resolution images enable the determination of high-accuracy topography for geologic studies. This paper presents details of APIC’s characteristics and functionalities as well as specific science objectives that APIC data can address, such as measuring a geometric tidal flexing through estimating the tidal Love number, h2 and l2, and small rotational effects, such as libration and precession, of natural satellites and small bodies (i.e., asteroids and comets) that are key to exploring a planetary body’s interior. Improved knowledge of spacecraft orbit via landmark tracking using the APIC data would also improve the recovery of low-degree gravitational parameters such as k2. In this paper, the performance of APIC is presented by showing how well the tidal deformation and libration measurements can be recovered with realistic mission scenarios and configurations.

An enhanced adaptive butterfly optimization algorithm rigorously verified on engineering problems and implemented to ISAR image motion compensation

PurposeThis study aims to evolve an enhanced butterfly optimization algorithm (BOA) with respect to convergence and accuracy performance for numerous benchmark functions, rigorous constrained engineering design problems and an inverse synthetic aperture radar (ISAR) image motion compensation.Design/methodology/approachAdaptive BOA (ABOA) is thus developed by incorporating spatial dispersal strategy to the global search and inserting the fittest solution to the local search, and hence its exploration and exploitation abilities are improved.FindingsThe accuracy and convergence performance of ABOA are well verified via exhaustive comparisons with BOA and its existing variants such as improved BOA (IBOA), modified BOA (MBOA) and BOA with Levy flight (BOAL) in terms of various precise metrics through 15 classical and 12 conference on evolutionary computation (CEC)-2017 benchmark functions. ABOA has outstanding accuracy and stability performance better than BOA, IBOA, MBOA and BOAL for most of the benchmarks. The design optimization performance of ABOA is also evaluated for three constrained engineering problems such as welded beam design, spring design and gear train design and the results are compared with those of BOA, MBOA and BOA with chaos. ABOA, therefore, optimizes engineering designs with the most optimal variables. Furthermore, a validation is performed through translational motion compensation (TMC) of the ISAR image for an aircraft, which includes blurriness. In TMC, the motion parameters such as velocity and acceleration of target are optimally predicted by the optimization algorithms. The TMC results are elaborately compared with BOA, IBOA, MBOA and BOAL between each other in view of images, motion parameter and numerical image measuring metrics.Originality/valueThe outperforming results reflect the optimization and design successes of ABOA which is enhanced by establishing better global and local search abilities over BOA and its existing variants.

Image Motion Compensation Control Technology of Unmanned Aerial Vehicle (UAV) Airborne Photoelectric Reconnaissance Platform

Airborne photoelectric platform has a wide application prospect in the field of imaging, especially in the field of UAV reconnaissance. However, in the process of image acquisition, the photoelectric platform equipment will produce obvious image shift because of its own attitude change or speed instability, so it is necessary to compensate the collected image. In this study, when the image motion compensation is carried out, the internal and external parameters of the camera are adjusted, and the geometric model of the image is established. The Camera Calibration Toolbox provided by MatLab is used to calibrate the internal parameters of camera. At the same time, when obtaining the camera's internal parameters, the corresponding flight attitude angle is obtained with the aid of the inertial navigation system. At the same time of acquiring the attitude of the aircraft by hardware, it is necessary to obtain the image shift coefficient by using the function. The coordinate system needs to be transformed into a coordinate system suitable for the inertial navigation system, so as to facilitate the calculation of external parameters of the camera. The transformation matrix from image moving image to normal image can be established by obtaining the parameters inside and outside the camera. Through bilinear interpolation, the matrix is derived in MatLab environment. In the experiment, 16 black-and-white grid images are selected to obtain the attitude angle, so as to complete the calculation process of the conversion matrix. Then, the images with image shift are screened out from the images taken in aerial state. The method proposed in this study is used for image compensation. According to the root mean square error analysis, the image compensation scheme proposed in this study is helpful to the imaging application of UAV airborne platform.

Magnetic resonance imaging for pathobiological assessment and interventional treatment of the coronary arteries.

X-ray-based fluoroscopy is the standard tool for diagnostics and intervention in coronary artery disease. In recent years, computed tomography has emerged as a non-invasive alternative to coronary angiography offering detection of coronary calcification and imaging of the vessel lumen by the use of iodinated contrast agents. Even though currently available invasive or non-invasive techniques can show the degree of vessel stenosis, they are unable to provide information about biofunctional plaque properties, e.g. plaque inflammation. Furthermore, the use of radiation and the necessity of iodinated contrast agents remain unfavourable prerequisites. Magnetic resonance imaging (MRI) is a radiation-free alternative to X-ray which offers anatomical and functional imaging contrasts fostering the idea of non-invasive biofunctional assessment of the coronary vessel wall. In combination with molecular contrast agents that target-specific epitopes of the vessel wall, MRI might reveal unique plaque properties rendering it, for example, ‘vulnerable and prone to rupture’. Early detection of these lesions may allow for early or prophylactic treatment even before an adverse coronary event occurs. Besides diagnostic imaging, advances in real-time image acquisition and motion compensation now provide grounds for MRI-guided coronary interventions. In this article, we summarize our research on MRI-based molecular imaging in cardiovascular disease and feature our advances towards real-time MRI-based coronary interventions in a porcine model.

Driving circuitry of a full-frame area array charge-coupled device (CCD) supporting multiple output modes and electronic image motion compensation

In order to design a charge-coupled device (CCD) driving circuit with a high frame rate and a large area array which can support the electronic image motion compensation function, first a basic driving circuit of the large-array CCD is reported in this paper. Next, electronic image motion compensation is realized by adding an image motion compensation timing generator and a main timing generator, Saa8103. The internal design structure of the image motion compensation generator is provided and is characterized as follows. The data are first sent by the system controller, the main timing generator and the image motion compensation timing generator. The generator, the vertical driver, the horizontal driver and the full-frame charge-coupled device chip are then composed. During the exposure period, the image motion compensation transfer clock is generated by the image motion compensation timing generator, and the charge-coupled device charge packet is driven to track the image movement after being amplified by the vertical and horizontal drivers to carry out electronic image motion compensation in the vertical direction. During the output period, the vertical and horizontal rotations generated by the main timing generator are carried out by the image motion compensation timing generator. The shift clock is transmitted to the vertical and horizontal drivers to drive the charge-coupled device to produce the output. The phase relationship of the four corners of the charge-coupled device is independently controlled by the image motion compensation timing generator, which supports not only the electronic image motion compensation but also the flexible selection of the output channel number and output mode. A field-programable gate array (FPGA)/complex programable logic device (CPLD) has been selected to be the timing generator of image motion compensation, and timing simulation is carried out. Lastly, the design of the driving circuit was verified by an indoor image motion compensation experiment, and a good compensation effect has been achieved. In this system, the charge-coupled device adopts a separate driving technology, and the charge transfer direction of each block can be established. Consequently, it is convenient to select the number of output channels and the output mode, which makes the camera suitable for different occasions, supports the charge-coupled device in terms of the output and electronic image motion compensation in various ways, and is capable of generating full-frame images. In order to meet the different requirements regarding the charge-coupled device frame frequency in different contexts, this proposed design overcomes the disadvantages of the complex structure and low reliability of traditional optical mechanical image motion compensation systems.

Efficient onboard image motion compensation for orbital inclination and eccentricity of geostationary weather satellites

Image registration is important in geostationary weather satellites. Achieving consistent registration of the images with respect to the geographical locations on the Earth is here of interest. The consistency in the registration between the images is affected whenever the orbital inclination and eccentricity are not zero. The imaging payload has a two-axis scanning mirror to capture the Earth image. The above orbital effects together with scan mirror pointing direction are the factors that cause the misregistration. This paper presents an onboard algorithm that provides the scan compensation angles due to the above factors and achieves consistent registration. The compensation varies every second, which is the time taken for each scan. Hence it is preferred to have computations onboard than to have ground based bulk uplinks for the scan compensation. The paper presents an algorithm that is useful, say, when (i) the onboard computing capabilities are limited, (ii) the navigation accuracies are coarse and (iii) the image resampling is not preferred on the ground and the payload data are directly used for weather applications. The paper also discusses the tests that were carried on the onboard software in order to validate its performance in achieving the consistent registration before launch. This is done by using another independent software tool which is also described in detail. Image motion algorithm was invoked for a couple of days in INSAT 3DR. The atmospheric wind vector deduced directly from the satellite images is given at the end.

快速反射镜在像移补偿中的应用

快速反射镜在像移补偿中的应用

Compensation of star image motion for a CMOS image sensor with a rolling shutter

The predecessor of China Optics was China Optics and Applied Optics Digest, which was founded in 1985. At that time, it was the only retrieval journal in the field of optics in China. At the end of 2008, China Optics and Applied Optics Digest was renamed

Development of image motion compensation system for 1.3m telescope at Vainu Bappu Observatory

We developed a tip-tilt system to compensate the turbulence induced image motion for the 1.3 m telescope at Vainu Bappu Observatory, at Kavalur. The instrument is designed to operate at the visible wavelength band (480 − 700 nm) with a field of view of 1′ × 1′. The tilt corrected images have demonstrated up to ≈ 57% improvement in image resolution and a corresponding peak intensity increase by a factor of ≈ 2.8. A closed-loop correction bandwidth of ≈ 26 Hz has been achieved with on-sky tests and the root mean square motion of the star image has been reduced by a factor of ∼ 14. These results are consistent with theoretical and numerical predictions of wave-front aberrations caused by atmospheric turbulence and image quality improvement expected from a real-time control system. In this paper, we present details of the instrument design, laboratory calibration studies and quantify its performance on the telescope.

Modeling and design of fast steering mirror in image motion compensation for backscanning step and stare imaging systems

To compensate for disturbances in an aircraft’s forward flight and variations in its attitude and angular velocity in the imaging process of photoelectric imaging system, a method of image motion compensation using a fast steering mirror was put forward. First, the working sequence of the image motion compensation system was designed. The position of the fast steering mirror is established by increasing the oblique wave to allow the fast steering mirror to move at the desired speed and compensate for the image motion speed. Then, a mathematical model of the fast steering mirror was established. A linear extended-state observer is designed to estimate the disturbance in real-time, and disturbance compensation is generated to offset the disturbance. Finally, a step response experiment and a tracking experiment were performed to test the characteristics. The step response curve shows that the system’s stable time is 1.8 ms. The tracking characteristic is determined by a given scan curve. The fast steering mirror can satisfy the requirement for a high-speed and highly precise tracking system. It improves the system’s interference rejection performance and reduces the difficulty of controller design.

Long Integral Time Continuous Panorama Scanning Imaging Based on Bilateral Control with Image Motion Compensation

Urban remote sensing with moving carriers enables comprehensive monitoring of an urban area. High spatial resolution and wide covering are always required to improve the performance and efficiency of remote sensing. Continuous scanning imaging is a feasible solution. However, imaging motion degrades the performance of a remote sensing system. Rotating motion leads to the loss of key urban morphology information of a panorama imaging. Image translation results in blurry images. For high spatial resolution and high efficiency imaging with low illumination condition, such as imaging at dusk, long-focus lens and long integral time must be further utilized, which makes the problem more severe. In this paper, a novel image motion compensation method is proposed to compensate for image rotation and image translation simultaneously. A quantitative description of image motion, including both image rotation and image translation, is first developed based on the principle of geometrical optics and then analyzed in detail through numerical simulations. Furthermore, a comprehensive image rotation compensation method is developed based on four-channel bilateral control with sliding mode controller, at the same time image translation compensation is performed according to the quantitative relationship of the motion of the scan mirror and image translation compensator. The experimental results show that the proposed method provides effective compensation for image rotation and image translation. This enables acquisition of high spatial resolution urban panoramic images.

Integrating Motion Compensation With Polar Format Interpolation for Enhanced Highly Squinted Airborne SAR Imagery

Spatial-variant motion compensation (MOCO) is critical for high resolution and highly squinted airborne synthetic aperture radar (SAR) imaging. Conventional imaging strategies generally perform systematic imaging algorithm and motion compensation algorithm in a separate way, the residual azimuth-variant motion error usually causes defocusing in the image. It is difficult for existing post-filtering strategies to realize high precision and high efficiency imaging simultaneously. To solve this problem, a novel parametric polar format algorithm (PPFA) is proposed in this paper. The polar format interpolation kernel is redefined and improved by inducing motion error parameter, so the proposed algorithm can realize fast imaging and precise spatial-variant motion compensation at the same time. The proposed algorithm has advantage of high processing efficiency as polar format algorithm (PFA) and effectively improves the focusing precision. Extensive comparisons with conventional algorithms illustrate the superiority of the proposed algorithm in both precision and efficiency.

Fast Compressive Sensing-Based SAR Imaging Integrated With Motion Compensation

Recently, compressive sensing (CS) has been applied in synthetic aperture radar (SAR) imaging, which is increasingly in the focus of study and shows great potential. In CS-based SAR imaging, motion errors of the moving platform introduce inaccuracies in the observation model, which cause various degradations in the final images. To accomplish accurate motion compensation during CS-based SAR imaging, we propose a fast CS-based SAR imaging integrated with motion compensation method. First, CS-based imaging based on the utilization of inverse observation deduced from the inversion of conventional imaging procedures is applied, which is much more computational efficient than the exact observation model. Then, an improved inverse observation model integrated with motion compensation is derived. In the improved model, spatially variant azimuth phase errors are taken into consideration. Joint SAR imaging and motion compensation are formulated as a sparse recovery problem and solved in an iterative way, wherein each iteration both image formation and motion compensation are carried out. The processing of SAR data shows that the proposed method can obtain better focused images compared with the existing SAR imaging and motion compensation methods.