Motion Blur Effects Research Articles

Blind 3D Video Stabilization with Spatio-Temporally Varying Motion Blur

Video stabilization is a challenging task that attempts to compensate for the overall frame shake during video acquisition. Existing three-dimensional video stabilization methods aim at modeling camera perspective projection through either data-driven training or explicit motion estimation. However, the above methods are difficult to effectively solve the issue of shaky videos with abrupt object movements, resulting in local motion blur in the direction of the movement. This phenomenon is prevalent in real-world scenarios featuring foreground blind motion scenes. Unfortunately, directly combining stabilization and deblurring methods poses challenges when dealing with this situation. In the video, the intensity of motion blur undergoes continuous changes, and the direct combination method inadequately utilizes spatiotemporal information, providing insufficient clues for cross-frame compensation. To alleviate this problem, the Cross-frame-temporal Module framework is proposed to address blind motion blur induced by various conditions, which utilizes cross-frame temporal features to estimate depth maps and camera motion. In this framework, a Blur Transform Network (BTNet) is designed to adapt to spatially varying motion blur, which transforms local regions according to the impact of blur intensities to adapt to the effects of non-uniform motion blur; furthermore, our Temporal-Aware Network (TANet) further suppresses motion blur by leveraging cross-frame temporal features. In addition, the limited availability of pair-training video data containing motion blur limits the application of this approach in practice. The Cross-frame-temporal Module framework adopts an un-pretrained in-test training strategy. Extensive experimental results have demonstrated that our method outperforms state-of-the-art methods.

SeaTrack: Rethinking Observation-Centric SORT for robust Nearshore Multiple Object Tracking

Nearshore Multiple Object Tracking (NMOT) aims to locate and associate nearshore objects. Current approaches utilize Automatic Identification Systems (AIS) and radar to accomplish this task. However, video signals can describe the visual appearance of nearshore objects without prior information such as identity, location, or motion. In addition, sea clutter will not affect the capture of living objects by visual sensors. Recognizing this, we analyzed three key long-term challenges of the vision-based NMOT and proposed a tracking pipeline that relies solely on motion information. Maritime objects are highly susceptible to being obscured or submerged by waves, resulting in fragmented tracklets. We first introduced guiding modulation to address the long-term occlusion and interaction of maritime objects. Subsequently, we modeled confidence, altitude, and angular momentum to mitigate the effects of motion blur, ringing, and overshoot artifacts to observations in unstable imaging environments. Additionally, we designed a motion fusion mechanism that combines long-term macro tracklets with short-term fine-grained tracklets. This correction mechanism helps reduce the estimation variance of the Kalman Filter (KF) to alleviate the substantial nonlinear motion of maritime objects. We call this pipeline SeaTrack, which remains simple, online, and real-time, demonstrating excellent performance and scalability in benchmark evaluations.

Real-Time Motion Blur Using Multi-Layer Motion Vectors

Traditional methods for motion blur, often relying on a single layer, deviate from the correct colors. We propose a multilayer rendering method that closely approximates the motion blur effect. Our approach stores motion vectors for each pixel, divides these vectors into multiple sample points, and performs a backward search from the current pixel. The color at a sample point is sampled if it shares the same motion vector as its origin. This procedure repeats across layers, with only the nearest color values sampled for depth testing. The average color sampled at each point becomes that of the motion blur. Our experimental results indicate that our method significantly reduces the color deviation commonly found in traditional approaches, achieving structural similarity index measures (SSIM) of 0.8 and 0.92, which represent substantial improvements over the accumulation method.

Fast and stable pedicel detection for robust visual servoing to harvest shaking fruits

This study introduces a fast and stable pedicel detection method for robust visual servoing (RVS), supplemented with video stabilization (ViS) and fast pedicel detection, to realize automated harvesting of shaking fruits. By incorporating ViS, the system effectively mitigates the effects of motion blur, thereby ensuring consistent and precise object detection. In addition, the Fourier spectrum-based band-stop filter (FSBF) is used to improve clarity. The proposed approach also leverages the fast point feature histogram (FPFH) for fast pedicel detection, achieving real-time detection rates of 15–37 fps. Furthermore, it incorporates 6D pose estimation, culminating in the implementation of a 6D pose-based robust visual servoing (6DRVS) system. The performance of this system is evaluated using standard metrics such as perception accuracy, approach accuracy, precision, recall, accuracy, and F1-score in both preliminary tests and on-site experiments at two cucumber farms in Korea. The 6DRVS, supplemented with fast pedicel detection and ViS, exhibited improvements across all evaluation metrics. It recorded 90.00% perception accuracy, 82.22% approach accuracy, 0.957 precision, 0.938 recall, 0.900 accuracy, and 0.947 F1-score, highlighting its essential role in ensuring precise and efficient harvesting.

Are chimpanzees futurists? Effects of motion lines and motion blur on the judgments of global motion direction in chimpanzees (Pan troglodytes).

Based on the invention and development of photography and movie in the 19th century, schools of contemporary art, such as Futurism, have emerged that express the dynamism of motion in painting. Painting techniques such as multiple stroboscopic images, motion blur, and motion lines are culturally based, but the biological basis of their perception has also been intensively investigated recently. Then what are the evolutionary origins of such pictorial representations of motion? Do nonhuman animals also have sensitivity to such representations? To address this question, we examined the effects of motion blur and motion lines on the judgments of global motion directions in chimpanzees. The results showed that the motion lines biased the chimpanzees' judgments toward the direction of motion implied by them, whereas the effect of the motion blur was either absent or weak (Experiment 1). In Experiment 2, we manipulated the length and number of motion lines to examine the effect of "speed" and "distance" in addition to the motion direction implied by the motion lines. The results showed that the effect of motion lines became stronger as the length and the number of lines increased within a specific range. These results indicate that the motion lines also imply the direction of motion in chimpanzees and provide a clue to the evolutionary basis for the pictorial representations of motion. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

A Collaborative Visual Sensing System for Precise Quality Inspection at Manufacturing Lines

Visual sensing has been widely adopted for quality inspection in production processes. This paper presents the design and implementation of a smart collaborative camera system, called BubCam , for automated quality inspection of manufactured ink bags in Hewlett-Packard (HP) Inc.’s factories. Specifically, BubCam estimates the volume of air bubbles in an ink bag, which may affect the printing quality. The design of BubCam faces challenges due to the dynamic ambient light reflection, motion blur effect, and data labeling difficulty. As a starting point, we design a single-camera system which leverages various deep learning (DL)-based image segmentation and depth fusion techniques. New data labeling and training approaches are proposed to utilize prior knowledge of the production system for training the segmentation model with a small dataset. Then, we design a multi-camera system which additionally deploys multiple wireless cameras to achieve better accuracy due to multi-view sensing. To save power of the wireless cameras, we formulate a configuration adaptation problem and develop the single-agent and multi-agent deep reinforcement learning (DRL)-based solutions to adjust each wireless camera’s operation mode and frame rate in response to the changes of presence of air bubbles and light reflection. The multi-agent DRL approach aims to reduce the retraining costs during the production line reconfiguration process by only retraining the DRL agents for the newly added cameras and the existing cameras with changed positions. Extensive evaluation on a lab testbed and real factory trial shows that BubCam outperforms six baseline solutions including the current manual inspection and existing bubble detection and camera configuration adaptation approaches. In particular, BubCam achieves 1.3x accuracy improvement and 300x latency reduction, compared with the manual inspection approach.

Edge smoothing optimization method in DMD digital lithography system based on dynamic blur matching pixel overlap technique.

Due to digital micromirrors device (DMD) digital lithography limited by non-integer pixel errors, the edge smoothness of the exposed image is low and the sawtooth defects are obvious. To improve the image edge smoothness, an optimized pixel overlay method was proposed, which called the DMD digital lithography based on dynamic blur effect matching pixel overlay technology. The core of this method is that motion blur effect is cleverly introduced in the process of pixel overlap to carry out the lithography optimization experiment. The simulation and experimental results showed that the sawtooth edge was reduced from 1.666 µm to 0.27 µm by adopting the 1/2 dynamic blur effect to match pixel displacement superposition, which is far less than half of the sawtooth edge before optimization. The results indicated that the proposed method can efficiently improve the edge smoothness of lithographic patterns. We believe that the proposed optimization method can provide great help for high fidelity and efficient DMD digital lithography microfabrication.

Supplemental Material for Are Chimpanzees Futurists? Effects of Motion Lines and Motion Blur on the Judgments of Global Motion Direction in Chimpanzees (Pan troglodytes)

Supplemental Material for Are Chimpanzees Futurists? Effects of Motion Lines and Motion Blur on the Judgments of Global Motion Direction in Chimpanzees (Pan troglodytes)

Cratering and Tectonic History of the Largest Uranian Satellite, Titania: New Insights Enabled by Image Reprocessing

From heavily cratered Umbriel to extensively tectonized Miranda, Titania is an intermediary of the Uranian system: heavily cratered, yet tectonically modified. An outstanding mystery in Titania's crater population is its apparent relative lack of large (>30 km) craters. However, progress has been limited by the coverage and quality of images available. Here, we present a new map of Titania enabled by reprocessing Voyager images to reduce the effects of motion blur. Of note, we identify a network of fractures, a set of lineaments that may represent a large multi-ring impact structure, and newly identified catenae. These findings suggest Titania's crater population is missing large craters due to viscous relaxation, tectonic resurfacing, and/or planetocentric debris, and does not necessarily require cryovolcanic resurfacing. In preparation for future missions to the Uranian system, this work presents foundations for identifying imaging targets that can contribute to furthering our understanding of the history and evolution of the Uranian system in a broader context of icy satellite evolution.

Dual-branch multi-information aggregation network with transformer and convolution for polyp segmentation

Computer-Aided Diagnosis (CAD) for polyp detection offers one of the most notable showcases. By using deep learning technologies, the accuracy of polyp segmentation is surpassing human experts. In such CAD process, a critical step is concerned with segmenting colorectal polyps from colonoscopy images. Despite remarkable successes attained by recent deep learning related works, much improvement is still anticipated to tackle challenging cases. For instance, the effects of motion blur and light reflection can introduce significant noise into the image. The same type of polyps has a diversity of size, color and texture. To address such challenges, this paper proposes a novel dual-branch multi-information aggregation network (DBMIA-Net) for polyp segmentation, which is able to accurately and reliably segment a variety of colorectal polyps with efficiency. Specifically, a dual-branch encoder with transformer and convolutional neural networks (CNN) is employed to extract polyp features, and two multi-information aggregation modules are applied in the decoder to fuse multi-scale features adaptively. Two multi-information aggregation modules include global information aggregation (GIA) module and edge information aggregation (EIA) module. In addition, to enhance the representation learning capability of the potential channel feature association, this paper also proposes a novel adaptive channel graph convolution (ACGC). To validate the effectiveness and advantages of the proposed network, we compare it with several state-of-the-art (SOTA) methods on five public datasets. Experimental results consistently demonstrate that the proposed DBMIA-Net obtains significantly superior segmentation performance across six popularly used evaluation matrices. Especially, we achieve 94.12% mean Dice on CVC-ClinicDB dataset which is 4.22% improvement compared to the previous state-of-the-art method PraNet. Compared with SOTA algorithms, DBMIA-Net has a better fitting ability and stronger generalization ability.

A Motion Deblurring Network for Enhancing UAV Image Quality in Bridge Inspection

Unmanned aerial vehicles (UAVs) have been increasingly utilized for facility safety inspections due to their superior safety, cost effectiveness, and inspection accuracy compared to traditional manpower-based methods. High-resolution images captured by UAVs directly contribute to identifying and quantifying structural defects on facility exteriors, making image quality a critical factor in achieving accurate results. However, motion blur induced by external factors such as vibration, low light conditions, and wind during UAV operation significantly degrades image quality, leading to inaccurate defect detection and quantification. To address this issue, this research proposes a deblurring network using a Generative Adversarial Network (GAN) to eliminate the motion blur effect in UAV images. The GAN-based motion deblur network represents an image inpainting method that leverages generative models to correct blurry artifacts, thereby generating clear images. Unlike previous studies, this proposed approach incorporates deblur and blur learning modules to realistically generate blur images required for training the generative models. The UAV images processed using the motion deblur network are evaluated using a quality assessment method based on local blur map and other well-known image quality assessment (IQA) metrics. Moreover, in the experiment of crack detection utilizing the object detection system, improved detection results are observed when using enhanced images. Overall, this research contributes to improving the quality and accuracy of facility safety inspections conducted with UAV-based inspections by effectively addressing the challenges associated with motion blur effects in UAV-captured images.

Presentation of a method for removal of motion blur effect in images by using GAN

Presentation of a method for removal of motion blur effect in images by using GAN

EFFNet: Element-wise feature fusion network for defect detection of display panels

Online or real-time defect detection of display panels after array process is of paramount importance for quality control and yield rate improvement of products in display industry. However, owing to the limitation in feature representation, the performances of traditional defect detection methods are not satisfactory. This paper develops a novel element-wise feature fusion network (EFFNet) to solve the issue and achieve high-accuracy real-time defect detection of display panels. The method adopts a transfer learning and fine-tuning strategy for feature extraction layers and a decoder with relatively less computational complexity. Particularly, a feature fusion module based on element-wise addition of pyramid features is proposed in skip connection to improve detection efficiency and accuracy. Our method is compared with many state-of-the-art CNN-based models. Additionally, the effects of training dataset size, motion blur, and different backgrounds on the performance of the proposed method are investigated. Extensive experiments, including the ablation study, demonstrate that the developed network can accurately detect defects with complex textures, ambiguous boundaries and low contrast. It also has good robustness against motion blur. It outperforms state-of-the-art methods in terms of mIoU, mPA, and F1-Measure. Moreover, it is able to detect defects at speeds of up to 159 fps with input images of size 256 × 256 pixels.

A review of video‐based rainfall measurement methods

AbstractAccurate and high spatiotemporal resolution rainfall observations are essential for hydrological forecasting and flood management, especially in urban hydrological applications. However, it is difficult for traditional rainfall gauges, weather radars, and satellites to accurately estimate rainfall while simultaneously capturing the spatial and temporal variability of rainfall well. In this context, video‐based rainfall measurement, a novel method, has the advantages of real‐time performance and low cost and may thus provide a new way to establish rainfall observation networks with high spatial and temporal resolution. In recent years, different algorithms have been developed to recognize raindrops and estimate rainfall from rainfall videos. It has been demonstrated that video‐based rainfall measurement methods can provide comprehensive rainfall information with fine spatial and temporal granularity. However, raindrop visibility and the depth of field effects are difficult to address. The motion blur effect of raindrops may result in substantial errors and uncertainties. A fundamental problem of video‐based rainfall measurements lies in locating raindrops and accurately calculating their actual size. Moreover, the effectiveness of deep learning‐based video rainfall measurement models is greatly influenced by the diversity of the training data. Therefore, enhancing the high robustness and accuracy of video‐based rainfall measurement algorithms and increasing the computational efficiency are paramount to further development, which are prerequisites for their application in practical rainfall monitoring and developing multicamera monitoring networks.This article is categorized under: Science of Water > Methods Science of Water > Hydrological Processes

Lidar Pose Tracking of a Tumbling Spacecraft Using the Smoothed Normal Distribution Transform

Lidar sensors enable precise pose estimation of an uncooperative spacecraft in close range. In this context, the iterative closest point (ICP) is usually employed as a tracking method. However, when the size of the point clouds increases, the required computation time of the ICP can become a limiting factor. The normal distribution transform (NDT) is an alternative algorithm which can be more efficient than the ICP, but suffers from robustness issues. In addition, lidar sensors are also subject to motion blur effects when tracking a spacecraft tumbling with a high angular velocity, leading to a loss of precision in the relative pose estimation. This work introduces a smoothed formulation of the NDT to improve the algorithm’s robustness while maintaining its efficiency. Additionally, two strategies are investigated to mitigate the effects of motion blur. The first consists in un-distorting the point cloud, while the second is a continuous-time formulation of the NDT. Hardware-in-the-loop tests at the European Proximity Operations Simulator demonstrate the capability of the proposed methods to precisely track an uncooperative spacecraft under realistic conditions within tens of milliseconds, even when the spacecraft tumbles with a significant angular rate.

Robust Face Hallucination Algorithm Using Motion Blur Embedded Nearest Proximate Patch Representation

Face hallucination (FH) techniques have received a lot of attention in recent years for generating high-resolution (HR) face images from captured low-resolution (LR), noisy, and blurry images. However, existing FH techniques are incapable of dealing with motion blur, which is commonly introduced in captured images due to camera defocussing and other factors. Therefore, to make the FH process more resistant to motion blur, in this article, we present a novel learning-based FH algorithm called Motion Blur Embedded Nearest Proximate Patch Representation (MBENPPR). The MBENPPR algorithm begins by estimating the motion blur kernel from a motion-blurred LR test face. The estimated kernel is then embedded in training images to make them compatible with test images. It assists in reducing the effect of motion blur in the reconstruction process. Furthermore, the nearest proximate patches are selected from the training space to represent the test image patches as a weighted linear combination of selected patches. It facilitates the proposed algorithm in preserving sharp edges and texture information in the resulting faces. The results of simulations on standard datasets and locally captured real-life faces show that the MBENPPR algorithm outperforms the compared existing algorithms.

Efficient, accurate and fast pupil segmentation for pupillary boundary in iris recognition

Iris recognition is a robust biometric system—user-friendly, accurate, fast, and reliable. This biometric system captures information in a contactless manner, making it suitable for use during the COVID-19 pandemic. Despite its advantages such as high security and high accuracy, iris recognition still suffers from pupil deformation, motion blur, eyelids blocking, reflection occlusion and eyelashes obscure. If the pupillary boundary is not accurately segmented, iris recognition may suffer tremendously. Moreover, reflections in iris image may lead to an incorrect pupillary boundary segmentation. The segmentation accuracy can also be affected and reduced because of the presence of an unwanted noise created by the motion blur effect in iris image. Additionally, the pupillary boundary might change from circular shape to uneven or irregular shape because of the interference and obstruction in pupil region. Therefore, this work is carried out to determine an accurate, efficient and fast algorithm for the segmentation of pupillary boundary. First, the iris image is pre-processed with Wiener filter. Next, the respective iris image is assigned with a specific threshold. After that, the pixel property in iris image is computed to determine the pupillary boundary coordinates which are acquired from the measured pixel list and area in iris image. Finally, morphological closing is used to remove reflections in the inner region of pupil boundary. All experiments are implemented with CASIA v4 database and Matlab R2020a.

Neural network based visual servo for quick-change device alignment in context of fusion reactor remote maintenance

The remote maintenance process for fusion reactor is complex, which can be very time-consuming and labor-consuming. This paper proposes a modified neural network based visual servo method to align a quick-change device with robot camera system. A classical position-based visual servo control law is used to guide the robot to reach the desired position. The key target for the above visual servo controller is to obtain a robust pose estimator to calculate the quick-change device pose with respect to the camera. This pose estimator is trained using RepVGG model with a self built image samples. An attention mechanism is added to the neural network to enhance the stability of pose prediction for reflective metal objects. The robot joint speed is also smoothed to reduce the image motion blur effect and make the visual servo process stable. The performance of the proposed visual servo controller was verified on an UR5 robot, and the results show that the stable and rough alignment of the quick-change device can be realized.

Random motion blur for optical image encryption

We present a compact optical encryption scheme by using a continuous-random-motion blurring model in an optical imaging system. Image encryption is performed by additive motion blur effects with continuous and random shifts of a camera. Real-time random phase modulation can be achieved without the use of random phase mask. Storage of the key is more convenient, which only requires parameters of motion. In addition, modulation characteristics are different from the traditional encryption schemes. On the premise of high security, modulation space is broadened, and flexibility of encryption is further improved. Simulations and experiments verify the validity of the motion blur-based crypto-system and demonstrate its security under several attacks. This novel method will be significant for the practical applications in the field of optical information security.

Dehazing and deblurring of underwater images with heavy-tailed priors.

The common problems of underwater images include color cast, haze effect, and the motion blur effect caused by turbulence and camera shake. To address these problems, research on color cast and haze and blur effects is carried out in this paper. Because red light has significant attenuation underwater, which could cause color cast of images, this paper proposes a red channel compensation method to solve this problem. This approach adaptively compensates for the red channel according to the pixel value of the red channel, successfully preventing excessive compensation. To address the haze effect of underwater images, combined with the physical model of underwater images, a variational method is introduced in the paper. This method can not only recover clear underwater images, but also refine the transmission map at the same time. Furthermore, the blind deconvolution method is adopted to deblur underwater images. First, the blur kernel of an underwater image is estimated, and then a clear underwater image is recovered based on the obtained blur kernel. Finally, qualitative and quantitative comparisons of the underwater images recovered by different methods are also carried out. From the qualitative perspective, the images recovered by our method have higher image sharpness and more outstanding details. The quantitative comparison results show that the images recovered using our method have higher scores according to various criteria. Therefore, on the whole, our method presents great advantages in comparison with others.