Blur Patterns Research Articles

Imaging simulation and restoration for a mobile-based long-focus camera with optical stabilization

Image stabilization technology of the mobile-based long-focus camera is an influential factor that limits the imaging quality. Among them, the periscope lens system is a common choice and is capable of compensating for the external vibration by rotating its prism. To further investigate the impact of residual vibration during the optical image stabilization process, we propose an imaging simulation method to quantify the degradation and construct the corresponding dataset for training. We decompose the degradation into translation perpendicular to the optical axis on the image plane and defocus parallel to the optical axis. The defocus blur pattern is modified to better reproduce the spatially variant deterioration caused by defocus. Constructing synthetic datasets through our proposed imaging simulation method significantly reduces the cost of training the post-processing network while enhancing the imaging quality. Extensive experiments demonstrate the accuracy of our proposed method and the superiority of our synthetic dataset.

Intriguing Findings of Frequency Selection for Image Deblurring

Blur was naturally analyzed in the frequency domain, by estimating the latent sharp image and the blur kernel given a blurry image. Recent progress on image deblurring always designs end-to-end architectures and aims at learning the difference between blurry and sharp image pairs from pixel-level, which inevitably overlooks the importance of blur kernels. This paper reveals an intriguing phenomenon that simply applying ReLU operation on the frequency domain of a blur image followed by inverse Fourier transform, i.e., frequency selection, provides faithful information about the blur pattern (e.g., the blur direction and blur level, implicitly shows the kernel pattern). Based on this observation, we attempt to leverage kernel-level information for image deblurring networks by inserting Fourier transform, ReLU operation, and inverse Fourier transform to the standard ResBlock. 1 × 1 convolution is further added to let the network modulate flexible thresholds for frequency selection. We term our newly built block as Res FFT-ReLU Block, which takes advantages of both kernel-level and pixel-level features via learning frequency-spatial dual-domain representations. Extensive experiments are conducted to acquire a thorough analysis on the insights of the method. Moreover, after plugging the proposed block into NAFNet, we can achieve 33.85 dB in PSNR on GoPro dataset. Our method noticeably improves backbone architectures without introducing many parameters, while maintaining low computational complexity. Code is available at https://github.com/DeepMed-Lab/DeepRFT-AAAI2023.

BANet: A Blur-Aware Attention Network for Dynamic Scene Deblurring.

Image motion blur results from a combination of object motions and camera shakes, and such blurring effect is generally directional and non-uniform. Previous research attempted to solve non-uniform blurs using self-recurrent multi-scale, multi-patch, or multi-temporal architectures with self-attention to obtain decent results. However, using self-recurrent frameworks typically leads to a longer inference time, while inter-pixel or inter-channel self-attention may cause excessive memory usage. This paper proposes a Blur-aware Attention Network (BANet), that accomplishes accurate and efficient deblurring via a single forward pass. Our BANet utilizes region-based self-attention with multi-kernel strip pooling to disentangle blur patterns of different magnitudes and orientations and cascaded parallel dilated convolution to aggregate multi-scale content features. Extensive experimental results on the GoPro and RealBlur benchmarks demonstrate that the proposed BANet performs favorably against the state-of-the-arts in blurred image restoration and can provide deblurred results in real-time.

The Impact of Image Contrast, Blur, and Presentation Pattern on Visual Sustained Attention

The Impact of Image Contrast, Blur, and Presentation Pattern on Visual Sustained Attention

Modulated Motion Blur-Based Vehicle Body Velocity and Pose Estimation Using an Optical Image Modulator

This paper presents a new velocity and pose sensor for a ground vehicle using modulated motion blur generated from an optical image modulator (OIM). The OIM repeatedly shakes the lens of a camera in the lateral direction of the vehicle body, which creates patterns of motion blur that are recorded in an image frame. The relationship between the blur pattern and vehicle velocity is determined using camera dynamics. The 2D velocity of the vehicle body is estimated by analyzing the motion blur pattern. The vehicle roll and pitch angle with respect to the ground are calculated by comparing the estimated average depths (heights) of the sections of a scene. We propose a signal processing and control system to precisely modulate the OIM, which is an electro-mechanical system consisting of a coil, magnet, spring and damper. In addition, we validate the influence of the control reference waveform through various experiments and confirm the robustness of the proposed system. The proposed system is suitable for many applications because it is insensitive to environmental changes and has a relatively low computation time while accurately measuring the speed.

The effect of astigmatic axis on visual acuity measured with different alphabets in Roman alphabet readers.

ObjectiveAstigmatism produces meridional variations in the retinal blur pattern, thus interacting with object spatial detail and altering visual performance as the axis changes. This study investigates the influence of astigmatic axis orientation on visual acuity (VA) for four alphabets used worldwide.MethodsVisual acuity was measured monocularly in 25 Roman alphabet users (mean age: 25.6±7.5 years) using computer-presented logarithm of the minimum angle of resolution (log-MAR) charts with letters from four different alphabets (Arabic, Chinese, Roman, and Tamil). VA was assessed under the effect of four optical conditions: best distance correction and three astigmatic conditions (using a +2.00 cylindrical diopter trial case lens with its axis oriented at 180, 45, or 90 degrees). For each alphabet, single optotypes were presented on a monitor viewed from a distance of 4.0 m, and a matching technique was used to identify the letters.ResultsThe degradation in VA with astigmatic defocus was influenced by the alphabet used (p<0.001) and by the astigmatic axis (p<0.001). Interactions in VA degradation between astigmatic axes and alphabet (p<0.001) showed differences within 0.10 logMAR. These interactions were more pronounced in alphabets with higher dominance of curves and vertical (Tamil) and horizontal (Arabic) detail.ConclusionInteractions between alphabet and type of astigmatism indicate that the effects of meridional blur on letter discrimination differ between alphabets. These findings have relevance in the way VA is assessed in populations using different typographies, and ultimately in the impact of astigmatic axis on their visual performance.

A Pattern Classification Based approach for Blur Classification

Blur type identification is one of the most crucial step of image restoration. In case of blind restoration of such images, it is generally assumed that the blur type is known prior to restoration of such images. However, it is not practical in real applications. So, blur type identification is extremely desirable before application of blind restoration technique to restore a blurred image. An approach to categorize blur in three classes namely motion, defocus, and combined blur is presented in this paper. Curvelet transform based energy features are utilized as features of blur patterns and a neural network is designed for classification. The simulation results show preciseness of proposed approach.

Probabilistic Deformation Models for Challenging Periocular Image Verification

The periocular region as a biometric trait has recently gained considerable traction, especially under challenging scenarios where reliable iris information is not available for human authentication. In this paper, we consider the problem of one-to-one (1 : 1) matching of highly nonideal periocular images captured in-the-wild under unconstrained imaging conditions. Such images exhibit considerable appearance variations, including nonuniform illumination variations, motion and defocus blur, off-axis gaze, and nonstationary pattern deformations. To address these challenges, we propose periocular probabilistic deformation models (PPDMs) that: 1) reduce the image matching problem to matching local image regions and 2) approximate the periocular distortions by local patch level spatial translations whose relationships are modeled by a Gaussian Markov random field. Given a periocular image pair, we determine the distortion-tolerant similarity metric by regularizing local match scores by the maximum a posteriori probability estimate of the relative local deformations between them. Unlike the existing global periocular image matching techniques, by accounting for local image deformations in the periocular matching process, PPDM exhibits greater tolerance to pattern variations. We demonstrate the effectiveness of our model via extensive evaluation on a large number of in-the-wild periocular images. We find that PPDMs outperform many benchmark 1 : 1 image matching techniques (improving verification rates at 0.1% false accept rate by ${\sim }30$ % over previous work and ${\sim }40$ % when compared with the best baseline) in challenging scenarios leading to state-of-the-art verification performance on multiple real-world periocular data sets.

Depth recovery and refinement from a single image using defocus cues

We present a technique to recover and refine the depth map from a single image captured by a conventional camera in this paper. Our method builds on the universal imaging principle: only scene at the focus distance will converge to a single sharp point on imaging sensor but other scene will yield different blur effects varying with its distance from the camera lens. We first estimate depth values at edge locations via spectrum contrast and then recover the full depth map using a depth matting optimization method. Due to the fact that some blur textures such as soft shadows or blur patterns will produce ambiguity results during the procedure of depth estimation, we use a total variation-based image smoothing method to smooth the original image, a smoothed image with detailed texture being suppressed can be generated. Taking this smoothed image as reference image, a guided filter is used to refine the final depth map.

Blur Classification Using Wavelet Transform and Feed Forward Neural Network

Image restoration deals with recovery of a sharp image from a blurred version. This approach can be defined as blind or non-blind based on the availability of blur parameters for deconvolution. In case of blind restoration of image, blur classification is extremely desirable before application of any blur parameters identification scheme. A novel approach for blur classification is presented in the paper. This work utilizes the appearance of blur patterns in frequency domain. These features are extracted in wavelet domain and a feed forward neural network is designed with these features. The simulation results illustrate the high efficiency of our algorithm.

Texture Features based Blur Classification in Barcode Images

Blur is an undesirable phenomenon which appears as image degradation. Blur classification is extremely desirable before application of any blur parameters estimation approach in case of blind restoration of barcode image. A novel approach to classify blur in motion, defocus, and co-existence of both blur categories is presented in this paper. The key idea involves statistical features extraction of blur pattern in frequency domain and designing of blur classification system with feed forward neural network.

Geometry of nanopore devices fabricated by electron beam lithography: Simulations and experimental comparisons

Fabrication of solid-state nanopore sensors for individual biomolecule analysis is a growing topic of interest. A number of recent studies demonstrate that engineered nanopore devices (ENDs) can be fabricated by electron beam lithography (EBL) with high density (on the order of 10 devices per cm2). The internal pore geometry of ENDs is a critical characteristic of the devices, and often obtained by some combination of SEM, TEM, AFM, and conductance measurements. However, experimental data alone is not sufficient to understand the nanopore geometry under a broad set of fabrication conditions. It is necessary to also examine the physical basis underlying the EBL-based fabrication of ENDs. In this work, the internal pore geometry of ENDs is calculated from electron energy distributions in EBL while investigating the effects of dose, operating blur, substrate, and dosing pattern. The photoresist is ZEP-520 on silicon or silicon nitride substrates. It is found that higher beam blur and lower dose cause a greater degree of pore tapering, with the most prominent tapering observed in sub-10nm pores. Nanopores in silicon nitride tapered more than those in silicon. The results also demonstrate that a combination of blur and dose can be chosen to achieve a target tapering angle and pore size at a given depth in the substrate. Because the pore tapering angle is non-uniform, the ICP (inductively coupled plasma) etch depth may also be used to tune pore size and geometry following EBL. The resist sensitivity is shown to increase with beam blur for pore sizes larger than 10nm. By comparing to our experimental data, it is found that beam intensity measured by the EBL instrument may not translate to the operating blur, as is often assumed in EBL simulations. Secondary electrons were found to be responsible for pore tapering and beam broadening in the resist.

Performance Capture of High‐Speed Motion Using Staggered Multi‐View Recording

AbstractWe present a markerless performance capture system that can acquire the motion and the texture of human actors performing fast movements using only commodity hardware. To this end we introduce two novel concepts: First, a staggered surround multi‐view recording setup that enables us to perform model‐based motion capture on motion‐blurred images, and second, a model‐based deblurring algorithm which is able to handle disocclusion, self‐occlusion and complex object motions. We show that the model‐based approach is not only a powerful strategy for tracking but also for deblurring highly complex blur patterns.

Blind deconvolution of quadratic time-frequency representations of musical signals for reverberation feature extraction.

An acoustic space is uniquely characterized by its reverberant behavior. Due to the complexity of the multiple reflections and diffraction of sound in an enclosure, currently available system identification algorithms cannot effectively estimate the impulse response of a concert hall simply from musical recordings in that space. This paper reports the use of blind image deconvolution methods to construct echo patterns from quadratic time-frequency representations of reverberant recordings of music. First, a quadratic time-frequency analysis is performed to decompose the musical signal into its constituent harmonic components. Quadratic time-frequency analysis methods are known to give enhanced resolution in the time-frequency plane in comparison to conventional Fourier analysis. Reverberant features then appear as blur in the time-frequency plane, which can be estimated by employing the methods of blind deconvolution of this “image” of the sound. The proposed algorithm retrieves both the blur pattern, which corresponds to the reverberation echogram, and the direct acoustic signal. By choosing the time-frequency frame scale and smoothing window, a multi-resolution analysis of the underlying acoustic impulse response is obtained. Various quadratic time-frequency analysis methods are evaluated and their relative performance is reported. The proposed methods are also compared to existing dereverberation algorithms.

The effect of acoustic velocity on phantom measurements

Urethane rubber ultrasound (US) phantoms have a much lower acoustic velocity (1430–1450 m/s) than the accepted soft tissue average of 1540 m/s. Two important questions arise: can the rod positions in these rubber phantoms be adjusted so that they may be used to test equipment distance measurement accuracy for all types of multielement transducers, and can they be used to measure beam focus (using the spread of the rod blur patterns)? These questions were addressed for linear-, phased-, convex- and vector-array transducers. Theoretical predictions for the different transducers’ distance measurement errors agreed with careful measurements obtained with a specially designed array of stainless-steel rods immersed in paraffin oil (1447 m/s). The conclusions of this study are that phantoms with acoustic velocities different from 1540 m/s cannot be used to check distance measurement accuracies of all the types of real-time transducers, nor to predict a transducer’s focusing performance in clinical scans.

Divergence thresholds as functions of temporal separation, spatial separation, and retinal locus.

Human thresholds were measured for the detection of angular divergence between straight lines using pairs of line segments. The dependence of these thresholds on temporal separation between the two lines, spatial separation, and retinal locus was assessed. Results were comparable to prior divergence thresholds obtained by Harrington and Harrington in their study of "blur patterns." In blur patterns motion parameters may be processed partly or wholly as form information rather than as motion information per se. Harrington and Harrington had used moderate blurring velocities, for which information on both motion and form were present. Observers may have been responding either to motion or to form. The study reported here used briefly presented two-line "blur patterns" with only form information. Analysis suggested that the form components of fast motion-produced blur patterns could be processed by the human visual system. Neither temporal nor spatial separation was a significant determiner of thresholds in accordance with Harrington and Harrington who found no effect of blur-line density in the range studied. Retinal locus was a factor as it was with blur patterns. Some possible mechanisms for the detection of divergence indicated by these results are discussed.

Perception of motion using blur pattern information in the moderate and high-velocity domains of vision

A series of studies of visual orientation at blurring velocities is integrated and summarized. Surfaces moving rapidly in depth were simulated and sensitivities of observers to depth-related geometric parameters of the resulting blur patterns, such as divergence and curvature, were measured. Examples of some ecological analyses of these parameters as sources of visual orientation information are given. The results indicate that human sensitivity to a number of blur pattern variables is high enough to be useful in guidance of locomotion. It was also found that some of the information about motion contained in blur patterns may actually be form information. It appears that visual motion processig may involve a complex interplay of visual form and motion analysing systems.

Visual orientation by motion-produced blur patterns: detection of divergence.

Blur patterns are physiological “streaks” of photochemical and neural activity that occur whenever an observer and his visual environment are in relative motion. When retinal velocities are high, the impression of visual “flow” gives way to one of a field of “blur lines” whose patterns are rich with information about the motions and the optical textures that produced them. Simulated blur patterns were produced and thresholds measured for the detection of divergence at nine retinal loci. Sensitivity was somewhat greater in the central retina. Thresh-olds remained the same despite variations in pattern velocity, number of elements, and the presence or absence of an internal velocity gradient. Observers were able to orient above-threshold patterns, but consistently underestimated the amount of slant.

Evaluation of image quality in tomographic imaging

Modulation transfer functions (MTE) for tomographic imaging with linear, circular, hypocycloidal and spiral motion are given. They may be determined experimentally from tomographs of slits or line pair test patterns. In some cases calculation of the MTF from the blur pattern by Fourier transformation is possible. All tomographic MTFs show the same oscillatory behaviour as MTFs of focal spots, resulting in spurious structures and contours in the tomogram. A quantity named signal/spurious signal ratio (SSR) is introduced as a figure of merit. Effective exposure angles are determined from the MTF and compared with the commonly used 'geometric' exposure angles which result from maximum tube elongations.