Sharp Frame Research Articles

Event-Assisted Blurriness Representation Learning for Blurry Image Unfolding.

The goal of blurry image deblurring and unfolding task is to recover a single sharp frame or a sequence from a blurry one. Recently, its performance is greatly improved with introduction of a bio-inspired visual sensor, event camera. Most existing event-assisted deblurring methods focus on the design of powerful network architectures and effective training strategy, while ignoring the role of blur modeling in removing various blur in dynamic scenes. In this work, we propose to implicitly model blur in an image by computing blurriness representation with an event-assisted blurriness encoder. The learning of blurriness representation is formulated as a ranking problem based on specially synthesized pairs. Blurriness-aware image unfolding is achieved by integrating blur relevant information contained in the representation into a base unfolding network. The integration is mainly realized by the proposed blurriness-guided modulation and multi-scale aggregation modules. Experiments on GOPRO and HQF datasets show favorable performance of the proposed method against state-of-the-art approaches. More results on real-world data validate its effectiveness in recovering a sequence of latent sharp frames from a blurry image.

Video reconstruction from a single motion blurred image using learned dynamic phase coding

Video reconstruction from a single motion-blurred image is a challenging problem, which can enhance the capabilities of existing cameras. Recently, several works addressed this task using conventional imaging and deep learning. Yet, such purely digital methods are inherently limited, due to direction ambiguity and noise sensitivity. Some works attempt to address these limitations with non-conventional image sensors, however, such sensors are extremely rare and expensive. To circumvent these limitations by simpler means, we propose a hybrid optical-digital method for video reconstruction that requires only simple modifications to existing optical systems. We use learned dynamic phase-coding in the lens aperture during image acquisition to encode motion trajectories, which serve as prior information for the video reconstruction process. The proposed computational camera generates a sharp frame burst of the scene at various frame rates from a single coded motion-blurred image, using an image-to-video convolutional neural network. We present advantages and improved performance compared to existing methods, with both simulations and a real-world camera prototype. We extend our optical coding to video frame interpolation and present robust and improved results for noisy videos.

Event-based video deblurring based on image and event feature fusion

Event-based video deblurring is a method that performs deblurring by taking the event sequence data obtained from an event camera, which is composed of bio-inspired sensors, along with blurry frames as input. Event-based video deblurring has gained attention as a method that can overcome the limitations of conventional frame-based video deblurring. In this study, we propose a novel event-based video deblurring network based on convolution neural networks (CNNs). Unlike the existing event-based deblurring methods that only use event data, the proposed method fuses all the available information from current blurry frames, previously recovered sharp frames, and event data to deblur a video. Specifically, we propose an image and event feature fusion (IEFF) module to fuse event data with current intensity frame information. Additionally, we propose a current-frame reconstruction from previous-frame (CRP) module for acquiring a pseudo sharp frame from a previously recovered sharp frame and a fusion-based residual estimation (FRE) module, which fuses the event features with the image features of the previous sharp frame extracted from the CRP module. We demonstrate through a verification experiment using synthetic and real datasets that the proposed method has superior quantitative and qualitative results compared to state-of-the-art methods.

Multi-Attention Convolutional Neural Network for Video Deblurring

Video deblurring, which aims at restoring the sharp video from blurry video, is drawing increasing attention in the field of computer vision. In this paper, a method called Multi-Attention Convolutional Neural Network (MACNN) consisting of the temporal-spatial attention module, the frame channel attention module, and the feature extraction-reconstruction module is proposed. First, we use the temporal-spatial attention module and the frame channel attention module to capture features with temporal and spatial information existing across neighboring frames. Then, these captured features are fused and reconstructed to restore the sharp frame. Last but not least, we train MACNN together with a content loss and a perceptual loss in an end-to-end manner to recover realistic video details. Both quantitative and qualitative evaluation results on standard benchmarks demonstrate the proposed MACNN is superior to the state-of-the-art methods in terms of accuracy, efficiency, and visual effect.

Continuous Facial Motion Deblurring

We introduce a novel framework for continuous facial motion deblurring that restores the continuous sharp moment latent in a single motion-blurred face image via a moment control factor. Although a motion-blurred image is the accumulated signal of continuous sharp moments during the exposure time, most existing single image deblurring approaches aim to restore a fixed number of frames using multiple networks and training stages. To address this problem, we propose a continuous facial motion deblurring network based on GAN (CFMD-GAN), which is a novel framework for restoring the continuous moment latent in a single motion-blurred face image with a single network and a single training stage. To stabilize the network training, we train the generator to restore continuous moments in the order determined by our facial motion-based reordering process (FMR) utilizing domain-specific knowledge of the face. Moreover, we propose an auxiliary regressor that helps our generator produce more accurate images by estimating continuous sharp moments. Furthermore, we introduce a control-adaptive (ContAda) block that performs spatially deformable convolution and channel-wise attention as a function of the control factor. Extensive experiments on the 300VW datasets demonstrate that the proposed framework generates a various number of continuous output frames by varying the moment control factor. Compared with the recent single-to-single image deblurring networks trained with the same 300VW training set, the proposed method show the superior performance in restoring the central sharp frame in terms of perceptual metrics, including LPIPS, FID and Arcface identity distance. The proposed method outperforms the existing single-to-video deblurring method for both qualitative and quantitative comparisons. In our experiments on the 300VW test set, the proposed framework reached 33.14 dB and 0.93 for recovery of 7 sharp frames in PSNR and SSIM, respectively.

Framing China's rise in the United States, Australia and the United Kingdom

Abstract Optimism about China's rise has in recent years given way to deep concern in the United States, Australia and the United Kingdom. Drawing on an original set of interviews with China experts from each country, and an array of primary and secondary sources, I show that shifting framings of China's rise reflect the dynamics of the US, Australian and UK national security fields. The article highlights three features specifically: first, the US field features a belief that China's rise can be arrested or prevented, absent in Australia and the UK. I root this dynamic in the system of professional appointments and the intense US ‘marketplace of ideas’, which gives rise to intense framing contestation and occasional sharp frame change. I then identify the key positions produced by each field, from which key actors have shaped the differing interpretations of China and its meaning. The election of Donald Trump, a strong China-critic, to the US presidency empowered key individuals across government who shifted the predominant framing of China from potential challenger to current threat. The smaller and more centralized fields in Australia and Britain feature fewer and less intense China-sceptical voices; responses have thereby remained largely pragmatic, despite worsening diplomatic relations in each case.

Recursive Neural Network for Video Deblurring

Video deblurring is still a challenging low-level vision task since spatio-temporal characteristics across both the spatial and temporal domains are difficult to model. In this article, to model the temporal information, we develop a non-local block which estimates inter-frame similarity and inter-frame difference. Specially, for modeling the spatial characteristics and restoring sharp frame details, we propose a recursive block that iteratively refines feature maps generated at the last iteration. In addition, a novel temporal loss function is introduced to ensure the temporal consistency of generated frames. Experimental results on public datasets demonstrate that our method achieves state-of-the-art performance both quantitatively and qualitatively.

Combining weighted curvelet accumulation with motion vector duty cycle for nonuniform video deblurring

Since object motion and camera shake can cause motion blur, there are some blurry frames in the captured videos by hand-held camera. In order to solve the above problem, we propose a nonuniform video deblurring method that combines weighted curvelet accumulation with motion vector duty cycle. In the proposed method, firstly we propose a weighted curvelet accumulation method that can synthesize the multiple adjacent frames in the frequency domain for estimating the initial latent sharp frame. Secondly, because the duty cycle has a close relation with the accuracy of the blur kernel, we propose a motion vector duty cycle estimation method by utilizing the inter-frame correlation information and the estimated initial latent sharp frame to improve the blur kernel accuracy. Finally, we build a novel nonblind video deblurring model by fully utilizing the spatiotemporal information and the estimated blur kernel for obtaining the deblurred frame. Experimental results on the numerous videos show that the proposed method achieves the state-of-the-art results either in subjective vision or in objective evaluation.

Combining Motion Compensation with Spatiotemporal Constraint for Video Deblurring.

We propose a video deblurring method by combining motion compensation with spatiotemporal constraint for restoring blurry video caused by camera shake. The proposed method makes effective full use of the spatiotemporal information not only in the blur kernel estimation, but also in the latent sharp frame restoration. Firstly, we estimate a motion vector between the current and the previous blurred frames, and introduce the estimated motion vector for deriving the motion-compensated frame with the previous restored frame. Secondly, we proposed a blur kernel estimation strategy by applying the derived motion-compensated frame to an improved regularization model for improving the quality of the estimated blur kernel and reducing the processing time. Thirdly, we propose a spatiotemporal constraint algorithm that can not only enhance temporal consistency, but also suppress noise and ringing artifacts of the deblurred video through introducing a temporal regularization term. Finally, we extend Fast Total Variation de-convolution (FTVd) for solving the minimization problem of the proposed spatiotemporal constraint energy function. Extensive experiments demonstrate that the proposed method achieve the state-of-the-art results either in subjective vision or objective evaluation.

Bundled Kernels for Nonuniform Blind Video Deblurring

We present a novel blind video deblurring approach by estimating a bundle of kernels and applying the residual deconvolution. Our approach adopts multiple kernels to represent spatially varying motion blur, and thus can cope with nonuniform video deblurring. For each blurred frame, we build a warping-based, space-variant motion blur model based on a bundle of homographies in between its adjacent frames. Then, the nearest sharp frame is employed to form an unblurred-blurred pair for solving the motion model and obtain a bundle of kernels at the blurred frame. Finally, we apply the deconvolution on the residual between the warped unblurred frame and blurred frame with the kernels. The blur kernel estimation and residual deconvolution are iteratively performed toward the deblurred frame, as well as significantly reducing artifacts such as ringings. Experiments show that our approach can efficiently remove the nonuniform video blurring, and achieves better deblurring results than some state-of-the-art methods.

Minimal Frame Operator Norms Via Minimal Theta Functions

We investigate sharp frame bounds of Gabor frames with chirped Gaussians and rectangular lattices or, equivalently, the case of the standard Gaussian and general lattices. We prove that for even redundancy and standard Gaussian window the hexagonal lattice minimizes the upper frame bound using a result by Montgomery on minimal theta functions.

Optimal Gabor frame bounds for separable lattices and estimates for Jacobi theta functions

We study sharp frame bounds of Gabor frames for integer redundancy with the standard Gaussian window and prove that the square lattice optimizes both the lower and the upper frame bound among all rectangular lattices. This proves a conjecture of Floch, Alard & Berrou (as reformulated by Strohmer & Beaver). The proof is based on refined log-convexity/concavity estimates for the Jacobi theta functions θ3 and θ4.

Al[sbnd]Al thermocompression bonding for wafer-level MEMS sealing

Al–Al thermocompression bonding has been studied using test structures relevant for wafer level sealing of MEMS devices. Si wafers with protruding frame structures were bonded to planar Si wafers, both covered with a sputtered Al film of 1μm thickness. The varied bonding process variables were the bonding temperature (400, 450 and 550°C) and the bonding force (18, 36 and 60kN). Frame widths 100μm, 200μm, with rounded or sharp frame corners were used. After bonding, laminates were diced into single chips and pull tested. The effect of process and design parameters was studied systematically with respect to dicing yield, bond strength and resulting fractured surfaces. The test structures showed an average strength of 20–50MPa for bonding at or above 450°C for all three bonding forces or bonding at 400°C with 60kN bond force. The current study indicates that strong AlAl thermocompression bonds can be achieved either at or above 450°C bonding temperature for low (18kN) and medium (36kN) bond force or by high bond force (60kN) at 400°C. The results show that an increased bond force is required to compensate for a reduced bonding temperature for AlAl thermocompression bonding in the studied temperature regime.

Role of stents in congenital heart disease

For more than 10 years, balloon-expandable intravascular stents have become an integral part of treatment for obstructive vascular lesions in children and adults with congenital heart disease. The initial problems with stents, such as sharp edges, rigid frame and unacceptable shortening when over-expanded have been overcome to a large extent with the newer designs. The problems related to delivery of stents, such as stent migration, balloon rupture, flaring of the edges of the stent, rupture of vessels and milking of the stent off the balloon, have also been overcome by newer designs of balloons. The failure of growth of balloon-expandable stents with the growth of the child is managed by redilation with or without additional stenting and newer growth stents. Self-expanding stents are not appropriate for use in growing children due to the limitation of their maximum diameters. The development of biodegradable stents may overcome these disadvantages in the future. Various new developments have recently occurred. Covered stents are ideal for treating acute vessel rupture and in isolating vascular aneurysms. Drug-eluting stents may prevent restenosis, but have not been used in children. Valved stents are a recent innovation for the treatment of regurgitant pulmonary valves.

A counterexample to “An extension of the Vitali-Hahn-Saks theorem” and a compactness result

We give a counterexample to An extension of the Vitali-HahnSaks theorem and from that highlight the sharp frame within which any attempt to change the version of such an extension should be possible. Lastly a sequential compactness criterion for Radon measures absolutely continuous with respect to a prescribed Radon measure defined on a locally compact separable metric space (taking into account the ideas of Hernandez-Lerma and Lasserre) is proved. The results deal with Radon measures but yield obvious corollaries on real (or vector-valued) Radon measures and so on functions with bounded variation on open subsets of Rn.