Sharpening Out of Focus Images using High‐Frequency Transfer

Abstract

AbstractFocus misses are common in image capture, such as when the camera or the subject moves rapidly in sports and macro photography. One option to sharpen focus‐missed photographs is through single image deconvolution, but high‐frequency data cannot be fully recovered; therefore, artifacts such as ringing and amplified noise become apparent. We propose a new method that uses assisting, similar but different, sharp image(s) provided by the user (such as multiple images of the same subject in different positions captured using a burst of photographs). Our first contribution is to theoretically analyze the errors in three sources of data—a slightly sharpened original input image that we call the target, single image deconvolution with an aggressive inverse filter, and warped assisting image(s) registered using optical flow. We show that these three sources have different error characteristics, depending on image location and frequency band (for example, aggressive deconvolution is more accurate in high‐frequency regions like edges). Next, we describe a practical method to compute these errors, given we have no ground truth and cannot easily work in the Fourier domain. Finally, we select the best source of data for a given pixel and scale in the Laplacian pyramid. We accurately transfer high‐frequency data to the input, while minimizing artifacts. We demonstrate sharpened results on out‐of‐focus images in macro, sports, portrait and wildlife photography.