No-flash Image Pairs Research Articles

Smartphone screening for neonatal jaundice via ambient-subtracted sclera chromaticity.

Jaundice is a major cause of mortality and morbidity in the newborn. Globally, early identification and home monitoring are significant challenges in reducing the incidence of jaundice-related neurological damage. Smartphone cameras are promising as colour-based screening tools as they are low-cost, objective and ubiquitous. We propose a novel smartphone method to screen for neonatal jaundice by imaging the sclera. It does not rely on colour calibration cards or accessories, which may facilitate its adoption at scale and in less economically developed regions. Our approach is to explicitly address three confounding factors in relating colour to jaundice: (1) skin pigmentation, (2) ambient light, and (3) camera spectral response. (1) The variation in skin pigmentation is avoided by imaging the sclera. (2) With the smartphone screen acting as an illuminating flash, a flash/ no-flash image pair is captured using the front-facing camera. The contribution of ambient light is subtracted. (3) In principle, this permits a device- and ambient-independent measure of sclera chromaticity following a one-time calibration. We introduce the concept of Scleral-Conjunctival Bilirubin (SCB), in analogy with Transcutaneous Bilirubin (TcB). The scleral chromaticity is mapped to an SCB value. A pilot study was conducted in the UCL Hospital Neonatal Care Unit (n = 37). Neonates were imaged using a specially developed app concurrently with having a blood test for total serum bilirubin (TSB). The better of two models for SCB based on ambient-subtracted sclera chromaticity achieved r = 0.75 (p<0.01) correlation with TSB. Ambient subtraction improved chromaticity estimates in proof-of-principle laboratory tests and screening performance within our study sample. Using an SCB decision threshold of 190μmol/L, the sensitivity was 100% (specificity 61%) in identifying newborns with TSB>250μmol/L (area under receiver operating characteristic curve, AUROC, 0.86), and 92% (specificity 67%) in identifying newborns with TSB>205μmol/L (AUROC 0.85). These results are comparable to modern transcutaneous bilirubinometers.

Siamese Dense Network for Reflection Removal with Flash and No-Flash Image Pairs

This work addresses the reflection removal with flash and no-flash image pairs to separate reflection from transmission. When objects are covered by glass, the no-flash image usually contains reflection, and thus flash is used to enhance transmission details. However, the flash image suffers from the specular highlight on the glass surface caused by flash. In this paper, we propose a siamese dense network (SDN) for reflection removal with flash and no-flash image pairs. SDN extracts shareable and complementary features via concatenated siamese dense blocks. We utilize an image fusion block for the SDN to fuse the intermediate output of two branches. Since severe information loss occurs in the specular highlight, we detect the specular highlight in the flash image based on gradient of the maximum chromaticity. Through observations, flash causes various artifacts such as tone distortion and inhomogeneous brightness. Thus, with synthetic datasets we collect 758 pairs of real flash and no-flash image pairs (including their ground truth) by different cameras to gain generalization. Various experiments show that the proposed method successfully removes reflections using flash and no-flash image pairs and outperforms state-of-the-art ones in terms of visual quality and quantitative measurements. Besides, we apply the SDN to color/depth image pairs and achieve both color reflection removal and depth filling.

Detail Enhanced Multi-Exposure Image Fusion Based On Edge Preserving Filters

Recent computational photography techniques play a significant role to overcome the limitation of standard digital cameras for handling wide dynamic range of real-world scenes contain brightly and poorly illuminated areas. In many of such techniques [1,2,3], it is often desirable to fuse details from images captured at different exposure settings, while avoiding visual artifacts. One such technique is High Dynamic Range (HDR) imaging that provides a solution to recover radiance maps from photographs taken with conventional imaging equipment. The process of HDR image composition needs the knowledge of exposure times and Camera Response Function (CRF), which is required to linearize the image data before combining Low Dynamic Range (LDR) exposures into HDR image. One of the long-standing challenges in HDR imaging technology is the limited Dynamic Range (DR) of conventional display devices and printing technology. Due to which these devices are unable to reproduce full DR. Although DR can be reduced by using a tone-mapping, but this comes at an unavoidable trade-off with increased computational cost. Therefore, it is desirable to maximize information content of the synthesized scene from a set of multi-exposure images without computing HDR radiance map and tone-mapping.This research attempts to develop a novel detail enhanced multi-exposure image fusion approach based on texture features, which exploits the edge preserving and intra-region smoothing property of nonlinear diffusion filters based on Partial Differential Equations (PDE). With the captured multi-exposure image series, we first decompose images into Base Layers (BLs) and Detail Layers (DLs) to extract sharp details and fine details, respectively. The magnitude of the gradient of the image intensity is utilized to encourage smoothness at homogeneous regions in preference to inhomogeneous regions. In the next step texture features of the BL to generate a decision mask (i.e., local range) have been considered that guide the fusion of BLs in multi-resolution fashion. Finally, well-exposed fused image is obtained that combines fused BL and the DL at each scale across all the input exposures. The combination of edge-preserving filters with Laplacian pyramid is shown to lead to texture detail enhancement in the fused image.Furthermore, Non-linear adaptive filter is employed for BL and DL decomposition that has better response near strong edges. The texture details are then added to the fused BL to reconstruct a detail enhanced LDR version of the image. This leads to an increased robustness of the texture details while at the same time avoiding gradient reversal artifacts near strong edges that may appear in fused image after DL enhancement.Finally, we propose a novel technique for exposure fusion in which Weighted Least Squares (WLS) optimization framework is utilized for weight map refinement of BLs and DLs, which lead to a new simple weighted average fusion framework. Computationally simple texture features (i.e. DL) and color saturation measure are preferred for quickly generating weight maps to control the contribution from an input set of multi-exposure images. Instead of employing intermediate HDR reconstruction and tone-mapping steps, well-exposed fused image is generated for displaying on conventional display devices. Simulation results are compared with a number of existing single resolution and multi-resolution techniques to show the benefits of the proposed scheme for the variety of cases. Moreover, the approaches proposed in this thesis are effective for blending flash and no-flash image pair, and multi-focus images, that is, input images photographed with and without flash, and images focused on different targets, respectively. A further advantage of the present technique is that it is well suited for detail enhancement in the fused image.

Occlusion-Aware Stereo Matching

Stereo vision systems with additional flash/no-flash cues have been demonstrated to be robust to depth discontinuities. The ratio of a flash and no-flash image pair naturally provides additional sc...

A Novel Imaging Method for Cell Phone Camera in Low Ambient Light Conditions Using Flash and No-Flash Image Pairs

A Novel Imaging Method for Cell Phone Camera in Low Ambient Light Conditions Using Flash and No-Flash Image Pairs

Constant Time Joint Bilateral Filtering Using Joint Integral Histograms

In this brief, we present a constant time method for the joint bilateral filtering. First, we propose an image data structure, coined as joint integral histograms (JIHs). Extending the classic integral images and the integral histograms, it represents the global information of two correlated images. In a JIH, the value at each bin indicates an integral determined by the two images. Then, the joint bilateral filtering is transformed to computation and manipulation of histograms. Utilizing the JIHs, we are capable of joint bilateral filtering in constant time. Its performance is validated in a digital photography approach using Flash-noFlash image pairs. Compared with the brute-force method, the proposed method achieves a speedup factor of 2-3 orders of magnitude while producing similar filtering results.

Removing photography artifacts using gradient projection and flash-exposure sampling

Flash images are known to suffer from several problems: saturation of nearby objects, poor illumination of distant objects, reflections of objects strongly lit by the flash and strong highlights due to the reflection of flash itself by glossy surfaces. We propose to use a flash and no-flash (ambient) image pair to produce better flash images. We present a novel gradient projection scheme based on a gradient coherence model that allows removal of reflections and highlights from flash images. We also present a brightness-ratio based algorithm that allows us to compensate for the falloff in the flash image brightness due to depth. In several practical scenarios, the quality of flash/no-flash images may be limited in terms of dynamic range. In such cases, we advocate using several images taken under different flash intensities and exposures. We analyze the flash intensity-exposure space and propose a method for adaptively sampling this space so as to minimize the number of captured images for any given scene. We present several experimental results that demonstrate the ability of our algorithms to produce improved flash images.

Digital photography with flash and no-flash image pairs

Digital photography has made it possible to quickly and easily take a pair of images of low-light environments: one with flash to capture detail and one without flash to capture ambient illumination. We present a variety of applications that analyze and combine the strengths of such flash/no-flash image pairs. Our applications include denoising and detail transfer (to merge the ambient qualities of the no-flash image with the high-frequency flash detail), white-balancing (to change the color tone of the ambient image), continuous flash (to interactively adjust flash intensity), and red-eye removal (to repair artifacts in the flash image). We demonstrate how these applications can synthesize new images that are of higher quality than either of the originals.