Radiance Map Research Articles

Estimation of Image Sensor Fill Factor Using a Single Arbitrary Image.

Achieving a high fill factor is a bottleneck problem for capturing high-quality images. There are hardware and software solutions to overcome this problem. In the solutions, the fill factor is known. However, this is an industrial secrecy by most image sensor manufacturers due to its direct effect on the assessment of the sensor quality. In this paper, we propose a method to estimate the fill factor of a camera sensor from an arbitrary single image. The virtual response function of the imaging process and sensor irradiance are estimated from the generation of virtual images. Then the global intensity values of the virtual images are obtained, which are the result of fusing the virtual images into a single, high dynamic range radiance map. A non-linear function is inferred from the original and global intensity values of the virtual images. The fill factor is estimated by the conditional minimum of the inferred function. The method is verified using images of two datasets. The results show that our method estimates the fill factor correctly with significant stability and accuracy from one single arbitrary image according to the low standard deviation of the estimated fill factors from each of images and for each camera.

Sun position estimation and tracking for virtual object placement in time-lapse videos

Realistic illumination of virtual objects placed in real videos is important in terms of achieving visual coherence. We propose a novel approach for illumination estimation on time-lapse videos and seamlessly insert virtual objects in these videos in a visually consistent way. The proposed approach works for both outdoor and indoor environments where the main light source is the Sun. We first modify an existing illumination estimation method that aims to obtain sparse radiance map of the environment in order to estimate the initial Sun position. We then track the hard ground shadows on the time-lapse video by using an energy-based pixel-wise method. The proposed method aims to track the shadows by utilizing the energy values of the pixels that forms them. We tested the method on various time-lapse videos recorded in outdoor and indoor environments and obtained successful results.

Radiance map construction based on spatial and intensity correlations between LE and SE images for HDR imaging

HDR image was developed for the reproduction of real scenes with an acquisition of large dynamic range. In general, HDR image consists of several different exposure images according to the exposure value of a digital camera. Before the construction of a single HDR image, each input image is calibrated using CRF to convert its image data to scene radiance. In order find CRF, conventional methods require special apparatus and reference targets, or several exposure images. This paper proposes a new HDR blending algorithm that uses only dual-exposure images. The proposed algorithm is based on the least squares method, and includes spatial and intensity weighting functions. Each weighting function is used to reduce error points and improve CRF computation accuracy. In addition, a constraint is added to correct white balance in the brightness level. The rendering results show that the proposed algorithm is superior to the conventional algorithm.

Adaptive exposure estimation for high dynamic range imaging applied to natural scenes and daylight skies.

Digital imaging of natural scenes and optical phenomena present on them (such as shadows, twilights, and crepuscular rays) can be a very challenging task because of the range spanned by the radiances impinging on the capture system. We propose a novel method for estimating the set of exposure times (bracketing set) needed to capture the full dynamic range of a scene with high dynamic range (HDR) content. The proposed method is adaptive to scene content and to any camera response and configuration, and it works on-line since the exposure times are estimated as the capturing process is ongoing. Besides, it requires no a priori information about scene content or radiance values. The resulting bracketing sets are minimal in the default method settings, but the user can set a tolerance for the maximum percentage of pixel population that is underexposed or saturated, which allows for a higher number of shots if a better signal-to-noise ratio (SNR) in the HDR scene is desired. This method is based on the use of the camera response function that is needed for building the HDR radiance map by stitching together several differently exposed low dynamic range images of the scene. The use of HDR imaging techniques converts our digital camera into a tool for measuring the relative radiance outgoing from each point of the scene, and for each color channel. This is important for accurate characterization of optical phenomena present in the atmosphere while not suffering any loss of information due to its HDR. We have compared our method with the most similar one developed so far [IEEE Trans. Image Process.17, 1864 (2008)]. Results of the experiments carried out for 30 natural scenes show that our proposed method equals or outperforms the previously developed best approach, with less shots and shorter exposure times, thereby asserting the advantage of being adaptive to scene content for exposure time estimation. As we can also tune the balance between capturing time and the SNR in our method, we have compared its SNR performance against that of Barakat's method as well as against a ground-truth HDR image of maximum SNR. Results confirm the success of the proposed method in exploiting its tunability to achieve the desired balance of total Δt and SNR.

Multi-camera platform for panoramic real-time HDR video construction and rendering

High dynamic range (HDR) images are usually obtained by capturing several images of the scene at different exposures. Previous HDR video techniques adopted the same principle by stacking HDR frames in time domain. We designed a new multi-camera platform which is able to construct and render HDR panoramic video in real time, with $$1{,}024 \times 256$$1,024?256 resolution and a frame rate of 25 fps. We exploit the overlapping fields of view between the cameras with different exposures to create an HDR radiance map. We propose a method for HDR frame reconstruction which merges the previous HDR imaging techniques with the algorithms for panorama reconstruction. The developed FPGA-based processing system is able to reconstruct the HDR frame using the proposed method and tone map the resulting image using a hardware-adapted global operator. The measured throughput of the system is 245 MB/s, which is, up to our knowledge, among the fastest HDR video processing system.

Ghost-Free Image Using Blur and Noise Estimation

This paper presents an efficient image enhancement method by fusion of two different exposure images in low-light condition. We use two degraded images with different exposures: one is a long-exposure image that preserves the brightness but contains blur and the other is a short-exposure image that contains a lot of noise but preserves object boundaries. The weight map used for image fusion without artifacts of blur and noise is computed using blur and noise estimation. To get a blur map, edges in a long-exposure image are detected at multiple scales and the amount of blur is estimated at detected edges. Also, we can get a noise map by noise estimation using a denoised short-exposure image. Ghost effect between two successive images is avoided according to the moving object map that is generated by a sigmoid comparison function based on the ratio of two input images. We can get result images by fusion of two degraded images using the weight maps. The proposed method can be extended to high dynamic range imaging without using information of a camera response function or generating a radiance map. Experimental results with various sets of images show the effectiveness of the proposed method in enhancing details and removing ghost artifacts.

Adapting iterative retinex computation for high-dynamic-range tone mapping

Retinex algorithms have been widely applied in many aspects of image processing. Based on the iterative Retinex algorithm, we propose an edge-preserving illumination estimation method. Inspired by the anisotropic diffusion, an edge-stopping function is introduced in the iterative computation. This modification enables the preservation of abrupt edges when computing the upper envelope of a given image. Based on the illumination-reflectance decomposition, a high-dynamic-range (HDR) radiance map can be easily tone-mapped to be a low-dynamic-range image by compressing the range of the estimated illumination. Artifacts are effectively suppressed using the proposed method. Meanwhile, we also propose a jumping-spiral iteration manner to improve the symmetry of the edge response. Experimental results show that the proposed tone mapping algorithm is very effective in reproducing HDR scenes, and has a better performance compared with some similar operators.

Local tone mapping using the K-means algorithm and automatic gamma setting

Tone mapping (TM) algorithms reproduce the high dynamic range (HDR) images on low dynamic range (LDR) display devices such as monitors or printers. In this paper, we propose a local TM algorithm, in which the HDR input is segmented using the K-means algorithm and a display gamma parameter is set automatically for each segmented region. The proposed TM algorithm computes the luminance of an input that is the radiance map generated from a set of LDR images acquired with varying exposure settings. Then, according to the bilateral filtered luminance, an image is divided into a number of regions using the K-means algorithm. The display gamma value is set automatically according to the mean value of each region. Then, the tone of HDR image is reproduced by a local TM method with adaptive gamma value. We generate the tone-mapped image using the proposed local TM. Computer simulation with real LDR images shows the effectiveness of the proposed local TM algorithm in terms of the visual quality as well as the local contrast. It can be used for contrast and color enhancement in various display and acquisition devices.

High Dynamic Range Imaging for Archaeological Recording

This paper notes the adoption of digital photography as a primary recording means within archaeology, and reviews some issues and problems that this presents. Particular attention is given to the problems of recording high-contrast scenes in archaeology and high dynamic range imaging using multiple exposures is suggested as a means of providing an archive of high-contrast scenes that can later be tone-mapped to provide a variety of visualisations. Exposure fusion is also considered, although it is noted that this has some disadvantages. Three case studies are then presented: (1) a very high-contrast photograph taken from within a rock-cut tomb at Cala Morell, Menorca, (2) an archaeological test pitting exercise requiring rapid acquisition of photographic records in challenging circumstances and (3) legacy material consisting of three differently exposed colour positive (slide) photographs of the same scene. In each case, high dynamic range (HDR) methods are shown to significantly aid the generation of a high-quality illustrative record photograph, and it is concluded that HDR imaging could serve an effective role in archaeological photographic recording, although there remain problems of archiving and distributing HDR radiance map data.

Tone mapping using color correction function and image decomposition in high dynamic range imaging

In this paper, we propose a tone mapping (TM) method using color correction function (CCF) and image decomposition in high dynamic range (HDR) imaging. The CCF in the proposed TM is derived from the luminance compression function with the color constraint under which the color ratios, between the three color channels of the radiance map and dynamic range compression term, are preserved and color saturation is controlled. The proposed CCF is developed to locally perform the luminance compression and color saturation control in local TM. For image decomposition, we use a bilateral filter and apply the adaptive weight to the base layer of the luminance. Computer simulations with various sets of real, low dynamic range images show the effectiveness of the proposed TM algorithm in terms of the visual quality as well the local contrast. It can be used for contrast and color enhancement in various display and acquisition devices.

ACCELERATING WAVES IN POLAR CORONAL HOLES AS SEEN BY EIS AND SUMER

We present EIS/Hinode and SUMER/SOHO observations of propagating disturbances detected in coronal lines in inter-plume and plume regions of a polar coronal hole. The observation was carried out on 2007 November 13 as part of the JOP196/HOP045 program. The SUMER spectroscopic observation gives information about fluctuations in radiance and on both resolved (Doppler shift) and unresolved (Doppler width) line-of-sight velocities, whereas EIS 40'' wide slot images detect fluctuations only in radiance but maximize the probability of overlapping field of view between the two instruments. From distance–time radiance maps, we detect the presence of propagating waves in a polar inter-plume region with a period of 15–20 minutes and a propagation speed increasing from 130 ± 14 km s−1 just above the limb to 330 ± 140 km s−1 around 160'' above the limb. These waves can be traced to originate from a bright region of the on-disk part of the coronal hole where the propagation speed is in the range of 25 ± 1.3 to 38 ± 4.5 km s−1, with the same periodicity. These on-disk bright regions can be visualized as the base of the coronal funnels. The adjacent plume region also shows the presence of propagating disturbances with the same range of periodicity but with propagation speeds in the range of 135 ± 18 to 165 ± 43 km s−1 only. A comparison between the distance–time radiance map of the two regions indicates that the waves within the plumes are not observable (may be getting dissipated) far off-limb, whereas this is not the case in the inter-plume region. A correlation analysis was also performed to find out the time delay between the oscillations at several heights in the off-limb region, finding results consistent with those from the analysis of the distance–time maps. To our knowledge, this result provides first spectroscopic evidence of the acceleration of propagating disturbances in the polar region close to the Sun (within 1.2 R/R☉), which provides clues to the understanding of the origin of these waves. We suggest that the waves are likely either Alfvénic or fast magnetoacoustic in the inter-plume region and slow magnetoacoustic in the plume region. This may lead to the conclusion that inter-plumes are a preferred channel for the acceleration of the fast solar wind.

一种新的图案表面投影显示彩色重现的辐射度补偿算法

A novel radiometric compensation algorithm is proposed to correct the color distortions of the projection display on patterned surface. Two choices of the objective image are investigated in this algorithm. A linear compression followed by a sigmoid-function transformation based on the statistical parameters of the compensating radiance map is performed to reduce the clippings of the compensating color and expected to properly reproduce the dynamic range and the visibility of the image content. Three types of projection displays are produced for each test image. The parameter analysis indicates that the compensated displays could well conceal the surface pattern and tend to exhibit higher lightness or lightness contrast in the camera. The comparison results in a psychophysical experiment demonstrate that all the compensated displays outperform the uncompensated ones. The original input images seem more appropriate to be selected as the objective images, and the predicted objectives on the white screen are just proper for the bright and low-contrast input images displayed on the faint surface.

Recovering high dynamic range by Multi-Exposure Retinex

The matter of generating high dynamic range (HDR) image from a number of differently exposed pictures arises to satisfy the needs of high-quality imaging and industrial applications. A number of HDR image generation algorithms have been proposed in the past. However, the HDR radiance map recovered by these classical methods cannot completely exclude the noisy pixels in the input images and thus are unable to produce the optimal result with highest possible SNR. In this paper we are going to introduce a new HDR generation algorithm based on the Multi-Exposure Retinex model deduced in this paper for HDR image composition. The luminance component L and the reflectance R are synthesized independently before being combined together. A novel R image composition method is introduced to help the composed result image reach the highest possible SNR. The method is tested on grey-level images in this paper, but it can be easily extended to the color-image version.

Image enhancement for increased dot‐counting efficiency in FISH

The most commonly used molecular cytogenetic technique is fluorescence in situ hybridization (FISH). It has been widely applied in many areas of diagnosis and research, including pre-natal and post-natal screening of chromosomal aberrations, pre-implantation genetic diagnosis, cancer cytogenetics, gene mapping, molecular pathology and developmental molecular biology. The analysis of FISH images consists of detecting fluorescent dots, after which the number of dots per cell can be counted or their relative positions can be measured. A major impediment in the analysis of FISH specimens is signal (dot) quality, which is influenced by the hybridization efficiency and/or the sensitivity of the camera that records the images. In this paper, we present an approach to improve the efficiency of detecting fluorescent signals in FISH images by recovering the radiance map of the camera. This allows us to generate a high-dynamic-range image wherein an extended range of the sample radiance captured by the camera can be visualized at distinct intensity values. The resulting higher-order numeric complexity of the transformed image is adjusted (or simplified) by examining the intensity distribution in each of the three colour channels (red, green and blue), and remapping the intensity values to generate a high-contrast image with a lower-order (compressed) dynamic range. The remapping is based on a criterion that optimizes the detection of the hybridized signals, allowing attenuation of saturated intensity values while amplifying low-intensity signals. A simple dot-counting algorithm is used to automatically process 2000 FISH images. The images are taken for lymphocytes from cultured blood specimens for cytogenetic testing. Images are manually analyzed by an expert to obtain ground truth for dot counts. A quantitative analysis is performed by comparing results of automated dot detection on images before and after enhancement with the developed algorithms. In addition, common errors in dot counting due to split dots, dust, poor segmentation and overlapping signals are analyzed and the robustness of the developed approach against these errors evaluated. It is observed that dot-detection efficiency is increased by an average of 9% across all colour channels while reducing errors in missed and false dot counts. Our proposed method and results demonstrate that dot-counting specificity and sensitivity can be improved by pre-processing and enhancing the image using the radiance curve of the camera and generating a high-contrast, remapped high-dynamic-range image prior to using any algorithm for dot counting.

Absolute radiance imaging using parametric image amplification

We show that parametric image amplification can be used to achieve a 2D radiance map directly expressed in photons per spatiotemporal mode. Radiance images of incoherent signals with less than one photon per mode (typically 10(-2)) are resolved.

Emission heights of coronal bright points on Fe XII radiance map

The study of coronal bright points (BPs) is important for understanding coronal heating and the origin of the solar wind. Previous studies indicated that coronal BPs have a highly significant tendency to coincide with magnetic neutral lines in the photosphere. Here we further studied the emission heights of the BPs above the photosphere in the bipolar magnetic loops that are apparently associated with them. As BPs are seen in projection against the disk their true emission heights are unknown. The correlation of the BP locations on the Fe XII radiance map from EIT with the magnetic field features (in particular neutral lines) was investigated in detail. The coronal magnetic field was determined by an extrapolation of the photospheric field (derived from 2-D magnetograms obtained from the Kitt Peak observatory) to different altitudes above the disk. It was found that most BPs sit on or near a photospheric neutral line, but that the emission occurs at a height of about 5 Mm. Some BPs, while being seen in projection, still seem to coincide with neutral lines, although their emission takes place at heights of more than 10 Mm. Such coincidences almost disappear for emissions above 20 Mm. We also projected the upper segments of the 3-D magnetic field lines above different heights, respectively, on to the tangent x– y plane, where x is in the east–west and y in the south–north direction. The shape of each BP was compared with the respective field-line segment nearby. This comparison suggests that most coronal BPs are actually located on the top of their associated magnetic loops. Finally, we calculated for each selected BP region the correlation coefficient between the Fe XII intensity enhancement and the horizontal component of the extrapolated magnetic field vector at the same x– y position in planes of different heights, respectively. We found that for almost all the BP regions we studied the correlation coefficient, with increasing height, increases to a maximal value and then decreases again. The height corresponding to this maximum was defined as the correlation height, which for most bright points was found to range below 20 Mm.

Overlay model for infrared scene generation

A dynamic model of IR missile engagements must combine the output of signature models into a scene of given resolution with a changing viewpoint and moving targets against some background. Some signature prediction models are stand-alone software packages, which currently cannot be dynamically interfaced to a running engagement model. They can be used to conveniently provide an image of an IR target at high resolution at a single viewpoint. An algorithm to solve the radiance map composition problem and combine target, sea, and sky signatures can be thought of intuitively as an idealized imaging radiometer model. We present the derivation of such a model from physical and signal processing considerations and its practical implementation; show that under reasonable assumptions, the radiance integral takes the form of a particularly simple computation; and demonstrate that the model behaves reasonably with translucent targets. The derived methodology provides very high radiometric accuracy with a rigorously controlled error and smooth integration of objects moving through the scene.

High dynamic range video

Typical video footage captured using an off-the-shelf camcorder suffers from limited dynamic range. This paper describes our approach to generate high dynamic range (HDR) video from an image sequence of a dynamic scene captured while rapidly varying the exposure of each frame. Our approach consists of three parts: automatic exposure control during capture, HDR stitching across neighboring frames, and tonemapping for viewing. HDR stitching requires accurately registering neighboring frames and choosing appropriate pixels for computing the radiance map. We show examples for a variety of dynamic scenes. We also show how we can compensate for scene and camera movement when creating an HDR still from a series of bracketed still photographs.

Infrared surveillance 1:Statistical model

A new procedure is presented for calculating the performance of scanning background-limited IR sensors with adaptive threshold signal-processing logic. The IR background scene is presumably specified as a radiance map of arbitrary (but deterministic) structure. The nonstationary statistical nature of the photocur-rent is taken into account by combining some results from optical communications theory with a formalism for the threshold-crossing statistics of nonstationary noises. The present work is extended in a companion paper in which various simplifying approximations and illustrative numerical examples are presented. The theory developed in these two papers is applicable to design studies for IR passive surveillance devices.