Wide Dynamic Range Image Research Articles

Retinex Jointed Multiscale CLAHE Model for HDR Image Tone Compression

Tone-mapping algorithms aim to compress a wide dynamic range image into a narrower dynamic range image suitable for display on imaging devices. A representative tone-mapping algorithm, Retinex theory, reflects color constancy based on the human visual system and performs dynamic range compression. However, it may induce halo artifacts in some areas or degrade chroma and detail. Thus, this paper proposes a Retinex jointed multiscale contrast limited adaptive histogram equalization method. The proposed algorithm reduces localized halo artifacts and detail loss while maintaining the tone-compression effect via high-scale Retinex processing. A performance comparison of the experimental results between the proposed and existing methods confirms that the proposed method effectively reduces the existing problems and displays better image quality.

Wide dynamic range and real-time reagent identification and imaging using multi-wavelength terahertz parametric generation and machine learning

In this study, we propose a technique for identifying and imaging reagents through shielding over a wide dynamic range using a real-time terahertz (THz) spectroscopy system with multi-wavelength THz parametric generation/detection and machine learning. To quickly identify reagents through shielding, the spectral information of the “detection Stokes beam” is used for reagent recognition via machine learning. In general THz wave-based reagent identification, continuous spectra are acquired and analyzed quantitatively by post-processing. In actual applications, however, such as testing for illicit drugs in mail, the technology must be able to quickly identify reagents as opposed to quantifying the amount present. In multi-wavelength THz parametric generation/detection, THz spectral information can be measured instantly using a “multi-wavelength detection Stokes beam” and near-infrared (NIR) camera. Moreover, machine learning enables reagent identification in real-time and over a wide dynamic range. Furthermore, by plotting the identification results as pixel values, the spatial distribution of reagents can be imaged at high speed without the need for post-processing.

Crystalline Selenium Layer Stacked Image Sensors With Pixel-Parallel A/D Converters Toward Photon Counting

We report crystalline selenium (c-Se) photo-conversion layer stacked image sensors with pixel-parallel A/D converters (ADCs) toward photon counting. The sensor separates photo-conversion and multiplication function from complementary metal–oxide–semiconductor (CMOS) circuits, which has advantages in terms of high sensitivity and high spatial resolution of the sensors. The sensor also has a wide dynamic range for input illuminance because the pixel-parallel 1-bit ADCs prevent saturation of the floating diffusion (FD). The circuits require no quenching operation, which is suitable for high-speed operation. The c-Se layer is directly stacked onto the CMOS sensor surface, which has been flattened by chemical mechanical polishing, and connected to each pixel with Au electrodes. A photo-generated electron charge can be multiplied in the c-Se layers, and introduced to 1-bit ADC, where the charges are converted into pulse signals to be counted by in-pixel counters. Linear output characteristics are confirmed with a wide dynamic range for illuminance of 96 dB. Video images of 128 <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\times96$ </tex-math></inline-formula> pixels are successfully demonstrated including avalanche multiplication mode in the supply voltage of more than 10 V, which exhibits the feasibility of high spatial resolution and high-speed image sensing applicable to photon counting and wide dynamic range imaging.

CMOS image sensor for wide dynamic range feature extraction in machine vision

This letter presents a wide dynamic range (WDR) feature extraction (FE) readout scheme for machine vision applications using CMOS image sensors (CISs). The proposed scheme with the proposed pixel structure has two operating modes, the normal and WDR modes. In the normal operating mode, the proposed CIS captures a normal image with high sensitivity. In addition, as a unique function, a bi-level image is obtained for real-time FE even if a pixel is saturated in strong illumination conditions. Thus, compared to typical CISs for machine vison, the proposed CIS can reveal object features that are blocked by light in real time. In the WDR operating mode, the proposed CIS produces a WDR image with its corresponding bi-level image. A prototype CIS was fabricated using a standard 0.35-μm 2P4M CMOS process with a 320 × 240 format (QVGA) with 10-μm pitch pixels. At 60 fps, the measured power consumption was 5.98 mW at 3.3 V for pixel readout and 2.8 V for readout circuitry. The dynamic range of 73.1 dB was achieved in the WDR operating mode.

On Wide Dynamic Range Tone Mapping CMOS Image Sensor

The dynamic range of a natural scene often covers over 6 decades of intensity from bright to dark areas. Typical image sensors, however, have limited ability to capture this dynamic range available in nature. Even after designing specific wide dynamic range (WDR) image sensors, displaying them on conventional media with limited ability requires computationally complex tone mapping.This paper proposed a novel CMOS pixel which can capture and perform tone mapping during data acquisition. The pixel requires a reference voltage to generate tone mapped response. A number of different reference signals are proposed and generated which can perform WDR operation. Nevertheless, fixed pattern noise (FPN) effects the performance of these pixel. A pixel model with simple parameter extraction procedure is described for a typical tone mapping operator. This model is then used to obtain a simple procedure for pixel calibration leading to reduced FPN. The new proposed pixel response is able to capture upto 6 decades of light intensity and reported FPN correction procedure produces 1% of FPN contrast error.

A Wide Dynamic Range CMOS Image Sensor with a Charge Splitting Gate and Two Storage Diodes.

In this paper, a wide dynamic range (WDR) CMOS image sensor (CIS) with a charge splitting gate (SG) and two storage diodes (SDs) is presented. By using single-gate on/off control with the SG, photocurrent path to the first (SD1) or second storage diodes (SD2) is switched alternatively and periodically during exposure and signal electrons generated in a photodiode (PD) are transferred to and accumulated in the SD1 or SD2. By setting a large ratio of the off-time to on-time of the SG, two different sensitivity signals, which are originated by the same photodiode, are generated and a WDR image signal is obtained. This technique has a distinct advantage on mitigating the problem of motion artifact in WDR imaging with high and low sensitivity signals and flexible dynamic control of the dynamic range. An experimental WDR CMOS image sensor with 280 (H) × 406 (V)-pixel array consisting of 14 sub-arrays, each of which have 20 (H) × 406 (V) pixels, was implemented and tested. For the SG on/off-time ratio of 30 and 279, the DR of 93 dB and 104 dB, respectively, was demonstrated. The effect of the proposed WDR imaging operation on the reduced motion artifact was experimentally confirmed.

Mantissa-Exponent-Based Tone Mapping for Wide Dynamic Range Image Sensors

The dynamic range of a scene is defined as the ratio between the maximum and minimum luminance in it. Wide dynamic range (WDR) means this ratio is so large that it exceeds the dynamic range of a traditional image sensor. Nowadays, WDR image sensors enable the capture of WDR scenes. However, the captured WDR image requires an additional tone mapping step to compress the high bit pixel of WDR image to low rate pixel so that it can be displayed on the screen. The tone mapping algorithm is mostly done in an image signal processor or with a specific software application. This brief proposes a tone mapping technique that is suitable for direct processing of the output of a WDR image sensor bitstream. The algorithm acquires statistics on the mantissa and exponent parts of the pixel value and then generates a refined histogram for tone mapping. Experiments that evaluate the image quality and hardware efficiency are carried out. The results indicate that the proposed mantissa exponent-based algorithm provides visually pleasing results and preserves details of the original WDR image better than other similar algorithms. The hardware resources’ efficiency of the algorithm makes the system on chip implementation possible.

Spike Coding for Dynamic Vision Sensor in Intelligent Driving

Dynamic vision sensor (DVS) as a bio-inspired camera, has shown great advantages in wide dynamic range and high temporal resolution imaging in contrast to conventional frame-based cameras. Its ability to capture high speed moving objects enables fast and accurate detection which plays a significant role in the emerging intelligent driving applications. The pixels in DVS independently respond to the luminance changes with output spikes. Thus, the spike stream conveying the ${x}$ -, ${y}$ -addresses, the firing time, and the polarity (ON/OFF), is quite different from conventional video frames. How to compress this kind of new data for efficient transmission and storage remains a big challenge, especially for on-board detection, monitoring and recording. To address this challenge, this paper first analyzes the spike firing mechanism and the spatiotemporal characteristics of the spike data, then introduces a cube-based spike coding framework for DVS. In the framework, an octree-based structure is proposed to adaptively partition the spike stream into coding cubes in both spatial and temporal dimensions, then several prediction modes are designed to exploit the spatial and temporal characteristics of spikes for compression, including address-prior mode and time-prior mode. To explore more flexibility, the intercube prediction is discussed extensively involving motion estimation and motion compensation. Finally, the experimental results demonstrate that our approach achieves an impressive coding performance with the average compression ratio of 2.6536 against the raw spike data, which is much higher than the results of conventional lossless coding algorithms.

광역 영상 재생을 위한 순응 밝기 특성 기반의 다중 광도 영상 합성 기법

HDR 영상 합성은 넓은 휘도 범위의 장면을 촬영하여 디스플레이에 실제 시각적인 장면을 재현하기 위한 방법이다. 일반적으로 HDR 영상은 다양한 카메라 노출 상태에서 영상을 촬영하고 여러 영상을 톤 합성하여 재생된다. 먼저 단일 HDR 영상을 구성하기 위해 각 다중 노출 영상을 방사 맵으로 변환한다. 이를 위해 여러 영상을 노출 비에 따라 혼합하거나 카메라 전달 함수를 유도하여 합성한다. 다음 합성된 방사 맵을 톤 맵핑함수를 이용해 시각적인 HDR 영상으로 재생해낸다. 기존 방법은 CRF 조사를 위해 기준 표시 샘플과 다수의 노출 영상을 필요로 하고, 여러 계층의 합성 단계를 거침으로 인해 오류가 누적되는 단점을 보였다. 본 논문에서는 두 개의 노출 영상을 이용하는 시각 밝기 함수 기반의 HDR 영상 합성 알고리즘을 제안하였다. 제안된 알고리즘은 기존 방법과 비교하여 경계부 오류 및 색 밸런스를 개선하고 또한 영상 혼합으로 인한 블러링 현상과 노이즈 증폭 현상을 개선하였다.

Reconfigurable hybrid vision enhancement system using tone mapping and adaptive gamma correction algorithm for night surveillance robot

Night vision system has become a critical component of modern warfare and the ability to see in nighttime conditions allows military maneuvers and a potential advantage to the forces equipped with this technology. These night vision systems rely on the very low light levels of night sky illumination to help image the targeted scene and its surroundings. Many research works have been undertaken to overcome issues in hardware implementation. In this paper, we contribute to enhance night vision sources by hybrid vision enhancement (HVE) system without affecting performance of hardware implementation. The proposed hybrid system consists of two algorithms such asoptimizedtone mapping (OTM) and adaptive gamma correction (AGC) algorithm. Normally, hybrid systems are not an area efficient, here we modify the tone mapping algorithm byoptimized filter design with the exponential basis. The differential evolution optimization algorithm is used to enhance the filter design. The proposed HVE system implementation is designed in Verilog language and synthesized with different FPGA device families in Xilinx tool. Simulation result shows that our proposed HVE system is able to enhance vision of wide dynamic range (WDR) images to good visual quality. The synthesis result shows that our proposed HVE system perform very efficient than existing system in terms of hardware utilization, maximum clock frequency, and power.

Local tone mapping algorithm and hardware implementation

A novel tone mapping algorithm and hardware implementation for displaying wide dynamic range (WDR) images are proposed. The algorithm processes WDR images in a pixel-by-pixel fashion in the logarithmic domain, and it uses the block-interpolated minimum and maximum pixel values. The hardware implementation can achieve real-time processing of WDR images and is resource-efficient. Experimental results show that the proposed algorithm and hardware implementation can produce images with good brightness and high contrast.

広ダイナミックレンジ撮像のための適応蓄積時間イメージセンサの設計・試作(高機能イメージセンサ)

広ダイナミックレンジ撮像のための適応蓄積時間イメージセンサの設計・試作(高機能イメージセンサ)

5-1 可変しきい値を用いたリニア出力型広ダイナミックレンジ撮像デバイス

5-1 可変しきい値を用いたリニア出力型広ダイナミックレンジ撮像デバイス

An FPGA implementation of a tone mapping algorithm with a halo-reducing filter

In this paper, we present a real-time hardware implementation of an exponent-based tone mapping algorithm of Hore et al., that uses both local and global image information for improving the contrast and increasing the brightness of tone-mapped images. Although there are several tone mapping algorithms available in the literature, most of them require manual tuning of their rendering parameters. However, in our implementation, the algorithm has an embedded automatic key parameter estimation block that controls the brightness of the tone-mapped images. We also present the implementation of a Gaussian-based halo-reducing filter. The hardware implementation is described in Verilog and synthesized for a field programmable gate array device. Experimental results performed on different wide dynamic range images show that we are able to get images which are of good visual quality and have good brightness and contrast. The good performance of our hardware architecture is also confirmed quantitatively with the high peak signal-to-noise ratio and structural similarity index.

Using polynomials to simplify fixed pattern noise and photometric correction of logarithmic CMOS image sensors.

An important class of complementary metal-oxide-semiconductor (CMOS) image sensors are those where pixel responses are monotonic nonlinear functions of light stimuli. This class includes various logarithmic architectures, which are easily capable of wide dynamic range imaging, at video rates, but which are vulnerable to image quality issues. To minimize fixed pattern noise (FPN) and maximize photometric accuracy, pixel responses must be calibrated and corrected due to mismatch and process variation during fabrication. Unlike literature approaches, which employ circuit-based models of varying complexity, this paper introduces a novel approach based on low-degree polynomials. Although each pixel may have a highly nonlinear response, an approximately-linear FPN calibration is possible by exploiting the monotonic nature of imaging. Moreover, FPN correction requires only arithmetic, and an optimal fixed-point implementation is readily derived, subject to a user-specified number of bits per pixel. Using a monotonic spline, involving cubic polynomials, photometric calibration is also possible without a circuit-based model, and fixed-point photometric correction requires only a look-up table. The approach is experimentally validated with a logarithmic CMOS image sensor and is compared to a leading approach from the literature. The novel approach proves effective and efficient.

An Energy/Illumination-Adaptive CMOS Image Sensor With Reconfigurable Modes of Operations

We present an energy/illumination-adaptive CMOS image sensor for distributed wireless sensor applications. The adaptive feature enables always-on imaging operation with extremely low power consumption for extended lifetime of wireless image sensor nodes and provides optimum images in a wide range of illuminations. For adaptive operation, the sensor employs reconfigurable modes of operation. Most of time, the sensor is in a monitoring mode, which keeps imaging at extremely low power consumption. The sensor turns into a high-sensitivity imaging mode or a wide dynamic range imaging mode when illumination varies and sufficient power supply is available from energy harvesting. The sensor changes its operation back to the monitoring mode in order to save energy in the battery. The sensor operates at 1.36 μW/frame in the monitoring mode from harvested energy and provides high-sensitive (24 V/lx·sec) and wide dynamic range images (99.2 dB) at 867 μW in battery operation. The chip achieved power FOM of 15.4 pW/pixel·frame in 0.18 μm technology.

High-dynamic-range coherent diffractive imaging: ptychography using the mixed-mode pixel array detector.

Coherent (X-ray) diffractive imaging (CDI) is an increasingly popular form of X-ray microscopy, mainly due to its potential to produce high-resolution images and the lack of an objective lens between the sample and its corresponding imaging detector. One challenge, however, is that very high dynamic range diffraction data must be collected to produce both quantitative and high-resolution images. In this work, hard X-ray ptychographic coherent diffractive imaging has been performed at the P10 beamline of the PETRA III synchrotron to demonstrate the potential of a very wide dynamic range imaging X-ray detector (the Mixed-Mode Pixel Array Detector, or MM-PAD). The detector is capable of single photon detection, detecting fluxes exceeding 1 × 10(8) 8-keV photons pixel(-1) s(-1), and framing at 1 kHz. A ptychographic reconstruction was performed using a peak focal intensity on the order of 1 × 10(10) photons µm(-2) s(-1) within an area of approximately 325 nm × 603 nm. This was done without need of a beam stop and with a very modest attenuation, while `still' images of the empty beam far-field intensity were recorded without any attenuation. The treatment of the detector frames and CDI methodology for reconstruction of non-sensitive detector regions, partially also extending the active detector area, are described.

Flicker removal for CMOS wide dynamic range imaging based on alternating current component analysis

This paper presents a novel wide dynamic range system to reduce flickering artifact produced by a high speed electronic rolling shutter in a CMOS image sensor. The proposed algorithm removes flicker artifact in the shortexposure image using a correlation between the pair of two differently exposed images to extend the dynamic range. Since the proposed method overcomes the electronic shutter problem in a conventional wide dynamic range imaging system, it can remove the flicker artifact under an alternating current light source such as a fluorescent bulb.

Wide Dynamic Range Technologies: For mobile imaging sensor systems

This article presents several wide dynamic range (WDR) technologies for CMOS image sensors. In photography, dynamic range is the range between the maximum and minimum measurable light intensities. The varying degree of light intensity depends on the device that is used as a capture device, which decides the overall performance of a dynamic range of an imaging sensor. WDR technology increases the dynamic range of an imaging sensor by physically improving a pixel performance or digitally applying multiple exposure times for each pixel. This enables users to easily take a picture in case of a backlit, indoor, or tunnel scene, all of which contain mixed high- and low-light conditions. We will review WDR technologies and describe our efforts on a mobile imaging sensor system considering pixel technology and digital technology supporting the performance of a WDR imaging sensor.

Hardware Implementation of an Automatic Rendering Tone Mapping Algorithm for a Wide Dynamic Range Display

Tone mapping algorithms are used to adapt captured wide dynamic range (WDR) scenes to the limited dynamic range of available display devices. Although there are several tone mapping algorithms available, most of them require manual tuning of their rendering parameters. In addition, the high complexities of some of these algorithms make it difficult to implement efficient real-time hardware systems. In this work, a real-time hardware implementation of an exponent-based tone mapping algorithm is presented. The algorithm performs a mixture of both global and local compression on colored WDR images. An automatic parameter selector has been proposed for the tone mapping algorithm in order to achieve good tone-mapped images without manual reconfiguration of the algorithm for each WDR image. Both algorithms are described in Verilog and synthesized for a field programmable gate array (FPGA). The hardware architecture employs a combination of parallelism and system pipelining, so as to achieve a high performance in power consumption, hardware resources usage and processing speed. Results show that the hardware architecture produces images of good visual quality that can be compared to software-based tone mapping algorithms. High peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) scores were obtained when the results were compared with output images obtained from software simulations using MATLAB.