RFG-HDR: Representative Feature-Guided Transformer For Multi-Exposure High Dynamic Range Imaging

Machine vision plays an important part in industrial online inspection. Owing to the nonuniform illuminance conditions and variable working distances, the captured image tends to be over-exposed or under-exposed. As a result, when processing the image such as crack inspection, the algorithm complexity and computing time increase. Multiexposure high dynamic range (HDR) image synthesis is used to improve the quality of the captured image, whose dynamic range is limited. Inevitably, camera shake will result in ghost effect, which blurs the synthesis image to some extent. However, existed exposure fusion algorithms assume that the input images are either perfectly aligned or captured in the same scene. These assumptions limit the application. At present, widely used registration based on Scale Invariant Feature Transform (SIFT) is usually time consuming. In order to rapidly obtain a high quality HDR image without ghost effect, we come up with an efficient Low Dynamic Range (LDR) images capturing approach and propose a registration method based on ORiented Brief (ORB) and histogram equalization which can eliminate the illumination differences between the LDR images. The fusion is performed after alignment. The experiment results demonstrate that the proposed method is robust to illumination changes and local geometric distortion. Comparing with other exposure fusion methods, our method is more efficient and can produce HDR images without ghost effect by registering and fusing four multi-exposure images.

High dynamic range (HDR) images can be generated by capturing a sequence of low dynamic range (LDR) images of the same scene with different exposures and then merging those images to create an HDR image. During capturing of LDR images, any changes in the scene or slightest camera movement results in ghost artifacts in the resultant HDR image. Over the past few years many algorithms have been proposed to produce ghost free HDR images of dynamic scenes. In this study we performed subjective psychophysical experiments to evaluate four algorithms for removing ghost artifacts in the final HDR image. To our best knowledge, no evaluation of deghosting algorithms for HDR imaging has been published. Thus, the aim of this paper is not only to evaluate different ghost removal algorithms but also to introduce a methodology to evaluate such algorithms and to present some of the challenges that exist in evaluating ghost removal algorithms in HDR images. Optical flow algorithms have been shown to produce successful results in aligning input images before merging them into an HDR image. As a result one of the state-of-the-art deghosting algorithm for HDR image alignment is based on optical flow. To test the limits of the evaluated deghosting algorithms the scenes used in our experiments were selected following the criteria proposed by Baker et al. [2011], which is considered as de facto standard for evaluating optical flow methodologies. The scenes used in the experiments serve to provide challenges that need to be dealt with by not only algorithms based on optical flow methodologies but also by other ghost removal algorithms for HDR imaging. The results reveal the scenes for which the evaluated algorithms fail and may serve as a guide for future research in this area.

Compared with the normal low dynamic range (LDR) images, the high dynamic range (HDR) images provide more dynamic range and image details. Although the existing techniques for generating the HDR images have a good effect for static scenes, they usually produce artifacts on the HDR images for dynamic scenes. In recent years, some learning-based approaches are used to synthesize the HDR images and obtain good results. However, there are also many problems, including the deficiency of explaining and the time-consuming training process. In this article, we propose a novel approach to synthesize multiview HDR images through fuzzy broad learning system (FBLS). We use a set of multiview LDR images with different exposure as input and transfer corresponding Takagi-Sugeno (TS) fuzzy subsystems; then, the structure is expanded in a wide sense in the "enhancement groups" which transfer from the TS fuzzy rules with nonlinear transformation. After integrating fuzzy subsystems and enhancement groups with the trained-well weight, the HDR image is generated. In FBLS, applying the incremental learning algorithm and the pseudoinverse method to compute the weights can greatly reduce the training time. In addition, the fuzzy system has better interpretability. In the learning process, IF-THEN fuzzy rules can effectively help the model to detect the artifacts and reject them in the final HDR result. These advantages solve the problem of existing deep-learning methods. Furthermore, we set up a new dataset of multiview LDR images with corresponding HDR ground truth to train our system. Our experimental results show that our system can synthesize high-quality multiview HDR images, which has a higher training speed than other learning methods.

The current HDR (High-Dynamic Range) images gets expensive display devices with low dynamic range of equipment problems, research objectives are presented methods for using ordinary camera fetching and displaying high dynamic range images. General three-color camera’s use is to obtain 3 different exposures of the same scene images, and binary image pyramid, followed by low-level image panning and rotation registration step by step, using HDR Darkroom Photomatix software obtains high dynamic range images ,tone mapping and detail enhancement, using Photoshop software to fine-tune to get the final high-dynamic range images. Visual evaluation and instrumental measurements shows the synthesis of high dynamic range images can increase reflects the brightness of the scene, details and colour information, application and promotion of the value of the method.

Multi-exposure high dynamic range imaging with informative content enhanced network

Rate–distortion optimization of multi-exposure image coding for high dynamic range image coding

To acquire a high dynamic range (HDR) image of a scene, several low dynamic range (LDR) images acquired from a digital camera with different exposure times are generally fused into one HDR image to cover the entire dynamic range of the scene. However, when capturing a scene, the scene dynamic range (SDR) is unknown. Consequently, the exposure times for the LDR images need to be as varied as possible to cover the unknown SDR. This paper proposes a method to estimate the SDR using two LDR im- ages. Using the SDR information, SDR-adaptive exposure times can then be selected to achieve the optimal HDR image. The SDR is defined as two exposure times when captured LDR images are mar- ginally clipped to black and white, indicating the lower and upper limits of the SDR, respectively. To identify these times, two LDR images, an overexposed and an underexposed image, are cap- tured. Using the opto-electronic conversion function of the camera used, the minimum gray level in the overexposed image is then used to estimate the exposure time to make the minimum gray level of the image just black, while the maximum gray level in the under- exposed image is used to estimate the exposure time to make the maximum gray level of the image just white. By evaluating the ac- quired HDR image error according to the exposure times of fused LDR images for various scenes, SDR-adaptive exposure times to acquire an optimal HDR image with the minimal error are selected. Experiments confirm that the quality of an HDR image based on fusing LDR images with the proposed SDR-adaptive exposure times is similar to that of an HDR image based on fusing LDR images with conventionally chosen exposure times, even though the number of LDR images used to acquire the HDR image with the proposed method is much smaller than that used by the conventional method. © 2009 Society for Imaging Science and Technology. DOI: 10.2352/J.ImagingSci.Technol.2009.53.2.020505

Multi-exposure high dynamic range (HDR) imaging aims to generate an HDR image from multiple differently exposed low dynamic range (LDR) images. It is a challenging task due to two major problems: (1) there are usually misalignments among the input LDR images, and (2) LDR images often have incomplete information due to under-/over-exposure. In this paper, we propose a disentangled feature-guided HDR network (DFGNet) to alleviate the above-stated problems. Specifically, we first extract and disentangle exposure features and spatial features of input LDR images. Then, we process these features through the proposed DFG modules, which produce a high-quality HDR image. Experiments show that the proposed DFGNet achieves outstanding performance on a benchmark dataset. Our code and more results are available at https://github.com/KeuntekLee/DFGNet.

High Dynamic Range (HDR) images are widely used in automotive, aerospace, AI, and other fields but are limited by the maximum dynamic range of a single data acquisition using CMOS image sensors. High dynamic range images are usually synthesized through multiple exposure techniques and image processing techniques. One of the most challenging task in multiframe Low Dynamic Range (LDR) images fusion for HDR is to eliminate ghosting artifacts caused by motion. In traditional algorithms, optical flow is generally used to align dynamic scenes before image fusion, which can achieve good results in cases of small-scale motion scenes but causes obvious ghosting artifacts when motion magnitude is large. Recently, attention mechanisms have been introduced during the alignment stage to enhance the network’s ability to remove ghosts. However, significant ghosting artifacts still occur in some scenarios with large-scale motion or oversaturated areas. We proposea novel Distilled Feature TransformerBlock (DFTB) structure to distill and re-extract information from deep image features obtained after U-Net downsampling, achieving ghost removal at the semantic level for HDR fusion. We introduce a Feature Distillation Transformer Block (FDTB), based on the Swin-Transformer and RFDB structure. FDTB uses multiple distillation connections to learn more discriminative feature representations. For the multiexposure moving scene image fusion HDR ghost removal task, in the previous method, the use of deep learning to remove the ghost effect in the composite image has been perfect, and it is almost difficult to observe the ghost residue of moving objects in the composite HDR image. The method in this paper focuses more on how to save the details of LDR image more completely after removing the ghost to synthesize high-quality HDR image. After using the proposed FDTB, the edge texture details of the synthesized HDR image are saved more perfectly, which shows that FDTB has a better effect in saving the details of image fusion. Futhermore, we propose a new depth framework based on DFTB for fusing and removing ghosts from deep image features, called TransU-Fusion. First of all, we use the encoder in U-Net to extract image features of different exposures and map them to different dimensional feature spaces. By utilizing the symmetry of the U-Net structure, we can ultimately output these feature images as original size HDR images. Then, we further fuse high-dimensional space features using Dilated Residual Dense Block (DRDB) to expand the receptive field, which is beneficial for repairing over-saturated regions. We use the transformer in DFTB to perform low-pass filtering on low-dimensional space features and interact with global information to remove ghosts. Finally, the processed features are merged and output as an HDR image without ghosting artifacts through the decoder. After testing on datasets and comparing with benchmark and state-of-the-art models, the results demonstrate our model’s excellent information fusion ability and stronger ghost removal capability.

In this paper, a light consistency solution for generating high dynamic range (HDR) images based on a single low dynamic range image(LDR) is proposed, and the virtual object is rendered by illumination. The solution can reduce the time of image acquisition and processing, and solve the problems caused by the limitations of image acquisition equipment. The solution is divided into three stages: image preprocessing, high dynamic range image generation and virtual object relighting. Firstly, in the stage of image pretreatment, the wavelet noise reduction method based on a Gaussian mixture model is used to remove image noise and avoid image detail distortion. The inverse camera response function is utilized to linearize the image, the pixel brightness range is expanded based on the inverse tone mapping function, and the threshold segmentation method is combined with flooding Gaussian smoothing to calculate the highlight spread diagram to compensate for scene highlights lost during camera shooting. Then, the extended dynamic range image is interpolated linearly by using the specular expansion image to get the high dynamic range image. Based on the analysis and experimental simulation, compared with other methods, it is found that using a single low-dynamic-range image can greatly reduce the time of image acquisition and processing and reduce the limitations of image acquisition equipment, while maintaining good light fusion. Based on the simulation results, the efficiency of light consistency processing is improved.

High dynamic range imaging is an important field in computer vision. Compared with general low dynamic range (LDR) images, high dynamic range (HDR) images represent a larger luminance range, making the images closer to the real scene. In this paper, we propose an approach for HDR image reconstruction from a single LDR image based on histogram learning. First, the dynamic range of an LDR image is expanded to an extended dynamic range (EDR) image. Then, histogram learning is established to predict the intensity distribution of an HDR image of the EDR image. Next, we use histogram matching to reallocate pixel intensities. The final HDR image is generated through regional adjustment using reinforcement learning. By decomposing low-frequency and high-frequency information, the proposed network can predict the lost high-frequency details while expanding the intensity ranges. We conduct the experiments based on HDR-Real and HDR-EYE datasets. The quantitative and qualitative evaluations have demonstrated the effectiveness of the proposed approach compared to the previous methods.

In this paper, a high dynamic range (HDR) imaging method based on the stereo vision system is presented. The proposed method uses differently exposed low dynamic range (LDR) images captured from a stereo camera. The stereo LDR images are first converted to initial stereo HDR images using the inverse camera response function estimated from the LDR images. However, due to the limited dynamic range of the stereo LDR camera, the radiance values in under/over-exposed regions of the initial main-view (MV) HDR image can be lost. To restore these radiance values, the proposed stereo matching and hole-filling algorithms are applied to the stereo HDR images. Specifically, the auxiliary-view (AV) HDR image is warped by using the estimated disparity between initial the stereo HDR images and then effective hole-filling is applied to the warped AV HDR image. To reconstruct the final MV HDR, the warped and hole-filled AV HDR image is fused with the initial MV HDR image using the weight map. The experimental results demonstrate objectively and subjectively that the proposed stereo HDR imaging method provides better performance compared to the conventional method.

Feature Point (FP) detection is a fundamental step in computer vision tasks. Although FP detectors are mostly designed to support Low Dynamic Range (LDR) images as input, interest in High Dynamic Range (HDR) images has increased recently due to their higher precision to register overexposed and underexposed areas in an image. As the detection of FPs is strongly dependent on the illumination of the scene, HDR images have the potential to be more robust than LDR images during FP detection. Known FP detectors, however, do not use the full potential of HDR images. In addition, few works have evaluated the performance of HDR images in this context. In this paper, we propose a modification of FP detectors aiming to improve FP detection on HDR images. To this end, we design a local mask based on the Coefficient of Variation (CV) of sets of pixels, creating thus a new step in FP detection. We compare our approach with popular FP detection methods using a standard evaluation metric, Repeatability Rate (RR) of FPs, and also Uniformity as a proposed new criterion. A dataset of images from scenes affected by camera transformations and substantial illumination changes was used as input. Experimental results show that our proposed algorithms give better Uniformity and RR in most HDR images from the dataset when compared to standard FP detectors. Moreover, they indicate that HDR images present a great potential to be explored in applications that rely on FP detection.

Real world scenes contain a large range of light intensities. To adapt to display device, High Dynamic Range (HDR) image should be converted into Low Dynamic Range (LDR) image. A common task of tone mapping algorithms is to reproduce high dynamic range images on low dynamic range display devices. In this paper, a new tone mapping algorithm is proposed for high dynamic range images. Based on the probabilistic model is proposed for high dynamic image's tone reproduction, the proposed method uses a logarithmic normal distribution instead of normal distribution. Therefore, the algorithm can preserve visibility and contrast impression of high dynamic range scenes in the common display devices. Experimental results show the superior performance of the app roach in terms of visual quality.

Current research works on high dynamic range (HDR) images put emphasis on the perception quality of the reconstructed image, where an enhanced low dynamic range (LDR) image is directly output as an HDR image from a sequence of LDR images. These works are useful to improve the limited ability of display devices. However, the dynamic range is not actually expanded and the physical properties of real scenes are unavailable in these works. For example, the radiance map of the surrounding scene cannot be recovered in such a direct way, which is an important issue in many industrial and aerospace applications. This paper proposes an efficient synthesizing and displaying system for HDR images. It focus on providing solutions of the following open problems: 1) LDR image registration under camera shaking and object motion; 2) HDR image reconstruction for physical property analysis of real scenes; 3) Structure preservation when compressing dynamic range of HDR image for LDR display devices.KeywordsMotion compensationCamera response functionHigh dynamic range image synthesisTone-Mapping