Camera Response Research Articles

An Optimized LIME Scheme for Medical Low Light Level Image Enhancement.

The role of medical image technology in the medical field is becoming more and more obvious. Doctors can use medical image technology to more accurately understand the patient's condition and make accurate judgments and diagnosis and treatment in order to make a large number of medical blurred images clear and easy to identify. Inspired by the human vision system (HVS), we propose a simple and effective method of low-light image enhancement. In the proposed method, first a sampler is used to get the optimal exposure ratio for the camera response model. Then, a generator is used to synthesize dual-exposure images that are well exposed in the regions where the original image is underexposed. Next, the enhanced image is processed by using a part of low-light image enhancement via the illumination map estimation (LIME) algorithm, and the weight matrix of the two images will be determined when fusing. After that, the combiner is used to get the synthesized image with all pixels well exposed, and finally, a postprocessing part is added to make the output image perform better. In the postprocessing part, the best gray range of the image is adjusted, and the image is denoised and recomposed by using the block machine 3-dimensional (BM3D) model. Experiment results show that the proposed method can enhance low-light images with less visual information distortions when compared with those of several recent effective methods. When it is applied in the field of medical images, it is convenient for medical workers to accurately grasp the details and characteristics of medical images and help medical workers analyze and judge the images more conveniently.

Representing Camera Response Function by a Single Latent Variable and Fully Connected Neural Network

Modelling the mapping from scene irradiance to image intensity is essential for many computer vision tasks. Such mapping is known as the camera response. Most digital cameras use a nonlinear function to map irradiance, as measured by the sensor to an image intensity used to record the photograph. Modelling of the response is necessary for the nonlinear calibration. In this paper, a new high-performance camera response model that uses a single latent variable and fully connected neural network is proposed. The model is produced using unsupervised learning with an autoencoder on real-world (example) camera responses. Neural architecture searching is then used to find the optimal neural network architecture. A latent distribution learning approach was introduced to constrain the latent distribution. The proposed model achieved state-of-the-art CRF representation accuracy in a number of benchmark tests, but is over twice as fast as the best current models when performing the maximum likelihood estimation during camera response calibration due to the simple yet efficient model representation.

Accuracy and Reproducibility of Above-Water Radiometry With Calibrated Smartphone Cameras Using RAW Data

Consumer cameras, especially on smartphones, are popular and effective instruments for above-water radiometry. The remote sensing reflectance Rrs is measured above the water surface and used to estimate inherent optical properties and constituent concentrations. Two smartphone apps, HydroColor and EyeOnWater, are used worldwide by professional and citizen scientists alike. However, consumer camera data have problems with accuracy and reproducibility between cameras, with systematic differences of up to 40% in intercomparisons. These problems stem from the need, until recently, to use JPEG data. Lossless data, in the RAW format, and calibrations of the spectral and radiometric response of consumer cameras can now be used to significantly improve the data quality. Here, we apply these methods to above-water radiometry. The resulting accuracy in Rrs is around 10% in the red, green, and blue (RGB) bands and 2% in the RGB band ratios, similar to professional instruments and up to 9 times better than existing smartphone-based methods. Data from different smartphones are reproducible to within measurement uncertainties, which are on the percent level. The primary sources of uncertainty are environmental factors and sensor noise. We conclude that using RAW data, smartphones and other consumer cameras are complementary to professional instruments in terms of data quality. We offer practical recommendations for using consumer cameras in professional and citizen science.

ADOP

In this paper we present ADOP, a novel point-based, differentiable neural rendering pipeline. Like other neural renderers, our system takes as input calibrated camera images and a proxy geometry of the scene, in our case a point cloud. To generate a novel view, the point cloud is rasterized with learned feature vectors as colors and a deep neural network fills the remaining holes and shades each output pixel. The rasterizer renders points as one-pixel splats, which makes it very fast and allows us to compute gradients with respect to all relevant input parameters efficiently. Furthermore, our pipeline contains a fully differentiable physically-based photometric camera model, including exposure, white balance, and a camera response function. Following the idea of inverse rendering, we use our renderer to refine its input in order to reduce inconsistencies and optimize the quality of its output. In particular, we can optimize structural parameters like the camera pose, lens distortions, point positions and features, and a neural environment map, but also photometric parameters like camera response function, vignetting, and per-image exposure and white balance. Because our pipeline includes photometric parameters, e.g. exposure and camera response function, our system can smoothly handle input images with varying exposure and white balance, and generates high-dynamic range output. We show that due to the improved input, we can achieve high render quality, also for difficult input, e.g. with imperfect camera calibrations, inaccurate proxy geometry, or varying exposure. As a result, a simpler and thus faster deep neural network is sufficient for reconstruction. In combination with the fast point rasterization, ADOP achieves real-time rendering rates even for models with well over 100M points. https://github.com/darglein/ADOP

Physics-Based TOF Imaging Simulation for Space Targets Based on Improved Path Tracing

Aiming at the application of close-up space measurement based on time-of-flight (TOF) cameras, according to the analysis of the characteristics of the space background environment and the imaging characteristics of the TOF camera, a physics-based amplitude modulated continuous wave (AMCW) TOF camera imaging simulation method for space targets based on the improved path tracing is proposed. Firstly, the microfacet bidirectional reflection distribution function (BRDF) model of several typical space target surface materials is fitted according to the measured BRDF data in the TOF camera response band to make it physics-based. Secondly, an improved path tracing algorithm is developed to adapt to the TOF camera by introducing a cosine component to characterize the modulated light in the TOF camera. Then, the imaging link simulation model considering the coupling effects of the BRDF of materials, the suppression of background illumination (SBI), optical system, detector, electronic equipment, platform vibration, and noise is established, and the simulation images of the TOF camera are obtained. Finally, ground tests are carried out, and the test shows that the relative error of the grey mean, grey variance, depth mean, and depth variance is 2.59%, 3.80%, 18.29%, and 14.58%, respectively; the MSE, SSIM, and PSNR results of our method are also better than those of the reference method. The ground test results verify the correctness of the proposed simulation model, which can provide image data support for the ground test of TOF camera algorithms for space targets.

Diverse image enhancer for complex underexposed image

The visual appearance of images changes in line with varying environmental light conditions. Adjusting the exposure of various distorted images is a highly complex process. Previous approaches have addressed this issue from different viewpoints and attained remarkable progress. However, they either failed to achieve visually pleasing results or were suitable for a single class of images (e.g., underexposed or nonuniform images). To fully consider the exposure in various distorted images, we proposed a diverse image enhancement model that improved the brightness and contrast, processed the colors, and eliminated the hazy effect. Accordingly, an input red green blue color image was transformed into a hue, saturation, value color image. The V component was inverted and enhanced using three steps. In the first step, the hyperbolic and statistical methods were applied, and then their results were combined using an adjusted logarithmic methodology. This method properly adjusted the high-contrast and low-contrast impact while preserving the vital image information. In the second step, the output of the first step was inverted back and fed into a complete optimization algorithm to estimate the illumination map. Then, the exposure ratio map was estimated using an illumination map, which was adjusted using the camera response function. In the third step, a nonlinear stretching function was introduced to control brightness and contrast. For instance, a lower value of α yielded maximum stretching, and a higher value of α eliminated haze in the image to a great extent. Finally, an empirical evaluation and comparison of the most recent state-of-the-art approaches on eight datasets revealed that the proposed model efficiently addressed the exposure in various degraded images.

Monte Carlo modelling of a compact CZT-based gamma camera with application to 177Lu imaging

BackgroundSemiconductor gamma-camera systems based on cadmium zinc telluride (CZT) detectors present new challenges due to an energy-response that includes effects of low-energy tailing. In particular, such energy tails produce effects that need to be considered when imaging radionuclides with multiple emissions such as ^{177}{mathrm {Lu}}. Monte Carlo simulation can be used to investigate the behaviour of such systems and optimise their use, provided that the detector model closely reflects the real physical detector. The aim of this work is to develop a CZT model applicable for simulation of CZT-based gamma cameras.MethodsThe equations describing the charge transport and signal induction are considered in three dimensions and are solved numerically, and the CZT model is then realised by coupling the detector-response to the photon-transport handled by the SIMIND Monte Carlo program. The CZT model is tuned to reproduce experimentally measured energy spectra of a hand-held gamma camera system for multiple radionuclides (^{99mathrm {m}}{mathrm {Tc}}, ^{123}{mathrm {I}} and ^{177}{mathrm {Lu}}) and parallel-hole collimators (MEGP, LEHR) as well as an uncollimated system.ResultsOverall, the model results agree well with measurements across the range of experimental conditions. The applicability of the model is demonstrated by separating energy spectra into components to investigate the interference of high-energy photons on lower energy-windows, where pronounced effects of low-energy tailing for ^{177}{mathrm {Lu}} are observed.ConclusionsThe developed model provides understanding of the specifics of the camera response and is expected to be helpful for future optimisation of gamma camera applications.

Research of Binocular Visual Inertial Algorithm Based on Point and Line Features

To solve the problem of poor performance of the binocular visual inertial odometer VINS-Fusion in scenes with low texture and large luminosity changes, a binocular visual inertial odometer PLVINS-Fusion is designed that integrates line feature measurement information, which use line features to easy to extract in low-texture scenes, and have the advantage of more robust tracking performance in scenes with large luminosity changes. Point and line features are extracted in the front-end visual extraction at the same time, and line feature residuals are added to the back-end nonlinear optimization, construct a bag-of-words model combining point and line features in the loop detection module. On this basis, a real-time photometric calibration algorithm is adopted to jointly optimize the exposure time, the camera response function and the vignetting factor, and the stability of KLT optical flow tracking is improved by correcting the image brightness. Experiments on benchmark dataset show that the optimized algorithm has higher robustness and effectively improves the positioning accuracy, and meets the real-time performance requirement.

Laboratory Hyperspectral Image Acquisition System Setup and Validation

Hyperspectral Imaging (HSI) techniques have demonstrated potential to provide useful information in a broad set of applications in different domains, from precision agriculture to environmental science. A first step in the preparation of the algorithms to be employed outdoors starts at a laboratory level, capturing a high amount of samples to be analysed and processed in order to extract the necessary information about the spectral characteristics of the studied samples in the most precise way. In this article, a custom-made scanning system for hyperspectral image acquisition is described. Commercially available components have been carefully selected in order to be integrated into a flexible infrastructure able to obtain data from any Generic Interface for Cameras (GenICam) compliant devices using the gigabyte Ethernet interface. The entire setup has been tested using the Specim FX hyperspectral series (FX10 and FX17) and a Graphical User Interface (GUI) has been developed in order to control the individual components and visualise data. Morphological analysis, spectral response and optical aberration of these pushbroom-type hyperspectral cameras have been evaluated prior to the validation of the whole system with different plastic samples for which spectral signatures are extracted and compared with well-known spectral libraries.

The optimal projection intensities determination strategy for robust strip-edge detection in adaptive fringe pattern measurement

Adaptive digital fringe projection (AFP) technique is a traditional approach to measure the high dynamic range (HDR) surface. However, the projected light intensities are difficult to determine. And there are some discrepancies occurred in the location of detected strip-edge. In this paper, a projection intensities determination strategy was proposed to minimize the influence of camera response nonlinear distortion and improve the robustness of strip-edges detection. To determine the relationship between the projected gray level and the captured gray level, the mapping function between the projector and the camera is established. To evaluate the distortion degree of the captured image, the camera response nonlinear distortion model is proposed. Compared with the traditional AFP method, the proposed method is relatively simple and flexible. And the proposed method could be satisfying the different user demands. The experiment shows that the influence of nonlinearity of camera response was reduced and the robustness of strip-edge detection is improved.

Driver drowsiness detection system based on infinite feature selection algorithm and support vector machine

In recent times, driver drowsiness is one of the major reasons for road accidents that leads to severe physical injuries, deaths and significant economic losses. Hence, the existing driver drowsiness detection systems require a countermeasure device for the prevention of sleepiness related accident. This research paper aims to perform drowsiness detection with the help of driver’s eye state, head pose, and mouth state information. Initially, the input data were collected from the public drowsy driver database. Then, the Camera Response Model (CRM) was applied to improve the quality of collected data. Also, viola-jones, and Kanade-Lucas-Tomasi (KLT) approaches were used to detect and track the driver’s face, eye, and mouth regions from the input video. In this research study, Online Region-Based Active Contour Model (ORACM) algorithm was used to segment the driver’s mouth region in order to obtain the threshold value. Successively, feature extraction; Histogram of Oriented Gradients (HOG) and Local Binary Pattern (LBP) was applied to extract the features from the detected eye region. The extracted features of the eye region were combined with the threshold value of mouth region and head pose angle. After extracting the feature vectors, infinite approach was utilized to choose the relevant feature vectors. Finally, the selected features were classified using Support Vector Machine (SVM) for classifying the stages of drowsiness detection. Simulation outcome illustrated that the proposed system increased the classification accuracy up to 5.52% as related to hybrid Convolutional Neural Network (CNN) and Long Short-Term Memory (LSTM).

Joint enhancement and denoising in electronic speckle pattern interferometry fringe patterns with low contrast or uneven illumination via an oriented variational Retinex model.

Simultaneous speckle reduction and contrast enhancement for electronic speckle pattern interferometry (ESPI) fringe patterns is a challenging task. In this paper, we propose a joint enhancement and denoising method based on the oriented variational Retinex model for ESPI fringe patterns with low contrast or uneven illumination. In our model, we use the structure prior to constrain the illumination and introduce a fractional-order differential to constrain the reflectance for enhancement, then use the second-order partial derivative of the reflectance as the denoising term to reduce noise. The proposed model is solved using the sequential method to obtain piecewise smoothed illumination and noise-suppressed reflectance sequentially, which avoids remaining noise in the illumination and reflectance map. After obtaining the refined illuminance and reflectance, we substitute the gamma-corrected illuminance into the camera response function to further adjust the reflectance as the final enhancement result. We test our proposed method on two non-uniform illumination computer-simulated and two low-contrast experimentally obtained ESPI fringe patterns. Finally, we compare our method with three other joint enhancement and denoising variational Retinex methods.

Camera response function assessment in multispectral HDR imaging

Recently, spatially varying Bidirectional Reflectance Distribution Functions (svBRDF) is widely used as a model to characterize the appearance of materials with varying visual properties over the surface. One of the challenges in image-based svBRDF capture systems rises for surfaces with high specularity and sparkles, which require a dynamic range higher than the dynamic range of cameras. High Dynamic Range Imaging (HDRI) for svBRDF systems with multispectral camera has not been addressed properly in the literature. In HDRI, Camera Response Function (CRF) plays a crucial role in the precision of results specially when measuring metrological data such as spectral svBRDF. In this work, we investigate the effect of CRF assessment on the precision of measurement. Therefore, we have conducted two experiments to measure absolute CRF using reflective chart method as well as estimate relative CRF by Debevec and Malik’s method for a filter wheel multispectral camera to be used in a svBRDF setup. Results are evaluated on two levels: radiance map construction and reflectance calculation, by comparing to telespectroradiometer measurements as ground truth data. Results showed that although the HDRI with measured absolute CRF outputs radiance measurements with the same physical units and in the same scale as ground truth data, HDRI with estimated relative CRF outperformed in terms of the precision of reflectance measurement.

Using near–surface temperature data to vicariously calibrate high-resolution thermal infrared imagery and estimate physical surface properties

Thermal response of the surface to solar insolation is a function of the topography and the thermal physical characteristics of the landscape, which include bulk density, heat capacity, thermal conductivity and surface albedo and emissivity. Thermal imaging is routinely used to constrain thermal physical properties by characterizing or modeling changes in the diurnal temperature profiles. Images need to be acquired throughout the diurnal cycle – typically this is done twice during a diurnal cycle, but we suggest multiple times. Comparison of images acquired over 24 hours requires that either the data be calibrated to surface temperature, or the response of the thermal camera is linear and stable over the image acquisition period. Depending on the type and age of the thermal instrument, imagery may be self-calibrated in radiance, corrected for atmospheric effects, and pixels converted to surface temperature. We used an experimental instrumentation where the calibration should be stable, but calibration coefficients are unknown. Cases may occur where one wishes to validate the camera's calibration. We present a method to validate and calibrate the instrument and characterize the thermal physical properties for areas of interest. Finally, in situ high-temporal-resolution oblique thermal imaging can be invaluable in preparation for conducting overflight missions. We present the following:•The use of oblique thermal high temporal resolution thermal imaging over diurnal or multiday periods for the characterization of landscapes has not been widespread but poses great potential.•A method of collecting and analyzing thermal data that can be used to either determine or validate thermal camera calibration coefficients.•An approach to characterize thermophysical properties of the landscape using oblique temporally high-resolution thermal imaging, combined with in situ ground measurements.

Alternate Iterative Least-Squares Algorithm Based on Nonuniform Phase Shifting for Suppressing Nonlinearity Errors in Fringe Projection Profilometry

In fringe projection technique, nonlinearities of used devices are recognized as major factors inducing phase errors, especially when the used camera and projector simultaneously have nonlinear features. In this case, ripple artifacts appear to be unevenly distributed over the reconstructed surface depending on the local reflectivity. For suppressing these errors, this paper suggests a phase-measuring technique which is characterized by two critical steps including anti-aliasing phase-shifting and iterative least-squares phase estimation. Anti-aliasing phase-shifting requires the selected phase shifts to be nonuniform and to have no common divisors. It provides a possibility of accurately recovering phases from few to four fringe patterns and eliminating influence of high order harmonics on the measurement results. By expanding the fringe signals from projector into harmonics and modeling the camera response as a polynomial, we derive a least-squares algorithm based on alternate iterations to calculate the fringe phase map and the related nonlinearity coefficients simultaneously. Through practical experiments, it has been demonstrated that this technique allows us to decrease artifacts caused by the simultaneous nonlinearities of the camera and the projector, thus improving measurement accuracy significantly.

Camera Response Based Nighttime Image Enhancement Using Concurrent Reflectance

This article proposes a novel scheme that enhances low-light images locally to prevail their details and concurrently adapts camera response models (CRMs) to maintain the naturalness of the image. Low-light images shadow significant information within them, which demands to be unveiled through image enhancement techniques. However, the contemporary methods prove deficient of one or more factors that determine a well-enhanced image. These methods inevitably include intensity distortions or globally enhance the image, which causes loss of local information. The method proposed through this article initially decomposes the image according to retinex theory by utilizing a measured adaption model. A novel CRM is utilized to enhance the illumination of the image, while a reconstructed histogram equalization method enhances the reflectance locally. The method has been devised with complete awareness of its practical compatibility for hardware or software purposes. Extensive quantitative, qualitative, and time complexity analyses confirm the supremacy of the proposed method over various state-of-the-art methods.

Color Alignment for Relative Color Constancy via Non-Standard References.

Relative colour constancy is an essential requirement for many scientific imaging applications. However, most digital cameras differ in their image formations and native sensor output is usually inaccessible, e.g., in smartphone camera applications. This makes it hard to achieve consistent colour assessment across a range of devices, and that undermines the performance of computer vision algorithms. To resolve this issue, we propose a colour alignment model that considers the camera image formation as a black-box and formulates colour alignment as a three-step process: camera response calibration, response linearisation, and colour matching. The proposed model works with non-standard colour references, i.e., colour patches without knowing the true colour values, by utilising a novel balance-of-linear-distances feature. It is equivalent to determining the camera parameters through an unsupervised process. It also works with a minimum number of corresponding colour patches across the images to be colour aligned to deliver the applicable processing. Three challenging image datasets collected by multiple cameras under various illumination and exposure conditions, including one that imitates uncommon scenes such as scientific imaging, were used to evaluate the model. Performance benchmarks demonstrated that our model achieved superior performance compared to other popular and state-of-the-art methods.

An accurate partial shading detection and global maximum power point tracking technique based on image processing

This paper aims to detect the existence of partial shading conditions on the PV array and to estimate the vicinity of the global maximum power point for shaded PV modules based solely on the captured image of the PV modules. Detecting the existence of partial shading is based on comparing the average image pixel intensity for each PV module to find any mismatch in the incident irradiance. Estimating the incident irradiance level of each PV module is based on the camera response function with the help of a reference module. Furthermore, after estimating the irradiance intensity on the modules, we used the captured image for each PV module to detect the shaded area percentage. Detect the presence of partial shading conditions and estimating the position of the Global Maximum Power Point under partial shading was achieved with simple and cheap procedure yet effective at various shading patterns regardless of the environmental circumstances.

Endmember-Assisted Camera Response Function Learning, Toward Improving Hyperspectral Image Super-Resolution Performance

Endmember-Assisted Camera Response Function Learning, Toward Improving Hyperspectral Image Super-Resolution Performance

Accurate estimation of camera response function for high dynamic range measurement.

A high dynamic range surface has large reflectivity variations and measuring it with a structured light technique could cause pixel saturation that will seriously degrade the measurement accuracy. Accurate identification of saturated pixels will significantly improve the measurement accuracy. This paper introduces an accurate estimation approach to the camera response function (CRF) for high dynamic range measurement and proposes a method that uses the estimated CRF to identify the saturated pixels in captured images. We perform experiments using the proposed saturation identification method and the other existing methods to compare both the saturation identification accuracy and measurement accuracy. The experiment results verify both the accuracy and suitability of the proposed method.