Radiometric Compensation Research Articles

Long-Term Radiometric Stability of Uncooled and Shutterless Microbolometer-Based Infrared Cameras

Uncooled and shutterless microbolometer cameras are good candidates for infrared imaging systems installed on small satellites or small unmanned aerial vehicles: they are light and passive since no cooling system or mechanical shutter is required and they can be operated at ambient temperatures. However, the radiometric compensation has to be carefully performed to make the system compatible with applications where the radiometric accuracy of the images is mandatory. In this paper, we study the impact of the camera environment to the radiometric accuracy of the images. We propose and test hardware and software solutions to improve this accuracy and the quality of the radiometric images. We show that the radiometric calibration of the camera with our model is valid over a long time period— about 3 years—using in-door experiments.

Physics-Based Efficient Full Projector Compensation Using Only Natural Images.

Achieving practical full projector compensation requires the projection display to adapt quickly to textured projection surfaces and unexpected movements without interrupting the display procedure. A possible solution to achieve this involves using a projector and an RGB camera and correcting both color and geometry by directly capturing and analyzing the projected natural image content, without the need for additional patterns. In this study, we approach full projector compensation as a numerical optimization problem and present a physics-based framework that can handle both geometric calibration and radiometric compensation for a Projector-camera system (Procams), using only a few sampling natural images. Within the framework, we decouple and estimate the Procams' factors, such as the response function of the projector, the correspondence between the projector and camera, and the reflectance of projection surfaces. This approach provides an interpretable and flexible solution to adapt to the changes in geometry and reflectance caused by movements. Benefitting from the physics-based scheme, our method guarantees both accurate color calculation and efficient movement and reflectance estimation. Our experimental results demonstrate that our method surpasses other state-of-the-art end-to-end full projector compensation methods, with superior image quality, reduced computational time, lower memory consumption, greater geometric accuracy, and a more compact network architecture.

Projection mapping technologies: A review of current trends and future directions.

This study summarizes current trends and future directions in projection mapping technologies. Projection mapping seamlessly merges the virtual and real worlds through projected imagery onto physical surfaces, creating an augmented reality environment. Beyond traditional applications in advertising, art, and entertainment, various fields, including medical surgery, product design, and telecommunications, have embraced projection mapping. This study categorizes recent techniques that address technical challenges in accurately replicating desired appearances on physical surfaces through projected imagery into four groups: geometric registration, radiometric compensation, defocus compensation, and shadow removal. It subsequently introduces unconventional projectors developed to resolve specific technical issues and discusses two approaches for overcoming the inherent limitations of projector hardware, such as the inability to display images floating above physical surfaces. Finally, this study concludes the discussion with possible future directions for projection mapping technologies.

Real-Time Radiometric Compensation on Deforming Non-Rigid Surfaces using Motion Prediction in Projection Mapping

Real-Time Radiometric Compensation on Deforming Non-Rigid Surfaces using Motion Prediction in Projection Mapping

A Monocular Projector-Camera System Using Modular Architecture.

This paper presents a monocular projector-camera (procam) system using modular architecture based on relay optics. Conventional coaxial procam systems cannot support (1) online changes to lens settings (zoom and focus) and (2) wide-angle projection mapping. We develop design guidelines for a proposed procam system that would solve these restrictions and address the proposed system's unique technical issue of crosstalk between the camera and projector pixels. We conducted experiments using prototypes to validate the feasibility of the proposed framework. First, we confirmed that the proposed crosstalk reduction technique worked well. Second, we found our technique could achieve correct alignment of a projected image onto a moving surface while changing the zoom and focus of the objective lens. The monocular procam system also achieved radiometric compensation where a surface texture was visually concealed by pixel-wise control of a projection color based on the captured results of offline color pattern projections. Finally, we demonstrated the high expandability of our modular architecture, through the creation of a high dynamic range projection.

Radiometric Compensation for Occluded Crops Imaged Using High-Spatial-Resolution Unmanned Aerial Vehicle System

Crop characterization is considered a prerequisite to devising effective strategies for ensuring successful implementation of sustainable agricultural management strategies. As such, remote-sensing technology has opened an exciting horizon for crop characterization at reasonable spatial, spectral, and temporal scales. However, the presence of shadows on croplands tends to distort radiometric properties of the crops, subsequently limiting the retrieval of crop-related information. This study proposes a simple and reliable approach for radiometrically compensating crops under total occlusion using brightness-based compensation and thresholding approaches. Unmanned aerial vehicle (UAV) imagery was used to characterize crops at the experimental site. In this study, shadow was demarcated through the computation and use of mean spectral radiance values as the threshold across spectral channels of UAV imagery. Several image classifiers, viz., k-nearest neighbor (KNN), maximum likelihood, multilayer perceptron (MLP), and image segmentation, were used to categorize land features, with a view to determine the areal coverage of crops prior to the radiometric compensation process. Radiometric compensation was then performed to restore radiometric properties of land features under occlusion by performing brightness tuning on the RGB imagery. Radiometric compensation results revealed maize and soil as land features subjected to occlusion. The relative error of the mean results for radiance comparison between lit and occluded regions revealed 26.47% deviation of the restored radiance of occluded maize from that of lit maize. On the other hand, the reasonable REM value of soil was noted to be 50.92%, implying poor radiometric compensation results. Postradiometric compensation classification results revealed increases in the areal coverage of maize cultivars and soil by 40.56% and 12.37%, respectively, after being radiometrically compensated, as predicted by the KNN classifier. The maximum likelihood, MLP, and segmentation classifiers predicted increases in area covered with maize of 18.03%, 22.42%, and 30.64%, respectively. Moreover, these classifiers also predicted increases in the area covered with soil of 1.46%, 10.05%, and 14.29%, respectively. The results of this study highlight the significance of brightness tuning and thresholding approaches in radiometrically compensating occluded crops.

Evaluation of P-Band SAR Tomography for Mapping Tropical Forest Vertical Backscatter and Tree Height

Low-frequency tomographic synthetic aperture radar (TomoSAR) techniques provide an opportunity for quantifying the dynamics of dense tropical forest vertical structures. Here, we compare the performance of different TomoSAR processing, Back-projection (BP), Capon beamforming (CB), and MUltiple SIgnal Classification (MUSIC), and compensation techniques for estimating forest height (FH) and forest vertical profile from the backscattered echoes. The study also examines how polarimetric measurements in linear, compact, hybrid, and dual circular modes influence parameter estimation. The tomographic analysis was carried out using P-band data acquired over the Paracou study site in French Guiana, and the quantitative evaluation was performed using LiDAR-based canopy height measurements taken during the 2009 TropiSAR campaign. Our results show that the relative root mean squared error (RMSE) of height was less than 10%, with negligible systematic errors across the range, with Capon and MUSIC performing better for height estimates. Radiometric compensation, such as slope correction, does not improve tree height estimation. Further, we compare and analyze the impact of the compensation approach on forest vertical profiles and tomographic metrics and the integrated backscattered power. It is observed that radiometric compensation increases the backscatter values of the vertical profile with a slight shift in local maxima of the canopy layer for both the Capon and the MUSIC estimators. Our results suggest that applying the proper processing and compensation techniques on P-band TomoSAR observations from space will allow the monitoring of forest vertical structure and biomass dynamics.

Real-Time Radiometric Compensation for Continuously Deforming Projection Surfaces

Real-Time Radiometric Compensation for Continuously Deforming Projection Surfaces

Radiometric compensation for non-rigid surfaces by continuously estimating inter-pixel correspondence

In recent years, radiometric compensation techniques for realizing ideal image projection, even for a patterned surface, have attracted attention. In radiometric compensation, the influence of the pattern can be canceled based on the input and output relation, referred to as the response function, which is determined by a projector–camera system. However, since the response function strongly depends on the inter-pixel correspondence between the projector and the camera, the projection surface is restricted to being a rigid body. In the present study, we achieve radiometric compensation for a non-rigid surface, such as a swaying curtain in a normal room, by estimating the inter-pixel correspondence in real time. The reflectance of the projection surface is estimated based on observation of the projected image by the camera without using special equipment, and the offset of the correspondence is estimated based on its validity. We evaluate the effectiveness of the proposed method for various combinations of curtain patterns and projected images.

P‐35: Human Visual System Inspired Artifact Reduction in Projector Compensation

Projection onto an arbitrary surface requires radiometric compensation to correct for distortions caused by the colors and texture of the underlying surface. Radiometric compensation modifies the projector input image with respect to the projection surface characteristics. Due to the limited dynamic range of any projector, the modified target image can easily go beyond the projector intensity limits, causing saturation artifacts. In this paper, a novel locally‐adaptive, spatial luminance modification framework is presented to reduce brightness/contrast demands on the projector during radiometric compensation.

Practical Radiometric Compensation for Projection Display on Textured Surfaces using a Multidimensional Model

AbstractRadiometric compensation methods remove the effect of the underlying spatially varying surface reflectance of the texture when projecting on textured surfaces. All prior work sample the surface reflectance dependent radiometric transfer function from the projector to the camera at every pixel that requires the camera to observe tens or hundreds of images projected by the projector. In this paper, we cast the radiometric compensation problem as a sampling and reconstruction of multi‐dimensional radiometric transfer function that models the color transfer function from the projector to an observing camera and the surface reflectance in a unified manner. Such a multi‐dimensional representation makes no assumption about linearity of the projector to camera color transfer function and can therefore handle projectors with non‐linear color transfer functions(e.g. DLP, LCOS, LED‐based or laser‐based). We show that with a well‐curated sampling of this multi‐dimensional function, achieved by exploiting the following key properties, is adequate for its accurate representation: (a) the spectral reflectance of most real‐world materials are smooth and can be well‐represented using a lower‐dimension function; (b) the reflectance properties of the underlying texture have strong redundancies – for example, multiple pixels or even regions can have similar surface reflectance; (c) the color transfer function from the projector to camera have strong input coherence. The proposed sampling allows us to reduce the number of projected images that needs to be observed by a camera by up to two orders of magnitude, the minimum being only two. We then present a new multi‐dimensional scattered data interpolation technique to reconstruct the radiometric transfer function at a high spatial density (i.e. at every pixel) to compute the compensation image. We show that the accuracy of our interpolation technique is higher than any existing methods.

Haze removal for new generation optical sensors

ABSTRACTHaze and cloud contamination is a common problem in optical remote-sensing imagery, as it can lead to the inaccurate estimation of physical properties of the surface derived from remote sensing and reduced accuracy of land cover classification and change detection. Haze optimized transform (HOT) is a methodology applicable to radiometric compensation of additive haze effects in visible bands that exhibits a spatially complex distribution over an image. The generic approach of HOT allows for the use of older satellite imaging sensors that include at least two visible bands (e.g. Landsat Thematic Mapper (TM) and Landsat Multispectral Scanner (MSS) sensors). This study proposes modifications to extend HOT applicability to new sensors. The improvements and extended functionality adapt the method to the higher radiometric resolution specifications of newer generation sensors and use percentile-based minimum in the correction procedure to avoid causing fake minimum. Alternative filters are also evaluated to smooth raw HOT output and the cloud mask is generated as an additional output. A Landsat 8 scene of Los Angeles is used to demonstrate the improved methodology. The methodology is applicable to sensors such as QuickBird, Worldview 2/3. More than 20 additional scenes were used to evaluate the effectiveness of the methodology.

Impact of Training Images on Radiometric Compensation

The increasing availability of both high-resolution projectors andimperfect displays make radiometric correction an essential componentin all modern projection systems. Particularly, projectingin casual locations, such as classrooms, open areas and homes,calls for the development of radiometric correction techniques thatare fully automatic and deal with display imperfections in real-time.This paper reviews the current radiometric compensation algorithmsand discusses the influence of different training images on theirperformance.

位相シフト法を用いた光学的映像補正の高精度化(セッション3,学生研究発表会)

位相シフト法を用いた光学的映像補正の高精度化(セッション3,学生研究発表会)

Fabricating Diminishable Visual Markers for Geometric Registration in Projection Mapping.

We propose a visual marker embedding method for the pose estimation of a projection surface to correctly map projected images onto the surface. Assuming that the surface is fabricated by a full-color or multi-material three-dimensional (3D) printer, we propose to automatically embed visual markers on the surface with mechanical accuracy. The appearance of the marker is designed such that the marker is detected by infrared cameras even when printed on a non-planar surface while its appearance can be diminished by the projection to be as imperceptible as possible to human observers. The marker placement is optimized using a genetic algorithm to maximize the number of valid viewpoints from which the pose of the object can be estimated correctly using a stereo camera system. We also propose a radiometric compensation technique to quickly diminish the marker appearance. Experimental results confirm that the pose of projection objects are correctly estimated while the appearance of the markers was diminished to an imperceptible level. At the same time, we confirmed the limitations of the current method; only one object can be handled, and pose estimation is not performed at interactive frame rates. Finally, we demonstrate the proposed technique to show that it works successfully for various surface shapes and target textures.

Paper] Compressive Inverse Light Transport for Radiometric Compensation in Projection-based Displays

Paper] Compressive Inverse Light Transport for Radiometric Compensation in Projection-based Displays

Real-Time Radiometric Compensation for Optical See-Through Head-Mounted Displays.

Optical see-through head-mounted displays are currently seeing a transition out of research labs towards the consumer-oriented market. However, whilst availability has improved and prices have decreased, the technology has not matured much. Most commercially available optical see-through head mounted displays follow a similar principle and use an optical combiner blending the physical environment with digital information. This approach yields problems as the colors for the overlaid digital information can not be correctly reproduced. The perceived pixel colors are always a result of the displayed pixel color and the color of the current physical environment seen through the head-mounted display. In this paper we present an initial approach for mitigating the effect of color-blending in optical see-through head-mounted displays by introducing a real-time radiometric compensation. Our approach is based on a novel prototype for an optical see-through head-mounted display that allows the capture of the current environment as seen by the user's eye. We present three different algorithms using this prototype to compensate color blending in real-time and with pixel-accuracy. We demonstrate the benefits and performance as well as the results of a user study. We see application for all common Augmented Reality scenarios but also for other areas such as Diminished Reality or supporting color-blind people.

Radiometric Compensation of Images Projected on Non-White Surfaces by Exploiting Chromatic Adaptation and Perceptual Anchoring.

Flat surfaces in our living environment to be used as replacements of a projection screen are not necessarily white. We propose a perceptual radiometric compensation method to counteract the effect of color projection surfaces on image appearance. It reduces color clipping while preserving the hue and brightness of images based on the anchoring property of human visual system. In addition, it considers the effect of chromatic adaptation on perceptual image quality and fixes the color distortion caused by non-white projection surfaces by properly shifting the color of the image pixels toward the complementary color of the projection surface. User ratings show that our method outperforms the existing methods in 974 out of 1020 subjective tests.

Paper] Radiometric Compensation Using Color-Mixing Matrix Reformed from Light Transport Matrix

Paper] Radiometric Compensation Using Color-Mixing Matrix Reformed from Light Transport Matrix

Radiometric compensation for cooperative distributed multi-projection system through 2-DOF distributed control.

This paper proposes a novel radiometric compensation technique for cooperative projection system based-on distributed optimization. To achieve high scalability and robustness, we assume cooperative projection environments such that 1. each projector does not have information about other projectors as well as target images, 2. the camera does not have information about the projectors either, while having the target images, and 3. only a broadcast communication from the camera to the projectors is allowed to suppress the data transfer bandwidth. To this end, we first investigate a distributed optimization based feedback mechanism that is suitable for the required decentralized information processing environment. Next, we show that this mechanism works well for still image projection, however not necessary for moving images due to the lack of dynamic responsiveness. To overcome this issue, we propose to implement an additional feedforward mechanism. Such a 2 Degree Of Freedom (2-DOF) control structure is well-known in control engineering community as a typical method to enhance not only disturbance rejection but also reference tracking capability, simultaneously. We theoretically guarantee and experimentally demonstrate that this 2-DOF structure yields the moving image projection accuracy that is overwhelming the best achievable performance only by the distributed optimization mechanisms.