Object Of Reflection Research Articles

Reflectance estimation for proximity sensing by vision-language models: utilizing distributional semantics for low-level cognition in robotics

Large language models (LLMs) and vision-language models (VLMs) have been increasingly used in robotics for high-level cognition, but their use for low-level cognition, such as interpreting sensor information, remains underexplored. In robotic grasping, estimating the reflectance of objects is crucial for successful grasping, as it significantly impacts the distance measured by proximity sensors. We investigate whether LLMs can estimate reflectance from object names alone, leveraging the embedded human knowledge in distributional semantics, and if the latent structure of language in VLMs positively affects image-based reflectance estimation. In this paper, we verify that (1) LLMs such as GPT-3.5 and GPT-4 can estimate an object's reflectance using only text as input; and (2) VLMs such as CLIP can increase their generalization capabilities in reflectance estimation from images. Our experiments show that GPT-4 can estimate an object's reflectance using only text input with a mean error of 14.7%, lower than the image-only ResNet. Moreover, CLIP achieved the lowest mean error of 11.8%, while GPT-3.5 obtained a competitive 19.9% compared to ResNet's 17.8%. These results suggest that the distributional semantics in LLMs and VLMs increases their generalization capabilities, and the knowledge acquired by VLMs benefits from the latent structure of language.

Accuracy Measurement of Hyperspectral Image Classification in Remote Sensing with the Light Spectrum-based Affinity Propagation Clustering-based Segmentation

The area of remote sensing and computer vision includes the challenge of hyperspectral image classification. It entails grouping pixels in hyperspectral pictures into several classes according to their spectral signature. Hyperspectral photographs are helpful for a variety of applications, including vegetation study, mineral mapping, and mapping urban land use, since they include information on an object's reflectance in hundreds of small, contiguous wavelength bands. This task's objective is to correctly identify and categorize several item categories in the image. Many approaches have been stated by several researchers in this field to enhance the accuracy of the segmentation and accuracy. However, fails to attain the optimal accuracy due to the intricate nature of the images. To tackle these issues, we propose a novel Modified Extreme Learning machine (M-ELM) approach for the credible hyperspectral image classification outcomes with the publicly available Kaggle datasets. Before the classification, the input images are segmented using the Light Spectrum-based modified affinity propagation clustering technique (LSO-MAPC). In the beginning, the images are pre-processed using the non-linear diffusion partial differential equations technique which effectively pre-processed the image spatially. Experiments are effectuated to analyze the performance of the proposed method and compared it with state-of-art works in a quantitative way. The proposed approach ensures a classification accuracy of 96%.

Multiview Textured Mesh Recovery by Differentiable Rendering

Although having achieved the promising results on shape and color recovery through self-supervision, the multi-layer perceptrons-based methods usually suffer from heavy computational cost on learning the deep implicit surface representation. Since rendering each pixel requires a forward network inference, it is very computational intensive to synthesize a whole image. To tackle these challenges, we propose an effective coarse-to-fine approach to recover the textured mesh from multi-views in this paper. Specifically, a differentiable Poisson Solver is employed to represent the object's shape, which is able to produce topology-agnostic and watertight surfaces. To account for depth information, we optimize the shape geometry by minimizing the differences between the rendered mesh and the predicted depth from multi-view stereo. In contrast to the implicit neural representation on shape and color, we introduce a physically based inverse rendering scheme to jointly estimate the environment lighting and object's reflectance, which is able to render the high resolution image at real-time. The texture of the reconstructed mesh is interpolated from a learnable dense texture grid. We have conducted the extensive experiments on several multi-view stereo datasets, whose promising results demonstrate the efficacy of our proposed approach. The code is available at https://github.com/l1346792580123/diff.

АBOUT THE SEMANTICS OF THE WORD ЗРAКЪ IN CHURCH SLAVONIC TEXTS

The material of the study was the Church Slavonic texts of the 18–20th centuries. As a result of an electronic search in digitized versions of these sources, 512 entries were obtained, mainly from liturgical texts, which distinguishes our material from illustrative examples for this lexeme in a number of other dictionaries. Lexicographic sources allow us to state that there is no monosemantic correspondence between the selected meanings of the Church Slavonic word and the Greek equivalents. As a result of the analysis of the studied material, the following meanings were distinguished for the word зракъ́ : 1) contemplation (process, action according to the verb зрѣти́ ); 2) look; 3) sight, eye (the ability to see); 4) appearance, face; 5) picture, icon; 6) image (refl ection of objects and phenomena of reality in human mind); 7) essence, nature; 8) vision. Th e meaning (1) naming the process is related to the verb зрѣти́ in its transitive meaning, as are the meanings (4, 5, 6, 7) naming the object. Meanings (2) and (3) are associated with the verb зрѣти́ in intransitive meanings: as the names of the ability to see — sight (3) and its immaterial tool — look (2). In the fi rst 7 meanings, the word зракъ́ is associated with the verb зрѣти́ , in the meaning (8) — with the verb зрѣтисѧ ́ . One can define the first three meanings as ‘internal’, subjective, associated with the characteristics of the looking subject, and the rest as ‘external’, objective, associated with the characteristics of the visible object. Th e complexity of the semantic structure of the word зракъ́ in the studied texts is partly due to the presence of polysemantic combinations. In several cases, an unusual use of the plural forms of this word is noted, accompanying some shades of its meanings.

Robust Reflectance Estimation for Projection-Based Appearance Control in a Dynamic Light Environment.

We present a novel method that robustly estimates the reflectance, even in an environment with dynamically changing light. To control the appearance of an object by using a projector-camera system, an appropriate estimate of the object's reflectance is vital to the creation of an appropriate projection image. Most conventional estimation methods assume static light conditions; however, in practice, the appearance is affected by both the reflectance and environmental light. In an environment with dynamically changing light, conventional reflectance estimation methods require calibration every time the conditions change. In contrast, our method requires no additional calibration because it simultaneously estimates both the reflectance and environmental light. Our method is based on the concept of creating two different light conditions by switching the projection at a rate higher than that perceived by the human eye and captures the images of a target object separately under each condition. The reflectance and environmental light are then simultaneously estimated by using the pair of images acquired under these two conditions. We implemented a projector-camera system that switches the projection on and off at 120 Hz. Experiments confirm the robustness of our method when changing the environmental light. Further, our method can robustly estimate the reflectance under practical indoor lighting conditions.

Waypoint shift manifolds for object color evaluation and comparison

AbstractVariability in object appearance is common in everyday experience and this variability can generally be attributed to the relationships between viewing conditions and the composition of an object's spectral reflectances. This paper proposes that objects, illuminants, and observers can be evaluated using a Wpt (Waypoint) shift manifold (WSM), which is defined as a set of Wpt points associated with one or more objects, observers, and/or illuminants. Since spectral reflectances of an object can be decomposed to three components (a wavelength invariant nonselective component, a characteristic reflectance, and a metameric black reflectance), WSMs of any object for an observing condition is composed of three Wpt vectors associated with a perfect reflecting diffuser, the characteristic reflectance, and the metameric black reflectance of the object. Material shift potential (MSP) can be defined as shapes and orientations of object WSMs for evaluation and comparison of object colors. Experimental results show that MSP are dramatically influenced by Wpt vectors associated with the characteristic reflectance and the metameric black reflectance for an object rather than the Wpt vectors associated with nonselective reflectance. Thus, MSP can be preserved for objects having similar characteristic reflectances and metameric black reflectances. In other words, spectral reflectances that have been manipulated to have similar overall shapes exhibit similar color inconstancy.

Computational luminance constancy from naturalistic images.

The human visual system supports stable percepts of object color even though the light that reflects from object surfaces varies significantly with the scene illumination. To understand the computations that support stable color perception, we study how estimating a target object's luminous reflectance factor (LRF; a measure of the light reflected from the object under a standard illuminant) depends on variation in key properties of naturalistic scenes. Specifically, we study how variation in target object reflectance, illumination spectra, and the reflectance of background objects in a scene impact estimation of a target object's LRF. To do this, we applied supervised statistical learning methods to the simulated excitations of human cone photoreceptors, obtained from labeled naturalistic images. The naturalistic images were rendered with computer graphics. The illumination spectra of the light sources and the reflectance spectra of the surfaces in the scene were generated using statistical models of natural spectral variation. Optimally decoding target object LRF from the responses of a small learned set of task-specific linear receptive fields that operate on a contrast representation of the cone excitations yields estimates that are within 13% of the correct LRF. Our work provides a framework for evaluating how different sources of scene variability limit performance on luminance constancy.

Spectral reflectance estimation from one RGB image using self-interreflections in a concave object.

Light interreflections occurring in a concave object generate a color gradient that is characteristic of the object's spectral reflectance. In this paper, we use this property in order to estimate the spectral reflectance of matte, uniformly colored, V-shaped surfaces from a single RGB image taken under directional lighting. First, simulations show that using one image of the concave object is equivalent to, and can even outperform, the state-of-the-art approaches based on three images taken under three lightings with different colors. Experiments on real images of folded papers were performed under unmeasured direct sunlight. The results show that our interreflection-based approach outperforms existing approaches, even when the latter are improved by a calibration step. The mathematical solution for the interreflection equation and the effect of surface parameters on the performance of the method are also discussed in this paper.

Application of a reflective microscope objective for multiphoton microscopy.

Reflective objectives (ROs) mitigate chromatic aberration across a broad wavelength range. Yet, a systematic performance characterisation of ROs has not been done. In this paper, we compare the performance of a 0.5 numerical-aperture (NA) reflective objective (RO) with a 0.55 NA standard glass objective (SO), using two-photon fluorescence (TPF) and second-harmonic generation (SHG). For experiments spanning ∼1 octave in the visible and NIR wavelengths, the SO leads to defocusing errors of 25-40% for TPF images of subdiffraction fluorescent beads and 10-12% for SHG images of collagen fibres. The corresponding error for the RO is ∼4% for both imaging modalities. This work emphasises the potential utility of ROs for multimodal multiphoton microscopy applications.

Computer-generated Moiré profilometry.

A new 3D measuring method based on computer-generated moiré fringes is proposed. The two AC components of the 0-degree and 90-degree phase-shifted fringe patterns on reference plane are prepared in advance. While the AC component of the single-shot deformed pattern is multiplied by the two prepared AC components, respectively, two computer-generated moiré fringes can be retrieved. The ratio of the two computer-generated moiré fringes is just the tangent of the phase modulated by the object. It is of great potential in real-time or even dynamical 3D measurement due to its single-shot deformed pattern feature, and it avoids the influences of the object's reflectivity simultaneously. Compared to the Fourier transform profilometry, its error is smaller due to its higher first-order spectrum. Experimental results show the feasibility and validity of the proposed method.

Reflectance and Natural Illumination from Single-Material Specular Objects Using Deep Learning.

In this paper, we present a method that estimates reflectance and illumination information from a single image depicting a single-material specular object from a given class under natural illumination. We follow a data-driven, learning-based approach trained on a very large dataset, but in contrast to earlier work we do not assume one or more components (shape, reflectance, or illumination) to be known. We propose a two-step approach, where we first estimate the object's reflectance map, and then further decompose it into reflectance and illumination. For the first step, we introduce a Convolutional Neural Network (CNN) that directly predicts a reflectance map from the input image itself, as well as an indirect scheme that uses additional supervision, first estimating surface orientation and afterwards inferring the reflectance map using a learning-based sparse data interpolation technique. For the second step, we suggest a CNN architecture to reconstruct both Phong reflectance parameters and high-resolution spherical illumination maps from the reflectance map. We also propose new datasets to train these CNNs. We demonstrate the effectiveness of our approach for both steps by extensive quantitative and qualitative evaluation in both synthetic and real data as well as through numerous applications, that show improvements over the state-of-the-art.

Optically coherent image formation and denoising using a plug and play inversion framework.

The performance of optically coherent imaging systems can be limited by measurement and speckle noise. In this paper, we develop an image formation framework for computing the maximum a posteriori estimate of an object's reflectivity when imaged using coherent illumination and detection. The proposed approach allows for the use of Gaussian denoising algorithms (GDAs), without modification, to mitigate the exponentially distributed and signal-dependent noise that occurs in coherent imaging. Several GDAs are compared using both simulated and experimental data. The proposed framework is shown to be robust to noise and significantly reduce reconstruction error compared to the standard inversion technique.

Energy-Saving Light Source Spectrum Optimization by Considering Object's Reflectance

When an object is illuminated, part of the incident light is absorbed by the object that does not contribute to the illumination. If the light source's spectrum can be optimized by decreasing the light energy along the absorbed wavelengths, substantial energy can be saved. In this study, a spectrum optimization method by means of a genetic algorithm to minimize the light energy along the absorbed wavelengths while maintaining the light quality of the light source was proposed. Under the constraint that the color differences of the object illuminated by the optimized illuminant and the reference illuminant do not exceed specified levels, maximum energy saving ratio (ESR) is achieved for various reference illuminants. It is found that through the proposed method effectively saving the energy, for a monochromatic object, the ESR can reach 49.5% with barely recognizable color difference, whereas for a polychromatic object, the ESR can reach 55.2% with an acceptable color difference for each patch of the object.

BRDF-dependent accuracy of array-projection-based 3D sensors.

In order to perform high-speed three-dimensional (3D) shape measurements with structured light systems, high-speed projectors are required. One possibility is an array projector, which allows pattern projection at several tens of kilohertz by switching on and off the LEDs of various slide projectors. The different projection centers require a separate analysis, as the intensity received by the cameras depends on the projection direction and the object's bidirectional reflectance distribution function (BRDF). In this contribution, we investigate the BRDF-dependent errors of array-projection-based 3D sensors and propose an error compensation process.

Differential processing of binocular and monocular gloss cues in human visual cortex.

The visual impression of an object's surface reflectance (“gloss”) relies on a range of visual cues, both monocular and binocular. Whereas previous imaging work has identified processing within ventral visual areas as important for monocular cues, little is known about cortical areas involved in processing binocular cues. Here, we used human functional MRI (fMRI) to test for brain areas selectively involved in the processing of binocular cues. We manipulated stereoscopic information to create four conditions that differed in their disparity structure and in the impression of surface gloss that they evoked. We performed multivoxel pattern analysis to find areas whose fMRI responses allow classes of stimuli to be distinguished based on their depth structure vs. material appearance. We show that higher dorsal areas play a role in processing binocular gloss information, in addition to known ventral areas involved in material processing, with ventral area lateral occipital responding to both object shape and surface material properties. Moreover, we tested for similarities between the representation of gloss from binocular cues and monocular cues. Specifically, we tested for transfer in the decoding performance of an algorithm trained on glossy vs. matte objects defined by either binocular or by monocular cues. We found transfer effects from monocular to binocular cues in dorsal visual area V3B/kinetic occipital (KO), suggesting a shared representation of the two cues in this area. These results indicate the involvement of mid- to high-level visual circuitry in the estimation of surface material properties, with V3B/KO potentially playing a role in integrating monocular and binocular cues.

Spectral reflectance luminous efficacy

Due to the lack of quantitative evaluation of lighting efficiency and lack of theoretical guidance on spectrum mechanism, there exists huge energy consumption in landscape lighting. This study aims to quantify landscape lighting effect and define efficiency evaluation indicator by studying the relationship between light sources’ spectrum and object's spectral reflectance curves from spectrum aspect. Traditional landscape lighting and Gaussian models of light sources’ spectrum's effect on objects show general rules. Relative high saturated light sources are good for reshaping lighting, while low saturated light sources for reproducing lighting. As most landscapes in a small region are very saturated but less in color types, the concept of spectral reflectance luminous efficacy rather than traditional luminous efficacy is proposed for evaluating landscape lighting efficiency. And comprehensive spectral reflectance luminous efficacy considering weights of landscapes is used to evaluate relative complex scenes. For different landscapes with different spectral reflectance curves, traditional high luminous efficacy lighting may not always achieve better efficiency. For some landscapes, matching light sources’ efficiency would be much higher than traditional light sources. Landscape lighting effect and lighting efficiency could be quantitatively predicted and evaluated.

Image Color Registration for Illumination-Robust Pattern Recognition

Image-based pattern recognition techniques have attracted much attention in vision-based applications. Color-based methods have shown several benefits. However, due to color variations resulting from illumination changes, many color-based techniques have yet to demonstrate stable performance. For illumination-robust pattern recognition, we propose an image color registration method based on an image acquisition model. Since the image acquisition model is created using the variables related to an illumination condition, camera characteristics, and an object's surface reflectance, the proposed method normalizes the image's color by taking into account both the illumination and camera characteristics. To evaluate the performance of the proposed method in terms of illumination-robust pattern recognition, we perform both an image similarity test and a feature similarity test between images acquired under different illumination conditions. Through the experiments, the superiority and the usefulness of the proposed method was validated.

Simple Measurement Method for a Surface Reflection Using a Fisheye Camera

An object's optical reflection properties are due to the microscopic structure of the object's surface. Therefore, optical reflection measurement is important to obtain a realistic representation in computer graphics and in the inspection of industrial parts. In this study, we developed a compact reflectance measurement system without a mechanical drive unit. The system has the advantage of being simple and highly portable compared to the conventional measuring device used for this purpose. We constructed a prototype camera system and carried out several experiments to test it. The experimental results showed that the reflection characteristics will vary depending on the surface roughness of objects, and we could recognize the state of the surface reflection visually.

Diffuse reflectance relations based on diffusion dipole theory for large absorption and reduced scattering

Diffuse reflectance spectra are used to determine the optical properties of biological samples. In medicine and forensic science, the turbid objects under study often possess large absorption and/or scattering properties. However, data analysis is frequently based on the diffusion approximation to the radiative transfer equation, implying that it is limited to tissues where the reduced scattering coefficient dominates over the absorption coefficient. Nevertheless, up to absorption coefficients of 20 mm-1 at reduced scattering coefficients of 1 and 11.5 mm-1, we observed excellent agreement (r2=0.994) between reflectance measurements of phantoms and the diffuse reflectance equation proposed by Zonios et al. [Appl. Opt.38, 6628-6637 (1999)], derived as an approximation to one of the diffusion dipole equations of Farrell et al. [Med. Phys.19, 879-888 (1992)]. However, two parameters were fitted to all phantom experiments, including strongly absorbing samples, implying that the reflectance equation differs from diffusion theory. Yet, the exact diffusion dipole approximation at high reduced scattering and absorption also showed agreement with the phantom measurements. The mathematical structure of the diffuse reflectance relation used, derived by Zonios et al. [Appl. Opt.38, 6628-6637 (1999)], explains this observation. In conclusion, diffuse reflectance relations derived as an approximation to the diffusion dipole theory of Farrell et al. can analyze reflectance ratios accurately, even for much larger absorption than reduced scattering coefficients. This allows calibration of fiber-probe set-ups so that the object's diffuse reflectance can be related to its absorption even when large. These findings will greatly expand the application of diffuse reflection spectroscopy. In medicine, it may allow the use of blue/green wavelengths and measurements on whole blood, and in forensic science, it may allow inclusion of objects such as blood stains and cloth at crime scenes.

IR-UWB Radar Demonstrator for Ultra-Fine Movement Detection and Vital-Sign Monitoring

<?Pub Dtl=""?> In this paper an impulse-radio ultra-wideband (IR-UWB) hardware demonstrator is presented, which can be used as a radar sensor for highly precise object tracking and breath-rate sensing. The hardware consists of an impulse generator integrated circuit (IC) in the transmitter and a correlator IC with an integrating baseband circuit as correlation receiver. The radiated impulse is close to a fifth Gaussian derivative impulse with <formula formulatype="inline" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex Notation="TeX">$\sigma=51\ {\hbox {ps}}$</tex></formula> , efficiently using the Federal Communications Commission indoor mask. A detailed evaluation of the hardware is given. For the tracking, an impulse train is radiated by the transmitter, and the reflections of objects in front of the sensor are collected by the receiver. With the reflected signals, a continuous hardware correlation is computed by a sweeping impulse correlation. The correlation is applied to avoid sampling of the RF impulse with picosecond precision. To localize objects precisely in front of the sensor, three impulse tracking methods are compared: Tracking of the maximum impulse peak, tracking of the impulse slope, and a slope-to-slope tracking of the object's reflection and the signal of the static direct coupling between transmit and receive antenna; the slope-to-slope tracking showing the best performance. The precision of the sensor is shown by a measurement with a metal plate of 1-mm sinusoidal deviation, which is clearly resolved. Further measurements verify the use of the demonstrated principle as a breathing sensor. The breathing signals of male humans and a seven-week-old infant are presented, qualifying the IR-UWB radar principle as a useful tool for breath-rate determination.