Photometric Appearance Research Articles

3D hypothesis clustering for cross-view matching in multi-person motion capture

We present a multiview method for markerless motion capture of multiple people. The main challenge in this problem is to determine cross-view correspondences for the 2D joints in the presence of noise. We propose a 3D hypothesis clustering technique to solve this problem. The core idea is to transform joint matching in 2D space into a clustering problem in a 3D hypothesis space. In this way, evidence from photometric appearance, multiview geometry, and bone length can be integrated to solve the clustering problem efficiently and robustly. Each cluster encodes a set of matched 2D joints for the same person across different views, from which the 3D joints can be effectively inferred. We then assemble the inferred 3D joints to form full-body skeletons for all persons in a bottom–up way. Our experiments demonstrate the robustness of our approach even in challenging cases with heavy occlusion, closely interacting people, and few cameras. We have evaluated our method on many datasets, and our results show that it has significantly lower estimation errors than many state-of-the-art methods.

MIRACLES: atmospheric characterization of directly imaged planets and substellar companions at 4–5 μm

Context. Directly imaged planets and substellar companions are key targets for the characterization of self-luminous atmospheres. Their photometric appearance at 4–5 μm is sensitive to the chemical composition and cloud content of their atmosphere. Aims. We aim to systematically characterize the atmospheres of directly imaged low-mass companions at 4–5 μm. We want to homogeneously process the data, provide robust flux measurements, and compile a photometric library at thermal wavelengths of these mostly young, low-gravity objects. In this way, we want to find trends related to their spectral type and surface gravity by comparing with isolated brown dwarfs and predictions from atmospheric models. Methods. We used the high-resolution, high-contrast capabilities of NACO at the Very Large Telescope (VLT) to directly image the companions of HIP 65426, PZ Tel, and HD 206893 in the NB4.05 and/or M′ filters. For the same targets, and additionally β Pic, we also analyzed six archival VLT/NACO datasets which were taken with the NB3.74, L′, NB4.05, and M′ filters. The data processing and photometric extraction of the companions was done with PynPoint while the species toolkit was used to further analyze and interpret the fluxes and colors. Results. We detect for the first time HIP 65426 b, PZ Tel B, and HD 206893 B in the NB4.05 filter, PZ Tel B and HD 206893 B in the M′ filter, and β Pic b in the NB3.74 filter. We provide calibrated magnitudes and fluxes with a careful analysis of the error budget, both for the new and archival datasets. The L′–NB4.05 and L′–M′ colors of the studied sample are all red while the NB4.05–M′ color is blue for β Pic b, gray for PZ Tel B, and red for HIP 65426 b and HD 206893 B (although typically with low significance). The absolute NB4.05 and M′ fluxes of our sample are all larger than those of field dwarfs with similar spectral types. Finally, the surface gravity of β Pic b has been constrained to log g = 4.17−0.13+0.10 dex from its photometry and dynamical mass. Conclusions. A red color at 3–4 μm and a blue color at 4–5 μm might be (partially) caused by H2O and CO absorption, respectively, which are expected to be the most dominant gaseous opacities in hot (Teff ≳ 1300 K) atmospheres. The red characteristics of β Pic b, HIP 65426 b, and HD 206893 B at 3–5μm, as well as their higher fluxes in NB4.05 and M′ compared to field dwarfs, indicate that cloud densities are enhanced close to the photosphere as a result of their low surface gravity.

Unified Diagnosis Framework for Automated Nuclear Cataract Grading Based on Smartphone Slit-Lamp Images

Cataract constitutes half of the blindness cases worldwide; hence, detecting and treating cataracts in a timely manner are effective strategies for blindness prevention. Recently, methods of detecting cataracts through deep learning are flourishing; however, the task of improving the grading mechanism is still the priority in the research field. This study evaluates the classification capability of the automated nuclear cataract detection algorithm using ocular images captured by smartphone-based slit-lamp. The task of the algorithm is to automatically detect cataract severity in terms of the photometric appearance of the nuclear region of the crystalline lens of the eyes. The nuclear region of the ocular lens was localized by YOLOv3. Subsequently, the combination of a deep learning network, ShuffleNet, and a support vector machine (SVM) classifier was used to grade cataract severity, evaluating the gray conjugate features of the nuclear region. Using the trained algorithm, 819 anterior ocular images captured by smartphone-based slit-lamp were utilized to evaluate the algorithm's performance. The accuracy was 93.5% with Kappa of 95.4% and F1 of 92.3%. The AUC was 0.9198. The proposed validation method could evaluate a cataract severity in 29 ms and the entire classification process in less than 1s. This study can improve the accuracy of the examination, reduce misdiagnosis rate and the difficulty of the doctor's examination. The addition of scoring system can improve the quality of pictures obtained by non-ophthalmologists. The method is especially suitable for cataract screening in the underdeveloped areas or areas which are in shortage of ophthalmic resources. It can also improve the accessibility of ophthalmic medical treatment.

Enhanced skeleton and face 3D data for person re-identification from depth cameras

Person re-identification is typically performed using 2D still images or videos, where photometric appearance is the main visual cue used to discover the presence of a target subject when switching from different camera views across time. This invalidates any application where a person may change dress across subsequent acquisitions as can be the case of patients monitoring at home. Differently from RGB data, 3D information as acquired by depth cameras can open the way to person re-identification based on biometric cues such as distinguishing traits of the body or face. However, the accuracy of skeleton and face geometry extracted from depth data is not always adequate to enable person recognition, since both these features are affected by the pose of the subject and the distance from the camera. In this paper, we propose a method to derive a robust skeleton representation from a depth sequence and to complement it with a highly discriminative face feature. This is obtained by selecting skeleton and face samples based on their quality and using the temporal redundancy across the sequence to derive and refine cumulated models for both of them. Extracting skeleton and face features from such cumulated models and combining them for the recognition allow us to improve rank-1 re-identification accuracy compared to individual cues. A comparative evaluation on three benchmark datasets also shows results at the state-of-the-art.

On the origins of enigmatic stellar populations in Local Group galactic nuclei

We consider the origins of enigmatic stellar populations in four Local Group galactic nuclei, specifically the Milky Way, M31, M32 and M33. These are centrally concentrated blue stars, found in three out of the four nuclear star clusters (NSCs) considered here. Their origins are unknown, but could include blue straggler (BS) stars, extended horizontal branch stars and young recently formed stars. Here, we calculate order-of-magnitude estimates for various collision rates, as a function of the host NSC environment and distance from the cluster centre. These rates are sufficiently high that BSs, formed via collisions between main sequence (MS) stars, could contribute non-negligibly ($\sim$ 1-10% in mass) to every surface brightness profile, with the exception of the Milky Way. Stellar evolution models show that the envelopes of red giant branch (RGB) stars must be nearly completely stripped to significantly affect their photometric appearance, which requires multiple collisions. Hence, the collision rates for individual RGB stars are only sufficiently high in the inner $\lesssim$ 0.1 pc of M31 and M32 for RGB destruction to occur. Collisions between white dwarfs and MS stars, which should ablate the stars, could offer a steady and significant supply of gas in every NSC in our sample. The gas could either fragment to form new stars, or accrete onto old MS stars already present. Thus, collisional processes could contribute significantly to the observed blue excesses in M31 and M33; future studies should be aimed at better constraining theoretical predictions to compliment existing and future observational data.

Theoretical evolutionary framework for low-mass stars in massive star clusters

AbstractRecent spectroscopic and photometric surveys of Galactic Globular Clusters have shown that these stellar systems host distinct sub-populations of stars characterised by peculiar chemical patterns. In the following we wish to address the issue of how these specific chemical patterns affect both the structural and evolutionary properties of stars as well as their spectral energy distribution. The implications of these effects on the photometric appearance of multiple stellar populations in different photometric planes are also briefly discussed.

Cross-calibration of time-of-flight and colour cameras

Time-of-flight cameras provide depth information, which is complementary to the photometric appearance of the scene in ordinary images. It is desirable to merge the depth and colour information, in order to obtain a coherent scene representation. However, the individual cameras will have different viewpoints, resolutions and fields of view, which means that they must be mutually calibrated. This paper presents a geometric framework for the resulting multi-view and multi-modal calibration problem. It is shown that three-dimensional projective transformations can be used to align depth and parallax-based representations of the scene, with or without Euclidean reconstruction. A new evaluation procedure is also developed; this allows the reprojection error to be decomposed into calibration and sensor-dependent components. The complete approach is demonstrated on a network of three time-of-flight and six colour cameras. The applications of such a system, to a range of automatic scene-interpretation problems, are discussed.

Making in-Front-of Cars Transparent: Sharing First-Person-Views via Dashcam

Visual obstruction caused by a preceding vehicle is one of the key factors threatening driving safety. One possible solution is to share the first-person-view of the preceding vehicle to unveil the blocked field-of-view of the following vehicle. However, the geometric inconsistency caused by the camera-eye discrepancy renders view sharing between different cars a very challenging task. In this paper, we present a first-person-perspective image rendering algorithm to solve this problem. Firstly, we contour unobstructed view as the transferred region, then by iteratively estimating local homography transformations and performing perspective-adaptive warping using the estimated transformations, we are able to locally adjust the shape of the unobstructed view so that its perspective and boundary could be matched to that of the occluded region. Thus, the composited view is seamless in both the perceived perspective and photometric appearance, creating an impression as if the preceding vehicle is transparent. Our system improves the driver's visibility and thus relieves the burden on the driver, which in turn increases comfort. We demonstrate the usability and stability of our system by performing its evaluation with several challenging data sets collected from real-world driving scenarios.

Editorial for the Special Issue on Photometric Analysis for Computer Vision

This special issue arises from papers presented at the Workshop on Photometric Analysis For Computer Vision, held on October 14, 2007 in conjunction with the 11th International Conference on Computer Vision Conference in Rio de Janeiro, Brazil. Photometric analysis is a central aspect of computer vision theory and practice. The way an image looks depends on many factors, including geometry, illumination and reflectance properties of the objects. For transparent or translucent objects, or for objects composed by multiple coatings, the factors are even more numerous (refraction, subsurface scattering, ...). The laws combining these components are diverse and complex. This complexity makes computer vision tasks even more difficult and typically causes the failure of methods based on simple models. A typical example could be problems caused by specularities in the stereo-vision problem; proposed methods usually assume that the scene is perfectly diffuse. Feature tracking and matching is another example since the photometric appearance of objects can change when they or the camera move. On the one hand, from a theoretical as well as from a computational point of view, a better understanding and handling of these factors should improve robustness to photometric effects. On the other hand, this allows not only to circumvent problems but also to gather valuable information which can be practically exploited in computer vision tasks. One example is the information provided by shading and shadows.

Mars Express High Resolution Stereo Camera spectrophotometric data: Characteristics and science analysis

The High Resolution Stereo Camera (HRSC) on the Mars Express (MEx) spacecraft in orbit about Mars is delivering images of the Mars surface and atmosphere from its high‐inclination, elliptical orbit that are intended to cover most of the Mars surface by the end of the mission. These are acquired in four specific spectral passbands that sample the scene at different parts of the extended visual spectrum from near 0.4 μm to about 1μm and in five additional channels, four used for stereo and one nadir, that use the same fifth spectral passband. These channels view from different angles and can be used for photometric and spectral analysis. The data allow a variety of studies of the surface and atmosphere of Mars, but these require detailed understanding of the data characteristics, as there are several effects, including of the observation geometry and the atmosphere, that need to be treated. This article is a report to the science community on the current understanding of the HRSC color data after an initial analysis and also a demonstration of their capabilities for study of Mars, which, from this study, appear to be considerable. We first discuss the characteristics and calibration of the HRSC spectral data. The atmospheric influence on the photometric appearance of the surface materials is explored, and methods of compensating are considered. Reflectances are calculated for surface units by assuming the dark material is basalt and scaling the entire HRSC scene using a laboratory basalt spectrum for the selected dark material area; the results and the correction factors for different orbits are compared. We present several examples of what could become a large volume of future science results utilizing the HRSC spectrophotometric data in conjunction with photo‐interpretation. For example, we find relatively few but very important distinct spectral components at the HRSC resolution, including what we interpret as iron oxide‐rich material, unoxidized basalt, and polar ice, consistent with past works. The major spectral components are distributed in ways that suggest compositional mixing and the need to revise the current geologic understanding of major regions.

Extinction Effects on Galaxy Counts for Bulge‐Disk Galaxies

We examine extinction effects of interstellar dust on galaxy counts for bulge-disk galaxies. Evolution of dust mass is calculated simultaneously with galaxy chemical evolution equations under the context of the so-called infall model, and three-dimensional radiative transfer calculation is performed with scattering effects. Prevailing importance of the bulge on dust mass evolution and photometric appearance in the early formation epoch of spiral galaxies is demonstrated. We here present the galaxy number-magnitude relation, N(m), in the B and K bands and the number-redshift relation, N(z), in the B band with internal absorption taken into account. We confirm that N(m) is differently affected by evolution and extinction between the B and K bands, and that the K band is more suitable to distinguish q0 values. We find that in the B band the extinction effects give a drastic change in N(m) in such a way that N(m) looks as if galaxies show only mild luminosity evolution. Three kinds of models—evolutionary, evolutionary with dust, and nonevolutionary—can be most clearly distinguished from the observations of N(z) in z = 0.8-1.2 in the range of magnitude B = 20-24. We conclude that the dusty evolution model of galaxies with properly adopted parameters can better satisfy both N(m) and N(z) distributions simultaneously than the others.