Appearance Of Scene Research Articles

Assessment of VINO filters for correcting red-green Color Vision Deficiency.

In our ongoing research on the effectiveness of different passive tools for aiding Color Vision Deficiency (CVD) subjects, we have analyzed the VINO 02 Amp Oxy-Iso glasses using two strategies: 1) 52 observers were studied using four color tests (recognition, arrangement, discrimination, and color-naming); 2) the spectral transmittance of the lenses were used to model the color appearance of natural scenes for different simulated CVD subjects. We have also compared VINO and EnChroma glasses. The spectral transmission of the VINO glasses significantly changed color appearance. This change would allow some CVD subjects, above all the deutan ones, to be able to pass recognition tests but not the arrangement tests. To sum up, our results support the hypothesis that glasses with filters are unable to effectively resolve the problems related to color vision deficiency.

Semantic–geometric visual place recognition: a new perspective for reconciling opposing views

Human drivers are capable of recognizing places from a previous journey even when viewing them from the opposite direction during the return trip under radically different environmental conditions, without needing to look back or employ a camera or LIDAR sensor. Such navigation capabilities are attributed in large part to the robust semantic scene understanding capabilities of humans. However, for an autonomous robot or vehicle, achieving such human-like visual place recognition capability presents three major challenges: (1) dealing with a limited amount of commonly observable visual content when viewing the same place from the opposite direction; (2) dealing with significant lateral viewpoint changes caused by opposing directions of travel taking place on opposite sides of the road; and (3) dealing with a radically changed scene appearance due to environmental conditions such as time of day, season, and weather. Current state-of-the-art place recognition systems have only addressed these three challenges in isolation or in pairs, typically relying on appearance-based, deep-learnt place representations. In this paper, we present a novel, semantics-based system that for the first time solves all three challenges simultaneously. We propose a hybrid image descriptor that semantically aggregates salient visual information, complemented by appearance-based description, and augment a conventional coarse-to-fine recognition pipeline with keypoint correspondences extracted from within the convolutional feature maps of a pre-trained network. Finally, we introduce descriptor normalization and local score enhancement strategies for improving the robustness of the system. Using both existing benchmark datasets and extensive new datasets that for the first time combine the three challenges of opposing viewpoints, lateral viewpoint shifts, and extreme appearance change, we show that our system can achieve practical place recognition performance where existing state-of-the-art methods fail.

Controlled Lighting and Illumination-Independent Target Detection for Real-Time Cost-Efficient Applications. The Case Study of Sweet Pepper Robotic Harvesting.

Current harvesting robots are limited by low detection rates due to the unstructured and dynamic nature of both the objects and the environment. State-of-the-art algorithms include color- and texture-based detection, which are highly sensitive to the illumination conditions. Deep learning algorithms promise robustness at the cost of significant computational resources and the requirement for intensive databases. In this paper we present a Flash-No-Flash (FNF) controlled illumination acquisition protocol that frees the system from most ambient illumination effects and facilitates robust target detection while using only modest computational resources and no supervised training. The approach relies on the simultaneous acquisition of two images—with/without strong artificial lighting (“Flash”/“no-Flash”). The difference between these images represents the appearance of the target scene as if only the artificial light was present, allowing a tight control over ambient light for color-based detection. A performance evaluation database was acquired in greenhouse conditions using an eye-in-hand RGB camera mounted on a robotic manipulator. The database includes 156 scenes with 468 images containing a total of 344 yellow sweet peppers. Performance of both color blob and deep-learning detection algorithms are compared on Flash-only and FNF images. The collected database is made public.

Light Shapes

In computer graphics, illuminating a scene is a complex task, typically consisting of cycles of adjusting and rendering the scene to see the effects. We propose a technique for visualization of light as a tensor field via extracting its properties (i.e., intensity, direction, diffuseness) from (virtual) radiance measurements and showing these properties as a grid of shapes over a volume of a scene. Presented in the viewport, our visualizations give an understanding of the illumination conditions in the measured volume for both the local values and the global variations of light properties. Additionally, they allow quick inferences of the resulting visual appearance of (objects in) scenes without the need to render them. In our evaluation, observers performed at least as well using visualizations as using renderings when they were comparing illumination between parts of a scene and inferring the final appearance of objects in the measured volume. Therefore, the proposed visualizations are expected to help lighting artists by providing perceptually relevant information about the structure of the light field and flow in a scene.

Genologia lingwistyczna a badania nad komentarzem e-sportowym (problemy metodologiczne)

Gry komputerowe są niezwykle istotnym elementem współczesnej rzeczywistości. Ich popularność przyczyniła się do powstania sceny e-sportowej, a tym samym do zaistnienia nowego gatunku medialnego, jakim jest komentarz e-sportowy. W powyższym artykule ukazuję zastosowanie w analizie komentarza e-sportowego do meczów Counter Strike: Global Offensive metody badań genologicznych autorstwa Marii Wojtak. Zgodnie z założeniami autorki metody analizuję cztery aspekty gatunkowe. Pierwszym z nich jest struktura, a więc rama delimitacyjna oraz poszczególne segmenty, z jakich składa się cały komentarz. Kolejny aspekt, pragmatyczny, dotyczy relacji między nadawcą – komentatorem a odbiorcą – internautą. Dalej konieczny jest opis aspektu poznawczego, czyli tematyki, wartościowania i innych elementów tworzących językowy obraz świata, występujący w danym gatunku. Ostatnim aspektem są cechy stylistyczne charakterystyczne dla komentarza e-sportowego, takie jak potoczność, emocjonalność czy leksyka żargonowa. Główny wniosek płynący z artykułu wskazuje na stosowność zaadaptowania metody Marii Wojtak do badań nad nowym gatunkiem medialnym, jakim jest komentarz e-sportowy.

Stage-Wise Learning of Reaching Using Little Prior Knowledge.

In some manipulation robotics environments, because of the difficulty of precisely modeling dynamics and computing features which describe well the variety of scene appearances, hand-programming a robot behavior is often intractable. Deep reinforcement learning methods partially alleviate this problem in that they can dispense with hand-crafted features for the state representation and do not need pre-computed dynamics. However, they often use prior information in the task definition in the form of shaping rewards which guide the robot toward goal state areas but require engineering or human supervision and can lead to sub-optimal behavior. In this work we consider a complex robot reaching task with a large range of initial object positions and initial arm positions and propose a new learning approach with minimal supervision. Inspired by developmental robotics, our method consists of a weakly-supervised stage-wise procedure of three tasks. First, the robot learns to fixate the object with a 2-camera system. Second, it learns hand-eye coordination by learning to fixate its end-effector. Third, using the knowledge acquired in the previous steps, it learns to reach the object at different positions and from a large set of initial robot joint angles. Experiments in a simulated environment show that our stage-wise framework yields similar reaching performances, compared with a supervised setting without using kinematic models, hand-crafted features, calibration parameters or supervised visual modules.

Deep image-based relighting from optimal sparse samples

We present an image-based relighting method that can synthesize scene appearance under novel, distant illumination from the visible hemisphere, from only five images captured under pre-defined directional lights. Our method uses a deep convolutional neural network to regress the relit image from these five images; this relighting network is trained on a large synthetic dataset comprised of procedurally generated shapes with real-world reflectances. We show that by combining a custom-designed sampling network with the relighting network, we can jointly learn both the optimal input light directions and the relighting function. We present an extensive evaluation of our network, including an empirical analysis of reconstruction quality, optimal lighting configurations for different scenarios, and alternative network architectures. We demonstrate, on both synthetic and real scenes, that our method is able to reproduce complex, high-frequency lighting effects like specularities and cast shadows, and outperforms other image-based relighting methods that require an order of magnitude more images.

Seeing Beyond Foreground Occlusion: A Joint Framework for SAP-Based Scene Depth and Appearance Reconstruction

Foreground occlusion is a significant challenge in three-dimensional (3-D) reconstruction. In this paper, we first characterize the differences between multiview reconstruction with and without foreground occlusion. In order to reconstruct target scene, we attempt to see through the foreground occlusion using synthetic aperture photography. Different from existing methods, we propose a more generalized model for scene reconstruction, in which the target scene may not be fully observed by any of reference view. Assuming both scene depth and appearance are unknown, we reconstruct 3-D scene from camera array data by selecting optimal views with pixel based clustering. Then, we propose an iterative reconstruction approach in global optimization framework, in which we refine the reconstruction results by applying a coarse-to-fine strategy. Even when all views are partially occluded, our approach can recover accurate depth map as well as scene appearance using camera array data. Experimental results have indicated that the proposed approach is more robust to foreground occlusions and outperforms state-of-the-art approaches.

Multi-State Self-Learning Template Library Updating Approach for Multi-Camera Human Tracking in Complex Scenes

In multi-camera video tracking, the tracking scene and tracking-target appearance can become complex, and current tracking methods use entirely different databases and evaluation criteria. Herein, for the first time to our knowledge, we present a universally applicable template library updating approach for multi-camera human tracking called multi-state self-learning template library updating (RS-TLU), which can be applied in different multi-camera tracking algorithms. In RS-TLU, self-learning divides tracking results into three states, namely steady state, gradually changing state, and suddenly changing state, by using the similarity of objects with historical templates and instantaneous templates because every state requires a different decision strategy. Subsequently, the tracking results for each state are judged and learned with motion and occlusion information. Finally, the correct template is chosen in the robust template library. We investigate the effectiveness of the proposed method using three databases and 42 test videos, and calculate the number of false positives, false matches, and missing tracking targets. Experimental results demonstrate that, in comparison with the state-of-the-art algorithms for 15 complex scenes, our RS-TLU approach effectively improves the number of correct target templates and reduces the number of similar templates and error templates in the template library.

Moving human tracking across multi-camera based on artificial immune random forest and improved colour-texture feature fusion

ABSTRACTIn multi-camera video tracking, the tracking scene and tracking-target appearance can become complex. To finish multi-camera tracking in these challenging, we first utilize an improved brightness transfer function to establish matching tasks of different cameras and to reduce the influence of brightness changes between multiple cameras. Then, we proposes an improved colour-texture feature fusion (ICTFF) that is composed of the colour features and texture features for multi-camera human tracking in non-overlapping field of view. It’s the first time to use artificial immune random forest with fully exploit the linear combination information of the feature and colour feature that can achieving the optimization of the number of decision trees and the effective classification of the target feature. Compared with state-of-the-art algorithms, our ICTFF algorithm can significantly improve the tracking accuracy in some complex scenes, such as changing the speed, changing the direction and appearance of the target.

24/7 Place Recognition by View Synthesis.

We address the problem of large-scale visual place recognition for situations where the scene undergoes a major change in appearance, for example, due to illumination (day/night), change of seasons, aging, or structural modifications over time such as buildings being built or destroyed. Such situations represent a major challenge for current large-scale place recognition methods. This work has the following three principal contributions. First, we demonstrate that matching across large changes in the scene appearance becomes much easier when both the query image and the database image depict the scene from approximately the same viewpoint. Second, based on this observation, we develop a new place recognition approach that combines (i) an efficient synthesis of novel views with (ii) a compact indexable image representation. Third, we introduce a new challenging dataset of 1,125 camera-phone query images of Tokyo that contain major changes in illumination (day, sunset, night) as well as structural changes in the scene. We demonstrate that the proposed approach significantly outperforms other large-scale place recognition techniques on this challenging data.

Refining raw pixel values using a value error model to drive texture synthesis

The goal in photography is generally the construction of a model of scene appearance. Unfortunately, statistical variations introduced by photon shot and other noise introduce errors in the raw value reported for each pixel sample. Rather than simply accepting those values as the best raw representation, the current work treats them as initial estimates of the correct values, and computes an error model for each pixel's value. The value error models for all pixels in an image are then used to drive a type of texture synthesis which refines the pixel value estimates, potentially increasing both accuracy and precision of each value. Each refined raw pixel value is synthesized from the value estimates of a plurality of pixels with overlapping error bounds and similar context within the same image. The error modeling and texture synthesis algorithms are implemented in and evaluated using KREMY (KentuckY Raw Error Modeler, pronounced "creamy"), a free software tool created for this purpose.

Depth-Compressed Expression for Providing Natural, Visual Experiences with Integral 3D Displays

Providing a natural 3D visualization is a major challenge in 3D display technologies. Although 3D displays with light-ray reconstruction have been demonstrated, displayable 3D scenes are selective because their depth-reconstruction range is restricted. Here, we attempt to expand the range virtually by introducing "depth-compressed expressions," in which the depth of 3D scenes are compressed or modified in the axial direction so that the appearances of depth-compressed scenes is kept natural for viewers. With a simulated system of an autostereoscopic 3D display with light-ray reconstruction, we investigated how large the depth range needed to be to show the depth-compressed scenes without inducing unnaturalness in viewers. Using a linear depthcompression method—the simplest way of depth-compression—we found that viewers did not feel unnaturalness for the depthcompressed scenes that were expressed within at most half the depth range of the originals. These results gave us a design goal in developing 3D displays for high quality 3D visualization.

Towards a Perceptually-Motivated Color Space for High Dynamic Range Imaging

To reproduce the appearance of real world scenes, a number of color appearance models have been proposed thanks to adapted psycho-visual experiments. Most of them were designed and intended for a limited dynamic range, or address only dynamic range compression applications. However, given the increasing availability of displays with higher luminance and contrast ranges, a detailed analysis of appearance attributes is also necessary for dynamic range expansion scenarios. In this study, we propose a psycho-visual experimental setup, designed by adapting and combining the adjustment and partition scaling methodologies, which we employ for measuring perceptual colorfulness of color patches with different levels of lightness, chroma and hue. The proposed setup reduces the complexity and increases the efficiency relative to previous experimental setups and allows both expert and non-expert participants to be included. From the collected data, a modified color space is obtained and a new saturation model for dynamic range compression and expansion is derived for high dynamic range imaging.

Reconstruction of Crime Scene, a Forensic Technique used in Substantiating Alleged Homicide into Accidental Death: A Case Study

Crime scene reconstruction is the process of determining or eliminating the events that occurred at the crime scene by analysis of physical evidences. Through this paper we will discuss a case of accident that was turned into murder (i.e. Under Section 302 of Indian Penal Court) because the family members of the deceased alleged that the deceased was murdered and then dragged by tying to a vehicle to dispose off the body and to destroy the evidences. In this paper we discuss how this sensational case was solved by the scientists of Forensic Science Laboratory, Haryana through the reconstruction of crime scene by analyzing crime scene appearance, the locations and positions of the physical evidence, and the injuries to the deceased. This involved scientific crime scene investigation, interpretation of patterned evidence at the scene, laboratory testing of the physical evidence, systematic study of related case information, and the logical formulation of a theory.

Detecting and representing predictable structure during auditory scene analysis.

We use psychophysics and MEG to test how sensitivity to input statistics facilitates auditory-scene-analysis (ASA). Human subjects listened to 'scenes' comprised of concurrent tone-pip streams (sources). On occasional trials a new source appeared partway. Listeners were more accurate and quicker to detect source appearance in scenes comprised of temporally-regular (REG), rather than random (RAND), sources. MEG in passive listeners and those actively detecting appearance events revealed increased sustained activity in auditory and parietal cortex in REG relative to RAND scenes, emerging ~400 ms of scene-onset. Over and above this, appearance in REG scenes was associated with increased responses relative to RAND scenes. The effect of temporal structure on appearance-evoked responses was delayed when listeners were focused on the scenes relative to when listening passively, consistent with the notion that attention reduces 'surprise'. Overall, the results implicate a mechanism that tracks predictability of multiple concurrent sources to facilitate active and passive ASA.

Appearance based background subtraction for PTZ cameras

Traditional background subtraction algorithms assume the camera is static and are based on simple per-pixel models of scene appearance. This leads to false detections when the camera moves. While this can sometimes be addressed by online image registration, this approach is prone to dramatic failures and long-term drift. We present a novel background subtraction algorithm designed for pan-tilt-zoom cameras that overcomes this challenge without the need for explicit image registration. The proposed algorithm automatically trains a discriminative background model, which is global in the sense that it is the same regardless of image location. Our approach first extracts multiple features from across the image and uses principal component analysis for dimensionality reduction. The extracted features are then grouped to form a Bag of Features. A global background model is then learned from the bagged features using Support Vector Machine. The proposed approach is fast and accurate. Having a single global model makes it computationally inexpensive in comparison to traditional pixel-wise models. It outperforms several state-of-the-art algorithms on the CDnet 2014 pan-tilt-zoom and baseline categories and Hopkins155 dataset. In particular, it achieves an F-Measure of 75.41% on the CDnet dataset PTZ category, significantly better than the previously reported best score of 62.07%. These results show that by removing the coupling between detection model and spatial location, we significantly increase the robustness to camera motion.

Cloudmaps from static ground-view video

Cloud shadows dramatically affect the appearance of outdoor scenes. We describe three approaches that use video of cloud shadows to estimate a cloudmap, a spatio-temporal function that represents the clouds passing over the scene. Two of the methods make assumptions about the camera and/or scene geometry. The third method uses techniques from manifold learning and does not require such assumptions. None of the methods require directly viewing the clouds, but instead use the pattern of intensity changes caused by the cloud shadows. An accurate estimate of the cloudmap has potential applications in solar power estimation and forecasting, surveillance, and graphics. We present a quantitative evaluation of our methods on synthetic scenes and show qualitative results on real scenes. We also demonstrate the use of a cloudmap for foreground object detection and video editing.

Nonrigid registration of remote sensing images via sparse and dense feature matching.

In this paper, we propose a novel formulation for building pixelwise alignments between remote sensing images under nonrigid transformation based on matching both sparsely and densely sampled features. Our formulation contains two coupling variables: the nonrigid geometric transformation and the discrete dense flow field. To match sparse features, we fit a geometric transformation specified in a reproducing kernel Hilbert space and impose a locally linear constraint to regularize the transformation. To match dense features, we compute a dense flow field by using a formulation analogous to scale invariant feature transform (SIFT) flow which allows nonrigid matching across different scene appearances. An additional term is introduced to ensure the coherence between the two variables, and we alternatively solve for one variable under the assumption that the other is known. Extensive experiments on both synthetic and real remote sensing images demonstrate that our approach greatly outperforms state-of-the-art methods, particularly when the data contain severe degradations.

Testing models of peripheral encoding using metamerism in an oddity paradigm.

Most of the visual field is peripheral, and the periphery encodes visual input with less fidelity compared to the fovea. What information is encoded, and what is lost in the visual periphery? A systematic way to answer this question is to determine how sensitive the visual system is to different kinds of lossy image changes compared to the unmodified natural scene. If modified images are indiscriminable from the original scene, then the information discarded by the modification is not important for perception under the experimental conditions used. We measured the detectability of modifications of natural image structure using a temporal three-alternative oddity task, in which observers compared modified images to original natural scenes. We consider two lossy image transformations, Gaussian blur and Portilla and Simoncelli texture synthesis. Although our paradigm demonstrates metamerism (physically different images that appear the same) under some conditions, in general we find that humans can be capable of impressive sensitivity to deviations from natural appearance. The representations we examine here do not preserve all the information necessary to match the appearance of natural scenes in the periphery.