Artificial Scenes Research Articles

Award for Distinguished Scientific Early Career Contributions to Psychology: Marc G. Berman.

The APA Awards for Distinguished Scientific Early Career Contributions to Psychology recognize psychologists who have demonstrated excellence early in their careers and have held a doctoral degree for no more than 9 years. One of the 2019 award winners is Marc G. Bergman "for innovative research delving into environmental influences on cognition and affect. Marc G. Berman's research has engaged behavioral, neuroscientific, and computational techniques to expose how natural and artificial scenes can alter cognitive processes and induce affective reactions. In addition, his research seeks to understand the underlying visual computational mechanisms that underlie these effects. This work spans the study of healthy individuals and individuals with various cognitive dysfunctions due to underlying disease states." (PsycINFO Database Record (c) 2019 APA, all rights reserved).

ParallelEye-CS: A New Dataset of Synthetic Images for Testing the Visual Intelligence of Intelligent Vehicles

Virtual simulation testing is becoming indispensable for the intelligence testing of intelligent vehicles. However, even the most advanced simulation software provides rather limited test conditions. In the long run, intelligent vehicles are expected to work at SAE (Society of Automotive Engineers) level 4 or level 5. Researchers should make full use of virtual simulation scenarios to test the visual intelligence algorithms of intelligent vehicles under various imaging conditions. In this paper, we create realistic artificial scenes to simulate the self-driving scenarios, and collect a dataset of synthetic images from the virtual driving scenes, named “ParallelEye-CS”. In the artificial scenes, we can flexibly change environmental conditions and automatically acquire accurate and diverse ground-truth labels. As a result, ParallelEye-CS has six ground-truth labels and includes twenty types of tests, which are divided into normal, environmental, and difficult tasks. Furthermore, we utilize ParallelEye-CS in combination with other publicly available datasets to conduct experiments for visual object detection. The experimental results indicate that: 1) object detection algorithms of intelligent vehicles can be tested under various scenario challenges; 2) mixed dataset can improve the accuracy of object detection algorithms, but domain shift is a serious issue worthy of attention.

The ParallelEye Dataset: A Large Collection of Virtual Images for Traffic Vision Research

Dataset plays an essential role in the training and testing of traffic vision algorithms. However, the collection and annotation of images from the real world is time-consuming, labor-intensive, and error-prone. Therefore, more and more researchers have begun to explore the virtual dataset, to overcome the disadvantages of real datasets. In this paper, we propose a systematic method to construct large-scale artificial scenes and collect a new virtual dataset (named “ParallelEye”) for the traffic vision research. The Unity3D rendering software is used to simulate environmental changes in the artificial scenes and generate ground-truth labels automatically, including semantic/instance segmentation, object bounding boxes, and so on. In addition, we utilize ParallelEye in combination with real datasets to conduct experiments. The experimental results show the inclusion of virtual data helps to enhance the per-class accuracy in object detection and semantic segmentation. Meanwhile, it is also illustrated that the virtual data with controllable imaging conditions can be used to design evaluation experiments flexibly.

Wide Field of View Microwave Interferometric Radiometer Imaging

In microwave interferometric radiometers with a large field of view, as for example the Microwave Imaging Radiometer with Aperture Synthesis (MIRAS) onboard the Soil Moisture and Ocean Salinity (SMOS) satellite, one of the major causes of reconstruction error is the contribution to the visibility of the brightness temperature outside the fundamental period, defined on the basis of reciprocal grids. A mitigation method consisting of estimating this contribution through the application of a brightness temperature model outside the fundamental period is proposed. The main advantage is that it does not require any a posteriori addition of artificial scenes to the reconstructed image. Additionally, a method to avoid the sophisticated matrix regularization and inversion techniques usually applied in microwave interferometry is presented. Image reconstruction algorithms are implemented on a minimum grid size in order to maximize their numerical efficiency. An improved method to apply an apodization window to the reconstructed image for reducing Gibbs oscillations is also proposed. All procedures are generally described considering the single polarization case and successively implemented applying the MIRAS layout in both its single polarization and full polarimetric modes. Results show similar performance of the proposed algorithm with respect to the nominal one applied by SMOS. All algorithms are implemented in the MIRAS Testing Software and have been successfully used for scientific studies by other teams.

CloudifierNet - Deep Vision Models for Artificial Image Processing

Today, more and more, it is necessary that most applications and documents developed in previous or current technologies to be accessible online on cloud-based infrastructures. That’s why the migration of legacy systems including their hosts of documents to new technologies and online infrastructures, using modern Artificial Intelligence techniques, is absolutely necessary.With the advancement of Artificial Intelligence and Deep Learning with its multitude of applications, a new area of research is emerging – that of automated systems development and maintenance. The underlying work objective that led to this paper aims to research and develop truly intelligent systems able to analyze user interfaces from various sources and generate real and usable inferences ranging from architecture analysis to actual code generation. One key element of such systems is that of artificial scene detection and analysis based on deep learning computer vision systems. Computer vision models and particularly deep directed acyclic graphs based on convolutional modules are generally constructed and trained based on natural images datasets. Due to this fact, the models will develop during the training process natural image feature detectors apart from the base graph modules that will learn basic primitive features. In the current paper, we will present the base principles of a deep neural pipeline for computer vision applied to artificial scenes (scenes generated by user interfaces or similar). Finally, we will present the conclusions based on experimental development and benchmarking against state-of-the-art transfer-learning implemented deep vision models.

Parallel Motion Planning: Learning a Deep Planning Model against Emergencies

To handle the issue of preventing emergencies for motion planning in autonomous driving, we present a novel parallel motion planning framework. Artificial traffic scenes are firstly constructed based on real traffic scenes. A deep planning model which can learn from both real and artificial scenes is developed and used to make planning decisions in an end-to-end mode. To prevent emergencies, a generative adversarial networks (GAN) model is designed and learns from the artificial emergencies from artificial traffic scenes. During deployment, the well-trained GAN model is used to generate multiple virtual emergencies based on the current real scene, and the well-trained planning model simultaneously makes different planning decisions for both virtual scenes and the current scenes. The final planning decision is made by comprehensively analyzing observations and virtual emergencies. Through parallel planning, the planner can timely make rational decision without a large number of calculations when an emergency occurs.

Improving the Spatial Bias Correction Algorithm in SMOS Image Reconstruction Processor: Validation of Soil Moisture Retrievals With In Situ Data

SMOS is a space mission led by the European Space Agency and designed to provide global maps of Soil Moisture and Ocean salinity, two important geophysical parameters for understanding the water cycle variations and climate change. The SMOS payload is a 2-D interferometer operating at L-band that consists of 69 elementary antennas located along a Y-shaped structure. Important spatial biases persist in the retrieved brightness temperature (BT) images mainly due to the phenomenon of aliasing inside the field of view of SMOS but also due to the Gibbs oscillations near land/ocean transitions. To minimize these biases, a differential image reconstruction algorithm is used in the operational processor that reduces the contrast of the image to be retrieved. To do that, the contribution of a constant artificial temperature map is removed from the measurements prior to reconstruction and then added back after the reconstruction. In this paper, we show that strong residual biases are still present in the retrieved images. To reduce them, we propose to improve the bias correction algorithm by using a more realistic artificial temperature scene based on separating the land and ocean regions and assigning a constant temperature over land and a Fresnel BT model over the ocean. The artificial scene is also improved by means of representing each pixel by its water fraction percentage to smooth the land/ocean transitions. The improved algorithm is validated over the ocean by comparing the retrieved temperatures to a forward geophysical model but also over land by comparing the retrieved soil moisture to in situ measurements.

Localizing salient body motion in multi-person scenes using convolutional neural networks

With modern computer vision techniques being successfully developed for a variety of tasks, extracting meaningful knowledge from complex scenes with multiple people still poses problems. Consequently, experiments with application-specific motion, such as gesture recognition scenarios, are often constrained to single person scenes in the literature. Therefore, in this paper we address the challenging task of detecting salient body motion in scenes with more than one person. We propose a neural architecture that only reacts to a specific kind of motion in the scene: A limited set of body gestures. The model is trained end-to-end, thereby avoiding hand-crafted features and the strong reliance on pre-processing as it is prevalent in similar studies. The presented model implements a saliency mechanism that reacts to body motion cues which have not been included in previous computational saliency systems. Our architecture consists of a 3D Convolutional Neural Network that receives a frame sequence as its input and localizes active gesture movement. To train our network with a large data variety, we introduce an approach to combine Kinect recordings of one person into artificial scenes with multiple people, yielding a large diversity of scene configurations in our dataset. We performed experiments using these sequences and show that the proposed model is able to localize the salient body motion of our gesture set. We found that 3D convolutions and a baseline model with 2D convolutions perform surprisingly similar on our task. Our experiments revealed the influence of gesture characteristics on how well they can be learned by our model. Given a distinct gesture set and computational restrictions, we conclude that using 2D convolutions might often perform equally well.

Binocular disparity can augment the capacity of vision without affecting subjective experience of depth

Binocular disparity results in a tangible subjective experience of three-dimensional world, but whether disparity also augments objective perceptual performance remains debated. We hypothesized that the improved coding of depth enabled by binocular disparity allows participants to individuate more objects at a glance as the objects can be more efficiently differentiated from each other and the background. We asked participants to enumerate objects in briefly presented naturalistic (Experiment 1) and artificial (Experiment 2) scenes in immersive virtual reality. This type of enumeration task yields well-documented capacity limits where up to 3–4 items can be enumerated rapidly and accurately, known as subitizing. Our results show that although binocular disparity did not yield a large general improvement in enumeration accuracy or reaction times, it improved participants’ ability to process the items right after the limit of perceptual capacity. Binocular disparity also sped-up response times by 27 ms on average when artificial stimuli (cubes) were used. Interestingly, the influence of disparity on subjectively experienced depth revealed a clearly different pattern than the influence of disparity on objective performance. This suggests that the functional and subjective sides of stereopsis can be dissociated. Altogether our results suggest that binocular disparity may increase the number of items the visual system can simultaneously process. This may help animals to better resolve and track objects in complex, cluttered visual environments.

Contribution of central vision and peripheral vision to categorization of natural and artificial scenes

Contribution of central vision and peripheral vision to categorization of natural and artificial scenes

Fractal dimension and the navigational information provided by natural scenes.

Recent work on virtual reality navigation in humans has suggested that navigational success is inversely correlated with the fractal dimension (FD) of artificial scenes. Here we investigate the generality of this claim by analysing the relationship between the fractal dimension of natural insect navigation environments and a quantitative measure of the navigational information content of natural scenes. We show that the fractal dimension of natural scenes is in general inversely proportional to the information they provide to navigating agents on heading direction as measured by the rotational image difference function (rotIDF). The rotIDF determines the precision and accuracy with which the orientation of a reference image can be recovered or maintained and the range over which a gradient descent in image differences will find the minimum of the rotIDF, that is the reference orientation. However, scenes with similar fractal dimension can differ significantly in the depth of the rotIDF, because FD does not discriminate between the orientations of edges, while the rotIDF is mainly affected by edge orientation parallel to the axis of rotation. We present a new equation for the rotIDF relating navigational information to quantifiable image properties such as contrast to show (1) that for any given scene the maximum value of the rotIDF (its depth) is proportional to pixel variance and (2) that FD is inversely proportional to pixel variance. This contrast dependence, together with scene differences in orientation statistics, explains why there is no strict relationship between FD and navigational information. Our experimental data and their numerical analysis corroborate these results.

Training and testing object detectors with virtual images

In the area of computer vision, deep learning has produced a variety of state-of-the-art models that rely on massive labeled data. However, collecting and annotating images from the real world is too demanding in terms of labor and money investments, and is usually inflexible to build datasets with specific characteristics, such as small area of objects and high occlusion level. Under the framework of Parallel Vision, this paper presents a purposeful way to design artificial scenes and automatically generate virtual images with precise annotations. A virtual dataset named ParallelEye is built, which can be used for several computer vision tasks. Then, by training the DPM U+0028 Deformable parts model U+0029 and Faster R-CNN detectors, we prove that the performance of models can be significantly improved by combining ParallelEye with publicly available real-world datasets during the training phase. In addition, we investigate the potential of testing the trained models from a specific aspect using intentionally designed virtual datasets, in order to discover the flaws of trained models. From the experimental results, we conclude that our virtual dataset is viable to train and test the object detectors.

Espaços privados e usos colectivos: Do admirável mundo novo do consumo às novas tipologias comerciais de Lisboa

La consolidación de nuevos hábitos asociados al aumento del tiempo libre, después de la segunda mitad del s. XX, intensifica y diversifica las actividades comerciales. Comercio, cultura y ocio organizan el espacio urbano contemporáneo. Los espacios comerciales son elementos estructurantes y polarizadores del tejido de la ciudad. El consumo está lleno de significado social. Todo aspira a ser transformado en consumo. Los centros comerciales se dotan de sistemas de vigilancia y control, usados como elemento de bienestar entre los ciudadanos. Se crean ambientes confortables donde las preocupaciones estén ausentes. Es el mundo de la imagen. El deseo de contacto con el sentido tradicional del espacio público hace que se creen copias de lugares, pedazos de realidad en escenarios prefabricados.En un principio surgen centros comerciales en la zona metropolitana de Lisboa y Oporto, pero ahora también se encuentran en ciudades más pequeñas. El comercio entra en la organización territorial, siendo usado como instrumento de intervención urbanística.

Referenceless Phase Holography for 3D Imaging

Referenceless phase holography generates the full (amplitude and phase) optical field if intensity and phase of this field are given as numerical data. It is based on the interference of two pure phase fields which are produced by reflection of two mutually coherent plane waves at two phase modulating spatial light modulators of the liquid crystal on silicon type. Thus any optical field of any real or artificial 3D scene can be displayed. This means that referenceless phase holography is a promising method for future 3D imaging, e. g. in 3D television. The paper introduces the theory of the method and presents three possible interferometer arrangements, for the first time the Mach–Zehnder and the grating interferometer adapted to this application. The possibilities and problems in calculating the diffraction fields of given 3D scenes are worked out, the best choice and modifications of the algorithms are given. Several novelty experimental examples are given proving the 3D cues depth of field, occlusion and parallax. The benefits and advantages over other holographic approaches are pointed out, open problems and necessary technological developments as well as possibilities and future prospects are discussed.

Human surface tilt estimation in natural and artificial 3D scenes

Human surface tilt estimation in natural and artificial 3D scenes

Parallel vision for perception and understanding of complex scenes: methods, framework, and perspectives

In the study of image and vision computing, the generalization capability of an algorithm often determines whether it is able to work well in complex scenes. The goal of this review article is to survey the use of photorealistic image synthesis methods in addressing the problems of visual perception and understanding. Currently, the ACP Methodology comprising artificial systems, computational experiments, and parallel execution is playing an essential role in modeling and control of complex systems. This paper extends the ACP Methodology into the computer vision field, by proposing the concept and basic framework of Parallel Vision. In this paper, we first review previous works related to Parallel Vision, in terms of synthetic data generation and utilization. We detail the utility of synthetic data for feature analysis, object analysis, scene analysis, and other analyses. Then we propose the basic framework of Parallel Vision, which is composed of an ACP trilogy (artificial scenes, computational experiments, and parallel execution). We also present some in-depth thoughts and perspectives on Parallel Vision. This paper emphasizes the significance of synthetic data to vision system design and suggests a novel research methodology for perception and understanding of complex scenes.

Assessing the Ability of Simulated Laboratory Scenes to Predict the Image Quality Performance of HDR Captures (and Rendering) of Exterior Scenes Using Mobile Phone Cameras

With the advent of computational photography, most cellphones include High Dynamic Range (HDR) modes or "apps" that capture and render high contrast scenes in-camera using techniques such as multiple exposures and subsequent "addition" of those exposures to render a properly exposed image. The results from different cameras vary. Testing the image quality of different cameras involves field-testing under dynamic lighting conditions that may involve moving objects. Such testing often becomes a cumbersome and time-consuming task. It would be more efficient to conduct such testing in a controlled, laboratory environment. This study investigates the feasibility of such testing. Natural exterior scenes, at day and night, some of which include "motion", were captured with a range of cellphone cameras using their native HDR modes. The luminance ratios of these scenes were accurately measured using various spectro-radiometers and luminance meters. Artificial scenes, which include characteristics of the natural exterior scenes and have similar luminance ratios, were created in a laboratory environment. These simulated scenes were captured using the same modes as the natural exterior scenes. A subjective image quality evaluation was conducted using some 20 observers to establish an observer preference scale separately for each scene. For each natural exterior scene, the correlation coefficients between its preference scale and the preference scale obtained for each laboratory scene were calculated, and the laboratory scene with the highest correlation was identified. It was determined that while it was difficult to accurately quantify the actual dynamic range of a natural exterior scene, especially at night, we could still simulate the luminance ratios of a wide range of natural exterior HDR scenes, from 266:1 to 15120:1, within a laboratory environment. Preliminary results of the subjective study indicated that reasonably good correlation (0.8 or higher on average) was obtained between the natural exterior and laboratory simulated scenes. However, such correlations were determined to be specific to the type of scene studied. The scope of this study needs to be narrowed. Another consideration, how moving objects in the scene would affect the results, needs further investigation.

You think you know where you looked? You better look again.

People are surprisingly bad at knowing where they have looked in a scene. We tested participants' ability to recall their own eye movements in 2 experiments using natural or artificial scenes. In each experiment, participants performed a change-detection (Exp.1) or search (Exp.2) task. On 25% of trials, after 3 seconds of viewing the scene, participants were asked to indicate where they thought they had just fixated. They responded by making mouse clicks on 12 locations in the unchanged scene. After 135 trials, observers saw 10 new scenes and were asked to put 12 clicks where they thought someone else would have looked. Although observers located their own fixations more successfully than a random model, their performance was no better than when they were guessing someone else's fixations. Performance with artificial scenes was worse, though judging one's own fixations was slightly superior. Even after repeating the fixation-location task on 30 scenes immediately after scene viewing, performance was far from the prediction of an ideal observer. Memory for our own fixation locations appears to add next to nothing beyond what common sense tells us about the likely fixations of others. These results have important implications for socially important visual search tasks. For example, a radiologist might think he has looked at "everything" in an image, but eye tracking data suggest that this is not so. Such shortcomings might be avoided by providing observers with better insights of where they have looked. (PsycINFO Database Record

Neural representation of scene boundaries

Three-dimensional environmental boundaries fundamentally define the limits of a given space. A body of research employing a variety of methods points to their importance as cues in navigation. However, little is known about the nature of the representation of scene boundaries by high-level scene cortices in the human brain (namely, the parahippocampal place area (PPA) and retrosplenial complex (RSC)). Here we use univariate and multivoxel pattern analysis to study classification performance for artificial scene images that vary in degree of vertical boundary structure (a flat 2D boundary, a very slight addition of 3D boundary, or full walls). Our findings present evidence that there are distinct neural components for representing two different aspects of boundaries: 1) acute sensitivity to the presence of grounded 3D vertical structure, represented by the PPA, and 2) whether a boundary introduces a significant impediment to the viewer’s potential navigation within a space, represented by RSC.

Additivity of Feature-Based and Symmetry-Based Grouping Effects in Multiple Object Tracking.

Multiple object tracking (MOT) is an attentional process wherein people track several moving targets among several distractors. Symmetry, an important indicator of regularity, is a general spatial pattern observed in natural and artificial scenes. According to the “laws of perceptual organization” proposed by Gestalt psychologists, regularity is a principle of perceptual grouping, such as similarity and closure. A great deal of research reported that feature-based similarity grouping (e.g., grouping based on color, size, or shape) among targets in MOT tasks can improve tracking performance. However, no additive feature-based grouping effects have been reported where the tracking objects had two or more features. “Additive effect” refers to a greater grouping effect produced by grouping based on multiple cues instead of one cue. Can spatial symmetry produce a similar grouping effect similar to that of feature similarity in MOT tasks? Are the grouping effects based on symmetry and feature similarity additive? This study includes four experiments to address these questions. The results of Experiments 1 and 2 demonstrated the automatic symmetry-based grouping effects. More importantly, an additive grouping effect of symmetry and feature similarity was observed in Experiments 3 and 4. Our findings indicate that symmetry can produce an enhanced grouping effect in MOT and facilitate the grouping effect based on color or shape similarity. The “where” and “what” pathways might have played an important role in the additive grouping effect.