Regions In Scene Research Articles

A Novel Hybrid Approach to Detect High Density Crowd Regions in Still Image Scenes

A Novel Hybrid Approach to Detect High Density Crowd Regions in Still Image Scenes

A NEW FRAMEWORK FOR OBJECT-BASED IMAGE ANALYSIS BASED ON SEGMENTATION SCALE SPACE AND RANDOM FOREST CLASSIFIER

Abstract. In this paper a new object-based framework is developed for automate scale selection in image segmentation. The quality of image objects have an important impact on further analyses. Due to the strong dependency of segmentation results to the scale parameter, choosing the best value for this parameter, for each class, becomes a main challenge in object-based image analysis. We propose a new framework which employs pixel-based land cover map to estimate the initial scale dedicated to each class. These scales are used to build segmentation scale space (SSS), a hierarchy of image objects. Optimization of SSS, respect to NDVI and DSM values in each super object is used to get the best scale in local regions of image scene. Optimized SSS segmentations are finally classified to produce the final land cover map. Very high resolution aerial image and digital surface model provided by ISPRS 2D semantic labelling dataset is used in our experiments. The result of our proposed method is comparable to those of ESP tool, a well-known method to estimate the scale of segmentation, and marginally improved the overall accuracy of classification from 79% to 80%.

A DAISY descriptor based multi-view stereo method for large-scale scenes

Normalized cross-correlation (NCC) has been widely used as the matching cost function in multi-view stereo methods. However, NCC is vulnerable in the occlusion area and edge region of large-scale scenes because of color distortion and illumination changes. To alleviate the above problems, we present an improved patch based multi-view stereo method by introducing a photometric discrepancy function based on DAISY descriptor. In the patch extraction stage, a new corresponding point matching method based on the DAISY descriptor is proposed and the epipolar constraint is used to filter mismatched points. In the patch optimization stage, a photometric discrepancy function based on DAISY descriptor is proposed to measure the photo-consistency among reconstructed patches to identify reliable patches. Finally, dense patches are obtained by expanding sparse patches with global visibility information and patch optimization. Experimental results show that the proposed algorithm obtains better reconstruction results in occlusion and edge regions of large-scale scenes.

Image Interpolation Based on Saliency Information

In this paper, we propose an image interpolation algorithm based on the characteristics of the Human Visual System (HVS). Firstly, the saliency map is extracted for the input image by the feature contrast. Based on the saliency detection algorithm, the input image can be divided into salient regions and non-salient regions. A non-local means based interpolation and the bicubic interpolation operations are adopted to implement the image interpolation for salient and non-salient regions in visual scenes, respectively. Experimental results show that the proposed method improves the perceptual quality of the salient regions during image interpolation. The PSNR and SSIM results demonstrate that the proposed method can obtain better performance than existing related image interpolation methods.

Motion parallax links visual motion areas and scene regions

When we move, the retinal velocities of objects in our surrounding differ according to their relative distances and give rise to a powerful three-dimensional visual cue referred to as motion parallax. Motion parallax allows us to infer our surrounding's 3D structure as well as self-motion based on 2D retinal information. However, the neural substrates mediating the link between visual motion and scene processing are largely unexplored. We used fMRI in human observers to study motion parallax by means of an ecologically relevant yet highly controlled stimulus that mimicked the observer's lateral motion past a depth-layered scene. We found parallax selective responses in parietal regions IPS3 and IPS4, and in a region lateral to scene selective occipital place area (OPA). The traditionally defined scene responsive regions OPA, the para-hippocampal place area (PPA) and the retrosplenial cortex (RSC) did not respond to parallax. During parallax processing, the occipital parallax selective region entertained highly specific functional connectivity with IPS3 and with scene selective PPA. These results establish a network linking dorsal motion and ventral scene processing regions specifically during parallax processing, which may underlie the brain's ability to derive 3D scene information from motion parallax.

Coding of object size and object category in scene regions

Coding of object size and object category in scene regions

Adaptive High Dynamic Range for Time-of-Flight Cameras

The limited dynamic range of conventional digital cameras is a well-known problem. A common solution is to apply high-dynamic-range (HDR) techniques over several images acquired using different exposure times. Time-of-flight (ToF) cameras using a photonic mixer device (PMD) are not an exception, since the dynamic range of its dual pixels is also limited. Furthermore, in this case, the saturation of the pixel channels leads to wrong depth measurements. An appropriate solution is the suppression of background illumination (SBI) system designed by the PMD. This system actually extends the dynamic range of the camera by hardware, but it also introduces noise when activated. In this paper, we present an adaptive HDR (AHDR) solution to the problem for the ToF case that overcomes the limited dynamic range of the system, allowing sensing along a theoretically infinite dynamic range with the only limitations of the power of the illumination system and the decay of the SNR with higher distances or lower illumination intensities. Our method is able to detect and segment relevant scene regions responsible for unexpected saturation, i.e., close foreground objects, from the rest of the scene and adjust the exposure times of the acquired images considering them. The results show a reduction in detail losses and a higher SNR in the AHDR raw images, with respect to single acquisitions. This results in a dramatic depth error reduction and effective axial resolution improvement in critical areas, while keeping a high frame rate. In addition, the SBI-related noise is eliminated.

Emotion regulation abnormalities in schizophrenia: Directed attention strategies fail to decrease the neurophysiological response to unpleasant stimuli.

Previous research provides evidence that individuals with schizophrenia (SZ) have emotion regulation abnormalities, particularly when attempting to use reappraisal to decrease negative emotion. The current study extended this literature by examining the effectiveness of a different form of emotion regulation, directed attention, which has been shown to be effective at reducing negative emotion in healthy individuals. Participants included outpatients with SZ (n = 28) and healthy controls (CN: n = 25), who viewed unpleasant and neutral images during separate event-related potential and eye-movement tasks. Trials included both passive viewing and directed attention segments. During directed attention, gaze was directed toward highly arousing aspects of an unpleasant image, less arousing aspects of an unpleasant image, or a nonarousing aspect of a neutral image. The late positive potential (LPP) event-related potential component indexed emotion regulation success. Directing attention to nonarousing aspects of unpleasant images decreased the LPP in CN; however, SZ showed similar LPP amplitude when attention was directed toward more or less arousing aspects of unpleasant scenes. Eye tracking indicated that SZ were more likely than CN to attend to arousing portions of unpleasant scenes when attention was directed toward less arousing scene regions. Furthermore, pupilary data suggested that SZ patients failed to engage effortful cognitive processes needed to inhibit the prepotent response of attending to arousing aspects of unpleasant scenes when attention was directed toward nonarousing scene regions. Findings add to the growing literature indicating that individuals with SZ display emotion regulation abnormalities and provide novel evidence that dysfunctional emotion-attention interactions and generalized cognitive control deficits are associated with ineffective use of directed attention strategies to regulate negative emotion. (PsycINFO Database Record

Contour Erasure and Filling-in: Old Simulations Account for Most New Observations.

Three recent studies used similar stimulus sequences to investigate mechanisms for brightness perception. Anstis and Greenlee (2014) demonstrated that adaptation to a flickering black and white outline erased the visibility of a subsequent target shape defined by a luminance increment or decrement. Robinson and de Sa (2012, 2013) used a flickering disk or annulus to show a similar effect. Here, a neural network model of visual perception (Francis & Kim, 2012), that previously explained properties of scene fading, is shown to also explain most of the erasure effects reported by Anstis and Greenlee and by Robinson and de Sa. The model proposes that in normal viewing conditions a brightness filling-in process is constrained by oriented boundaries, which thereby define separate regions of a visual scene. Contour adaptation can weaken the boundaries and thereby allow brightness signals to merge together, which renders target stimuli indistinguishable from the background. New simulations with the stimuli used by Anstis and Greenlee and Robinson and de Sa produce model output very similar to the perceptual experience of human observers. Finally, the model predicts that adaptation to illusory contours will not produce contour erasure.

Disentangling the effects of spatial inconsistency of targets and distractors when searching in realistic scenes.

Previous research has suggested that correctly placed objects facilitate eye guidance, but also that objects violating spatial associations within scenes may be prioritized for selection and subsequent inspection. We analyzed the respective eye guidance of spatial expectations and target template (precise picture or verbal label) in visual search, while taking into account any impact of object spatial inconsistency on extrafoveal or foveal processing. Moreover, we isolated search disruption due to misleading spatial expectations about the target from the influence of spatial inconsistency within the scene upon search behavior. Reliable spatial expectations and precise target template improved oculomotor efficiency across all search phases. Spatial inconsistency resulted in preferential saccadic selection when guidance by template was insufficient to ensure effective search from the outset and the misplaced object was bigger than the objects consistently placed in the same scene region. This prioritization emerged principally during early inspection of the region, but the inconsistent object also tended to be preferentially fixated overall across region viewing. These results suggest that objects are first selected covertly on the basis of their relative size and that subsequent overt selection is made considering object-context associations processed in extrafoveal vision. Once the object was fixated, inconsistency resulted in longer first fixation duration and longer total dwell time. As a whole, our findings indicate that observed impairment of oculomotor behavior when searching for an implausibly placed target is the combined product of disruption due to unreliable spatial expectations and prioritization of inconsistent objects before and during object fixation.

Weakly-supervised region annotation for understanding scene images

Scene image understanding has drawn much attention for its intriguing applications in the past years. In this paper, we propose a unified probabilistic graphical model called Topic-based Coherent Region Annotation (TCRA) for weakly-supervised scene region annotation. The multiscale over-segmented regions within a scene image are considered as the "words" of our topic model, which impose neighborhood contextual constraints on topic level through spatial MRF modeling, and incorporate an annotation reasoning mechanism for learning and inferring region labels automatically. Mean field variational inference is provided for model learning. The proposed TCRA has the following two main advantages for understanding natural scene images. First, spatial information of multiscale over-segmented regions is explicitly modeled to obtain coherent region annotations. Second, only image-level labels are needed for automatically inferring the label of every region within the scene. This is particularly helpful in reducing human burden on manually labeling pixel-level semantics in the scene understanding research. Thus, given a scene image that has no textual prior, the regions in it can be automatically labeled using the learned TCRA model. The experimental results conducted on three benchmarks consisting of the MSRCORID image dataset, the UIUC Events image dataset and the SIFT FLOW dataset show that the proposed model outperforms the recent state-of-the-art methods.

Perceptual load in different regions of the visual scene and its relevance for driving.

The aim of this study was to better understand the role played by perceptual load, at both central and peripheral regions of the visual scene, in driving safety. Attention is a crucial factor in driving safety, and previous laboratory studies suggest that perceptual load is an important factor determining the efficiency of attentional selectivity. Yet, the effects of perceptual load on driving were never studied systematically. Using a driving simulator, we orthogonally manipulated the load levels at the road (central load) and its sides (peripheral load), while occasionally introducing critical events at one of these regions. Perceptual load affected driving performance at both regions of the visual scene. Critically, the effect was different for central versus peripheral load: Whereas load levels on the road mainly affected driving speed, load levels on its sides mainly affected the ability to detect critical events initiating from the roadsides. Moreover, higher levels of peripheral load impaired performance but mainly with low levels of central load, replicating findings with simple letter stimuli. Perceptual load has a considerable effect on driving, but the nature of this effect depends on the region of the visual scene at which the load is introduced. Given the observed importance of perceptual load, authors of future studies of driving safety should take it into account. Specifically, these findings suggest that our understanding of factors that may be relevant for driving safety would benefit from studying these factors under different levels of load at different regions of the visual scene.

Video Saliency Detection Using Contrast of Spatiotemporal Directional Coherence

Saliency detection in video sequences has attracted great attention in recent years due to its promising contributions for various computer vision applications. However, most existing methods often fail to correctly find salient regions in complex scenes due to ambiguities between salient motions and irrelevant ones generated from the background. In this letter, we present a simple and powerful framework for video saliency detection based on the contrast of the spatiotemporal directional coherence. Our approach is designed in a general way so that it can be applied to videos taken under various environments including dynamic illuminations and textures in the background, which still bother previous saliency detection methods. Based on various challenging datasets, we compare ours with several competitive approaches proposed in literature and results demonstrate that the proposed method is effective and robust for detecting salient regions in diverse video sequences.

A Precision-Recall Criterion Based Consensus Model For Fusing Multiple Segmentations

This paper presents a general framework for seamlessly combining multiple low cost and inaccurate estimated segmentation maps (with an arbitrary number of regions) of the same scene to achieve a final improved segmentation. The proposed fusion model is derived from the well-known precision-recall criterion, specially dedicated to the specific clustering problem of any spatially indexed data and which is also efficient and widely used in the vision community for evaluating both a region-based segmentation and the quality of contours produced by this segmentation map compared to one or multiple ground-truth segmentations of the same image. The proposed combination framework is here specifically designed to be robust with respect to outlier segmentations (that appear to be inconsistent with the remainder of the segmentation ensemble) and includes an explicit internal regularization factor reflecting the inherent ill-posed nature of the segmentation problem. We propose also a hierarchical and efficient way to optimize the consensus energy function related to this fusion model that exploits a simple and deterministic iterative relaxation strategy combining the different segments or individual regions belonging to the segmentation ensemble in the final solution. The experimental results on the Berkeley database with manual ground truth segmentations show the effectiveness of our combination model.

지능 영상 감시를 위한 흑백 영상 데이터에서의 효과적인 이동 투영 음영 제거

In detection of moving objects from video sequences, an essential process for intelligent visual surveillance, the cast shadows accompanying moving objects are different from background so that they may be easily extracted as foreground object blobs, which causes errors in localization, segmentation, tracking and classification of objects. Most of the previous research results about moving cast shadow detection and removal usually utilize color information about objects and scenes. In this paper, we proposes a novel cast shadow removal method of moving objects in gray level video data for visual surveillance application. The proposed method utilizes observations about edge patterns in the shadow region in the current frame and the corresponding region in the background scene, and applies Laplacian edge detector to the blob regions in the current frame and the corresponding regions in the background scene. Then, the product of the outcomes of application determines moving object blob pixels from the blob pixels in the foreground mask. The minimal rectangle regions containing all blob pixles classified as moving object pixels are extracted. The proposed method is simple but turns out practically very effective for Adative Gaussian Mixture Model-based object detection of intelligent visual surveillance applications, which is verified through experiments.

A Region of Interest (ROI) Sharing Protocol for Multirobot Cooperation With Distributed Sensing Based on Semantic Stability

This paper introduces a region of interest (ROI) sharing protocol for robots, each of which observes different regions in the same scene. This type of observation is generally referred to as “distributed sensing.” The ROI sharing protocol is implemented based on a concept of semantic stability, which helps to find a stable context by selecting sufficient information from an image or a scene. The semantic stability is formulated based on semantic relatedness and the information gap in the relations found in the ROI. Moreover, the necessary and sufficient information is determined by selecting the best ROI from a given image. Finally, the best ROI is shared between the robots for making cooperation. Experimental results show that the ROI sharing is efficient in terms of task performance and robustness.

Spatiotemporal Saliency Detection Using Textural Contrast and Its Applications

Saliency detection has been extensively studied due to its promising contributions for various computer vision applications. However, most existing methods are easily biased toward edges or corners, which are statistically significant, but not necessarily relevant. Moreover, they often fail to find salient regions in complex scenes due to ambiguities between salient regions and highly textured backgrounds. In this paper, we present a novel unified framework for spatiotemporal saliency detection based on textural contrast. Our method is simple and robust, yet biologically plausible; thus, it can be easily extended to various applications, such as image retargeting, object segmentation, and video surveillance. Based on various datasets, we conduct comparative evaluations of 12 representative saliency detection models presented in the literature, and the results show that the proposed scheme outperforms other previously developed methods in detecting salient regions of the static and dynamic scenes.

Highly Informative Natural Scene Regions Increase Microsaccade Production during Visual Scanning

Classical image statistics, such as contrast, entropy, and the correlation between central and nearby pixel intensities, are thought to guide ocular fixation targeting. However, these statistics are not necessarily task relevant and therefore do not provide a complete picture of the relationship between informativeness and ocular targeting. Moreover, it is not known whether either informativeness or classical image statistics affect microsaccade production; thus, the role of microsaccades in information acquisition is also unknown. The objective quantification of the informativeness of a scene region is a major challenge, because it can vary with both image features and the task of the viewer. Thus, previous definitions of informativeness suffered from subjectivity and inconsistency across studies. Here we developed an objective measure of informativeness based on fixation consistency across human observers, which accounts for both bottom-up and top-down influences in ocular targeting. We then analyzed fixations in more versus less informative image regions in relation to classical statistics. Observers generated more microsaccades on more informative than less informative image regions, and such regions also exhibited low redundancy in their classical statistics. Increased microsaccade production was not explained by increased fixation duration, suggesting that the visual system specifically uses microsaccades to heighten information acquisition from informative regions.

Detection of salient regions in crowded scenes

The increasing number of cameras and a handful of human operators to monitor the video inputs from hundreds of cameras leave the system ill equipped to fulfil the task of detecting anomalies. Thus, there is a dire need to automatically detect the regions that require immediate attention for more effective and proactive surveillance. A framework that utilises the temporal variations in the flow field of a crowd scene to automatically detect salient regions is proposed, while eliminating the need to have prior knowledge of the scene or training. The flow fields are deemed to be a dynamic system and adopt the stability theory of dynamic systems, to determine the motion dynamics within a given area. In this context, the salient regions refer to the areas with high motion dynamics, where the points in a particular region are unstable. The experimental results on public, crowd scenes have shown the effectiveness of the proposed method in detecting salient regions which correspond to an unstable flow, occlusions, bottlenecks, and entries and exits.

Labeling Spain With Stanford

We present an end-to-end framework for outdoor scene region decomposition, learned on a small set of randomly selected images that generalizes well to multiple data sets containing images from around the world. We discuss the different aspects of the framework especially a generalized variational inference method with better approximations to the true marginals of a graphical model. Experimentally, we explain why the framework is robust and performs competitively on many diverse scene data sets, including several unseen scene types. We have obtained high pixel-level accuracies ( ≈ 80%) in three of the four data sets, which include a benchmark data set known as the Stanford background data set. Our model obtained over 70% accuracy on the fourth data set, which contained a number of indoor and close-up images that are significantly different from our training examples.