Regions In Scene Research Articles

Hiding information in videos using motion clues of feature points

Motivated by the weaknesses of human visual system in understanding the changes in dynamic scenes, a new video steganography method is proposed. There are two main ideas in this method: (1) detecting highly dynamic regions in video scenes and use them to hide the data, and (2) determining the right amount of data to be embedded in the selected regions. Motion clues of the feature points are used to study the dynamics of the scenes, and then, select the regions of interest accordingly. To determine the embedding capacity for each pixel, some statistical indicators extracted from behaviors of the feature points are used. The experimental results on a large database show that the proposed method achieves a substantial performance improvement over the existing methods; the average embedding capacity and the perceptual invisibility rate values are 0.52 bpp and 50.66 dB respectively.

Robust Detection and Affine Rectification of Planar Homogeneous Texture for Scene Understanding

Man-made environments tend to be abundant with planar homogeneous texture, which manifests as regularly repeating scene elements along a plane. In this work, we propose to exploit such structure to facilitate high-level scene understanding. By robustly fitting a texture projection model to optimal dominant frequency estimates in image patches, we arrive at a projective-invariant method to localize such generic, semantically meaningful regions in multi-planar scenes. The recovered projective parameters also allow an affine-ambiguous rectification in real-world images marred with outliers, room clutter, and photometric severities. Comprehensive qualitative and quantitative evaluations are performed that show our method outperforms existing representative work for both rectification and detection. The potential of homogeneous texture for two scene understanding tasks is then explored. Firstly, in environments where vanishing points cannot be reliably detected, or the Manhattan assumption is not satisfied, homogeneous texture detected by the proposed approach is shown to provide alternative cues to obtain a scene geometric layout. Second, low-level feature descriptors extracted upon affine rectification of detected texture are found to be not only class-discriminative but also complementary to features without rectification, improving recognition performance on the 67-category MIT benchmark of indoor scenes. One of our configurations involving deep ConvNet features outperforms most current state-of-the-art work on this dataset, achieving a classification accuracy of 76.90%. The approach is additionally validated on a set of 31 categories (mostly outdoor man-made environments exhibiting regular, repeating structure), being a subset of the large-scale Places2 scene dataset.

Memory shapes visual search strategies in large-scale environments

Search is a central visual function. Most of what is known about search derives from experiments where subjects view 2D displays on computer monitors. In the natural world, however, search involves movement of the body in large-scale spatial contexts, and it is unclear how this might affect search strategies. In this experiment, we explore the nature of memory representations developed when searching in an immersive virtual environment. By manipulating target location, we demonstrate that search depends on episodic spatial memory as well as learnt spatial priors. Subjects rapidly learned the large-scale structure of the space, with shorter paths and less head rotation to find targets. These results suggest that spatial memory of the global structure allows a search strategy that involves efficient attention allocation based on the relevance of scene regions. Thus spatial memory may allow less energetically costly search strategies.

Hierarchical semantic segmentation of image scene with object labeling

Semantic segmentation of an image scene provides semantic information of image regions while less information of objects. In this paper, we propose a method of hierarchical semantic segmentation, including scene level and object level, which aims at labeling both scene regions and objects in an image. In the scene level, we use a feature-based MRF model to recognize the scene categories. The raw probability for each category is predicted via a one-vs-all classification mode. The features and raw probability of superpixels are embedded into the MRF model. With the graph-cut inference, we get the raw scene-level labeling result. In the object level, we use a constraint-based geodesic propagation to get object segmentation. The category and appearance features are utilized as the prior constraints to guide the direction of object label propagation. In this hierarchical model, the scene-level labeling and the object-level labeling have a mutual relationship, which regions and objects are optimized interactively. The experimental results on two datasets show the well performance of our method.

Improving MISR AOD Retrievals With Low-Light-Level Corrections for Veiling Light

Operational retrievals of aerosol optical depth (AOD) from Multi-angle Imaging SpectroRadiometer (MISR) data have been shown to have a high bias in pristine oceanic areas. One line of evidence involves comparison with Maritime Aerosol Network (MAN) observations, including the areas of low aerosol loading close to Antarctica. In this paper, a principal reason for the AOD overestimation is identified, which is stray light measured by the MISR cameras in dark regions of high-contrast scenes. A small fraction of the light from surrounding bright areas, such as clouds or sea ice, is redistributed to dark areas, artificially increasing their brightness. Internal reflections and light scattering from optical elements in MISR’s pushbroom cameras contribute to this veiling light effect. A simple correction model is developed that relies on the average scene brightness and an empirically determined set of veiling light coefficients for each MISR camera and wavelength. Several independent methods are employed to determine these coefficients. Three sets of coefficients are further implemented and tested in prototype MISR 4.4-km AOD retrievals. The results show dramatic improvements in retrieved AODs compared against MAN observations and the currently operational V22 MISR retrievals. For the best performing set of coefficients, the bias is reduced by 51%, from 0.039 to 0.019, the RMSE is lowered by 19%, from 0.062 to 0.050, and 84% of retrievals fall within the uncertainty envelope compared with 66% of retrievals in V22. The best performing set will be implemented operationally in the next V23 MISR AOD product release.

Contrasting gist-based and template-based guidance during real-world visual search.

Visual search through real-world scenes is guided both by a representation of target features and by knowledge of the sematic properties of the scene (derived from scene gist recognition). In 3 experiments, we compared the relative roles of these 2 sources of guidance. Participants searched for a target object in the presence of a critical distractor object. The color of the critical distractor either matched or mismatched (a) the color of an item maintained in visual working memory for a secondary task (Experiment 1), or (b) the color of the target, cued by a picture before search commenced (Experiments 2 and 3). Capture of gaze by a matching distractor served as an index of template guidance. There were 4 main findings: (a) The distractor match effect was observed from the first saccade on the scene, (b) it was independent of the availability of scene-level gist-based guidance, (c) it was independent of whether the distractor appeared in a plausible location for the target, and (d) it was preserved even when gist-based guidance was available before scene onset. Moreover, gist-based, semantic guidance of gaze to target-plausible regions of the scene was delayed relative to template-based guidance. These results suggest that feature-based template guidance is not limited to plausible scene regions after an initial, scene-level analysis. (PsycINFO Database Record

Intelligent Detail Enhancement for Exposure Fusion

Multiscale exposure fusion is a fast approach to fuse several differently exposed images captured at the same high dynamic range (HDR) scene into a high-quality low-dynamic range (LDR) image. The fused image is expected to include all details of the input images. However the details in the brightest and darkest regions are usually not well preserved. Adding details that are extracted from the input images to the fused image is an efficient approach to overcome the problem. In this paper a new gradient domain weighted least square based image smoothing algorithm is proposed to extract the details in the brightest and darkest regions of the HDR scene. The extracted details are then added to an image that is produced using an edge-preserving smoothing pyramid based multiscale exposure fusion algorithm. Experimental results show that the proposed detail enhanced exposure fusion algorithm can preserve details in saturated regions especially the brightest regions better than the state-of-the-art multiscale exposure fusion algorithms.

Differential Sampling of Visual Space in Ventral and Dorsal Early Visual Cortex.

A fundamental feature of cortical visual processing is the separation of visual processing for the upper and lower visual fields. In early visual cortex (EVC), the upper visual field is processed ventrally, with the lower visual field processed dorsally. This distinction persists into several category-selective regions of occipitotemporal cortex, with ventral and lateral scene-, face-, and object-selective regions biased for the upper and lower visual fields, respectively. Here, using an elliptical population receptive field (pRF) model, we systematically tested the sampling of visual space within ventral and dorsal divisions of human EVC in both male and female participants. We found that (1) pRFs tend to be elliptical and oriented toward the fovea with distinct angular distributions for ventral and dorsal divisions of EVC, potentially reflecting a radial bias; and (2) pRFs in ventral areas were larger (∼1.5×) and more elliptical (∼1.2×) than those in dorsal areas. These differences potentially reflect a tendency for receptive fields in ventral temporal cortex to overlap the fovea with less emphasis on precise localization and isotropic representation of space compared with dorsal areas. Collectively, these findings suggest that ventral and dorsal divisions of EVC sample visual space differently, likely contributing to and/or stemming from the functional differentiation of visual processing observed in higher-level regions of the ventral and dorsal cortical visual pathways.SIGNIFICANCE STATEMENT The processing of visual information from the upper and lower visual fields is separated in visual cortex. Although ventral and dorsal divisions of early visual cortex (EVC) are commonly assumed to sample visual space equivalently, we demonstrate systematic differences using an elliptical population receptive field (pRF) model. Specifically, we demonstrate that (1) ventral and dorsal divisions of EVC exhibit diverging distributions of pRF angle, which are biased toward the fovea; and (2) ventral pRFs exhibit higher aspect ratios and cover larger areas than dorsal pRFs. These results suggest that ventral and dorsal divisions of EVC sample visual space differently and that such differential sampling likely contributes to different functional roles attributed to the ventral and dorsal pathways, such as object recognition and visually guided attention, respectively.

Exercise-Induced Physiological Arousal Biases Attention Toward Threatening Scene Details.

The present experiment examined whether physiological arousal induced by acute bouts of aerobic exercise would influence attention and memory for scenes depicting or not depicting weapons. In a repeated-measures design, participants exercised at either low or high exertion levels. During exercise, they were presented with images, some of which depicted weapons; immediately following exercise, they completed a recognition test for portions of central and peripheral scene regions. Two primary results emerged. First, in the low exertion condition, we replicated extant research showing inferior peripheral scene memory when images contained, versus did not contain, weapons. Second, the high exertion condition increased central scene memory relative to low exertion, and this effect was specific to images containing weapons. Thus, we provide evidence for accentuated weapon focus effects during states of exercise-induced physiological arousal. These results contribute new applied and theoretical understandings regarding the interactions between physiological state, breadth of attention, and memory.

Meaning-based guidance of attention in scenes as revealed by meaning maps.

Real-world scenes comprise a blooming, buzzing confusion of information. To manage this complexity, visual attention is guided to important scene regions in real time 1-7 . What factors guide attention within scenes? A leading theoretical position suggests that visual salience based on semantically uninterpreted image features plays the critical causal role in attentional guidance, with knowledge and meaning playing a secondary or modulatory role 8-11 . Here we propose instead that meaning plays the dominant role in guiding human attention through scenes. To test this proposal, we developed 'meaning maps' that represent the semantic richness of scene regions in a format that can be directly compared to image salience. We then contrasted the degree to which the spatial distributions of meaning and salience predict viewers' overt attention within scenes. The results showed that both meaning and salience predicted the distribution of attention, but that when the relationship between meaning and salience was controlled, only meaning accounted for unique variance in attention. This pattern of results was apparent from the very earliest time-point in scene viewing. We conclude that meaning is the driving force guiding attention through real-world scenes.

ELD-Net: An Efficient Deep Learning Architecture for Accurate Saliency Detection.

Recent advances in saliency detection have utilized deep learning to obtain high-level features to detect salient regions in scenes. These advances have yielded results superior to those reported in past work, which involved the use of hand-crafted low-level features for saliency detection. In this paper, we propose ELD-Net, a unified deep learning framework for accurate and efficient saliency detection. We show that hand-crafted features can provide complementary information to enhance saliency detection that uses only high-level features. Our method uses both low-level and high-level features for saliency detection. High-level features are extracted using GoogLeNet, and low-level features evaluate the relative importance of a local region using its differences from other regions in an image. The two feature maps are independently encoded by the convolutional and the ReLU layers. The encoded low-level and high-level features are then combined by concatenation and convolution. Finally, a linear fully connected layer is used to evaluate the saliency of a queried region. A full resolution saliency map is obtained by querying the saliency of each local region of an image. Since the high-level features are encoded at low resolution, and the encoded high-level features can be reused for every query region, our ELD-Net is very fast. Our experiments show that our method outperforms state-of-the-art deep learning-based saliency detection methods.

Human Eye Movements After Viewpoint Shifts in Edited Dynamic Scenes are Under Cognitive Control.

We tested whether viewers have cognitive control over their eye movements after cuts in videos of real-world scenes. In the critical conditions, scene cuts constituted panoramic view shifts: Half of the view following a cut matched the view on the same scene before the cut. We manipulated the viewing task between two groups of participants. The main experimental group judged whether the scene following a cut was a continuation of the scene before the cut. Results showed that following view shifts, fixations were determined by the task from 250 ms until 1.5 s: Participants made more and earlier fixations on scene regions that matched across cuts, compared to nonmatching scene regions. This was evident in comparison to a control group of participants that performed a task that did not require judging scene continuity across cuts, and did not show the preference for matching scene regions. Our results illustrate that viewing intentions can have robust and consistent effects on gaze behavior in dynamic scenes, immediately after cuts.

Superior colliculus neurons encode a visual saliency map during free viewing of natural dynamic video

Models of visual attention postulate the existence of a saliency map whose function is to guide attention and gaze to the most conspicuous regions in a visual scene. Although cortical representations of saliency have been reported, there is mounting evidence for a subcortical saliency mechanism, which pre-dates the evolution of neocortex. Here, we conduct a strong test of the saliency hypothesis by comparing the output of a well-established computational saliency model with the activation of neurons in the primate superior colliculus (SC), a midbrain structure associated with attention and gaze, while monkeys watched video of natural scenes. We find that the activity of SC superficial visual-layer neurons (SCs), specifically, is well-predicted by the model. This saliency representation is unlikely to be inherited from fronto-parietal cortices, which do not project to SCs, but may be computed in SCs and relayed to other areas via tectothalamic pathways.

Detail-Enhanced Multi-Scale Exposure Fusion.

Multi-scale exposure fusion is an effective image enhancement technique for a high dynamic range (HDR) scene. In this paper, a new multi-scale exposure fusion algorithm is proposed to merge differently exposed low dynamic range (LDR) images by using the weighted guided image filter to smooth the Gaussian pyramids of weight maps for all the LDR images. Details in the brightest and darkest regions of the HDR scene are preserved better by the proposed algorithm without relative brightness change in the fused image. In addition, a new weighted structure tensor is introduced to the differently exposed images and it is adopted to design a detail extraction component for the proposed fusion algorithm, such that users are allowed to manipulate fine details in the enhanced image according to their preference. The proposed multi-scale exposure fusion algorithm is also applied to design a simple single image brightening algorithm for both low-light imaging and back-light imaging.

Reappraisal and Distraction Emotion Regulation Strategies Are Associated with Distinct Patterns of Visual Attention and Differing Levels of Cognitive Demand.

Multiple emotion regulation strategies have been identified and found to differ in their effectiveness at decreasing negative emotions. One reason for this might be that individual strategies are associated with differing levels of cognitive demand and require distinct patterns of visual attention to achieve their effects. In the current study, we tested this hypothesis in a sample of psychiatrically healthy participants (n = 25) who attempted to down-regulate negative emotion to photographs from the International Affective Picture System using cognitive reappraisal or distraction. Eye movements, pupil dilation, and subjective reports of negative emotionality were obtained for reappraisal, distraction, unpleasant passive viewing, and neutral passive viewing conditions. Behavioral results indicated that reappraisal and distraction successfully decreased self-reported negative affect relative to unpleasant passive viewing. Successful down regulation of negative affect was associated with different patterns of visual attention across regulation strategies. During reappraisal, there was an initial increase in dwell time to arousing scene regions and a subsequent shift away from these regions during later portions of the trial, whereas distraction was associated with reduced total dwell time to arousing interest areas throughout the entire stimulus presentation. Pupil dilation was greater for reappraisal than distraction or unpleasant passive viewing, suggesting that reappraisal may recruit more effortful cognitive control processes. Furthermore, greater decreases in self-reported negative emotion were associated with a lower proportion of dwell time within arousing areas of interest. These findings suggest that different emotion regulation strategies necessitate different patterns of visual attention to be effective and that individual differences in visual attention predict the extent to which individuals can successfully decrease negative emotion using reappraisal and distraction.

Salient object detection with low-rank approximation and ℓ2,1-norm minimization

Salient object detection is an important issue in computer vision and image procession in that it can facilitate humans to locate conspicuous visual regions in complex scenes rapidly and improve the performance of object detection and video tracking. In recent years, low-rank matrix approximation has been proved to be favorable in image saliency detection and gained a great deal of attention. An underlying assumption of low-rank recovery is that an image is a combination of background regions being low-rank and salient objects being sparse, which corresponds to tough non-smooth optimization problems. In this paper, by incorporating ℓ2,1-norm minimization, we obtain the corresponding smooth optimization problems and propose two effective algorithms with proved convergence. To guarantee the robustness of the proposed methods, the input image is divided into patches and each patch is approximately represented by its mean value. Besides, multi-scale visual features of each patch of the given image are extracted to capture common low-level features such as color, edge, shape and texture. The salient objects of a given image are indicated with sparse coefficients solved by the low-rank matrix approximation problem. Saliency maps are further produced with integration of the high-level prior knowledge. Finally, extensive experiments in four real-world datasets demonstrate that the proposed methods come with competitive performance over the eight compared state-of-the-arts.

A High-Fidelity Haze Removal Method Based on HOT for Visible Remote Sensing Images

Spatially varying haze is a common feature of most satellite images currently used for land cover classification and mapping and can significantly affect image quality. In this paper, we present a high-fidelity haze removal method based on Haze Optimized Transformation (HOT), comprising of three steps: semi-automatic HOT transform, HOT perfection and percentile based dark object subtraction (DOS). Since digital numbers (DNs) of band red and blue are highly correlated in clear sky, the R-squared criterion is utilized to search the relative clearest regions of the whole scene automatically. After HOT transform, spurious HOT responses are first masked out and filled by means of four-direction scan and dynamic interpolation, and then homomorphic filter is performed to compensate for loss of HOT of masked-out regions with large areas. To avoid patches and halo artifacts, a procedure called percentile DOS is implemented to eliminate the influence of haze. Scenes including various land cover types are selected to validate the proposed method, and a comparison analysis with HOT and Background Suppressed Haze Thickness Index (BSHTI) is performed. Three quality assessment indicators are selected to evaluate the haze removed effect on image quality from different perspective and band profiles are utilized to analyze the spectral consistency. Experiment results verify the effectiveness of the proposed method for haze removal and the superiority of it in preserving the natural color of object itself, enhancing local contrast, and maintaining structural information of original image.

Object-guided motion estimation

Motion estimation in image sequences is classically addressed under one of the two following forms: estimation of optical flow (instantaneous apparent motion over the whole image) and visual tracking (estimation of the motion of a certain scene region over time). Major progresses have been recently achieved on both fronts, with robust and accurate techniques available for each problem. However, these problems are mostly studied as if they were independent, while they address in fact two faces of the same reality. This paper analyzes the benefits and consequences of combining tracking methods for estimation of per-object dense motion trajectories. We show experimentally that studying the global translation of an object will benefit the motion estimation accuracy of sample points inside the larger structure.

Pinpointing the peripheral bias in neural scene-processing networks during natural viewing.

Peripherally presented stimuli evoke stronger activity in scene-processing regions than foveally presented stimuli, suggesting that scene understanding is driven largely by peripheral information. We used functional MRI to investigate whether functional connectivity evoked during natural perception of audiovisual movies reflects this peripheral bias. For each scene-sensitive region--the parahippocampal place area (PPA), retrosplenial cortex, and occipital place area--we computed two measures: the extent to which its activity could be predicted by V1 activity (connectivity strength) and the eccentricities within V1 to which it was most closely related (connectivity profile). Scene regions were most related to peripheral voxels in V1, but the detailed nature of this connectivity varied within and between these regions. The retrosplenial cortex showed the most consistent peripheral bias but was less predictable from V1 activity, while the occipital place area was related to a wider range of eccentricities and was strongly coupled to V1. We divided the PPA along its posterior-anterior axis into retinotopic maps PHC1, PHC2, and anterior PPA, and found that a peripheral bias was detectable throughout all subregions, though the anterior PPA showed a less consistent relationship to eccentricity and a substantially weaker overall relationship to V1. We also observed an opposite foveal bias in object-perception regions including the lateral occipital complex and fusiform face area. These results show a fine-scale relationship between eccentricity biases and functional correlation during natural perception, giving new insight into the structure of the scene-perception network.

Super-resolution in a defocused plenoptic camera: a wave-optics-based approach.

Plenoptic cameras enable the capture of a light field with a single device. However, with traditional light field rendering procedures, they can provide only low-resolution two-dimensional images. Super-resolution is considered to overcome this drawback. In this study, we present a super-resolution method for the defocused plenoptic camera (Plenoptic 1.0), where the imaging system is modeled using wave optics principles and utilizing low-resolution depth information of the scene. We are particularly interested in super-resolution of in-focus and near in-focus scene regions, which constitute the most challenging cases. The simulation results show that the employed wave-optics model makes super-resolution possible for such regions as long as sufficiently accurate depth information is available.