Spatial Envelope Research Articles

Natural scene classification using overcomplete ICA

Principal component analysis (PCA) has been widely used to extract features for pattern recognition problems such as object recognition [Turk and Pentland, J. Cognitive Neurosci. 3(1) (1991)]. In natural scene classification, Oliva and Torralba presented such an algorithm in Oliva and Torralba [Int. J. Comput. Vision 42(3) (2001) 145–175] for representing images by their “spatial envelope” properties, including naturalness, openness, and roughness. Our implementation closely matched the original algorithm in accuracy for naturalness classification (or “manmade–natural” classification) on a similar (Corel) dataset [Dong and Luo, Towards holistic scene descriptors for semantic scene classification, Eastman Kodak Company Technical Report, October 1, 2003]. However, we found that consumer photos, which are far more unconstrained in content and imaging conditions, present a greater challenge for the algorithm (as they typically do for image understanding algorithms). In this paper, we present an alternative approach to more robust naturalness classification, using overcomplete independent components analysis (ICA) directly on the Fourier-transformed image to derive sparse representations as more effective features for classification. Using both heuristic and support vector machine classifiers, we demonstrated that our ICA-based features are superior to the PCA-based features used in Oliva and Torrabla [Int. J. Comput. Vision 42(3) (2001) 145–175]; Dong and Luo [Towards holistic scene descriptors for semantic scene classification, Eastman Kodak Company Technical Report, October 1, 2003]. In addition, we augment ICA-based features with camera metadata related to image capture conditions to further improve the performance of our algorithm.

Degeneracy breaking and intervalley scattering due to short-ranged impurities in finite single-wall carbon nanotubes

We present a theoretical study of degeneracy breaking due to short-ranged impurities in finite, single-wall, metallic carbon nanotubes. The effective mass model is used to describe the slowly varying spatial envelope wave functions of spinless electrons near the Fermi level at two inequivalent valleys ($K$-points) in terms of the four component Dirac equation for massless fermions, with the role of spin assumed by pseudospin due to the relative amplitude of the wave function on the sublattice atoms (``$A$'' and ``$B$''). Using boundary conditions at the ends of the tube that neither break valley degeneracy nor mix pseudospin eigenvectors, we use degenerate perturbation theory to show that the presence of impurities has two effects. First, the position of the impurity with respect to the spatial variation of the envelope standing waves results in a sinusoidal oscillation of energy level shift as a function of energy. Second, the position of the impurity within the hexagonal graphite unit cell produces a particular $4\ifmmode\times\else\texttimes\fi{}4$ matrix structure of the corresponding effective Hamiltonian. The symmetry of this Hamiltonian with respect to pseudospin flip is related to degeneracy breaking and, for an armchair tube, the symmetry with respect to mirror reflection in the nanotube axis is related to pseudospin mixing.

Linear s-polarized surface waves in a medium inhomogeneous in one dimension

Linear s-polarized surface waves can exist at the boundary between an isotropic homogeneous medium and a medium inhomogeneous in one dimension (1D-inhomogeneous medium). This is related to deformation of the spatial envelope of the electric and magnetic components of the surface wave propagating in the 1D-inhomogeneous medium (in particular, in a plane-stratified medium). Such linear s-polarized surface waves can appear only provided that the refractive index of the inhomogeneous medium increases with the distance from the interface.

Complex spatial envelopes, invariants and matched field processing

The waveguide invariant was introduced in the Russian Literature (S. D. Chuprov, Ocean Acoustics, 1982) to explain range-frequency interference patterns of intensity. The concept of the spatial envelope is introduced here to show how the waveguide invariant can be derived and interpreted in terms of pressure rather than intensity. It is shown how this leads to range-frequency scaling that can be used to generate finely spaced (in frequency) matched field replicas from course spaced propagation calculations. For a downward refracting sound speed profile the acoustic propagating modes can be separated into groups (refracted bottom interacting, surface-bottom interacting). These mode groups have very similar waveguide invariants. The frequency scaling can be applied to these mode groups separately. An example is presented where 150 Hz shallow water replicas are computed every 20 Hz and interpolated, yielding a factor of 30 in reduction of replica computation for MFP.

Spatial noise shaping based on human visual sensitivity and its application to image coding

A spatial noise shaping (SNS) method based on human visual sensitivity is presented. The method exploits the capability of frequency domain linear prediction for spatial envelope retrieval. It effectively shapes (or hides) the noise of an image in areas which are not sensitive to human vision so that the resulting image is more pleasant to human eyes. The noise comes from the processing of the image, and it can be either separable like the additive noise pattern in image watermarking or nonseparable like the quantization noise in image coding. An application of the algorithm is demonstrated in the paper by using it to enhance image coders. Images decoded from the SNS incorporated coders have superior perceived quality than those without using SNS.

Real-time identification of smoke images by clustering motions on a fractal curve with a temporal embedding method

Automatic forest fire detection with CCD cameras requires a landscape image analysis in two stages: first the tracking of local dynamic envelopes of pixels, and second the discrimination between the various natural phenomena that may cause such envelopes. For this second process, we have to deal with restrictive conditions: lack of spatial information, complexity of motions, and real-time constraints on detection. We present here a fast algorithm adapted to the extraction of complex motions in small spatial envelopes. The principle of the method is to extract local motions from cluster analysis of points in a multidimensional temporal embedding space. We detail the four successive steps of this method: temporal embedding of gray-levels, fractal indexing of points, chaining points into a linked list, and motion extraction from point sequences of the linked list.

Modeling the Shape of the Scene: A Holistic Representation of the Spatial Envelope

In this paper, we propose a computational model of the recognition of real world scenes that bypasses the segmentation and the processing of individual objects or regions. The procedure is based on a very low dimensional representation of the scene, that we term the Spatial Envelope. We propose a set of perceptual dimensions (naturalness, openness, roughness, expansion, ruggedness) that represent the dominant spatial structure of a scene. Then, we show that these dimensions may be reliably estimated using spectral and coarsely localized information. The model generates a multidimensional space in which scenes sharing membership in semantic categories (e.g., streets, highways, coasts) are projected closed together. The performance of the spatial envelope model shows that specific information about object shape or identity is not a requirement for scene categorization and that modeling a holistic representation of the scene informs about its probable semantic category.

Multi-component optical solitary waves

We discuss several novel types of multi-component (temporal and spatial) envelope solitary waves that appear in fiber and waveguide nonlinear optics. In particular, we describe multi-channel solitary waves in bit-parallel-wavelength fiber transmission systems for high-performance computer networks, multi-color parametric spatial solitary waves due to cascaded nonlinearities of quadratic materials, and quasiperiodic envelope solitons due to quasi-phase-matching in Fibonacci optical superlattices.

Procedure for the spatial resolution of multipath signals in urban radio links

The growing use and demand for microwave communication systems has led to an increase in system density, especially in urban areas where the abundant presence of potential obstacles increases the risk of propagation. For efficient frequency planning, knowledge of the multipath geometries involved is required. This can be obtained by resolving the multipath field into its constituent components, yielding their amplitudes and angles of arrivals at a receiver terminal. Investigations into the resolution of multipath height gain curves are documented in the literature. These generally concentrate on resolving the two predominant signal components. In an urban environment, the number of distinct signal paths generally exceeds two. Resolution of more than two components is more complex and requires considerably more effort. The paper presents a component resolution procedure which determines the individual amplitudes and angles of arrival of the constituent signal paths. The procedure is applied to the spatial amplitude envelope observed when displacing a receiver incrementally over a distance of a few wavelengths through a multipath field. The presented solution has the advantages of relatively simple measurements and post-processing of measured data. In stationary measurement environments and discrete ray distributions, the solution requires only the recording of envelope amplitude values and no phase information. The performance of the resolution procedure was evaluated using a receiver with a standard antenna. Examples including simulated and experimental radio paths supported by measurements are presented.

Visually Controlled Locomotion: Its Dependence on Optic Flow, Three-Dimensional Space Perception, and Cognition

Gibson (1958/this issue) and his followers have emphasized the role of optic flow in the control of locomotion. In recent years much research has been devoted to the visual control of aiming and braking, mainly in connection with terrestrial locomotion. The goal of this article is to broaden the topic empirically and theoretically. At the empirical level, we argue that there are a number of visually controlled maneuvers that need to be addressed for their own sake, for they involve more than can be learned from research on aiming and braking. At the theoretical level, we argue that optic flow needs to be supplemented by other explanatory primitives, including the actor's perception of three-dimensional spatial layout and the actor's cognitive representations of the spatial envelope and plant dynamics of his or her body or vehicle.

Pressure Transients within MCS Mesohighs and Wake Lows

By animating enhanced coarse surface pressure observations of 12 1985 Preliminary Regional Experiment for Storm-Scale Operational Research Meteorology (PRE-STORM) mesoscale convective systems (MCSs) the authors exposed 92 transitory highs and lows living within virtually all of the systems’ mesohighs and wake lows. A quasi-Lagrangian (feature following, not material following) analysis of the pressure fields produced five primary results. First, these transients, with magnitudes of a few millibars, horizontal dimensions of order 100 km, and average lifetimes of about 2 h, collectively composed spatial and temporal envelopes that contributed at least part of the total pressure field within mesohighs and wake lows. Transients did not apparently favor formation or dissipation in any location of the envelopes. Second, as the MCSs matured, the difference between each complex’s transitory highs’ mean pressure and transitory lows’ mean pressure increased in 78% of the conclusive cases. Apparently, one frequent role of MCSs is locally to magnify storm-scale pressure gradients. Third, transient paths reflect the frequent symmetric-to-asymmetric metamorphoses of the MCSs. Fourth, the temporal fluctuations of the numbers and apparent sizes of transients within a composite MCS partially support theories of MCS upscale evolution. Finally, the composite’s transient numbers and apparent sizes varied almost identically with time in a pattern that closely resembles the fluctuation of stratiform and convective volumetric rain rates of MCSs.

Permissible Three-Dimensional Testing in a Two-Dimensional Adaptive Wall Wind Tunnel

We adress and extend the useful spatial test envelope of the University of Waterloo Flexible Wall Wind Tunnel (UWFWWT). The work confirms that three-dimensional models, such as a sphere, can be located asymmetrically to either side of the test section centerline and yet be satisfactorily examined using only the roof and floor centerline static pressure data and the unmodified two-dimensional wall ajustment strategy.

Spatial-temporal turbulent flow-field and heat transfer behavior in end-wall junctions

Abstract The spatial-temporal velocity and surface heat transfer behavior in a rectangular end-wall junction are examined experimentally for an impinging turbulent boundary layer. The study is done in a water channel on a fast response, constant heat-flux surface using flow visualization, scanned-laser particle image velociemtry (PIV), and thermochromic liquid crystals (LC). Particle image visualizations and PIV-derived vorticity isocontours in the junction region of a rectangular block illustrate that a dominant unsteady horseshoe vortex resides in the junction region, as suggested by previous studies 2 (Devenport and Simpson 1990; Pierce and Shin 1992). The horseshoe vortex moves sporadically within a spatial envelope, due to the interaction with both (1) vortices from the impinging turbulent boundary layer; and (2) intermittent, opposite rotation vortices spawned by eruptive vortex-surface interactions (Walker 1978; Peridier et al. 1991). Companion surface heat transfer studies using thermochromic liquid crystals of both a canonical turbulent boundary layer and the rectangular block junction illustrate the presence of initially discrete spanwise variations in heat transfer on the approach surface caused by the presence of the streamwise, near-wall “streaks” in the impinging turbulent boundary layer; in the junction region, these streaks undergo a rapid metamorphosis into discrete, transverse regions of high heat transfer. These localized regions display significant transient, three-dimensional (3-D) behavior, appearing to be directly associated with vortex-surface interactions by the resident horseshoe vortex. This milieu of turbulent vortex development/interaction results in increases of 62% in spatially averaged Stanton number and increases of over 200% in local Stanton number compared to the canonical turbulent boundary layer.

Speed Judgments of Transparent Stimuli

When two moving patterns are combined additively, observers often perceive two transparent surfaces, even when there are no cues supporting this segmentation in a frozen snapshot. The ability of observers to make quantitative judgments about the speed of one of the patterns under these conditions was examined. The component patterns consisted of band-pass-filtered random noise presented in a spatial Gaussian contrast envelope, displayed for 250 ms. On each trial a standard pattern appeared on one side of the fixation point, while a test pattern appeared on the other. The test pattern moved in the same direction as the standard, but with a speed which varied from trial to trial according to a staircase procedure. The subjects' task was to report the side of the fixation point on which faster motion was seen. In some conditions the test stimulus was made to appear transparent by adding a mask pattern. When the mask was stationary, or moved slowly with respect to the test, no significant biases were introduced and discrimination performance was comparable to the no-mask condition (typically 3%). But if the mask moved over the test with similar speed, the task became much harder, regardless of whether the mask moved in a direction opposite or orthogonal to the test. (Some subjects commented on a perceived directional repulsion between tests and orthogonally moving masks.) These results suggest the use of nondirectional temporal channels in the performance of the speed discrimination task.

Maximum-likelihood method for focus-variation image reconstruction in high resolution transmission electron microscopy

This paper deals with image reconstruction in high-resolution transmission electron microscopy (HRTEM) via focus variation. This technique aims at the reconstruction of the complex-valued electron wave at the exit plane of the specimen including correction for the aberrations of the microscope, with a focal series of HRTEM images as input. The general case with linear and non-linear contributions to the images is considered, requiring recursive optimization of the wave function, which is obtained by a non-linear least-squares fit to the measured image intensities. Based upon the pioneering work of Kirkland (1984) in this field, we discuss the following improvements. In the first place, a solution of the maximum-likelihood (MAL) equations for image reconstruction is derived in which the - for HRTEM inherent - coupling between the wave function and its complex conjugate is accounted for. Secondly, it is outlined how optimum convergence for the MAL solution is obtained. Thirdly, a computationally efficient implementation of the recursive algorithm to solve the MAL equations is presented. The algorithm is based on a factorization of the spatial coherence envelope, and on focal integration approach for the temporal coherence envelope in the transmission-cross-coefficient. The convergence behavior of this algorithm is studied using simulated focal series as input.

Spatial properties of envelope-responsive cells in area 17 and 18 neurons of the cat.

1. Many neurons in areas 17 and 18 respond to spatial contrast envelope stimuli whose Fourier components fall outside the cell's spatial-frequency-selective range. The spatial properties of such envelope responses are investigated here and compared with responses to conventional luminance-defined gratings to explore the underlying receptive-field mechanism. 2. Three spatial properties of envelope responses are reported more extensively in this paper. First, the envelope responses were selective to the carrier spatial frequency in a narrow range of frequencies higher than a given cell's luminance spatial frequency selective range (luminance passband). Second, a given cell's dependence on envelope spatial frequency often differed from its luminance passband. Last, the optimal carrier spatial frequency did not shift systematically with the envelope spatial frequency, supporting the hypothesis that the carrier and envelope spatial-frequency dependencies were mediated by distinct mechanisms. 3. In contrast to the direction selectivity to the envelope motion in many envelope-responsive cells, no direction preference to carrier motion was found for envelope responses. The direction of carrier motion did not alter the direction selectivity for envelope motion, further supporting the hypothesis that the carrier and envelope temporal properties were mediated by separate mechanisms. 4. The distributions of the optimal carrier and luminance spatial frequencies among envelope-responsive cells were analyzed. The optimal carrier spatial frequencies were randomly distributed from five times the cell's optimal luminance spatial frequency to the upper resolution limit of the X-retinal ganglion cells at the same retinal eccentricity, suggesting that the selective ranges of envelope responses and luminance responses are not strongly correlated over the population of envelope-responsive cells. 5. Our data support a "two-stream" receptive-field model for envelope-responsive cells. One stream is a conventional, spatially linear receptive-field mechanism, mediating luminance responses for the cell; the other mediates envelope responses and consists of a two-stage processing: a set of spatially small and distributed nonlinear neural subunits whose outputs are spatially pooled at the second stage. 6. In conclusion, this study indicates that envelope responses in area 17 and 18 neurons cannot be due to a nonlinearity that is common to all visual stimuli before narrowband spatial-frequency-selective filtering; instead, a specialized processing stream, parallel to the conventional luminance response stream, is needed to supplement the traditional luminance processing stream in these cells. This specialized stream responds to the envelope stimuli and is selective to their carrier and envelope spatial frequencies. The distributions of the optimal luminance and carrier spatial frequencies indicate a rich variety of possible integration between luminance and envelope information.

Nonlinear filtering in object and Fourier space in a joint transform optical correlator: comparison and experimental realization

The use of different kinds of nonlinear filtering in a joint transform correlator are studied and compared. The study is divided into two parts, one corresponding to object space and the second to the Fourier domain of the joint power spectrum. In the first part, phase and inverse filters are computed; their inverse Fourier transforms are also computed, thereby becoming the reference in the object space. In the Fourier space, the binarization of the power spectrum is realized and compared with a new procedure for removing the spatial envelope. All cases are simulated and experimentally implemented by a compact joint transform correlator.

Generation of supersonic and subsonic solitons in the Fermi-Pasta-Ulam chain

Abstract The dynamics of the Fermi-Pasta-Ulam chain with quartic anharmonicity are numerically investigated by means of a stimulating force of Gaussian time shape, which acts on a single atom. Depending on the strength and duration of this force, several phenomena have been observed. A pulse of originally continuous envelope disintegrates into solitary objects (‘multifurcative process’), which may be ultrasonic solitons of kink type, or subsonic and self-localized solitons of oscillating nature. In the oscillations of the latter two, frequencies are involved, above the Debye frequency, and the spatial envelope displays a ‘breathing’ behaviour. Some analytical background is also presented.

Effect of contrast and adaptation on the perception of the direction and speed of drifting gratings

Three experiments were conducted to analyse the effect of contrast and adaptation state on the ability of human observers to discriminate the motion of drifting gratings. In the first experiment, subjects judged the direction of briefly presented gratings, which slowly drifted leftward or rightward. The test gratings were enveloped in space by a raised cosine function and in time by a Gaussian. The centre of the spatial envelope was either 2 deg left or right of the fixation point. An adaptive staircase procedure was used to find the velocities, at which the observer judged the motion direction in 75% of the presentations as leftwards or rightwards, respectively. In the second experiment, subjects judged the relative speed of two simultaneously presented gratings. Stimulus contrast was varied in both experiments from 0.01 to 0.32. Discrimination threshold vs contrast functions were measured before and after adaptation to a high-contrast (0.4) grating drifting at rates between 2 and 32 Hz. In a third experiment, subjects matched, before and after adaptation, the relative speed of a test stimulus, which had a constant contrast (0.04 or 0.08) and a variable speed, to that of a reference stimulus having a variable contrast but a constant speed. The results indicate that, before adaptation, direction and speed discrimination thresholds are independent of test contrast, except when test contrast approaches the detection threshold level. Adaptation to a drifting grating increases the lower threshold of motion (LTM) and the speed discrimination threshold (ΔV/V) for low test contrasts. In addition, the point of subjective stationarity (PSS) shifts towards the adapted direction and this shift is more pronounced for low test contrasts. The perceived speed of a drifting grating increases with increasing contrast level. Adaptation to a drifting grating shifts the perceived speed vs log contrast function downwards and to the right (toward higher contrast levels) and this shift is greatest for adaptation frequencies between 8 and 16 Hz. We further explored the effects of adaptation contrast (0.04, 0.4 and 0.9) and adaptation drift direction (iso- or contra-directional) on the perceived speed versus contrast function. The effect of adaptation is greatest for iso-directional drift and increases with increasing adaptation contrast. The results are discussed in terms of a contrast gain control model of adaptation.

Propagation of optical pulses with spatial and temporal dependence

A convenient approximate formula is proposed for the study of free-space propagation of spatial and temporal pulses with an identifiable carrier frequency. It does not contain a time derivative operation on the pulse's temporal envelope explicitly. It is shown that once a short (for example, picosecond or subpicosecond) pulse is created with a spatial and a temporal structure, it does not last forever. The approximation discussed is valid over a certain distance as dictated by the wave equation. Beyond this distance, the spatial and temporal characteristics begin to influence each other significantly. Two examples are presented. The first example is that of a pulse with a factored form of a spatial envelope times a temporal envelope. The second example is that of a clear aperture with a grating, by which pulse stretching or temporal distortion is examined and the result is in agreement with that found in the literature.