Natural Scene Imagery Research Articles

A review of methods for text detection in imagery of natural scenes

Text detection in natural scenes plays a key role in recognition and understanding of text in computer vision, while providing the basic possibilities for combining with Natural Language Processing (NLP). The ability of detection for small text and adjacent text is weak although the text detection in natural scenes has evolved from the horizontal text detection initially to the arbitrary shape text detection. Meanwhile, the environment of natural scenes is complex and changeable, which is a huge challenge for text detection. In this paper, we discuss and compare the horizontal and multi-directional detection methods based on target detection, as well as the curved and arbitrary shape detection methods based on the instance segmentation and semantic segmentation. The algorithm ideas are described and their advantages and disadvantages are analysed. Moreover, we found two research routes of curved text and arbitrary shape text detection known as Top-Down and Bottom-Up. According to the evaluations, PMTD [20] performs best in multi-directional text detection, while SPCNet [16] wins in the curve and arbitrary shape text detection.

Calibration and Characterization of Hyperspectral Imaging Systems Used for Natural Scene Imagery

Calibration and characterization of focal plane hyperspectral imaging systems play an important role in natural scene imagery. Illumination plays a major role during imaging, as both the camera and electronically tunable filter may suffer low transmission at the ends of the visible spectrum, resulting in a low signal to noise ratio. It is important that the spectral characteristics of the imaging system as well as its geometric properties be well characterized and its radiometric performance known. The aim of this article is to identify the main sources of errors in a common design of focal-plane hyperspectral imaging system and devise ways of compensating for these errors. Calibration and characterization of a focal-plane hyperspectral imaging system include nominal wavelength accuracy analysis. This was carried out by capturing images of a mercury vapour lamp to study principal emission lines in the visible spectrum. The linearity of the hyperspectral imaging system was investigated by recording an input-output function. This was accomplished by comparing signals captured by the hyperspectral imaging system and luminance data recorded using a luminance meter. System noise characterization was done by repeated acquisitions of dark noise images captured under identical conditions. Main meridian analysis was accomplished by obtaining sample edge patches from the centre and near-boundary of hyperspectral image and then constructing edge and line spread functions. The final test image analysis involved verifying system calibration, image correction and compensation algorithms. Results show that with proper calibration and characterization of imaging systems, high quality images are obtained and can be used for research works which include hyperspectral image registration and hyperspectral image recognition for natural scenes.

Different spatial frequency bands selectively signal for natural image statistics in the early visual system

Early visual evoked potentials (VEPs) measured in humans have recently been observed to be modulated by the image statistics of natural scene imagery. Specifically, the early VEP is dominated by a strong positivity when participants view minimally complex natural scene imagery, with the magnitude of that component being modulated by luminance contrast differences across spatial frequency (i.e., the slope of the amplitude spectrum). For scenes high in structural complexity, the early VEP is dominated by a prominent negativity that exhibits little dependency on luminance contrast. However, since natural scene imagery is broad band in terms of spatial frequency, it is not known whether the above-mentioned modulation results from a complex interaction within or between the early neural processes tuned to different bands of spatial frequency. Here, we sought to address this question by measuring early VEPs (specifically, the C1, P1, and N1 components) while human participants viewed natural scene imagery that was filtered to contain specific bands of spatial frequency information. The results show that the C1 component is largely unmodulated by the luminance statistics of natural scene imagery (being only measurable when such stimuli were made to contain high spatial frequencies). The P1 and N1, on the other hand, were observed to exhibit strong spatial frequency-dependent modulation to the luminance statistics of natural scene imagery. The results therefore suggest that the dependency of early VEPs on natural image statistics results from an interaction between the early neural processes tuned to different bands of spatial frequency.

A Semi-Automatic Approach for the Extraction of Sandy Bodies (Sand Spits) From IKONOS-2 Data

The automatic or semi-automatic extraction of features from satellite images has always become a challenge for remote sensing researchers. The analysis of satellite imagery of natural scenes presents many unique problems, because they cannot be represented easily by regular rules or grammars. A sand spit does not present a well-defined topographic boundary, because they are influenced by tides, waves and wind. Moreover, the bubbles and foam caused by the breaking waves and the turbidity of the water difficult an accurate extraction of the boundary between land (sand spit) and water. This paper presents different approaches in order to extract sand spits from IKONOS-2 data. A novel semi-automatic approach is proposed, which is based on global thresholding through the Otsu's method, further refined through detected edges (GThE). The performance of GThE is compared with traditional pixel-based and object-based classification algorithms. The dataset is composed by six IKONOS-2 images, acquired between 2001 and 2007, covering a sand spit (Cabedelo) located in Portugal. The performance of the three methods used in the estimation of the sand spit area was evaluated through two sets of reference values of the sand spit area. The proposed GThE method presented better results than the other traditional methods, with a clear advantage of a considerable faster performance, beyond requiring a minimum operator intervention. Finally, the relation of the sand spit area with several hydrodynamic and agitation parameters was investigated, where it was demonstrated that the river discharge was the parameter with higher influence in the Cabedelo area.

On the effectiveness of noise masks: Naturalistic vs. un-naturalistic image statistics

It has been argued that the human visual system is optimized for identification of broadband objects embedded in stimuli possessing orientation averaged power spectra fall-offs that obey the 1/fβ relationship typically observed in natural scene imagery (i.e., β=2.0 on logarithmic axes). Here, we were interested in whether individual spatial channels leading to recognition are functionally optimized for narrowband targets when masked by noise possessing naturalistic image statistics (β=2.0). The current study therefore explores the impact of variable β noise masks on the identification of narrowband target stimuli ranging in spatial complexity, while simultaneously controlling for physical or perceived differences between the masks. The results show that β=2.0 noise masks produce the largest identification thresholds regardless of target complexity, and thus do not seem to yield functionally optimized channel processing. The differential masking effects are discussed in the context of contrast gain control.

From spatial frequency contrast to edge preponderance: the differential modulation of early visual evoked potentials by natural scene stimuli

The contrast response function of early visual evoked potentials elicited by sinusoidal gratings is known to exhibit characteristic potentials closely associated with the processes of parvocellular and magnocellular pathways. Specifically, the N1 component has been linked with parvocellular processes, while the P1 component has been linked with magnocellular processes. However, little is known regarding the response properties of the N1 and P1 components during the processing and encoding of complex (i.e., broadband) stimuli such as natural scenes. Here, we examine how established physical characteristics of natural scene imagery modulate the N1 and P1 components in humans by providing a systematic investigation of component modulation as visual stimuli are gradually built up from simple sinusoidal gratings to highly complex natural scene imagery. The results suggest that the relative dominance in signal output of the N1 and P1 components is dependent on spatial frequency (SF) luminance contrast for simple stimuli up to natural scene imagery possessing few edges. However, such a dependency shifts to a dominant N1 signal for natural scenes possessing abundant edge content and operates independently of SF luminance contrast.

An Intelligent Framework for Natural Object Identification in Images

Human superiority over computers in identifying natural objects like clouds, water, grass etc. comes from two capabilities: the capability to maintain a growing knowledge base pertaining to natural object attributes as experienced over a period of time, and the capability to tolerate variations inherently present in the attributes of natural objects. In this paper we aim to mimic both these capabilities for efficient machine identification of local objects in natural scene imagery through a combined application of Case Based Reasoning (CBR) and Fuzzy Logic (FL).Training data of natural objects is used offline to build a case base of various natural object attributes. The imprecision in the definition of natural objects is accommodated by employing fuzzy linguistic variables for object attribute description in the case base as well as in the similarity calculation and case retrieval procedures. When presented with a new image, the reasoning system applies fuzzy similarity concept to retrieve approximately similar cases from the case base and output the corresponding object label. Experimental results endorse validity of proposed approach.

A Model for the Detection of Moving Targets in Visual Clutter Inspired by Insect Physiology

We present a computational model for target discrimination based on intracellular recordings from neurons in the fly visual system. Determining how insects detect and track small moving features, often against cluttered moving backgrounds, is an intriguing challenge, both from a physiological and a computational perspective. Previous research has characterized higher-order neurons within the fly brain, known as ‘small target motion detectors’ (STMD), that respond robustly to moving features, even when the velocity of the target is matched to the background (i.e. with no relative motion cues). We recorded from intermediate-order neurons in the fly visual system that are well suited as a component along the target detection pathway. This full-wave rectifying, transient cell (RTC) reveals independent adaptation to luminance changes of opposite signs (suggesting separate ON and OFF channels) and fast adaptive temporal mechanisms, similar to other cell types previously described. From this physiological data we have created a numerical model for target discrimination. This model includes nonlinear filtering based on the fly optics, the photoreceptors, the 1st order interneurons (Large Monopolar Cells), and the newly derived parameters for the RTC. We show that our RTC-based target detection model is well matched to properties described for the STMDs, such as contrast sensitivity, height tuning and velocity tuning. The model output shows that the spatiotemporal profile of small targets is sufficiently rare within natural scene imagery to allow our highly nonlinear ‘matched filter’ to successfully detect most targets from the background. Importantly, this model can explain this type of feature discrimination without the need for relative motion cues.

Discrimination of amplitude spectrum slope in the fovea and parafovea and the local amplitude distributions of natural scene imagery

A number of studies have investigated whether human visual performance can be related to the general form of the amplitude spectra (i.e., 1/f(alpha)) of natural scenes. Here, it is argued that there are some discrepancies in the data between some of those studies and that one possible explanation for the discrepancies may be related to differences in methodology (e.g., stimuli presented to the fovea as opposed to the parafovea). We sought to resolve some of the discrepancies with two psychophysical paradigms involving alpha discrimination with visual noise and natural scene image patches presented to the fovea or parafovea. Fovea-parafovea threshold differences were apparent for stimuli possessing alpha values < 1.0, with the parafovea typically showing highest thresholds for reference alpha values in the 0.74-0.85 range. Both fovea and parafovea thresholds were lowest in the 1.2-1.4 range. In addition, we conducted a local amplitude distribution analysis (i.e., assessed local alpha) with a large set of high-resolution natural scene imagery and found that the results of that analysis provided a better account of the alpha discrimination thresholds for stimuli presented to the fovea as opposed to the parafovea.

Human perceptual performance with nonliteral imagery: region recognition and texture-based segmentation.

In this study the authors address the issue of how the perceptual usefulness of nonliteral imagery should be evaluated. Perceptual performance with nonliteral imagery of natural scenes obtained at night from infrared and image-intensified sensors and from multisensor fusion methods was assessed to relate performance on 2 basic perceptual tasks to fundamental characteristics of the imagery. Specifically, single-sensor imagery and fused multisensor imagery (both achromatic and false color) were used to test performance on a region recognition task and a texture segmentation task. Results indicate that the use of color rendering and type of scene content play specific roles in determining perceptual performance allowed by nonliteral imagery. The authors argue that the usefulness of various image-rendering methods should be evaluated with respect to multiple perceptual tasks.

Segmentation of satellite imagery of natural scenes using data mining

The authors describe a segmentation technique that integrates traditional image processing algorithms with techniques adapted from knowledge discovery in databases (KDD) and data mining to analyze and segment unstructured satellite images of natural scenes. They have divided their segmentation task into three major steps. First, an initial segmentation is achieved using dynamic local thresholding, producing a set of regions. Then, spectral, spatial, and textural features for each region are generated from the thresholded image. Finally, given these features as attributes, an unsupervised machine learning methodology called conceptual clustering is used to cluster the regions found in the image into N classes-thus, determining the number of classes in the image automatically. They have applied the technique successfully to ERS-1 synthetic aperture radar (SAR). Landsat thematic mapper (TM), and NOAA advanced very high resolution radiometer (AVHRR) data of natural scenes.