Natural Scene Images Research Articles

Reduced annotation based on deep active learning for arabic text detection in natural scene images

Providing labeled Arabic text images dataset for scene text detection is inherently difficult and costly at the same time. Consequently, only few small datasets are available for this task. Previous work has only focused on the data augmentation technique of small datasets; however, the images generated with these techniques cannot reproduce the complexity and variability of natural images. In this paper, we propose a new Arabic text images dataset using the Google Street View service named Tunisia Street View Dataset (TSVD). The dataset contains 7k images collected from different Tunisian cities. It is much more diverse and complex than current image datasets. Taking advantage of this dataset to train Convolutional Neural Network (CNN) models, annotation is required for building high performance models. The annotation task consumes a lot of time and effort for researchers due to its repetitiveness. The development time of text detection systems in natural images is valuable with an effective use. We believe that we have developed a Deep Active Learning algorithm for the annotation phase. A Deep Active Learning algorithm for the annotation phase has been developed by approaching the annotation suggestion task using a deep learning text detector. CNN are used to perform the text detection in natural scene images. Our deep active learning framework combines CNN and active learning approach. This reduces annotation effort by making pertinent suggestions on the most effective annotation areas. We utilize uncertainty provided by CNN models to determine the maximum uncertain areas for annotation. Deep active learning is shown in order to reduce significantly the number of training samples required and also to minimize the annotation work of our dataset up to 1/5. Our dataset is publicly available in IEEE DataPort https://dx.doi.org/10.21227/extw-0k60.

How much can we differentiate at a brief glance: revealing the truer limit in conscious contents through the massive report paradigm (MRP).

Upon a brief glance, how well can we differentiate what we see from what we do not? Previous studies answered this question as ‘poorly’. This is in stark contrast with our everyday experience. Here, we consider the possibility that previous restriction in stimulus variability and response alternatives reduced what participants could express from what they consciously experienced. We introduce a novel massive report paradigm that probes the ability to differentiate what we see from what we do not. In each trial, participants viewed a natural scene image and judged whether a small image patch was a part of the original image. To examine the limit of discriminability, we also included subtler changes in the image as modification of objects. Neither the images nor patches were repeated per participant. Our results showed that participants were highly accurate (accuracy greater than 80%) in differentiating patches from the viewed images from patches that are not present. Additionally, the differentiation between original and modified objects was influenced by object sizes and/or the congruence between objects and the scene gists. Our massive report paradigm opens a door to quantitatively measure the limit of immense informativeness of a moment of consciousness.

Contrast discrimination in images of natural scenes.

Contrast discrimination determines the threshold contrast required to distinguish between two suprathreshold visual stimuli. It is typically measured using sine-wave gratings. We first present a modification to Barten's semi-mechanistic contrast discrimination model to account for spatial frequency effects and demonstrate how the model can successfully predict visual thresholds obtained from published classical contrast discrimination studies. Contrast discrimination functions are then measured from images of natural scenes, using a psychophysical paradigm based on that employed in our previous study of contrast detection sensitivity. The proposed discrimination model modification is shown to successfully predict discrimination thresholds for structurally very different types of natural image stimuli. A comparison of results shows that, for normal contrast levels in natural scene viewing, contextual contrast detection and discrimination are approximately the same and almost independent of spatial frequency within the range of 1-20 c/deg. At higher frequencies, both sensitivities decrease in magnitude due to optical limitations of the eye. The results are discussed in relation to current image quality models.

Application Research of Deep Learning Technology in Natural Landscape Animation Design

Due to the limitation of technology and cost, the animation design of natural landscape in the past was often dealt with relative simplicity. With the increasing level of audience appreciation and the continuous development of animation technology, new requirements are put forward for the design of natural landscape animation. In order to make the animation design effect of natural landscape more real, the synthetic aperture radar image is firstly analyzed to obtain the location of mountains, farmland, rivers, villages, roads, and buildings. Considering the superiority of U-Net network in image semantic segmentation, this paper constructs a semantic segmentation model based on U-Net structure. In this model, dense connection module is introduced in downsampling, and spatial void pyramid structure is introduced in upsampling to retain more image features, to achieve accurate segmentation of satellite images. Experimental results show that the proposed algorithm has higher segmentation accuracy than other algorithms. After accurate classification of natural scene images, it can provide a guarantee for designing more real natural landscape animation design effects.

Color vision deficiencies and camouflage: a comparative study between normal and CVD observers

There is a belief that observers with color vision deficiencies (CVD) perform better in detecting camouflaged objects than normal observers. Some studies have concluded contradictory findings when studying the performance of normal and CVD observers in the camouflage detection tasks in different conditions. This work presents a literature review on this topic, dividing it into three different and contradictory types of results: better performance for CVD, for normal observers, or same performance. Besides, two psychophysical experiments have been designed and carried out in a calibrated computer monitor on both normal and CVD human observers to measure the searching times of the different types of observers needed to find camouflaged stimuli in two different types of stimuli. Results show the trend that, in our experimental conditions, normal observers need shorter searching times than CVD observers in finding camouflaged stimuli both in images of natural scenes and in images with synthetic stimuli.

Bilingual text detection from natural scene images using faster R-CNN and extended histogram of oriented gradients

Bilingual text detection from natural scene images using faster R-CNN and extended histogram of oriented gradients

Manipulating image luminance to improve eye gaze and verbal behavior in autistic children

Autism has been characterized by a tendency to attend to the local visual details over surveying an image to understand the gist–a phenomenon called local interference. This sensory processing trait has been found to negatively impact social communication. Although much work has been conducted to understand these traits, little to no work has been conducted to intervene to provide support for local interference. Additionally, recent understanding of autism now introduces the core role of sensory processing and its impact on social communication. However, no interventions to the end of our knowledge have been explored to leverage this relationship. This work builds on the connection between visual attention and semantic representation in autistic children. In this work, we ask the following research questions: RQ1: Does manipulating image characteristics of luminance and spatial frequency increase likelihood of fixations in hot spots (Areas of Interest) for autistic children? RQ2: Does manipulating low-level image characteristics of luminance and spatial frequency increase the likelihood of global verbal responses for autistic children? We sought to manipulate visual attention as measured by eye gaze fixations and semantic representation of verbal response to the question “What is this picture about?”. We explore digital strategies to offload low-level, sensory processing of global features via digital filtering. In this work, we designed a global filter to reduce image characteristics found to be distracting for autistic people and compared baseline images to featured images in 11 autistic children. Participants saw counterbalanced images way over 2 sessions. Eye gaze in areas of interest and verbal responses were collected and analyzed. We found that luminance in non-salient areas impacted both eye gaze and verbal responding–however in opposite ways (however versus high levels of luminance). Additionally, the interaction of luminance and spatial frequency in areas of interest is also significant. This is the first empirical study in designing an assistive technology aimed to augment global processing that occurs at a sensory-processing and social-communication level. Contributions of this work include empirical findings regarding the quantification of local interference in images of natural scenes for autistic children in real-world settings; digital methods to offload global visual processing to make this information more accessible via insight on the role of luminance and spatial frequency in visual perception of and semantic representation in images of natural scenes.

Discriminative Cervical Lesion Detection in Colposcopic Images With Global Class Activation and Local Bin Excitation.

Accurate cervical lesion detection (CLD) methods using colposcopic images are highly demanded in computer-aided diagnosis (CAD) for automatic diagnosis of High-grade Squamous Intraepithelial Lesions (HSIL). However, compared to natural scene images, the specific characteristics of colposcopic images, such as low contrast, visual similarity, and ambiguous lesion boundaries, pose difficulties to accurately locating HSIL regions and also significantly impede the performance improvement of existing CLD approaches. To tackle these difficulties and better capture cervical lesions, we develop novel feature enhancing mechanisms from both global and local perspectives, and propose a new discriminative CLD framework, called CervixNet, with a Global Class Activation (GCA) module and a Local Bin Excitation (LBE) module. Specifically, the GCA module learns discriminative features by introducing an auxiliary classifier, and guides our model to focus on HSIL regions while ignoring noisy regions. It globally facilitates the feature extraction process and helps boost feature discriminability. Further, our LBE module excites lesion features in a local manner, and allows the lesion regions to be more fine-grained enhanced by explicitly modelling the inter-dependencies among bins of proposal feature. Extensive experiments on a number of 9888 clinical colposcopic images verify the superiority of our method (AP .75 = 20.45) over state-of-the-art models on four widely used metrics.

Seeing better in nature: contrast enhancement by near infrared imaging

The present paper deals with the question: why is long-distance contrast improved when imaging in the near infrared? This is intimately related to the even broader question, why do we see what we see? The latter is a fundamental general question of optics, connecting the fields of imaging with human and/or camera vision. We first describe the concepts of perception and contrast in general and then discuss the physics background by comparing visible and near infrared images of various natural scenes. Our focus is on the often utilized contrast enhancement in the near infrared range, i.e. on the question of how much clearer we might be able to see if we could see in the near infrared.

Dense depth estimation from multiple 360-degree images using virtual depth

In this paper, we propose a dense depth estimation pipeline for multiview 360∘ images. The proposed pipeline leverages a spherical camera model that compensates for radial distortion in 360∘ images. The key contribution of this paper is the extension of a spherical camera model to multiview by introducing a translation scaling scheme. Moreover, we propose an effective dense depth estimation method by setting virtual depth and minimizing photonic reprojection error. We validate the performance of the proposed pipeline using the images of natural scenes as well as the synthesized dataset for quantitive evaluation. The experimental results verify that the proposed pipeline improves estimation accuracy compared to the current state-of-art dense depth estimation methods.

An efficient ROI detection algorithm for Bangla text extraction and recognition from natural scene images

This research work plays a significant role in finding information from the scene images to fulfill the demand of real life applications like detection of license plate, navigation of robot and helping the visually impaired persons. Here, a new algorithm has been proposed and applied on the scene images to extract Region of Interest (ROI). All the Bangla words are then separated from a sentence by analyzing and applying the Connected Component (CC) method along with bounding box technology. Another new algorithm has been proposed and applied to apart and bring-out Bangla characters from the Bangla words. This algorithm works by the method of vertical scanning of the images of Bangla words. Finally, the extracted characters are recognized by using the Support Vector Machine (SVM) as a classifier which works with Histogram of Oriented Gradient (HOG) features. There are 500 scene images with variations in colors, writing styles and orientations in our designed database. The proposed algorithm yields the accuracy 92.70% and 93.23% in extraction of ROI and character respectively. In the recognition of Bangla characters (digits, Basic characters, and joined characters), the average accuracy is 99.16%. The recognition accuracy of Bangla characters using Convolutional Neural Network (CNN) is also calculated and the obtained result is 83.52%.

How to handle bi/tri-lingual Indic texts in a single image? A new dataset of natural scene and born-digital images

How to handle bi/tri-lingual Indic texts in a single image? A new dataset of natural scene and born-digital images

The spatiotemporal neural dynamics of object location representations in the human brain

To interact with objects in complex environments, we must know what they are and where they are in spite of challenging viewing conditions. Here, we investigated where, how and when representations of object location and category emerge in the human brain when objects appear on cluttered natural scene images using a combination of functional magnetic resonance imaging, electroencephalography and computational models. We found location representations to emerge along the ventral visual stream towards lateral occipital complex, mirrored by gradual emergence in deep neural networks. Time-resolved analysis suggested that computing object location representations involves recurrent processing in high-level visual cortex. Object category representations also emerged gradually along the ventral visual stream, with evidence for recurrent computations. These results resolve the spatiotemporal dynamics of the ventral visual stream that give rise to representations of where and what objects are present in a scene under challenging viewing conditions.

Identification of maize leaf diseases by using the SKPSNet-50 convolutional neural network model

Maize leaf disease is the main factor that affects the yield and quality of maize. Early diagnosis and control of maize leaf diseases help ensure maize yield and quality. However, early maize leaf diseases show small spots and irregular shapes in natural scene images, resulting in current models having difficulty identifying maize leaf diseases accurately. This study proposes a SKPSNet-50 convolutional neural network model to realize the accurate identification of maize leaf diseases in natural scene images. The proposed model replaces the 3 × 3 convolution kernel in the backbone network ResNet-50 with Select Kernel–Point–Swish_B (SKPS), which is an improved building block of the selected kernel (SK) unit, and replaces the ReLU activation function with the Swish_B activation function to improve the feature extraction capability of diseased leaves with small spots and irregular shapes. In addition, this study adopts the combined focal loss function to guide the parameter adjustment of the model to solve the problem of data imbalance. Experimental results show that compared with classic machine learning models (such as BP neural network and support vector machine) and deep neural network models (such as AlexNet, VGG16, ResNet-50, SENet-50, SKNet-50 and GhostNet), the proposed model is more effective in the identification of maize leaf diseases in natural scene images, with an average identification accuracy of 92.9%, which is about 6% higher than that of the SKNet-50 model.

Detection and identification of tea leaf diseases based on AX-RetinaNet

The accurate detection and identification of tea leaf diseases are conducive to its precise prevention and control. Convolutional neural network (CNN) can automatically extract the features of diseased tea leaves in the images. However, tea leaf images taken in natural environments have problems, such as complex backgrounds, dense leaves, and large-scale changes. The existing CNNs have low accuracy in detecting and identifying tea leaf diseases. This study proposes an improved RetinaNet target detection and identification network, AX-RetinaNet, which is used for the automatic detection and identification of tea leaf diseases in natural scene images. AX-RetinaNet uses an improved multiscale feature fusion module of the X-module and adds a channel attention module, Attention. The feature fusion module of the X-module obtains feature maps with rich information through multiple fusions of multi-scale features. The attention module assigns a network adaptively optimized weight to each feature map channel so that the network can select more effective features and reduce the interference of redundant features. This study also uses data augmentation methods to solve the problem of insufficient samples. Experimental results show the detection and identification accuracy of AX-RetinaNet for tea leaf diseases in natural scene images is better than the existing target detection and identification networks, such as SSD, RetinaNet, YOLO-v3, YOLO-v4, Centernet, M2det, and EfficientNet. The AX-RetinaNet detection and identification results indicated the mAP value of 93.83% and the F1-score value of 0.954. Compared with the original network, the mAP value, recall value, and identification accuracy increased by nearly 4%, by 4%, and by nearly 1.5%, respectively.

What can we experience and report on a rapidly presented image? Intersubjective measures of specificity of freely reported contents of consciousness.

Background: A majority of previous studies appear to support a view that human observers can only perceive coarse information from a natural scene image when it is presented rapidly (<100ms, masked). In these studies, participants were often forced to choose an answer from options that experimenters preselected. These options can underestimate what participants experience and can report on it. The current study aims to introduce a novel methodology to investigate how detailed information participants can report after briefly seeing a natural scene image. Methods: We used a novel free-report paradigm to examine what people can freely report following a rapidly presented natural scene image (67/133/267ms, masked). N = 670 online participants typed up to five words to report what they saw in the image together with confidence of the respective responses. We developed a novel index, Intersubjective Agreement (IA). IA quantifies how specifically the response words were used to describe the target image, with a high value meaning the word is not often reported for other images. Importantly, IA eliminates the need for experimenters to preselect response options. Results: The words with high IA values are often something detailed (e.g., a small object) in a particular image. With IA, unlike commonly believed, we demonstrated that participants reported highly specific and detailed aspects of the briefly (even at 67ms, masked) shown image. Further, IA is positively correlated with confidence, indicating metacognitive conscious access to the reported aspects of the image. Conclusion: These new findings challenge the dominant view that the content of rapid scene experience is limited to global and coarse gist. Our novel paradigm opens a door to investigate various contents of consciousness with a free-report paradigm.

Natural image statistics for mouse vision.

The mouse has dichromatic color vision based on two different types of opsins: short (S)- and middle (M)-wavelength-sensitive opsins with peak sensitivity to ultraviolet (UV; 360 nm) and green light (508 nm), respectively. In the mouse retina, cone photoreceptors that predominantly express the S-opsin are more sensitive to contrasts and denser towards the ventral retina, preferentially sampling the upper part of the visual field. In contrast, the expression of the M-opsin gradually increases towards the dorsal retina that encodes the lower visual field. Such a distinctive retinal organization is assumed to arise from a selective pressure in evolution to efficiently encode the natural scenes. However, natural image statistics of UV light remain largely unexplored. Here we developed a multi-spectral camera to acquire high-quality UV and green images of the same natural scenes, and examined the optimality of the mouse retina to the image statistics. We found that the local contrast and the spatial correlation were both higher in UV than in green for images above the horizon, but lower in UV than in green for those below the horizon. This suggests that the dorsoventral functional division of the mouse retina is not optimal for maximizing the bandwidth of information transmission. Factors besides the coding efficiency, such as visual behavioral requirements, will thus need to be considered to fully explain the characteristic organization of the mouse retina.

Contour feature learning for locating text in natural scene images

Contour feature learning for locating text in natural scene images

SIRA: Scale illumination rotation affine invariant mask R-CNN for pedestrian detection.

In this paper, we resolve the challenging obstacle of detecting pedestrians with the ubiquity of irregularities in scale, rotation, and the illumination of the natural scene images natively. Pedestrian instances with such obstacles exhibit significantly unique characteristics. Thus, it strongly influences the performance of pedestrian detection techniques. We propose the new robust Scale Illumination Rotation and Affine invariant Mask R-CNN (SIRA M-RCNN) framework for overcoming the predecessor’s difficulties. The first phase of the proposed system deals with illumination variation by histogram analysis. Further, we use the contourlet transformation, and the directional filter bank for the generation of the rotational invariant features. Finally, we use Affine Scale Invariant Feature Transform (ASIFT) to find points that are translation and scale-invariant. Extensive evaluation of the benchmark database will prove the effectiveness of SIRA M-RCNN. The experimental results achieve state-of-the-art performance and show a significant performance improvement in pedestrian detection.

Cursive Text Recognition in Natural Scene Images Using Deep Convolutional Recurrent Neural Network

Text recognition in natural scene images is a challenging problem in computer vision. Different than the optical character recognition (OCR), text recognition in natural scene images is more complex due to variations in text size, colors, fonts, orientations, complex backgrounds, occlusion, illuminations and uneven lighting conditions. In this paper, we propose a segmentation-free method based on a deep convolutional recurrent neural network to solve the problem of cursive text recognition, particularly focusing on Urdu text in natural scenes. Compared to the non-cursive scripts, Urdu text recognition is more complex due to variations in the writing styles, several shapes of the same character, connected text, ligature overlapping, stretched, diagonal and condensed text. The proposed model gets a whole word image as an input without pre-segmenting into individual characters, and then transforms into the sequence of the relevant features. Our model is based on three components: a deep convolutional neural network (CNN) with shortcut connections to extract and encode the features, a recurrent neural network (RNN) to decode the convolutional features, and a connectionist temporal classification (CTC) to map the predicted sequences into the target labels. To increase the text recognition accuracy further, we explore deeper CNN architectures like VGG-16, VGG-19, ResNet-18 and ResNet-34 to extract more appropriate Urdu text features, and compare the recognition results. To conduct the experiments, a new large-scale benchmark dataset of cropped Urdu word images in natural scenes is developed. The experimental results show that the proposed deep CRNN network with shortcut connections outperform than other network architectures. The dataset is publicly available and can be downloaded from <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://data.mendeley.com/datasets/k5fz57zd9z/1</uri> .