Information In Natural Scenes Research Articles

Subjective Affective Responses to Natural Scenes Require Understanding, Not Spatial Frequency Bands.

It is debated whether emotional processing and response depend on semantic identification or are preferentially tied to specific information in natural scenes, such as global features or local details. The present study aimed to further examine the relationship between scene understanding and affective response while manipulating visual content. To this end, we presented affective and neutral natural scenes which were progressively band-filtered to contain global features (low spatial frequencies) or local details (high spatial frequencies) and assessed both affective response and scene understanding. We observed that, if scene content was correctly reported, subjective ratings of arousal and valence were modulated by the affective content of the scene, and this modulation was similar across spatial frequency bands. On the other hand, no affective modulation of subjective ratings was observed if picture content was not correctly reported. The present results indicate that subjective affective response requires content understanding, and it is not tied to a specific spatial frequency range.

Reinforced Res-Unet transformer for underwater image enhancement

Light propagation through water is subject to varying degrees of energy loss, causing captured images to display characteristics of color distortion, reduced contrast, and indistinct details and textures. The data-driven approach offers significant advantages over traditional algorithms, such as improved accuracy and reduced computational costs. However, challenges such as optimizing network architecture, refining coding techniques, and expanding database resources must be addressed to ensure the generation of high-quality reconstructed images across diverse tasks. In this paper, an underwater image enhancement network based on feature fusion is proposed named RUTUIE, which integrates feature fusion techniques. It leverages the strengths of both Resnet and U-shape architecture, primarily structured around a streamlined up-and-down sampling mechanism. Specifically, the U-shaped structure serves as the backbone of ResNet, equipped with two feature transformers at both the encoding and decoding ends, which are linked by a single-stage up-and-down sampling structure. This architecture is designed to minimize the omission of minor features during feature scale transformations. Furthermore, the improved Transformer encoder leverages a feature-level attention mechanism and the advantages of CNNs, endowing the network with both local and global perceptual capabilities. Then, we propose and demonstrate that embedding an adaptive feature selection module at appropriate locations can retain more learned feature representations. Moreover, the application of a previously proposed color transfer method for synthesizing underwater images and augmenting network training. Extensive experiments demonstrate that our work effectively corrects color casts, reconstructs the rich texture information in natural scenes, and improves the contrast.

A new deep CNN for 3D text localization in the wild through shadow removal

Text localization in the wild is challenging due to the presence of 2D and 3D texts, the presence of shadows, arbitrary orientated text with non-linear arrangements, varying lighting conditions as well as complex background. This paper proposes the first approach for 3D text localization in natural scene images through shadow removal and a new deep CNN model. In a first step, exploiting the observation that 3D text generates shadow information in natural scenes, the proposed model detects and removes the shadow pixels of 3D text based on the Generalized Gradient Vector Flow concept and a new clustering approach. The performance of the classification of 2D and 3D texts in the scene images is strengthened by using key features, including pixel strength, sharpness and edge potential, which are extracted to eliminate false text and shadow pixels. For text localization after removing shadow information, EfficientNet is used as an encoder (backbone) and UNet as a decoder in a novel way employing differential binarization. Experimental validation and comparative analysis with state-of-the-art approaches on both a new purpose-built dataset as well as on the benchmark datasets of ICDAR MLT 2019, ICDAR ArT 2019, CTW1500, DAST1500, Total-Text, and MSRATD500 for each of the different steps of the method, show that the proposed approach outperforms the existing methods.

Detection of Oriented Scene Text in Natural Images

Recently, in the field of computer vision, people are paying more attention to extracting and understanding text information in natural scenes, which are widely used in real life, such as image OCR, image retrieval, and multi-language translation. Text detection plays a vital role in the process of text information extraction and understanding. Diverse text patterns and highly cluttered backgrounds are the main challenges of precise text detection. This paper studies oriented scene text detection, the horizontal scene text detector, and CTPN network architecture, which is improved and extended for oriental scene text detection. The main proposed improvements include: 1. The pre-processing method and parameter setting in the post-processing are improved to make the annotation more accurate; 2. The anchor mechanism is improved by setting a series of different anchor widths depending on the anchor heights instead of the fixed width. The improved model was trained with the training set of ICDAR2015 and part of ICDAR2017 MLT. Then it was tested on the ICDAR online platform with test sets of ICDAR2013 and ICDAR2015. Finally, it was tested with some images outside the datasets. The results show that the improved model can detect text with a tilt angle more precisely.

A Convolutional Recurrent Neural-Network-Based Machine Learning for Scene Text Recognition Application

Optical character recognition (OCR) is the process of acquiring text and layout information through analysis and recognition of text data image files. It is also a process to identify the geometric location and orientation of the texts and their symmetrical behavior. It usually consists of two steps: text detection and text recognition. Scene text recognition is a subfield of OCR that focuses on processing text in natural scenes, such as streets, billboards, license plates, etc. Unlike traditional document category photographs, it is a challenging task to use computer technology to locate and read text information in natural scenes. Imaging sequence recognition is a longstanding subject of research in the field of computer vision. Great progress has been made in this field; however, most models struggled to recognize text in images of complex scenes with high accuracy. This paper proposes a new pattern of text recognition based on the convolutional recurrent neural network (CRNN) as a solution to address this issue. It combines real-time scene text detection with differentiable binarization (DBNet) for text detection and segmentation, text direction classifier, and the Retinex algorithm for image enhancement. To evaluate the effectiveness of the proposed method, we performed experimental analysis of the proposed algorithm, and carried out simulation on complex scene image data based on existing literature data and also on several real datasets designed for a variety of nonstationary environments. Experimental results demonstrated that our proposed model performed better than the baseline methods on three benchmark datasets and achieved on-par performance with other approaches on existing datasets. This model can solve the problem that CRNN cannot identify text in complex and multi-oriented text scenes. Furthermore, it outperforms the original CRNN model with higher accuracy across a wider variety of application scenarios.

Multi-focus image fusion based on multi-scale sparse representation

Although colorful information in natural scenes can be collected, due to the limitation of camera depth of field, it is hard to capture an image with all-in-focus. Sparse representation (SR)-based methods have shown their powerful potentiality and ability in multi-focus image fusion. However, because of sparse coding and information compress, the existing fusion methods based on SR are imperfect to seize the rich details and significant texture information in source images. As a result, a fusion method based on multi-scale sparse representation for registered multi-focus images (MIF-MsSR) is proposed in this paper, where an adaptive fusion rule for sparse coefficients is presented. At first, source images are processed by multi-scale decomposition and sub-images with different scales can be obtained. According to image features with different richness in these sub-images, dictionaries with different sizes and redundancy are thereby trained. By comprehensively considering the relationships of focused areas, out-of-focused areas and boundary areas between the source images, an adaptive fusion rule based on l0−max and Sum Modified Laplacian (SML) is proposed. Finally, a fused image with all-in-focus can be obtained by sparse reconstruction and inverse multi-scale decomposition. Excessive experiments on multi-focus images have demonstrated that the proposed MIF-MsSR not only reserves the integrity of the information in source images, but also has better fusion performance on subjective and objective indicators than other state-of-the-art methods.

Cortical networks of dynamic scene category representation in the human brain

Humans have an impressive ability to rapidly process global information in natural scenes to infer their category. Yet, it remains unclear whether and how scene categories observed dynamically in the natural world are represented in cerebral cortex beyond few canonical scene-selective areas. To address this question, here we examined the representation of dynamic visual scenes by recording whole-brain blood oxygenation level-dependent (BOLD) responses while subjects viewed natural movies. We fit voxelwise encoding models to estimate tuning for scene categories that reflect statistical ensembles of objects and actions in the natural world. We find that this scene-category model explains a significant portion of the response variance broadly across cerebral cortex. Cluster analysis of scene-category tuning profiles across cortex reveals nine spatially-segregated networks of brain regions consistently across subjects. These networks show heterogeneous tuning for a diverse set of dynamic scene categories related to navigation, human activity, social interaction, civilization, natural environment, non-human animals, motion-energy, and texture, suggesting that the organization of scene category representation is quite complex.

Observing animals and humans: dogs target their gaze to the biological information in natural scenes.

BackgroundThis study examines how dogs observe images of natural scenes containing living creatures (wild animals, dogs and humans) recorded with eye gaze tracking. Because dogs have had limited exposure to wild animals in their lives, we also consider the natural novelty of the wild animal images for the dogs.MethodsThe eye gaze of dogs was recorded while they viewed natural images containing dogs, humans, and wild animals. Three categories of images were used: naturalistic landscape images containing single humans or animals, full body images containing a single human or an animal, and full body images containing a pair of humans or animals. The gazing behavior of two dog populations, family and kennel dogs, were compared.ResultsAs a main effect, dogs gazed at living creatures (object areas) longer than the background areas of the images; heads longer than bodies; heads longer than background areas; and bodies longer than background areas. Dogs gazed less at the object areas vs. the background in landscape images than in the other image categories. Both dog groups also gazed wild animal heads longer than human or dog heads in the images. When viewing single animal and human images, family dogs focused their gaze very prominently on the head areas, but in images containing a pair of animals or humans, they gazed more at the body than the head areas. In kennel dogs, the difference in gazing times of the head and body areas within single or paired images failed to reach significance.DiscussionDogs focused their gaze on living creatures in all image categories, also detecting them in the natural landscape images. Generally, they also gazed at the biologically informative areas of the images, such as the head, which supports the importance of the head/face area for dogs in obtaining social information. The natural novelty of the species represented in the images as well as the image category affected the gazing behavior of dogs. Furthermore, differences in the gazing strategy between family and kennel dogs was obtained, suggesting an influence of different social living environments and life experiences.

Natural Scene Nutrition Information Acquisition and Analysis Based on Deep Learning

Background: In the field of personalized health, it is often difficult for individuals to obtain professional knowledge to solve their practical problems timely and accurately. While there are some applications that can get targeted information, they often fail to function properly in nonideal environments, and they cannot achieve precise answers to individual users. Therefore, how to establish an information capture model based on big data and combine it with intelligent search is an important issue in the field of personalized health. Objective: This paper starts with the information acquisition and intelligent recommendation in the field of personalized health, and proposes a natural scene information acquisition and analysis model based on deep learning, focusing on improving the recognition rate of text in natural scenes and achieving targeted smart search to allow users to get more accurate personalized health advice. Methods: In this model, natural scene information is processed from four aspects: targeted big data collection and search, connected text proposal network text detection algorithm and projectionbased text segmentation, capsule network text recognition and result analysis. The model reduces recognition bias due to problems such as special filming conditions and photographic techniques by using deep learning algorithms. At the same time, the data mining has also improved the pertinence of the results analysis. Conclusion: This model combines deep learning and data mining methods to obtain intelligent solutions at a professional level by uploading target information images in non-ideal environments, and is suitable for accurate analysis of problems in personalized health area. Results: The proposed model is applied to analyze the user's nutrient intake requirements. The results show that the method achieves 83% prediction accuracy on the nutrient composition table dataset, and its performance is better than current convolutional neural network applications. And the model can get accurate personalized data to provide users with dietary advice.

APP Medical Diagnostic Check-up Consultation System Based on Speech Recognition

Background:Medical test orders can display the physiological functions of patients by using medical means. The medical staff determines the patient's condition through medical test orders and completes the treatment. However, for most patients and their families, there are so many terminologies in the medical test list and they are inconvenient to understand and query, which would affect the patients’ cognition and treatment effect. Therefore, it is especially necessary to develop a consulting system that can provide related analysis after getting medical test data.Objective :This paper starts with information acquisition and speech recognition. It proposes a natural scene information acquisition and analysis model based on deep learning, focusing on improving the recognition rate of routine test list and achieving targeted smart search to allow users to get more accurate personalized health advice.Methods :Based on medical characteristics, considering the needs of patients, this paper constructs an APP-based conventional medical test consultation system, using artificial intelligence and voice recognition technology to collect user input; analyzing user needs with the help of conventional medical information knowledge database.Results:This model combines speech recognition and data mining methods to obtain routine test list data and is suitable for accurate analysis of problems in routine check-up procedure. The app provides effective explanations and guidance for the treatment and rehabilitation of patients.Conclusion:It organically links the Internet with personalized medicine, which can effectively improve the popularity of medical knowledge and provide a reference for the application of medical services on the Internet. Meanwhile, this app can contribute to the improvement of medical standards and provide new models for modern medical management.

Blind Tone-Mapped Image Quality Assessment Based on Regional Sparse Response and Aesthetics.

High dynamic range (HDR) images give a strong disposition to capture all parts of natural scene information due to their wider brightness range than traditional low dynamic range (LDR) images. However, to visualize HDR images on common LDR displays, tone mapping operations (TMOs) are extra required, which inevitably lead to visual quality degradation, especially in the bright and dark regions. To evaluate the performance of different TMOs accurately, this paper proposes a blind tone-mapped image quality assessment method based on regional sparse response and aesthetics (RSRA-BTMI) by considering the influences of detail information and color on the human visual system. Specifically, for the detail loss in a tone-mapped image (TMI), multi-dictionaries are first designed for different brightness regions and whole TMI. Then regional sparse atoms aggregated by local entropy and global reconstruction residuals are presented to characterize the regional and global detail distortion in TMI, respectively. Besides, a few efficient aesthetic features are extracted to measure the color unnaturalness of TMI. Finally, all extracted features are linked with relevant subjective scores to conduct quality regression via random forest. Experimental results on the ESPL-LIVE HDR database demonstrate that the proposed RSRA-BTMI method is superior to the existing state-of-the-art blind TMI quality assessment methods.

How redundant are luminance and chrominance information in natural scenes?

Color vision is an impressive ability of the human visual system. Three different types of cones in the retina, and then three postreceptoral cone-opponent mechanisms enable us to extract both luminance and chromatic information from our environment. Theories of efficient coding argue that the visual system evolved this way to optimally process natural scenes. Therefore, the study of statistical properties of natural images has great potential value to expand our understanding of visual perception. From that perspective, a central question is to characterize the role of chromatic information: in natural scenes, what information can be extracted from chromatic channels that is not already present in luminance? The literature on this point is contradictory. On the one hand, several studies have investigated the joint distribution of luminance and chrominance, finding that luminance and chromatic edges are not independent (Fine et al. 2003) and that most edges are defined by luminance contrast with color information being redundant (Zhou & Mel 2008). On the other hand, Hansen and Gegenfurtner (2009) showed using mutual information that luminance and chromatic edges constitute relatively independent sources of information and that their independence increases along successive stages of visual processing. Here we improve and extend their analysis, using ~1000 images taken from the McGill calibrated color image database (Olmos & Kingdom, 2004) to quantify the redundancy of luminance and chrominance information in each image individually. While we replicate the main findings of Hansen and Gegenfurtner (2009), we also find that the estimated amount of mutual information depends on how the images are processed. The most critical step is divisive normalization, a late stage in the processing pipeline. How redundant chrominance and luminance are, may thus depend on the precise definition of these two quantities, explaining some inconsistencies in the literature.

Texture MTF from images of natural scenes

The implementation of automatic, adaptive filters in consumer imaging devices represents challenges to sharpness and resolution evaluation. The widely used e-SFR and other methods based on sine-waves and line targets are not necessarily representative of the capture of natural scene information. The recent dead leaves target is aimed at producing texture-MTFs that describe the capture of image detail under automatic non-linear, and contentaware processes. A newer approach to the texture-MTF measurement that substitutes the dead leaves target with pictorial images is presented in this paper. The aim of the proposed method is to measure effective-MTFs indicative of system characteristics for given scenes and camera processes. Nine pictorial images, portraying a variety of subjects and textures, were set as targets for a DSLR camera and a high-end smartphone camera. Computed MTFs were found to be congruent with the dead leaves MTF. Scene dependency was reported mainly for the smartphone camera measurements, providing insight into the performance of the content-dependent processes. Results from the DLSR camera images, captured with minimum non-adaptive operations, were reasonably consistent for the majority of the scenes. Based on variations in scene-dependent MTFs, we make recommendations for scene content that is best for texture-MTF analysis.

Functional Subdomains within Scene-Selective Cortex: Parahippocampal Place Area, Retrosplenial Complex, and Occipital Place Area.

Visual scene perception is a critical ability to survive in the real world. It is therefore reasonable to assume that the human brain contains neural circuitry selective for visual scenes. Here we show that responses in three scene-selective areas-identified in previous studies-carry information about many object and action categories encountered in daily life. We identify two subregions in each area: one that is selective for categories of man-made objects, and another that is selective for vehicles and locomotion-related action categories that appear in dynamic scenes. This consistent functional subdivision may reflect an anatomical organization into two processing streams, one biased toward static stimuli and one biased toward dynamic stimuli.

Influence of environmental information in natural scenes and the effects of motion adaptation on a fly motion-sensitive neuron during simulated flight.

ABSTRACTGaining information about the spatial layout of natural scenes is a challenging task that flies need to solve, especially when moving at high velocities. A group of motion sensitive cells in the lobula plate of flies is supposed to represent information about self-motion as well as the environment. Relevant environmental features might be the nearness of structures, influencing retinal velocity during translational self-motion, and the brightness contrast. We recorded the responses of the H1 cell, an individually identifiable lobula plate tangential cell, during stimulation with image sequences, simulating translational motion through natural sceneries with a variety of differing depth structures. A correlation was found between the average nearness of environmental structures within large parts of the cell's receptive field and its response across a variety of scenes, but no correlation was found between the brightness contrast of the stimuli and the cell response. As a consequence of motion adaptation resulting from repeated translation through the environment, the time-dependent response modulations induced by the spatial structure of the environment were increased relatively to the background activity of the cell. These results support the hypothesis that some lobula plate tangential cells do not only serve as sensors of self-motion, but also as a part of a neural system that processes information about the spatial layout of natural scenes.

From image processing to computational neuroscience: a neural model based on histogram equalization.

There are many ways in which the human visual system works to reduce the inherent redundancy of the visual information in natural scenes, coding it in an efficient way. The non-linear response curves of photoreceptors and the spatial organization of the receptive fields of visual neurons both work toward this goal of efficient coding. A related, very important aspect is that of the existence of post-retinal mechanisms for contrast enhancement that compensate for the blurring produced in early stages of the visual process. And alongside mechanisms for coding and wiring efficiency, there is neural activity in the human visual cortex that correlates with the perceptual phenomenon of lightness induction. In this paper we propose a neural model that is derived from an image processing technique for histogram equalization, and that is able to deal with all the aspects just mentioned: this new model is able to predict lightness induction phenomena, and improves the efficiency of the representation by flattening both the histogram and the power spectrum of the image signal.

Contrast-dependent OFF-dominance in cat primary visual cortex facilitates discrimination of stimuli with natural contrast statistics.

Both theoretical and experimental studies suggest that response properties in the visual system are shaped by signals in the natural environment. Recent studies showed that, in the primary visual cortex (V1), neurons preferring light decrements (OFF stimuli) outnumber those preferring light increments (ON stimuli). However, it is not clear whether the OFF-dominance in V1 neurons is related to the contrast statistics in natural images. By analysing the distribution of negative and positive contrasts in natural images at several spatial scales, we showed that optimal coding of the natural contrast signals would lead to a contrast-dependent OFF-dominant response, with a stronger degree of OFF-dominance at a higher contrast. Using bright and dark stimuli at various contrast levels to measure the receptive fields of neurons in cat V1, we found an increasing degree of OFF-dominance of the neuronal population as the contrast was increased. By modeling receptive fields exhibiting OFF- and ON-dominance, we found that contrast-dependent OFF-dominance facilitated the discrimination of stimuli with natural contrast distribution. Thus, by matching contrast-dependent OFF-dominance to the statistics of contrast distribution in natural images, V1 neurons may better discriminate contrast information in natural scenes.

Attention-free integration of spatial frequency-based information in natural scenes

The integration of visual image information provided by low and high spatial frequency channels is critical to rapid perception of natural scenes. However, little is known about the role of attention in integrating this information. In two experiments, using attention-demanding tasks, we examined the advantage of integration, i.e. the superior categorization accuracies for images, using a wide range of different spatial frequencies. In Experiment 1, a spatially central to-be-identified letter and a peripheral filtered image of a natural scene appeared simultaneously. In Experiment 2, the letter and the image were presented sequentially at the same spatial location. In both experiments results consistently showed an advantage of integration in categorization behavior that was not influenced by attention-demanding tasks. This finding suggests that the integration of frequency-based information in natural scenes is attention-free.

Preattentive integration of spatial frequency-based information in natural scenes

Preattentive integration of spatial frequency-based information in natural scenes

Manipulating the content of dynamic natural scenes to characterize response in human MT/MST

Optic flow is one of the most important sources of information for enabling human navigation through the world. A striking finding from single-cell studies in monkeys is the rapid saturation of response of MT/MST areas with the density of optic flow type motion information. These results are reflected psychophysically in human perception in the saturation of motion aftereffects. We began by comparing responses to natural optic flow scenes in human visual brain areas to responses to the same scenes with inverted contrast (photo negative). This changes scene familiarity while preserving local motion signals. This manipulation had no effect; however, the response was only correlated with the density of local motion (calculated by a motion correlation model) in V1, not in MT/MST. To further investigate this, we manipulated the visible proportion of natural dynamic scenes and found that areas MT and MST did not increase in response over a 16-fold increase in the amount of information presented, i.e., response had saturated. This makes sense in light of the sparseness of motion information in natural scenes, suggesting that the human brain is well adapted to exploit a small amount of dynamic signal and extract information important for survival.