Features Of Natural Images Research Articles

An Improved Skeleton Extraction Method via Multi-Task and Variable Coefficient Loss Function in Natural Images

Extracting object skeleton and its scale features in natural images is helpful for object detection and recognition in computer vision. In order to advance the location accuracy of object skeleton pixels, a new method via multi-task and variable coefficient loss function is proposed in this paper. Adopting the hierarchical integration mechanism to mutually refine captured features at different network layers; a specific variable coefficient loss function is designed for multi-class imbalanced data handling problem, such as the skeleton pixels in natural images are always far less than the non-skeleton pixels; the regression algorithm is an added deep learning branch in the skeleton extraction network assisting the improvement of recognition accuracy. Besides, not only the skeleton pixels and its classification can be obtained, but also its scales are predicted without disturbing skeleton acquisition process. The experimental results verify that both the skeleton accuracy and the generalization abilities are promoted benefiting from the regression task and the new loss function in the new method, as satisfactory results are achieved on three public datasets, i.e., SK-LARGE, SK-SMALL, and WH-SYMMAX, which are indicated by F-measures and precision/recall curves. The results further demonstrate that the proposed method is superior to the best skeleton extraction method available currently.

A Content-Adaptive Method for Image Denoising

In this paper, we firstly analyze the statistical distribution of simultaneous sparse coding errors (SSCE), which reflects the local correlation and non-local correlation characteristics of natural images. Based on the observation, we establish the optimal denoising problem which uses $L_{1}$ norm to constrain the image prior. According to the close-form solution of the proposed problem, we find that the denoising limit is only determined by the patch number, patch size and the variances of different bands in SSCE. Then we exploit the relationships between the patch complexity, patch size, patch number and the denoising limit. The study shows that a content-adaptive strategy may be useful to obtain better denoising performance. We then design a content-adaptive method for image denoising in which the groups are adaptively determined according to the structural complexities of reference patches. Experimental results show that the proposed scheme achieves competitive performance with several state-of-the-art methods in both subjective and objective aspects.

CloudifierNet - Deep Vision Models for Artificial Image Processing

Today, more and more, it is necessary that most applications and documents developed in previous or current technologies to be accessible online on cloud-based infrastructures. That’s why the migration of legacy systems including their hosts of documents to new technologies and online infrastructures, using modern Artificial Intelligence techniques, is absolutely necessary.With the advancement of Artificial Intelligence and Deep Learning with its multitude of applications, a new area of research is emerging – that of automated systems development and maintenance. The underlying work objective that led to this paper aims to research and develop truly intelligent systems able to analyze user interfaces from various sources and generate real and usable inferences ranging from architecture analysis to actual code generation. One key element of such systems is that of artificial scene detection and analysis based on deep learning computer vision systems. Computer vision models and particularly deep directed acyclic graphs based on convolutional modules are generally constructed and trained based on natural images datasets. Due to this fact, the models will develop during the training process natural image feature detectors apart from the base graph modules that will learn basic primitive features. In the current paper, we will present the base principles of a deep neural pipeline for computer vision applied to artificial scenes (scenes generated by user interfaces or similar). Finally, we will present the conclusions based on experimental development and benchmarking against state-of-the-art transfer-learning implemented deep vision models.

End-to-End Airport Detection in Remote Sensing Images Combining Cascade Region Proposal Networks and Multi-Threshold Detection Networks

Fast and accurate airport detection in remote sensing images is important for many military and civilian applications. However, traditional airport detection methods have low detection rates, high false alarm rates and slow speeds. Due to the power convolutional neural networks in object-detection systems, an end-to-end airport detection method based on convolutional neural networks is proposed in this study. First, based on the common low-level visual features of natural images and airport remote sensing images, region-based convolutional neural networks are chosen to conduct transfer learning for airport images using a limited amount of data. Second, to further improve the detection rate and reduce the false alarm rate, the concepts of “divide and conquer” and “integral loss’’ are introduced to establish cascade region proposal networks and multi-threshold detection networks, respectively. Third, hard example mining is used to improve the object discrimination ability and the training efficiency of the network during sample training. Additionally, a cross-optimization strategy is employed to achieve convolution layer sharing between the cascade region proposal networks and the subsequent multi-threshold detection networks, and this approach significantly decreases the detection time. The results show that the method established in this study can accurately detect various types of airports in complex backgrounds with a higher detection rate, lower false alarm rate, and shorter detection time than existing airport detection methods.

A no-reference quality assessment for contrast-distorted image based on improved learning method

No-reference image quality assessment (NR-IQA), which aims to predict image quality without accessing to reference images, is a fundamental and challenging problem in the field of image processing. Nevertheless, there are few researches about contrast-distorted images and results of the existing NR-IQA methods which cannot be in accordance with the subjective assessment results further. Therefore, an effective NR-IQA method for contrast-distorted image is proposed in this paper. Firstly, the proposed method extracts five features of all images from the database based on the natural scene statistics (NSS) model. Then the curve fitting method is subsequently represented to calculate values of natural image features. Finally, an improved Support Vector Regression (SVR) learning method based on grid search is proposed to establish the mapping between image feature values and the quality score of a test image. Experiments proved that the proposed method is effective when compared with other related state-of-the-art image quality assessment (IQA) methods based on three standard databases.

Common Object Representations for Visual Production and Recognition.

Production and comprehension have long been viewed as inseparable components of language. The study of vision, by contrast, has centered almost exclusively on comprehension. Here we investigate drawing-the most basic form of visual production. How do we convey concepts in visual form, and how does refining this skill, in turn, affect recognition? We developed an online platform for collecting large amounts of drawing and recognition data, and applied a deep convolutional neural network model of visual cortex trained only on natural images to explore the hypothesis that drawing recruits the same abstract feature representations that support natural visual object recognition. Consistent with this hypothesis, higher layers of this model captured the abstract features of both drawings and natural images most important for recognition, and people learning to produce more recognizable drawings of objects exhibited enhanced recognition of those objects. These findings could explain why drawing is so effective for communicating visual concepts, they suggest novel approaches for evaluating and refining conceptual knowledge, and they highlight the potential of deep networks for understanding human learning.

Amygdala lesions eliminate viewing preferences for faces in rhesus monkeys

In free-viewing experiments, primates orient preferentially toward faces and face-like stimuli. To investigate the neural basis of this behavior, we measured the spontaneous viewing preferences of monkeys with selective bilateral amygdala lesions. The results revealed that when faces and nonface objects were presented simultaneously, monkeys with amygdala lesions had no viewing preference for either conspecific faces or illusory facial features in everyday objects. Instead of directing eye movements toward socially relevant features in natural images, we found that, after amygdala loss, monkeys are biased toward features with increased low-level salience. We conclude that the amygdala has a role in our earliest specialized response to faces, a behavior thought to be a precursor for efficient social communication and essential for the development of face-selective cortex.

Strabismus Recognition Using Eye-Tracking Data and Convolutional Neural Networks.

Strabismus is one of the most common vision diseases that would cause amblyopia and even permanent vision loss. Timely diagnosis is crucial for well treating strabismus. In contrast to manual diagnosis, automatic recognition can significantly reduce labor cost and increase diagnosis efficiency. In this paper, we propose to recognize strabismus using eye-tracking data and convolutional neural networks. In particular, an eye tracker is first exploited to record a subject's eye movements. A gaze deviation (GaDe) image is then proposed to characterize the subject's eye-tracking data according to the accuracies of gaze points. The GaDe image is fed to a convolutional neural network (CNN) that has been trained on a large image database called ImageNet. The outputs of the full connection layers of the CNN are used as the GaDe image's features for strabismus recognition. A dataset containing eye-tracking data of both strabismic subjects and normal subjects is established for experiments. Experimental results demonstrate that the natural image features can be well transferred to represent eye-tracking data, and strabismus can be effectively recognized by our proposed method.

Publisher Correction: Letter perception emerges from unsupervised deep learning and recycling of natural image features

In the version of this Letter originally published, in the sentence beginning “Written symbols are culture specific...”, in the second example, ‘Φ’ was used instead of ‘F’; it should have read ‘(for example, ℱ versus F)’. This has now been corrected in all versions of the Letter.

Letter perception emerges from unsupervised deep learning and recycling of natural image features.

The use of written symbols is a major achievement of human cultural evolution. However, how abstract letter representations might be learned from vision is still an unsolved problem 1,2 . Here, we present a large-scale computational model of letter recognition based on deep neural networks 3,4 , which develops a hierarchy of increasingly more complex internal representations in a completely unsupervised way by fitting a probabilistic, generative model to the visual input 5,6 . In line with the hypothesis that learning written symbols partially recycles pre-existing neuronal circuits for object recognition 7 , earlier processing levels in the model exploit domain-general visual features learned from natural images, while domain-specific features emerge in upstream neurons following exposure to printed letters. We show that these high-level representations can be easily mapped to letter identities even for noise-degraded images, producing accurate simulations of a broad range of empirical findings on letter perception in human observers. Our model shows that by reusing natural visual primitives, learning written symbols only requires limited, domain-specific tuning, supporting the hypothesis that their shape has been culturally selected to match the statistical structure of natural environments 8 .

Deep Transfer Learning for Modality Classification of Medical Images

Medical images are valuable for clinical diagnosis and decision making. Image modality is an important primary step, as it is capable of aiding clinicians to access required medical image in retrieval systems. Traditional methods of modality classification are dependent on the choice of hand-crafted features and demand a clear awareness of prior domain knowledge. The feature learning approach may detect efficiently visual characteristics of different modalities, but it is limited to the number of training datasets. To overcome the absence of labeled data, on the one hand, we take deep convolutional neural networks (VGGNet, ResNet) with different depths pre-trained on ImageNet, fix most of the earlier layers to reserve generic features of natural images, and only train their higher-level portion on ImageCLEF to learn domain-specific features of medical figures. Then, we train from scratch deep CNNs with only six weight layers to capture more domain-specific features. On the other hand, we employ two data augmentation methods to help CNNs to give the full scope to their potential characterizing image modality features. The final prediction is given by our voting system based on the outputs of three CNNs. After evaluating our proposed model on the subfigure classification task in ImageCLEF2015 and ImageCLEF2016, we obtain new, state-of-the-art results—76.87% in ImageCLEF2015 and 87.37% in ImageCLEF2016—which imply that CNNs, based on our proposed transfer learning methods and data augmentation skills, can identify more efficiently modalities of medical images.

The Effects of misregistration on the dead leaves crosscorrelation texture blur analysis

The dead leaves image model is often used for measurement of the spatial frequency response (SFR) of digital cameras, where response to fine texture is of interest. It has a power spectral density (PSD) similar to natural images and image features of varying sizes, making it useful for measuring the texture-blurring effects of non-linear noise reduction which may not be well analyzed by traditional methods. The standard approach for analyzing images of this model is to compare observed PSDs to the analytically known one. However, recent works have proposed a cross-correlation based approach which promises more robust measurements via full-reference comparison with the known true pattern. A major assumption of this method is that the observed image and reference image can be aligned (registered) with subpixel accuracy. In this paper we study the effects of registration errors on the calculation of texture-based SFR and its derivative metrics (such as MTF50), in order to determine how accurate this registration must be for reliable results. We also propose a change to the dead leaves cross-correlation algorithm, recommending the use of the absolute value of the transfer function rather than its real part. Simulations of registration error on both real and simulated observed images reveal that small amounts of misregistration (as low as 0.15px) can cause large variability in MTF curves derived using the real part of the transfer function, while MTF curves derived from the absolute value are significantly less affected.

Second-Generation Curvelets on the Sphere

Curvelets are efficient to represent highly anisotropic signal content, such as a local linear and curvilinear structure. First-generation curvelets on the sphere, however, suffered from blocking artefacts. We present a new second-generation curvelet transform, where scale-discretised curvelets are constructed directly on the sphere. Scale-discretised curvelets exhibit a parabolic scaling relation, are well-localised in both spatial and harmonic domains, support the exact analysis and synthesis of both scalar and spin signals, and are free of blocking artefacts. We present fast algorithms to compute the exact curvelet transform, reducing computational complexity from $\mathcal{O}(L^5)$ to $\mathcal{O}(L^3\log_{2}{L})$ for signals band-limited at $L$. The implementation of these algorithms is made publicly available. Finally, we present an illustrative application demonstrating the effectiveness of curvelets for representing directional curve-like features in natural spherical images.

Using Matrix and Tensor Factorizations for the Single-Trial Analysis of Population Spike Trains.

Advances in neuronal recording techniques are leading to ever larger numbers of simultaneously monitored neurons. This poses the important analytical challenge of how to capture compactly all sensory information that neural population codes carry in their spatial dimension (differences in stimulus tuning across neurons at different locations), in their temporal dimension (temporal neural response variations), or in their combination (temporally coordinated neural population firing). Here we investigate the utility of tensor factorizations of population spike trains along space and time. These factorizations decompose a dataset of single-trial population spike trains into spatial firing patterns (combinations of neurons firing together), temporal firing patterns (temporal activation of these groups of neurons) and trial-dependent activation coefficients (strength of recruitment of such neural patterns on each trial). We validated various factorization methods on simulated data and on populations of ganglion cells simultaneously recorded in the salamander retina. We found that single-trial tensor space-by-time decompositions provided low-dimensional data-robust representations of spike trains that capture efficiently both their spatial and temporal information about sensory stimuli. Tensor decompositions with orthogonality constraints were the most efficient in extracting sensory information, whereas non-negative tensor decompositions worked well even on non-independent and overlapping spike patterns, and retrieved informative firing patterns expressed by the same population in response to novel stimuli. Our method showed that populations of retinal ganglion cells carried information in their spike timing on the ten-milliseconds-scale about spatial details of natural images. This information could not be recovered from the spike counts of these cells. First-spike latencies carried the majority of information provided by the whole spike train about fine-scale image features, and supplied almost as much information about coarse natural image features as firing rates. Together, these results highlight the importance of spike timing, and particularly of first-spike latencies, in retinal coding.

Robust video fingerprinting scheme based on contourlet hidden Markov tree model

Video fingerprints are feature vectors that uniquely characterize a video clip and can be used for video indexing and copyright applications. This paper presents a novel video fingerprinting method based on the Contourlet hidden Markov tree model and Singular Value Decomposition. The Contourlet transform is a two-dimensional extension of the wavelet transform using multiscale and directional filter banks. It effectively captures the smooth contours that are the dominant features in natural images. The Contourlet HMT model can capture all inter-scale, inter-direction, and inter-location dependencies of the contourlet coefficients using a few statistics parameters. These parameters are robust against common content- preserving operations such as lossy compression, additive noise, filtering etc. We introduce SVD to compress the parameter matrix for generating the video fingerprint. The performance of the proposed fingerprint is experimentally evaluated and compared to the Centroid of Gradient Orientations based method. The proposed method is shown to have a better overall performance in robustness and video identification.

Data-Driven Learning of a Union of Sparsifying Transforms Model for Blind Compressed Sensing

Compressed sensing is a powerful tool in applications such as magnetic resonance imaging (MRI). It enables accurate recovery of images from highly undersampled measurements by exploiting the sparsity of the images or image patches in a transform domain or dictionary. In this work, we focus on blind compressed sensing (BCS), where the underlying sparse signal model is a priori unknown, and propose a framework to simultaneously reconstruct the underlying image as well as the unknown model from highly undersampled measurements. Specifically, our model is that the patches of the underlying image(s) are approximately sparse in a transform domain. We also extend this model to a union of transforms model that better captures the diversity of features in natural images. The proposed block coordinate descent type algorithms for blind compressed sensing are highly efficient, and are guaranteed to converge to at least the partial global and partial local minimizers of the highly non-convex BCS problems. Our numerical experiments show that the proposed framework usually leads to better quality of image reconstructions in MRI compared to several recent image reconstruction methods. Importantly, the learning of a union of sparsifying transforms leads to better image reconstructions than a single adaptive transform.

Fast image quality assessment via supervised iterative quantization method

No-reference/Blind image quality assessment (NR-IQA/BIQA) is significant for image processing and yet very challenging, especially for real-time application and big image data processing. Traditional NR-IQA metrics usually train complex models such as support vector machine, neural network, and probability graph model, which result in long computational time and poor robustness. To overcome these weaknesses, the paper proposes a fast no-reference image quality assessment via hash coding method, named NRHC. First, the image is divided into overlapped patches to extract the spatial statistical features of natural scene images. Then the features are encoded to produce binary hash codes via supervised iterative quantization (SITQ) method. Finally, the Hamming distances between the hash code of the test image and those of original undistorted images are calculated to obtain the final image quality. Thorough experiments on benchmark databases demonstrate that the proposed approach achieves comparable performance and has higher computational efficiency and stronger robustness compared with the state-of-the-art NR-IQA methods.

Vision-based adaptive assistance and haptic guidance for safe wheelchair corridor following

In case of motor impairments, steering a wheelchair can become a hazardous task. Joystick jerks induced by uncontrolled motions may lead to wall collisions when a user steers a wheelchair along a corridor. This work introduces a low-cost assistive and guidance system for indoor corridor navigation in a wheelchair, which uses purely visual information, and which is capable of providing automatic trajectory correction and haptic guidance in order to avoid wall collisions. A visual servoing approach to autonomous corridor following serves as the backbone to this system. The algorithm employs natural image features which can be robustly extracted in real time. This algorithm is then fused with manual joystick input from the user so that progressive assistance and trajectory correction can be activated as soon as the user is in danger of collision. A force feedback in conjunction with the assistance is provided on the joystick in order to guide the user out of his dangerous trajectory. This ensures intuitive guidance and minimal interference from the trajectory correction system. In addition to being a low-cost approach, it can be seen that the proposed solution does not require an a-priori environment model. Experiments on a robotised wheelchair equipped with a monocular camera prove the capability of the system to adaptively guide and assist a user navigating in a corridor.

Representation of Maximally Regular Textures in Human Visual Cortex.

Naturalistic textures with an intermediate degree of statistical regularity can capture key structural features of natural images (Freeman and Simoncelli, 2011). V2 and later visual areas are sensitive to these features, while primary visual cortex is not (Freeman et al., 2013). Here we expand on this work by investigating a class of textures that have maximal formal regularity, the 17 crystallographic wallpaper groups (Fedorov, 1891). We used texture stimuli from four of the groups that differ in the maximum order of rotation symmetry they contain, and measured neural responses in human participants using functional MRI and high-density EEG. We found that cortical area V3 has a parametric representation of the rotation symmetries in the textures that is not present in either V1 or V2, the first discovery of a stimulus property that differentiates processing in V3 from that of lower-level areas. Parametric responses were also seen in higher-order ventral stream areas V4, VO1, and lateral occipital complex (LOC), but not in dorsal stream areas. The parametric response pattern was replicated in the EEG data, and source localization indicated that responses in V3 and V4 lead responses in LOC, which is consistent with a feedforward mechanism. Finally, we presented our stimuli to four well developed feedforward models and found that none of them were able to account for our results. Our results highlight structural regularity as an important stimulus dimension for distinguishing the early stages of visual processing, and suggest a previously unrecognized role for V3 in the visual form-processing hierarchy. Significance statement: Hierarchical processing is a fundamental organizing principle in visual neuroscience, with each successive processing stage being sensitive to increasingly complex stimulus properties. Here, we probe the encoding hierarchy in human visual cortex using a class of visual textures--wallpaper patterns--that are maximally regular. Through a combination of fMRI and EEG source imaging, we find specific responses to texture regularity that depend parametrically on the maximum order of rotation symmetry in the textures. These parametric responses are seen in several areas of the ventral visual processing stream, as well as in area V3, but not in V1 or V2. This is the first demonstration of a stimulus property that differentiates processing in V3 from that of lower-level visual areas.

Trilateral constrained sparse representation for Kinect depth hole filling

Due to measurement errors or interference noise, Kinect depth maps exhibit severe defects of holes and noise, which significantly affect their applicability to stereo visions. Filtering and inpainting techniques have been extensively applied to hole filling. However, they either fail to fill in large holes or introduce other artifacts near depth discontinuities, such as blurring, jagging, and ringing. The emerging reconstruction-based methods employ underlying regularized representation models to obtain relatively accurate combination coefficients, leading to improved depth recovery results. Sparse representation facilitates retaining the saliency features of natural images and is thus more favorite than other regression models in image restoration, e.g. ridge regression. However, its naive applicability to depth map recovery hardly affords satisfactory depth prediction. Motivated by locality learning and bilateral filtering, this paper advocates a trilateral constrained sparse representation for Kinect depth recovery, which considers the constraints of intensity similarity and spatial distance between reference patches and target one on sparsity penalty term, as well as position constraint of centroid pixel in the target patch on data-fidelity term. Learning from the accompanied color image, this method can produce optimal solution to hole-filling problem in terms of depth prediction accuracy. Various experimental results on real-world Kinect maps and public datasets show that the proposed method outperforms state-of-the-art methods in filling effects of both flat and discontinuous regions.