Natural Image Recognition Research Articles

Deep Learning-Based Pulmonary Nodule Screening: A Narrative Review

AbstractGiven its capacity to generate three-dimensional pictures, computed tomography is the most effective means of detecting lung nodules with more excellent resolution of detected nodules. Small lung nodules can easily be overlooked on chest X-rays, making interpretation difficult. Artificial intelligence algorithms have recently demonstrated remarkable progress in medical imaging, especially with deep learning techniques such as convolutional neural networks (CNNs). CNN produces excellent results in natural image recognition and classification using abundant available data and the computational abilities of modern computers. It further reduces false-positive pulmonary nodules in medical image processing. This review article provides a detailed and inclusive review of recent advances, challenges, performance comparisons, and possible future directions for the problem of pulmonary nodule screening using deep learning methods.

Pathological Images Classification based on the Pretrained Convolutional Neural Network

Convolutional neural network is more and more used to solve the problem of images classification, but the scale of the dataset in this filed is too small compared with that of natural image recognition, which leads to the slow convergence speed of the neural network, and the instability of the model, which is difficult to be used to predict the results. In this paper, the residual network is applied to the PCam dataset to realize the recognition of tumour cells, and pre-trained through the ImageNet dataset. In the same training epochs, the performance of the pre-train network is much higher than that of the randomly initialized network, which significantly improves the convergence speed of the neural network. In addition, the performance of the pre-train network can converge faster when the amount of data is sufficient, and in the case oof medical image recognition with less data, the performance of the pre-train network is much higher than that of the random initialization network.

Similarities and differences in the spatio-temporal neural dynamics underlying the recognition of natural images and line drawings

Similarities and differences in the spatio-temporal neural dynamics underlying the recognition of natural images and line drawings

Explainability of Convolutional Neural Networks for Dermatological Diagnosis

Background Convolutional neural networks (CNNs) are regarded as state-of-the-art artificial intelligence (AI) tools for dermatological diagnosis, and they have been shown to achieve expert-level performance when trained on a representative dataset. CNN explainability is a key factor to adopting such techniques in practice and can be achieved using attention maps of the network. However, evaluation of CNN explainability has been limited to visual assessment and remains qualitative, subjective, and time consuming. Objective This study aimed to provide a framework for an objective quantitative assessment of the explainability of CNNs for dermatological diagnosis benchmarks. Methods We sourced 566 images available under the Creative Commons license from two public datasets—DermNet NZ and SD-260, with reference diagnoses of acne, actinic keratosis, psoriasis, seborrheic dermatitis, viral warts, and vitiligo. Eight dermatologists with teledermatology expertise annotated each clinical image with a diagnosis, as well as diagnosis-supporting characteristics and their localization. A total of 16 supporting visual characteristics were selected, including basic terms such as macule, nodule, papule, patch, plaque, pustule, and scale, and additional terms such as closed comedo, cyst, dermatoglyphic disruption, leukotrichia, open comedo, scar, sun damage, telangiectasia, and thrombosed capillary. The resulting dataset consisted of 525 images with three rater annotations for each. Explainability of two fine-tuned CNN models, ResNet-50 and EfficientNet-B4, was analyzed with respect to the reference explanations provided by the dermatologists. Both models were pretrained on the ImageNet natural image recognition dataset and fine-tuned using 3214 images of the six target skin conditions obtained from an internal clinical dataset. CNN explanations were obtained as activation maps of the models through gradient-weighted class-activation maps. We computed the fuzzy sensitivity and specificity of each characteristic attention map with regard to both the fuzzy gold standard characteristic attention fusion masks and the fuzzy union of all characteristics. Results On average, explainability of EfficientNet-B4 was higher than that of ResNet-50 in terms of sensitivity for 13 of 16 supporting characteristics, with mean values of 0.24 (SD 0.07) and 0.16 (SD 0.05), respectively. However, explainability was lower in terms of specificity, with mean values of 0.82 (SD 0.03) and 0.90 (SD 0.00) for EfficientNet-B4 and ResNet-50, respectively. All measures were within the range of corresponding interrater metrics. Conclusions We objectively benchmarked the explainability power of dermatological diagnosis models through the use of expert-defined supporting characteristics for diagnosis. Acknowledgments This work was supported in part by the Danish Innovation Fund under Grant 0153-00154A. Conflict of Interest None declared.

Clinically relevant autistic traits predict greater reliance on detail for image recognition

Individuals with an autism spectrum disorder (ASD) diagnosis are often described as having an eye for detail. But it remains to be shown that a detail-focused processing bias is a ubiquitous property of vision in individuals with ASD. To address this question, we investigated whether a greater number of autistic traits in neurotypical subjects is associated with an increased reliance on image details during a natural image recognition task. To this end, we use a novel reverse correlation-based method (feature diagnosticity mapping) for measuring the relative importance of low-level image features for object recognition. The main finding of this study is that image recognition in participants with an above-median number of autistic traits benefited more from the presence of high-spatial frequency image features. Furthermore, we found that this reliance-on-detail effect was best predicted by the presence of the most clinically relevant autistic traits. Therefore, our findings suggest that a greater number of autistic traits in neurotypical individuals is associated with a more detail-oriented visual information processing strategy and that this effect might generalize to a clinical ASD population.

RSI-CB: A Large-Scale Remote Sensing Image Classification Benchmark Using Crowdsourced Data.

Image classification is a fundamental task in remote sensing image processing. In recent years, deep convolutional neural networks (DCNNs) have experienced significant breakthroughs in natural image recognition. The remote sensing field, however, is still lacking a large-scale benchmark similar to ImageNet. In this paper, we propose a remote sensing image classification benchmark (RSI-CB) based on massive, scalable, and diverse crowdsourced data. Using crowdsourced data, such as Open Street Map (OSM) data, ground objects in remote sensing images can be annotated effectively using points of interest, vector data from OSM, or other crowdsourced data. These annotated images can, then, be used in remote sensing image classification tasks. Based on this method, we construct a worldwide large-scale benchmark for remote sensing image classification. This benchmark has large-scale geographical distribution and large total image number. It contains six categories with 35 sub-classes of more than 24,000 images of size pixels. This classification system of ground objects is defined according to the national standard of land-use classification in China and is inspired by the hierarchy mechanism of ImageNet. Finally, we conduct numerous experiments to compare RSI-CB with the SAT-4, SAT-6, and UC-Merced data sets. The experiments show that RSI-CB is more suitable as a benchmark for remote sensing image classification tasks than other benchmarks in the big data era and has many potential applications.

Deep Convolutional Neural Network for Mapping Smallholder Agriculture Using High Spatial Resolution Satellite Image.

In classification of satellite images acquired over smallholder agricultural landscape with complex spectral profiles of various crop types, exploring image spatial information is important. The deep convolutional neural network (CNN), originally designed for natural image recognition in the computer vision field, can automatically explore high level spatial information and thus is promising for such tasks. This study tried to evaluate different CNN structures for classification of four smallholder agricultural landscapes in Heilongjiang, China using pan-sharpened 2 m GaoFen-1 (meaning high resolution in Chinese) satellite images. CNN with three pooling strategies: without pooling, with max pooling and with average pooling, were evaluated and compared with random forest. Two different numbers (~70,000 and ~290,000) of CNN learnable parameters were examined for each pooling strategy. The training and testing samples were systematically sampled from reference land cover maps to ensure sample distribution proportional to the reference land cover occurrence and included 60,000–400,000 pixels to ensure effective training. Testing sample classification results in the four study areas showed that the best pooling strategy was the average pooling CNN and that the CNN significantly outperformed random forest (2.4–3.3% higher overall accuracy and 0.05–0.24 higher kappa coefficient). Visual examination of CNN classification maps showed that CNN can discriminate better the spectrally similar crop types by effectively exploring spatial information. CNN was still significantly outperformed random forest using training samples that were evenly distributed among classes. Furthermore, future research to improve CNN performance was discussed.

A Survey on Deep Learning-Driven Remote Sensing Image Scene Understanding: Scene Classification, Scene Retrieval and Scene-Guided Object Detection

As a fundamental and important task in remote sensing, remote sensing image scene understanding (RSISU) has attracted tremendous research interest in recent years. RSISU includes the following sub-tasks: remote sensing image scene classification, remote sensing image scene retrieval, and scene-driven remote sensing image object detection. Although these sub-tasks have different goals, they share some communal hints. Hence, this paper tries to discuss them as a whole. Similar to other domains (e.g., speech recognition and natural image recognition), deep learning has also become the state-of-the-art technique in RSISU. To facilitate the sustainable progress of RSISU, this paper presents a comprehensive review of deep-learning-based RSISU methods, and points out some future research directions and potential applications of RSISU.

Analyzing the influence of contrast in large-scale recognition of natural images

This paper analyzes both the isolated influence of illumination quality in 2D facial recognition and also the influence of contrast measures in large-scale recognition of low-resolution natural images. First, using the Yale Face Database B, we have shown that by separately estimating the illumination quality of facial images (through a fuzzy inference system that combines av- erage brightness and global contrast of the patterns) and by recognizing the same images using a multilayer perceptron, there ex- ists a nearly-linear correlation between both illumination and recognition results. Second, we introduced a new contrast measure, called Harris Points Measured Contrast (HPMC), which assigns values of contrast in a more consistent form to images, according to their recognition rate than other global and local compared contrast analysis methods. For our experiments on image contrast analysis, we have used the CIFAR-10 dataset with 60,000 images and convolutional neural networks as classification models. Our results can be considered to decide if it is worth using a given test image, according to its calculated contrast applying the proposed HPCM metric, for further recognition tasks.

Fine-tuned convolutional neural nets for cardiac MRI acquisition plane recognition

In this paper, we propose a convolutional neural network-based method to automatically retrieve missing or noisy cardiac acquisition plane information from magnetic resonance imaging and predict the five most common cardiac views. We fine-tune a convolutional neural network (CNN) initially trained on a large natural image recognition data-set (Imagenet ILSVRC2012) and transfer the learnt feature representations to cardiac view recognition. We contrast this approach with a previously introduced method using classification forests and an augmented set of image miniatures, with prediction using off the shelf CNN features, and with CNNs learnt from scratch. We validate this algorithm on two different cardiac studies with 200 patients and 15 healthy volunteers, respectively. We show that there is value in fine-tuning a model trained for natural images to transfer it to medical images. Our approach achieves an average F1 score of 97.66% and significantly improves the state-of-the-art of image-based cardiac view recognition. This is an important building block to organise and filter large collections of cardiac data prior to further analysis. It allows us to merge studies from multiple centres, to perform smarter image filtering, to select the most appropriate image processing algorithm, and to enhance visualisation of cardiac data-sets in content-based image retrieval.

Natural scene recognition with increasing time-on-task: the role of typicality and global image properties.

Human observers can recognize natural images very effectively. Yet, in the literature there is a debate about the extent to which the recognition of natural images requires controlled attentional processing. In the present study we address this topic by testing whether natural scene recognition is affected by mental fatigue. Mental fatigue is known to particularly compromise high-level, controlled attentional processing of local features. Effortless, automatic processing of more global features of an image stays relatively intact, however. We conducted a natural image categorization experiment (N = 20) in which mental fatigue was induced by time-on-task (ToT). Stimuli were images from 5 natural scene categories. Semantic typicality (high or low) and the magnitude of 7 global image properties were determined for each image in separate rating experiments. Significant performance effects of typicality and global properties on scene recognition were found, but, despite a general decline in performance, these effects remained unchanged with increasing ToT. The findings support the importance of the global property processing in natural scene recognition and suggest that this process is insensitive to mental fatigue.

Two Phases of V1 Activity for Visual Recognition of Natural Images

Present theories of visual recognition emphasize the role of interactive processing across populations of neurons within a given network, but the nature of these interactions remains unresolved. In particular, data describing the sufficiency of feedforward algorithms for conscious vision and studies revealing the functional relevance of feedback connections to the striate cortex seem to offer contradictory accounts of visual information processing. TMS is a good method to experimentally address this issue, given its excellent temporal resolution and its capacity to establish causal relations between brain function and behavior. We studied 20 healthy volunteers in a visual recognition task. Subjects were briefly presented with images of animals (birds or mammals) in natural scenes and were asked to indicate the animal category. MRI-guided stereotaxic single TMS pulses were used to transiently disrupt striate cortex function at different times after image onset (SOA). Visual recognition was significantly impaired when TMS was applied over the occipital pole at SOAs of 100 and 220 msec. The first interval has consistently been described in previous TMS studies and is explained as the interruption of the feedforward volley of activity. Given the late latency and discrete nature of the second peak, we hypothesize that it represents the disruption of a feedback projection to V1, probably from other areas in the visual network. These results provide causal evidence for the necessity of recurrent interactive processing, through feedforward and feedback connections, in visual recognition of natural complex images.

Why is Real-World Visual Object Recognition Hard?

Progress in understanding the brain mechanisms underlying vision requires the construction of computational models that not only emulate the brain's anatomy and physiology, but ultimately match its performance on visual tasks. In recent years, “natural” images have become popular in the study of vision and have been used to show apparently impressive progress in building such models. Here, we challenge the use of uncontrolled “natural” images in guiding that progress. In particular, we show that a simple V1-like model—a neuroscientist's “null” model, which should perform poorly at real-world visual object recognition tasks—outperforms state-of-the-art object recognition systems (biologically inspired and otherwise) on a standard, ostensibly natural image recognition test. As a counterpoint, we designed a “simpler” recognition test to better span the real-world variation in object pose, position, and scale, and we show that this test correctly exposes the inadequacy of the V1-like model. Taken together, these results demonstrate that tests based on uncontrolled natural images can be seriously misleading, potentially guiding progress in the wrong direction. Instead, we reexamine what it means for images to be natural and argue for a renewed focus on the core problem of object recognition—real-world image variation.

Discussion on the Meeting on the Gibbs Sampler and Other Markov Chain Monte Carlo Methods

Discussion on the Meeting on the Gibbs Sampler and Other Markov Chain Monte Carlo Methods

Perception and recognition of photographic quality facial caricatures: Implications for the recognition of natural images

Abstract The perception and recognition of photographic images of famous faces was compared with the same images transformed to produce caricatures of different degrees of exaggeration. Following Brennan (1982; 1985). caricatures were produced by first comparing the position of facial features in a frame-grabbed image with the average position for a series of faces; deviations from the average were then accentuated by a constant fraction (16, 32 or 48%). Photographic quality caricatures for seven famous faces were generated by distorting regions of the original images in accordance with the change in feature positions. Images reducing the distinctiveness of faces (anticaricatures) were produced by decreasing deviations from the norm. In Experiment 1, perceptual ratings of the degree to which images resembled the individuals depicted was found to vary with the degree of caricaturing (−32, −16, 0, +16, +32%). Interpolation from the data indicated that the best likeness occurred for images with a small degree of positive exaggeration (+4.4% on average). The magnitude of this caricature advantage correlated with the familiarity with the target faces and with the quality of the caricaturing proccss as assessed independently by caricature artists. Experiment 2 examined the recognition of normal and caricatured images in a namdface matching task. Overall, the subjects' fastest reaction times occurred for images with positive caricaturing. The caricature advantage was primarily attributed to improved performance on trials where the name and face did not match. The results suggest that both the precise metric proportions of faces and the way faces deviate from average are represented in memory. The results also indicate that the “super-fidelity” of caricatures found (Modes, Brennan & Carey, 1987) is not restricted to line drawings and may, therefore, have implications for how we recognise natural facial images. SUMMARY OF RESULTS Experiment 1 Perceptual ratings of the degree to which images resembled depicted individuals was found to vary with level of caricaturing. Interpolation indicated the best likeness would occur with a small degree of positive caricaturing (+4.4% on average). The magnitude of the caricature advantage at the perceptual level correlated with the familiarity of the faces and with the quality of the caricaturing process as judged by caricature experts. Experiment 2 Overall analysis of the degree of image manipulation producing the fastest reaction times for individual subjects revealed a caricature advantage. This increased speed of processing for caricatured images did not reflect any speed-accuracy trade-off. Caricaturing images can therefore produce more efficient processing in a task requiring matching of a person's face and name. In the overall analysis of variance of reaction times (containing match and non-match trials), the caricature advantage did not achieve statistical significance. Three factors might have contributed to the lack of effect. First, the caricature advantage was relatively small in magnitude amounting to a 3% increase in speed. Secondly, the amount of caricaturing producing optimal speed of processing varied across subjects, some performing best with +16% caricatures, others with +32% caricatures. Finally, and of more theoretical interest, the effects of caricaturing appeared to depend on the type of trial. There was no caricature advantage on congruous trials when the name matched the subsequently presented face image. The caricature advantage was prevalent, however, on incongruous trials where the face and name did not match. With non-match trials, + l6% caricatures were processed significantly faster than the veridical images. Again the increase in speed of processing was not an artefact produced by a speed-accuracy trade-off.