Visual Similarity Research Articles

Developmental Trends of Visual Processing of Letters and Objects Using Naming Speed Tasks.

Studying the typical development of reading is key to understanding the precise deficits that underlie reading disabilities. An important correlate of efficient reading is the speed of naming arrays of simple stimuli such as letters and pictures. In this cross-sectional study, we examined developmental changes in visual processing that occurs during letter and object naming from childhood to early adulthood in terms of behavioral task efficiency, associated articulation and eye movement parameters, and the coordination between them, as measured by eye-voice span in both the spatial and temporal domains. We used naming speed (NS) tasks, in which participants were required to name sets of letters or simple objects as quickly and as accurately as possible. Single stimulus manipulations were made to these tasks to make the stimuli either more visually and/or phonologically similar to one another in order to examine how these manipulations affected task performance and the coordination between speech and eye movements. Across development there was an increased efficiency in speech and eye movement performance and their coordination in both the spatial and temporal domains. Furthermore, manipulations to the phonological and visual similarity of specific letter and object stimuli revealed that orthographic processing played a greater role than phonological processing in performance, with the contribution of phonological processing diminishing across development. This comprehensive typical developmental trajectory provides a benchmark for clinical populations to elucidate the nature of the cognitive dysfunction underlying reading difficulties.

Fine-Grained Recognition of Surface Targets with Limited Data

Recognition of surface targets has a vital influence on the development of military and civilian applications such as maritime rescue patrols, illegal-vessel screening, and maritime operation monitoring. However, owing to the interference of visual similarity and environmental variations and the lack of high-quality datasets, accurate recognition of surface targets has always been a challenging task. In this paper, we introduce a multi-attention residual model based on deep learning methods, in which channel and spatial attention modules are applied for feature fusion. In addition, we use transfer learning to improve the feature expression capabilities of the model under conditions of limited data. A function based on metric learning is adopted to increase the distance between different classes. Finally, a dataset with eight types of surface targets is established. Comparative experiments on our self-built dataset show that the proposed method focuses more on discriminative regions, avoiding problems like gradient disappearance, and achieves better classification results than B-CNN, RA-CNN, MAMC, and MA-CNN, DFL-CNN.

MD-NDNet: a multi-dimensional convolutional neural network for false-positive reduction in pulmonary nodule detection

Pulmonary nodule false-positive reduction is of great significance for automated nodule detection in clinical diagnosis of low-dose computed tomography (LDCT) lung cancer screening. Due to individual intra-nodule variations and visual similarities between true nodules and false positives as soft tissues in LDCT images, the current clinical practices remain subject to shortcomings of potential high-risk and time-consumption issues. In this paper, we propose a multi-dimensional nodule detection network (MD-NDNet) for automatic nodule false-positive reduction using deep convolutional neural network (DCNNs). The underlying method collaboratively integrates multi-dimensional nodule information to complementarily and comprehensively extract nodule inter-plane volumetric correlation features using three-dimensional CNNs (3D CNNs) and spatial nodule correlation features from sagittal, coronal, and axial planes using two-dimensional CNNs (2D CNNs) with attention module. To incorporate different sizes and shapes of nodule candidates, a multi-scale ensemble strategy is employed for probability aggregation with weights. The proposed method is evaluated on the LUNA16 challenge dataset in ISBI 2016 with ten-fold cross-validation. Experiment results show that the proposed framework achieves classification performance with a CPM score of 0.9008. All of these indicate that our method enables an efficient, accurate and reliable pulmonary nodule detection for clinical diagnosis.

Maṇḍala within the Rock: The Visualization of the Mahāmāyūrī-vidyārājñī and its Altar in Southwestern China, 9th–13th Centuries

Abstract This paper considers visualizations in Chinese medieval esoteric Buddhism in seven sculptural tableaux of the Mahāmāyūrī-vidyārājñī (Peacock Wisdom King 孔雀明王) from rock carving sites in the Sichuan Basin, Southwestern China. Early scholars highlighted the authority of Amoghavajra’s ritual manual for the Mahāmāyūrī images in this area, yet divergences between text and image hold them back from further interpretation. This paper reinvestigates these Mahāmāyūrī shrines “dialectically” by considering the text-image relationship. While keeping Amoghavajra’s ritual manual as a reference, it attempts to decode the meaning of the images and sites based on their own content, and to extrapolate from the text-image divergences how artistic productions and esoteric practices could lead to the presence of such divergences. This involves discussing artistic forms and decorative elements appropriated from exoteric Buddhism, as well as adjustments to the central icon and adjacent narrative scenes weaved within the temporal and spatial transitions. It also includes observations on the grouping between the Mahāmāyūrī and other deities in the larger iconographic program in their affiliated rock-cut sites, which reflects the interaction between this esoteric teaching and other popular beliefs. At least four out of seven examples share the same hierarchical iconographic programs or signature spatial structures similar to the Mahāmāyūrī altar prescribed in Amoghavajra’s ritual manual. It takes these visual or spatial similarities as concrete evidences that the construction of these shrines intended to make altars/maṇḍalas, although in two different ways to represent the esoteric altar and to create a space to conduct such a ritual.

Deep invariant texture features for water image classification

Detecting potential issues in naturally captured images of water is a challenging task due to visual similarities between clean and polluted water, as well as causes posed by image acquisition with different camera angles and placements. This paper presents novel deep invariant texture features along with a deep network for detecting clean and polluted water images. The proposed method first divides an input image into H, S and V components to extract finer details. For each of the color spaces, the proposed approach generates two directional coherence images based on Eigen value analysis and gradient distribution, which results in enhanced images. Then the proposed method extracts scale invariant gradient orientations based on Gaussian first order derivative filters on different standard deviations to study texture of each smoothed image. To strengthen the above features, we explore the combination of Gabor-wavelet-binary pattern for extracting texture of the input water image. The proposed method integrates merits of aforementioned features and the features extracted by VGG16 deep learning model to obtain a single feature vector. Furthermore, the extracted feature is fed to a gradient boosting decision tree for water image detection. A variety of experimental results on a large dataset containing different types of clean and stagnant water images show that the proposed method outperforms the existing methods in terms of classification rate and accuracy.

Annotation-Free Word Spotting with Bag-of-Features HMMs

The annotation-free word spotting method that is proposed in this paper makes document images searchable without requiring any labeled training data. Thus, our method supports the exploration of a document collection directly without demanding any manual efforts from the users for the preparation of a training dataset. Our method works in the query-by-example scenario where the user selects an exemplary occurrence of the query word. Afterwards, the entire collection of document images is searched according to visual similarity to the query. The proposed method requires only minimal assumptions about the visual appearance of text. This is achieved by processing document images as a whole without requiring a given segmentation of the images on word level or on line level. Therefore, the method is also segmentation-free. Word size variabilities can be handled by representing the sequential structure of text with a statistical sequence model. In order to make the computationally costly application of the sequence model feasible in practice, regions are retrieved according to approximate similarity with an efficient model decoding algorithm. Re-ranking these regions according to the visual similarity obtained with the sequence model leads to highly accurate word spotting results. The method is evaluated on five benchmark datasets. In the segmentation-free query-by-example scenario where no annotated training data is available, the method outperforms all other methods that have been evaluated on any of these five benchmarks.

Joint Person Objectness and Repulsion for Person Search.

Person search targets to search the probe person from the unconstrainted scene images, which can be treated as the combination of person detection and person matching. However, the existing methods based on the Detection-Matching framework ignore the person objectness and repulsion (OR) which are both beneficial to reduce the effect of distractor images. In this paper, we propose an OR similarity by jointly considering the objectness and repulsion information. Besides the traditional visual similarity term, the OR similarity also contains an objectness term and a repulsion term. The objectness term can reduce the similarity of distractor images that not contain a person and boost the performance of person search by improving the ranking of positive samples. Because the probe person has a different person ID with its neighbors, the gallery images having a higher similarity with the neighbors of probe should have a lower similarity with the probe person. Based on this repulsion constraint, the repulsion term is proposed to reduce the similarity of distractor images that are not most similar to the probe person. Treating the Faster R-CNN as the person detector, the OR similarity is evaluated on PRW and CUHK-SYSU datasets by the Detection-Matching framework with six description models. The extensive experiments demonstrate that the proposed OR similarity can effectively reduce the similarity of distractor samples and further boost the performance of person search, e.g., improve the mAP from 92.32% to 93.23% for CUHK-SYSY dataset, and from 50.91% to 52.30% for PRW datasets.

Serial and syntactic processing in the visual analysis of multi-digit numbers

The visual analysis of letter strings and digit strings is done by two separate cognitive processes. Recent studies have hypothesized that these processes are not only separate but also qualitatively different, in that they may encode information specific to numbers or to words. To examine this hypothesis and to shed further light on the visual analysis of numbers, we asked adults to read aloud multi-digit strings presented to them for brief durations. Their performance was better in digits on the number's left side than in digits farther to the right, with better performance in the two outer digits than their neighbors. This indicates the digits were processed serially, from left to right. Visual similarity of digits increased the likelihood of errors, and when a digit migrated to an incorrect position, it was most often to an adjacent location. Interestingly, the positions of 0 and 1 were encoded better than the positions of 2–9, and 2–9 were identified better when they were next to 0 or 1. To accommodate these findings, we propose a detailed model for the visual analysis of digit strings. The model assumes imperfect digit detectors in which a digit's visual information leaks to adjacent locations, and a compensation mechanism that inhibits this leakage. Crucially, the compensating inhibition is stronger for 0 and 1 than for the digits 2–9, presumably because of the importance of 0 and 1 in the number system. This sensitivity to 0 and 1 makes the visual analyzer specifically adapted to numbers, not words, and may be one of the brain's reasons to implement the visual analysis of numbers and words in two separate cognitive processes.

Separable Influences of Reward on Visual Processing and Choice.

Primate vision is characterized by constant, sequential processing and selection of visual targets to fixate. Although expected reward is known to influence both processing and selection of visual targets, similarities and differences between these effects remain unclear mainly because they have been measured in separate tasks. Using a novel paradigm, we simultaneously measured the effects of reward outcomes and expected reward on target selection and sensitivity to visual motion in monkeys. Monkeys freely chose between two visual targets and received a juice reward with varying probability for eye movements made to either of them. Targets were stationary apertures of drifting gratings, causing the end points of eye movements to these targets to be systematically biased in the direction of motion. We used this motion-induced bias as a measure of sensitivity to visual motion on each trial. We then performed different analyses to explore effects of objective and subjective reward values on choice and sensitivity to visual motion to find similarities and differences between reward effects on these two processes. Specifically, we used different reinforcement learning models to fit choice behavior and estimate subjective reward values based on the integration of reward outcomes over multiple trials. Moreover, to compare the effects of subjective reward value on choice and sensitivity to motion directly, we considered correlations between each of these variables and integrated reward outcomes on a wide range of timescales. We found that, in addition to choice, sensitivity to visual motion was also influenced by subjective reward value, although the motion was irrelevant for receiving reward. Unlike choice, however, sensitivity to visual motion was not affected by objective measures of reward value. Moreover, choice was determined by the difference in subjective reward values of the two options, whereas sensitivity to motion was influenced by the sum of values. Finally, models that best predicted visual processing and choice used sets of estimated reward values based on different types of reward integration and timescales. Together, our results demonstrate separable influences of reward on visual processing and choice, and point to the presence of multiple brain circuits for the integration of reward outcomes.

Multi-Target Multi-Camera Tracking With Optical-Based Pose Association

Multi-target multi-camera tracking (MTMCT) targets to generate trajectories of the object that appeared under multiple cameras automatically. MTMCT can be treated as a combination of intra-camera tracking and cross-camera tracking. The existing work only employs the global description to perform the tracklet generating. However, the global description cannot model the local similarity between targets, leading to existing methods not to be robust to occlusion and fast motion. To handle the mentioned problem, we propose an online Optical-based Pose Association (OPA) for multi-target multi-camera tracking. The proposed method utilizes local pose matching to solve the occlusion problem, and applies optical flow to reduce the distance caused by fast motion. For optical-based pose association, we firstly employ OpenPose to generate human pose for each proposal. Then, we utilize the optical flow generated by PWC-Net to adjust the estimated pose for the previous frame. Finally, the modified Object Keypoint Similarity is used to compute the similarity between the pose of the current frame and adjusted pose in the prior frame. Once obtaining the optical-based pose similarity, we combine it with the visual and bounding box spatial similarities to generate the final similarity matrix, and apply the Kuhn-Munkras algorithm for data association. The experiments on the MTMCT and MOT datasets verify the rationality of using human pose information and prove the superiority of the proposed method.

Дендрохронологическое исследование древесины в исторически значимых

Studies of wood in the Arctic were carried out on Vaygach Island, Hooker Island (Franz Joseph Land) and Novaya Zemlya. During the expedition “Arctic Floating University – 2013”, wood samples (cores) were taken from the buildings in key areas of the Arctic islands. Conforming to the wood samples we constructed tree-ring series and obtained floating chronologies, which were linked to dendroscales of standing trees with known year of each annual ring formation. Crossdating of floating chronologies was performed as follows. Dendroscale and floating chronology were overlapped and the synchronicity coefficient was calculated, then the dendroscale was shifted for a year and the coefficient was calculated again; the procedure was repeated until all possible options were checked. Visual similarity was assessed and the annual rings of floating chronology were dated based on the results with the maximum synchronicity coefficient, and the object was dated by the outermost ring. The year of formation of each annual ring of the buildings logs was determined by the dendrochronological method, resulting in assumption of the time of their construction, taking into account that it occurred a year or two after the tree felling. All the studied objects were located in areas with preserved history of their creation, which is highly important for confirming the objectivity of the obtained results. The dates of the buildings were determined as follows: Vaigach Island, 1945; Hooker Island (Franz Josef Land), 1936; the former bathhouse on Cape Zhelaniya (Novaya Zemlya), 1935; timber from the pillbox, 1938; the lighthouse, 1952; the frame on the coast of Ledyanaya Gavan’ Bay (Novaya Zemlya), 1938; a relatively modern building on the coast of the bay, 1991. Interesting to note that according to the obtained historical records the Lighthouse was built in year 1954, while dating by annual rings shows that the logs were in 1952 and they were probably imported in advance. The years for the construction research based on the use of the dendrochronological method opens up great opportunities at determining the age of unnamed wooden in the Arctic.

Multi-View Multi-Label Learning With Sparse Feature Selection for Image Annotation

In image analysis, image samples are always represented by multiple view features and associated with multiple class labels for better interpretation. However, multiple view data may include noisy, irrelevant and redundant features, while multiple class labels can be noisy and incomplete. Due to the special data characteristic, it is hard to perform feature selection on multi-view multi-label data. To address these challenges, in this paper, we propose a novel multi-view multi-label sparse feature selection (MSFS) method, which exploits both view relations and label correlations to select discriminative features for further learning. Specifically, the multi-labeled information is decomposed into a reduced latent label representation to capture higher level concepts and correlations among multiple labels. Multiple local geometric structures are constructed to exploit visual similarities and relations for different views. By taking full advantage of the latent label representation and multiple local geometric structures, the sparse regression model with an $l_{2,1}$ -norm and an Frobenius norm ( $F$ -norm) penalty terms is utilized to perform hierarchical feature selection, where the $F$ -norm penalty performs high-level (i.e., view-wise) feature selection to preserve the informative views and the $l_{2,1}$ -norm penalty conducts low-level (i.e., row-wise) feature selection to remove noisy features. To solve the proposed formulation, we also devise a simple yet efficient iterative algorithm. Experiments and comparisons on real-world image datasets demonstrate the effectiveness and potential of MSFS.

The Idiot as Artist:The Fantasy Boats of James Henry Pullen

The Idiot as Artist:The Fantasy Boats of James Henry Pullen

False memories for scenes using the DRM paradigm

People are remarkably good at remembering photographs. To further investigate the nature of the stored representations and the fidelity of human memories, it would be useful to evaluate the visual similarity of stimuli presented in experiments. Here, we explored the possible use of convolutional neural networks (CNN) as a measure of perceptual or representational similarity of visual scenes with respect to visual memory research. In Experiment 1, we presented participants with sets of nine images from the same scene category and tested whether they were able to detect the most distant scene in the image space defined by CNN. Experiment 2 was a visual variant of the Deese-Roediger-McDermott paradigm. We asked participants to remember a set of photographs from the same scene category. The photographs were preselected based on their distance to a particular visual prototype (defined as centroid of the image space). In the recognition test, we observed higher false alarm rates for scenes closer to this visual prototype. Our findings show that the similarity measured by CNN is reflected in human behavior: people can detect odd-one-out scenes or be lured to false alarms with similar stimuli. This method can be used for further studies regarding visual memory for complex scenes.

Visual and semantic similarity norms for a new object and scene photographic image set

Visual and semantic similarity norms for a new object and scene photographic image set

Object-selective cortex shows distinct representational formats along the posterior-to-anterior axis: evidence from brain-behavior correlations

Object information is transformed along the ventral visual stream from early formats into increasingly invariant representations. To gain insight into the nature of these representations, we measured different facets of similarity with three behavioral tasks and related them to neural responses. First, we measured similarity implicitly using a visual search task, measuring the time it took participants to find one target amongst distractors in a large-scale online experiment (72 images of inanimate objects, 2,556 pairs, n=1,272). Next, we measured similarity with an explicit shape-sorting task, asking participants to arrange items in a circular arena based on their shape similarity (n=25). Then, we measured similarity with the same task, but without any guidance on what object properties to focus on while placing more similar objects nearby (n=26). Finally, brain responses to each image were obtained using functional magnetic resonance imaging (n=10). Our results show that (i) there are clear differences in the measured similarity space of objects across tasks, (ii) the implicit visual search similarity was most strongly correlated with the posterior half of object-selective cortex, (iii) the explicit shape-sorting similarity was most strongly correlated to the anterior half of object-selective cortex, and (iv) the similarity measured with the unguided sorting task was not well-correlated with any responses along occipitotemporal cortex. These observations were confirmed quantitatively with linear mixed-effect modeling: the brain similarity structure along the hierarchy was better explained when including the interaction between behavioral task and location along the hierarchy, compared to a model without the interaction: 𝜒2(34)=178.73, p<0.001). Broadly, these results reveal a clear dissociation in the way posterior and anterior occipitotemporal cortex represents the similarity of objects. We hypothesize that this relatively sharp transition along the ventral stream hierarchy may reflect a passage from more pictorial to more structural representations of inanimate object information.

Dilated conditional GAN for bone suppression in chest radiographs with enforced semantic features.

The purpose of this essay is to improve computer-aided diagnosis of lung diseases by the removal of bone structures imagery such as ribs and clavicles, which may shadow a clinical view of lesions. This paper aims to develop an algorithm to suppress the imaging of bone structures within clinical x-ray images, leaving a residual portrayal of lung tissue; such that these images can be used to better serve applications, such as lung nodule detection or pathology based on the radiological reading of chest x rays. We propose a conditional Adversarial Generative Network (cGAN) (Mirza and Osindero, Conditional generative adversarial nets, 2014.) model, consisting of a generator and a discriminator, for the task of bone shadow suppression. The proposed model utilizes convolutional operations to expand the size of the receptive field of the generator without losing contextual information while downsampling the image. It is trained by enforcing both the pixel-wise intensity similarity and the semantic-level visual similarity between the generated x-ray images and the ground truth, via optimizing an L-1 loss of the pixel intensity values on the generator side and a feature matching loss on the discriminator side, respectively. The framework we propose is trained and tested on an open-access chest radiograph dataset for benchmark. Results show that our model is capable of generating bone-suppressed images of outstanding quality with a limited number of training samples (N=272). Our approach outperforms current state-of-the-art bone suppression methods using x-ray images. Instead of simply downsampling images at different scales, our proposed method mitigates the loss of contextual information by utilizing dilated convolutions, which gains a noticeable quality improvement for the outputs. On the other hand, our experiment shows that enforcing the semantic similarity between the generated and the ground truth images assists the adversarial training process and achieves better perceptual quality.

Perirhinal and Anterolateral Entorhinal Cortex Activity Patterns Reflect Perceived Visual Similarity of Highly Similar Objects

Perirhinal and Anterolateral Entorhinal Cortex Activity Patterns Reflect Perceived Visual Similarity of Highly Similar Objects

Letter Features as Predictors of Letter-Name Acquisition in Four Languages with Three Scripts

ABSTRACT To expand our understanding of script-general and script-specific principles in the learning of letter names, we examined how three characteristics of alphabet letters – their frequency in printed materials, order in the alphabet, and visual similarity to other letters – relate to children’s letter-name knowledge in four languages with three distinct scripts (English [N = 318; M age = 4.90], Portuguese [N = 366; M age = 5.80], Korean [N = 168; M age = 5.48], and Hebrew [N = 645; M age = 5.42]). Explanatory item response modeling analysis showed that the frequency of letters in printed materials was consistently related to letter difficulty across the four languages. There were also moderation effects for letter difficulty in English and Korean, and for discriminatory power of letters in Korean. The results suggest that exposure to letters as measured by letter frequency is a language-general mechanism in the learning of alphabet letters.

Visual similarity and psychological closeness are neurally dissociable in the brain response to vicarious pain

Personal and vicarious experience of pain activate partially overlapping brain networks. This brain activity is further modulated by low- and high-order factors, e.g., the perceived intensity of the model's pain and the model's similarity with the onlooker, respectively. We investigated which specific aspect of similarity modulates such empathic reactivity, focusing on the potential differentiation between visual similarity and psychological closeness between the onlooker and different types of models. To this aim, we recorded fMRI data in neurotypical participants who observed painful and tactile stimuli delivered to an adult human hand, a baby human hand, a puppy dog paw, and an anthropomorphic robotic hand. The interaction between type of vicarious experience (pain, touch) and nature of model (adult, baby, dog, robot) showed that the right supramarginal gyrus (rSMG) was selectively active for visual similarity (more active during vicarious pain for the adult and baby models), while the anterior cingulate cortex (ACC) was more sensitive to psychological closeness (specifically linked to vicarious pain for the baby model). These findings indicate that visual similarity and psychological closeness between onlooker and model differentially affect the activity of brain regions specifically implied in encoding interindividual sharing of sensorimotor and affective aspects of vicarious pain, respectively.