Visual Word Research Articles

Novel Object Captioning with Semantic Match from External Knowledge

Automatically describing the content of an image is a challenging task that is on the edge between natural language and computer vision. The current image caption models can describe the objects that are frequently seen in the training set very well, but they fail to describe the novel objects that are rarely seen or never seen in the training set. Despite describing novel objects being important for practical applications, only a few works investigate this issue. Furthermore, those works only investigate rarely seen objects, but ignore the never-seen objects. Meanwhile, the number of never-seen objects is more than the number of frequently seen and rarely seen objects. In this paper, we propose two blocks that incorporate external knowledge into the captioning model to solve this issue. Initially, in the encoding phase, the Semi-Fixed Word Embedding block is an improvement for the word embedding layer that enables the captioning model to understand the meaning of the arbitrary visual words rather than a fixed number of words. Furthermore, the Candidate Sentences Selection block chooses candidate sentences by semantic matching rather than probability, avoiding the influence of never-seen words. In experiments, we qualitatively analyze the proposed blocks and quantitatively evaluate several captioning models with the proposed blocks on the Nocaps dataset. The experimental results show the effectiveness of the proposed blocks for novel objects, especially when describing never-seen objects, CIDEr and SPICE improved by 13.1% and 12.0%, respectively.

A Novel Visual Indoor Positioning Method With Efficient Image Deblurring

Aiming to improve the accuracy of visual indoor positioning, an efficient deep multi-patch network based image deblurring algorithm (DMPID) is proposed to eliminate the effect of blurred images on the positioning accuracy. Meanwhile, the self-sorting visual word based image retrieval algorithm and the block based improved eight-point method are also proposed in this paper to improve the retrieval accuracy and positioning accuracy. The proposed image deblurring algorithm adopts the deep multi-patch network and the weight selective sharing scheme to acquire the deblurred images. Then, we propose the self-sorting visual word and add two additional elements representing the relative spatial information into every obtained feature point to utilize the intrinsic relationships between extracted features and physical positions in order to improve the retrieval accuracy. Meanwhile, the visual word filtering is also employed to eliminate redundant visual words and reduce the time consumption of image retrieval. Finally, the position estimation of query camera can be achieved by the epipolar constraint based on the improved eight-point method. Simulation results and performance analysis show that the proposed method can restore the image details effectively and improve the accuracy of visual indoor positioning.

Teaching categories via examples and explanations

People often learn categories through interaction with knowledgeable others who may use verbal explanations, visual exemplars, or both, to share their knowledge. Verbal and nonverbal means of pedagogical communication are commonly used in conjunction, but their respective roles are not fully understood. In this work, we studied how well these modes of communication work with different category structures. We conducted two experiments to investigate the effect of perceptual confusability and stimulus dimensionality on the effectiveness of verbal, exemplar-based, and mixed communication. One group of participants – teachers – learned a categorization rule and prepared learning materials for the students. Students studied the materials prepared for them and then demonstrated their knowledge on test stimuli. All communication modes were generally successful, but not equivalent, with mixed communication consistently showing best results. When teachers were free to generate as many visual exemplars or words as they wish, verbal and exemplar-based communication showed similar performance, although the verbal channel was slightly less reliable in situations requiring high perceptual precision. At the same time, verbal communication was better suited to handling high-dimensional stimuli when communication volume was restricted. We believe that our work serves as an important step towards studying language as a means for pedagogical category leaning.

A quantitative heterogeneity analysis approach to molecular subtype recognition of breast cancer in dynamic contrast-enhanced magnetic imaging images from radiomics data.

Breast cancer is a major cause of mortality among women worldwide. Dynamic contrast-enhanced breast magnetic resonance imaging (DCE-MRI) is a good imaging technique that can show temporal information about the kinetics of the contrast agent in suspicious breast lesions as well as acceptable spatial resolution. Computer-aided detection systems assist in the detection of lesions through medical image processing techniques combined with computerized analysis and calculation, which in turn helps radiologists recognize molecular subtypes of breast lesions that will be beneficial for better treatment plan decisions. In this paper, a computer-aided diagnosis method is proposed to automatically locate breast cancer lesions and identify molecular subtypes of breast cancer with heterogeneity analysis from radiomics data. A fast region-based convolutional network (Faster R-CNN) framework is first applied to images to detect breast cancer lesions. Then, the heterogeneous regions of every breast cancer lesion are extracted. Based on the multiple visual and kinetic radiomics features extracted from the heterogeneous regions, a temporal bag of visual word model is proposed, which takes into account the dynamic characteristics of both lesion and heterogeneous regions in images over time. The recognition task of molecular subtypes of breast lesions is realized based on a stacking classification model. At the genetic level, breast cancer is divided into four molecular subtypes, namely, luminal epithelial type A (Luminal A), luminal epithelial type B (Luminal B), HER-2 overexpression and basal cell type. The experimental results show that the precision of the four subtypes is 93%, 94%, 83%, 86%; the recall is 96%, 80%, 91%, 94%; and the F1-score is 95%, 86%, 87%. The experimental results denote the influence of heterogeneous regions on the recognition task. The DCE-MRI-based approach to identify molecular typing of breast cancer for noninvasive diagnosis will contribute to the development of breast cancer treatment, improved outcomes and reduced mortality.

A Food Recognition System for Diabetic Patients Based on an Optimized Bag-of-Features Model

Healthcare delivery might be completely changed by computer vision technologies. It might be used, in instance, to assess the carbohydrate content of meals for 1 diabetes patients. Based on the bag-of-features (BoF) paradigm, this paper suggests a way for automated food recognition. A thorough technical analysis was carried out to identify and optimize the components that make up the BoF architecture in order to guarantee the best outcomes. Also estimated were the associated parameters. A visual dataset with about 5000 food photos that was divided into 11 groups was developed in order to construct and test the prototype system. Using the scale-invariant 3 feature transform on the HSV colour space, the optimised method calculates dense local features. After that, it utilizes a visual dictionary with 10,000 visual words to create a linear support vector machine classifier is then used to categorize the food photos. The system demonstrated the viability of the suggested strategy in a very difficult picture dataset by achieving a classification accuracy of about 88%. By giving diabetes patients a more precise and effective means to control their carbohydrate consumption, this cutting-edge technology might have a profound effect on their quality of life.

Semantic Retrieval of Remote Sensing Images Based on the Bag-of-Words Association Mapping Method

With the increasing demand for remote sensing image applications, extracting the required images from a huge set of remote sensing images has become a hot topic. The previous retrieval methods cannot guarantee the efficiency, accuracy, and interpretability in the retrieval process. Therefore, we propose a bag-of-words association mapping method that can explain the semantic derivation process of remote sensing images. The method constructs associations between low-level features and high-level semantics through visual feature word packets. An improved FP-Growth method is proposed to achieve the construction of strong association rules to semantics. A feedback mechanism is established to improve the accuracy of subsequent retrievals by reducing the semantic probability of incorrect retrieval results. The public datasets AID and NWPU-RESISC45 were used to validate these experiments. The experimental results show that the average accuracies of the two datasets reach 87.5% and 90.8%, which are 22.5% and 20.3% higher than VGG16, and 17.6% and 15.6% higher than ResNet18, respectively. The experimental results were able to validate the effectiveness of our proposed method.

Does linear position matter for morphological processing? Evidence from a Tagalog masked priming experiment

ABSTRACT This study investigated morphological decomposition of Tagalog infixed, prefixed, and suffixed words using the masked priming paradigm. We directly compared morphological priming of <in> infixed, ni- prefixed and -in suffixed words to examine whether infixes are processed similarly to other affixes during early and automatic decomposition. We found significant priming effects for infixed, prefixed, and suffixed words, but no semantic or orthographic similarity priming. Magnitudes of priming effects for infixed and prefixed words were not significantly different, suggesting that decomposition of infixed words was not more costly for Tagalog speakers, contrary to phonological readjustment-based accounts of infixation. This is the first psycholinguistic experiment showing that infixed words are decomposed into morphological units during visual word recognition. We provide evidence that the imperfect edge-alignment of the stem within infixed words does not hamper the early morphological decomposition mechanisms, suggesting that edge-alignment might not be critical to trigger activation of morphological units.

Self-regulation of visual word form area activation with real-time fMRI neurofeedback

The Visual Word Form Area (VWFA) is a key region of the brain’s reading network and its activation has been shown to be strongly associated with reading skills. Here, for the first time, we investigated whether voluntary regulation of VWFA activation is feasible using real-time fMRI neurofeedback. 40 adults with typical reading skills were instructed to either upregulate (UP group, N = 20) or downregulate (DOWN group, N = 20) their own VWFA activation during six neurofeedback training runs. The VWFA target region was individually defined based on a functional localizer task. Before and after training, also regulation runs without feedback (“no-feedback runs”) were performed. When comparing the two groups, we found stronger activation across the reading network for the UP than the DOWN group. Further, activation in the VWFA was significantly stronger in the UP group than the DOWN group. Crucially, we observed a significant interaction of group and time (pre, post) for the no-feedback runs: The two groups did not differ significantly in their VWFA activation before neurofeedback training, but the UP group showed significantly stronger activation than the DOWN group after neurofeedback training. Our results indicate that upregulation of VWFA activation is feasible and that, once learned, successful upregulation can even be performed in the absence of feedback. These results are a crucial first step toward the development of a potential therapeutic support to improve reading skills in individuals with reading impairments.

Convolutional Neural Networks Trained to Identify Words Provide a Surprisingly Good Account of Visual Form Priming Effects

A wide variety of orthographic coding schemes and models of visual word identification have been developed to account for masked priming data that provide a measure of orthographic similarity between letter strings. These models tend to include hand-coded orthographic representations with single unit coding for specific forms of knowledge (e.g., units coding for a letter in a given position). Here we assess how well a range of these coding schemes and models account for the pattern of form priming effects taken from the Form Priming Project and compare these findings to results observed with 11 standard deep neural network models (DNNs) developed in computer science. We find that deep convolutional networks (CNNs) perform as well or better than the coding schemes and word recognition models, whereas transformer networks did less well. The success of CNNs is remarkable as their architectures were not developed to support word recognition (they were designed to perform well on object recognition), they classify pixel images of words (rather than artificial encodings of letter strings), and their training was highly simplified (not respecting many key aspects of human experience). In addition to these form priming effects, we find that the DNNs can account for visual similarity effects on priming that are beyond all current psychological models of priming. The findings add to the recent work of Hannagan et al. (2021) and suggest that CNNs should be given more attention in psychology as models of human visual word recognition.

Modality effects in free recall: A retrieved-context account.

The modality effect refers to the robust finding that memory performance differs for items presented aurally, as compared with visually. Whereas auditory presentation leads to stronger recency performance in immediate recall, visual presentation often produces better primacy performance (the inverse modality effect). To investigate and model these differences, we conducted two large-scale web-based immediate free recall experiments. In both experiments, participants studied visual and auditory word lists of varying lengths and rates of presentation. We observed typical modality and inverse modality effects, while also discovering that participants were more likely to initiate recall from recent items on auditory trials than on visual trials. However, modality effects persisted regardless of the first item recalled. Meanwhile, an analysis of intrusion errors revealed that participants were more likely on visual trials than on auditory trials to erroneously recall words from one list prior. Furthermore, words presented in the same modality as the present list intruded more often than those presented in a different modality. We next developed a retrieved-context account of the modality effect by fitting the Context Maintenance and Retrieval model to data across multiple list lengths. Through our simulations, we demonstrate that the modality effect can be explained by faster contextual drift and stronger context-to-item association formation during auditory presentation, relative to visual. Our modeling shows that modality effects can arise without hypothesizing distinct memory stores for recent and remote information. Finally, we propose that modality effects may derive primarily from the temporal dynamics of stimuli, rather than their modality. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Emotionality effects in Korean visual word recognition: Evidence from lab-based and web-based lexical decision tasks

Previous studies have shown that processes of word recognition are influenced by the emotional content of a word. This pattern is most readily explained by the motivated attention and affective states model (Lang, Bradley & Cuthbert, 1997), which states that emotional stimuli are motivationally significant and capture attention. Drawing on this theoretical account, the current study compared lexical decision response times to positive and negative emotion words versus neutral words across two experimental environments - a traditional lab-based environment and a web-based environment. In addition, the experiment was conducted using Korean words presented to native Korean speakers in order to test whether the emotionality effect emerges in a non-English language. The results revealed faster response times to emotion words versus neutral words across both experimental environments with no evidence of a difference between the two environments. These findings provide important evidence that emotion words successfully attract attention and facilitate word processing even in situations where participants might be more easily distracted than they would be in a traditional lab setting. This work also constitutes the first demonstration of an emotionality effect in Korean word recognition, thus providing further evidence that the emotionality effect may be a language-universal phenomenon.

Investigating app icon recognition with event-related potentials.

In modern society, visual symbols such as logos, icons, and letters have become essential for communication and cognition, playing a crucial role in daily life. This study focuses on app icons, a frequently encountered type of symbol, and aims to investigate the neural mechanisms involved in their recognition. Specifically, our objective is to identify the timing and location of brain activity associated with this process. We presented participants with familiar and unfamiliar app icons and asked them to perform a repetition detection task while recording the event-related potentials (ERPs) elicited by these stimuli. Statistical analysis revealed a significant difference in the ERPs between familiar and unfamiliar icons, occurring around 220 ms in the parietooccipital scalp region. The source analysis indicated that this ERP difference originated in the ventral occipitotemporal cortex, specifically the fusiform gyrus. These findings suggest that the recognition of familiar app icons results in the activation of the ventral occipitotemporal cortex approximately 220 ms after exposure. Additionally, our findings, in conjunction with previous research on visual word recognition, suggest that the lexical orthographic processing of visual words is dependent on general visual processing mechanisms that are also involved in the recognition of familiar app icons. In essence, the ventral occipitotemporal cortex likely plays a crucial role in memorizing and recognizing visual symbols and objects, including familiar visual words.

Semantic Bimodal Presentation Differentially Slows Working Memory Retrieval.

Although evidence has shown that working memory (WM) can be differentially affected by the multisensory congruency of different visual and auditory stimuli, it remains unclear whether different multisensory congruency about concrete and abstract words could impact further WM retrieval. By manipulating the attention focus toward different matching conditions of visual and auditory word characteristics in a 2-back paradigm, the present study revealed that for the characteristically incongruent condition under the auditory retrieval condition, the response to abstract words was faster than that to concrete words, indicating that auditory abstract words are not affected by visual representation, while auditory concrete words are. Alternatively, for concrete words under the visual retrieval condition, WM retrieval was faster in the characteristically incongruent condition than in the characteristically congruent condition, indicating that visual representation formed by auditory concrete words may interfere with WM retrieval of visual concrete words. The present findings demonstrated that concrete words in multisensory conditions may be too aggressively encoded with other visual representations, which would inadvertently slow WM retrieval. However, abstract words seem to suppress interference better, showing better WM performance than concrete words in the multisensory condition.

Zero-derived forms in the mental lexicon: Experimental evidence from Italian

Abstract Our paper investigates how zero-affixed verbal forms (specifically, denominal and deadjectival verbs) are processed by native speakers of Italian during visual word recognition. More specifically, we verify whether, and possibly to what extent, the processing of such forms differs from or resembles the processing of verbs resulting from two other productive word formation schemas in Italian, i. e., suffixation and parasynthesis. The interest of such a comparison lies in the fact that the three considered schemas behave similarly from a functional point of view, i. e.: i) they all change the category of the base, ii) they select the same base category (nouns and adjectives), and iii) they show similar semantic characteristics. However, this shared function is realized by means of different formal exponents (both overt and covert) which might imply different degrees of complexity. Crucially, suffixation and parasynthesis display an overt derivational marking, while zero-affixation obviously does not. On such premises, to verify whether the strength of connections between morphologically complex words and their bases is affected by the nature of the transcategorization marking (overt v s vs covert) or whether other factors play a more determining role, we conducted a psycholinguistic experiment involving a lexical decision task combined with the masked priming paradigm, which allows focusing on such relations in the mental lexicon. Results indicate that the processing of zero-affixed verbs resembles that of suffixed but differs from that of parasynthetic forms. We interpret such results discussing the possibility that the perception of a morphological relationship might depend less on the overtness of the marking, and more on the overall derivational complexity of the morphological process.

Semantic-to-autobiographical memory priming is ubiquitous.

It is now well established that the activation of semantic memories leads to the activation of autobiographical memories. Studies have shown that semantic processing of words or pictures primes autobiographical memories on voluntary and involuntary autobiographical memory tasks (the Crovitz cue-word task and the vigilance task). Known as semantic-to-autobiographical memory priming, our goal in the current study was to demonstrate the ubiquitous nature of this form of priming by showing that a wide variety of stimuli will prime involuntary autobiographical memories on the vigilance task. In Experiment 1, semantic-to-autobiographical priming was obtained on the vigilance task following the processing of sounds (e.g., the sound of bowling) and spoken words (e.g., the word bowling). In Experiment 2, semantic-to-autobiographical priming was observed on the vigilance task following tactile processing (e.g., the objects ball, glasses) and visual word processing (e.g., the words ball, glasses). In Experiment 3, semantic-to-autobiographical priming was observed on the vigilance task following the processing of videos (e.g., videos of a marching parade) and visual word processing (e.g., the word parade). The results of these experiments support the idea that semantic-to-autobiographical activations occur across a wide variety of stimuli (e.g., linguistic, perceptual). The results also further support the idea that semantic-to-autobiographical memory priming may play an important role in the production of involuntary memories in everyday life. Additional implications (for priming theory and autobiographical memory functions) are discussed.

Reduction or enhancement? Repetition effects on early brain potentials during visual word recognition are frequency dependent.

Most studies on word repetition have demonstrated that repeated stimuli yield reductions in brain activity. Despite the well-known repetition reduction effect, some literature reports repetition enhancements in electroencephalogram (EEG) activities. However, although studies of object and face recognition have consistently demonstrated both repetition reduction and enhancement effects, the results of repetition enhancement effects were not consistent in studies of visual word recognition. Therefore, the present study aimed to further investigate the repetition effect on the P200, an early event-related potential (ERP) component that indexes the coactivation of lexical candidates during visual word recognition. To achieve a high signal-to-noise ratio, EEG signals were decomposed into various modes by using the Hilbert-Huang transform. Results demonstrated a repetition enhancement effect on P200 activity in alpha-band oscillation and that lexicality and orthographic neighborhood size would influence the magnitude of the repetition enhancement effect on P200. These findings suggest that alpha activity during visual word recognition might reflect the coactivation of orthographically similar words in the early stages of lexical processing. Meantime, there were repetition reduction effects on ERP activities in theta-delta band oscillation, which might index that the lateral inhibition between lexical candidates would be omitted in repetition.

A Letter is a Letter and its Co-Occurrences: Cracking the Emergence of Position-Invariance Processing

Visual word recognition requires encoding letter identities and positions (orthographic processing). The present study focuses on the emergence of the mechanism responsible for encoding letter order in a word: position invariance. Reading experience leads to developing a flexible mechanism that encodes the information of the position of letters, explaining why jugde and judge are easily confused. Critically, orthographic regularities (e.g., frequent letter co-occurrences) modulate letter position encoding: the pseudoword mohter is extremely similar to mother because, in middle positions, the bigram TH is much more frequent than HT. Here, we tested whether position invariance emerges rapidly after the exposition to orthographic regularities—bigrams—in a novel script. To that end, we designed a study with two phases. In Phase 1, following Chetail (2017; Experiment 1b, Cognition, 163, 103–120), individuals were first exposed to a flow of artificial words for a few minutes, with four bigrams occurring frequently. Afterward, participants judged the strings with trained bigrams as more wordlike (i.e., readers quickly picked up subtle new orthographic regularities) than the strings with untrained bigrams, replicating Chetail (2017). In Phase 2, participants performed a same–different matching task in which they had to decide whether pairs of five-letter strings were the same or not. The critical comparison was between pairs with a transposition of letters in a frequent (trained) versus infrequent (untrained) bigram. Results showed that participants were more prone to make errors with frequent bigrams than with infrequent bigrams with a letter transposition. These findings reveal that position invariance emerges rapidly, after continuous exposure to orthographic regularities.

Detecting the visual word form area in a bilingual brain.

Utilizing a millimeter-scale fMRI technique and individual-based analysis, Zhan and colleagues drew a new cortical map of the visual word form area (VWFA) and examined how it processes diverse languages among different bilinguals. This research advances the current understanding of cortical language organization in the bilingual brain.

A Study of Factors Influencing the Volume of Responses to Posts in Physician Online Community.

Today's diverse health needs place greater demands on physicians. However, individual doctors have limited capabilities and may encounter many unsolvable medical problems. The physician online community provides a platform for physicians to communicate with each other and help each other. Physicians can post for help about problems they encounter at work. The number of responses to physicians' posts is critical to whether or not the problem is resolved. This study collected information on 13,226 posts from a well-known physician online community in China to analyze the factors that influence the number of post replies. In the analysis of the post content of the physician online community, this study innovatively introduces word usage features in the medical field. TextMind was used to extract the rate of several types of words in posts that frequently appear when describing medical information. Ultimately, we found that the rate of time words, visual words, auditory words, and physiological process words used in posts had a positive and significant effect on the number of post responses. A series of new post features has been found to have an impact on the number of post replies in physician online communities. This finding is beneficial for physicians to quickly obtain peer assistance through online platforms, increasing the likelihood of solving workplace challenges and improving physician care, as well as the success of physician online communities.

Feature Extraction Based on ORB- AKAZE for Echocardiogram View Classification

In computer vision, the extraction of robust features from images to construct models that automate image recognition and classification tasks is a prominent field of research. Handcrafted feature extraction and representation techniques become critical when dealing with limited hardware resource settings, low-quality images, and larger datasets. We propose two state-of-the-art handcrafted feature extraction techniques, Oriented FAST and Rotated BRIEF (ORB) and Accelerated KAZE (AKAZE), in combination with Bag of Visual Word (BOVW), to classify standard echocardiogram views using Machine learning (ML) algorithms. These novel approaches, ORB and AKAZE, which are rotation, scale, illumination, and noise invariant methods, outperform traditional methods. The despeckling algorithm Speckle Reduction Anisotropic Diffusion (SRAD), which is based on the Partial Differential Equation (PDE), was applied to echocardiogram images before feature extraction. Support Vector Machine (SVM), decision tree, and random forest algorithms correctly classified the feature vectors obtained from the ORB with accuracy rates of 96.5%, 76%, and 97.7%, respectively. Additionally, AKAZE's SVM, decision tree, and random forest algorithms outperformed state-of-the-art techniques with accuracy rates of 97.7%, 90%, and 99%, respectively.