Semantic Associations Research Articles

A Survey of Full-Cycle Cross-Modal Retrieval: From a Representation Learning Perspective

Cross-modal retrieval aims to elucidate information fusion, imitate human learning, and advance the field. Although previous reviews have primarily focused on binary and real-value coding methods, there is a scarcity of techniques grounded in deep representation learning. In this paper, we concentrated on harmonizing cross-modal representation learning and the full-cycle modeling of high-level semantic associations between vision and language, diverging from traditional statistical methods. We systematically categorized and summarized the challenges and open issues in implementing current technologies and investigated the pipeline of cross-modal retrieval, including pre-processing, feature engineering, pre-training tasks, encoding, cross-modal interaction, decoding, model optimization, and a unified architecture. Furthermore, we propose benchmark datasets and evaluation metrics to assist researchers in keeping pace with cross-modal retrieval advancements. By incorporating recent innovative works, we offer a perspective on potential advancements in cross-modal retrieval.

Kumlbúa þáttr, an Eschatological Event: The Traumatic Ontology of Being in Time in Medieval Iceland

<i>Kumlbúa þáttr</i>, an Eschatological Event: The Traumatic Ontology of Being in Time in Medieval Iceland

A selection-pattern-aware recommendation model with colored-motif attention network

Heterogeneous information networks (HIN) based models are now widely used to fuse auxiliary information for personalized recommendation. However, existing works mainly focus on capturing high-order connections between heterogeneous nodes through predefined pattern (such as meta-path, etc.), which depends on relevant domain knowledge and ignores the statistical characteristics of the substructures in HIN. How to capture meaningful personalized behavior patterns, especially the selection patterns (i.e. how a user select an item), and incorporate them into the preference model is still a challenge in the research of HIN-based recommendation. Therefore, in this paper, a specifically-designed Colored-Motif Attention Network (CMoAN) is proposed to deal with this problem. In the proposed CMoAN, colored motif are used as the elemental building blocks to represent context-based selection patterns, and then by constructing a motif-based adjacency matrix to capture higher-order semantic association between nodes in HIN. Besides, an attentive graph neural network is designed to efficiently model the semantics and high-order relations information. Extensive experiments on three real-world datasets demonstrate that CMoAN consistently outperforms state-of-the-art methods. Furthermore, experimental results also verify the effectiveness of using colored motif for capturing users’selection patterns.

A Comparative Study of Korean and Chinese Counter-expectation Markers ‘Gikkotaeya’ and ‘Dabuliao’

This paper mainly takes the typical permissive expectation markers ‘gikkotaeya’ and ‘dabuliao’ in Korean and Chinese as the research objects, and makes a comparative analysis of the expectation characteristics and subjective quantities, and the following conclusions can be drawn: In terms of similarities, these two expressions have commonalities in semantic types, internal semantic associations, and lexicalization processes. In terms of differences, in terms of semantic types, the semantic type of ‘gikkotaeya’ representing subjective emotional attitude is more typical, and the semantic type of ‘dabuliao’ representing ‘small quantity’ and ‘situation maintain’ is more characteristic; in terms of internal semantic association, ‘gikkotaeya’ is more typical than ‘dabuliao’ in counter-expectaion and subjective expression; in the process of lexicalization, ‘gikkotaeya’ has a variety of extensions, and ‘dabuliao’ has a relatively single path, which is mainly affected by objective quantities. In terms of the motives for differences, it is mainly related to the strength of subjectivity, the interaction between subjective quantity and expectation category.

Multi‐graph learning‐based software defect location

AbstractSoftware quality is key to the success of software systems. Modern software systems are growing in their worth based on industry needs and becoming more complex, which inevitably increases the possibility of more defects in software systems. Software repairing is time‐consuming, especially locating the source files related to specific software defect reports. To locate defective source files more quickly and accurately, automated software defect location technology is generated and has a huge application value. The existing deep learning‐based software defect location method focuses on extracting the semantic correlation between the source file and the corresponding defect reports. However, the extensive code structure information contained in the source files is ignored. To this end, we propose a software defect location method, namely, multi‐graph learning‐based software defect location (MGSDL). By extracting the program dependency graphs for functions, each source file is converted into a graph bag containing multiple graphs (i.e., multi‐graph). Further, a multi‐graph learning method is proposed, which learns code structure information from multi‐graph to establish the semantic association between source files and software defect reports. Experiments' results on four publicly available datasets, AspectJ, Tomcat, Eclipse UI, and SWT, show that MGSDL improves on average 3.88%, 5.66%, 13.23%, 9.47%, and 3.26% over the competitive methods in five evaluation metrics, rank@10, rank@5, MRR, MAP, and AUC, respectively.

The Role of Semantic Associations as a Metacognitive Cue in Creative Idea Generation.

Is my idea creative? This question directs investing in companies and choosing a research agenda. Following previous research, we focus on the originality of ideas and consider their association with self-assessments of idea generators regarding their own originality. We operationalize the originality score as the frequency (%) of each idea within a sample of participants and originality judgment as the self-assessment of this frequency. Initial evidence suggests that originality scores and originality judgments are produced by separate processes. As a result, originality judgments are prone to biases. So far, heuristic cues that lead to such biases are hardly known. We used methods from computational linguistics to examine the semantic distance as a potential heuristic cue underlying originality judgments. We examined the extent to which the semantic distance would contribute additional explanatory value in predicting originality scores and originality judgments, above and beyond cues known from previous research. In Experiment 1, we re-analyzed previous data that compared originality scores and originality judgments after adding the semantic distance of the generated ideas from the stimuli. We found that the semantic distance contributed to the gap between originality scores and originality judgments. In Experiment 2, we manipulated the examples given in task instructions to prime participants with two levels of idea originality and two levels of semantic distance. We replicated Experiment 1 in finding the semantic distance as a biasing factor for originality judgments. In addition, we found differences among the conditions in the extent of the bias. This study highlights the semantic distance as an unacknowledged metacognitive cue and demonstrates its biasing power for originality judgments.

대구방언 ‘카다’의 문법

This study aims to investigate various usages and those semantic associations of ‘kada’ in Daegu Dialect. Though a number of previous researches already dealt with different functions of ‘kada’, such as quotation verb, utterance verb, and pro-verb, this study tries to analyze in detail individual grammatical characteristics and explore comprehensively their mutual relation. In addition, by discussing together ‘ikada, geukada, jeokada’, which had been developed in the opposite direction with ‘kada’, it is confirmed that these words have a close and systematic correspondence with each other in terms of grammar.

A time sequence coding based node-structure feature model oriented to node classification

Graph data mining is an important method for managing complex systems in artificial intelligence (AI). As an important branch of graph data mining, node classification is widely applied in paper classification in citation networks, user classification in social networks, etc. At present, many graph neural networks (GNNs) have been proposed to realize node classification by obtaining node embeddings encoded by aggregating neighborhood features. Most GNNs use one-hot encoding as the initial node feature when there is no node attribute in the graph. However, one-hot encoding is a global encoding technique that cannot reflect the environmental context of specific nodes. To this end, we proposed a graph structure sequential coding (GSSC) model, including a learnable node-structure feature X, sampler, encoder and classifier, to obtain the structural embeddings which can better reflect the topological structure of non-attribute graphs. For non-attribute graphs, we utilized the learnable feature X as the initial node-structure feature, which could be simultaneously trained with the GSSC model. In addition, we used a decoupling scheme to separate the sampling and encoding processes of GSSC. Therefore, we could use different samplers for sparse and dense graphs. In addition, different sampling orders (i.e., temporality) in node sequences could reflect different semantic associations between the nodes. Therefore, we innovatively employed time sequence models (TSMs) as encoders to encode node sequences with temporality. Based on these TSMs, semantic information of the head node in the node sequence can be captured. The experimental results on five datasets confirmed that our GSSC model performs better than other representative methods in node classification for graphs without node attributes.

Multi-Source Knowledge Reasoning Graph Network for Multi-Modal Commonsense Inference

As a crucial part of natural language processing, event-centered commonsense inference task has attracted increasing attention. With a given observed event, the intention and reaction of the people involved in the event are required to be inferred with artificial intelligent algorithms. To solve this problem, sequence-to-sequence methods are widely studied, where the event is first encoded into a specific representation and then decoded to generate the results. However, all the existing methods learn the event representation only with the textual information, while the visual information is ignored, which is actually helpful for the commonsense reference. In this article, we first define a new task of multi-modal commonsense reference with both textual and visual information. A new event-centered multi-modal dataset is also provided. Then we propose a multi-source knowledge reasoning graph network to solve this task, where three kinds of relational knowledge are considered. Multi-modal correlations are learned to get the event’s multi-modal representation from a global perspective. Intra-event object relations are explored to capture the fine-grained event feature with an object graph. Inter-event semantic relations are also explored through the external knowledge to understand the semantic associations among events with an event graph. We conduct extensive experiments on the new dataset, and the results show the effectiveness of our method.

"Shattering peace”

Based on an analysis of the main trends expressed by some of the most influential Colombian media, this research addresses the dynamics of public opinion regarding peace and peace agreements in Colombia. In light of the fact that the country is going through the implementation stage of the agreements signed with the Fuerzas Armadas Revolucionarias de Colombia (FARC), we analyze the question of whether there have been substantial changes in how the subject of peace is presented and established by the media during the government of the Iván Duque and how these changes potentially impact public discussion. The proposed approach is based on mixed methods applied to the analysis of media discourse. This approach allows us to combine an analysis of trends to form an analysis of content drawn from semantic association networks.

HAAN: Learning a Hierarchical Adaptive Alignment Network for Image-Text Retrieval

Image-text retrieval aims to search related results of one modality by querying another modality. As a fundamental and key problem in cross-modal retrieval, image-text retrieval is still a challenging problem owing to the complementary and imbalanced relationship between different modalities (i.e., Image and Text) and different granularities (i.e., Global-level and Local-level). However, existing works have not fully considered how to effectively mine and fuse the complementarities between images and texts at different granularities. Therefore, in this paper, we propose a hierarchical adaptive alignment network, whose contributions are as follows: (1) We propose a multi-level alignment network, which simultaneously mines global-level and local-level data, thereby enhancing the semantic association between images and texts. (2) We propose an adaptive weighted loss to flexibly optimize the image-text similarity with two stages in a unified framework. (3) We conduct extensive experiments on three public benchmark datasets (Corel 5K, Pascal Sentence, and Wiki) and compare them with eleven state-of-the-art methods. The experimental results thoroughly verify the effectiveness of our proposed method.

How Prophecy Works: A Study of the Semantic Field of נביא and a Close Reading of Jeremiah 1:4–19, 23:9–40 and 27:1–28:17. By William Kelly

This monograph, the author’s published PhD thesis from the University of Edinburgh, sets out to explore ‘how prophecy works’. As the subheading clarifies, though, its aims are more modest: it investigates how prophecy works in three selected passages in the MT of the book of Jeremiah (1:4–19, 23:9–40, and 27:1–28:17). Chapter 1 summarizes the scholarly debate pertaining to the meaning of the Hebrew word ‘prophet’ (נביא) and its associated verb forms. Influenced by especially Barr’s cautionary words, Kelly argues the answer to the questions ‘how prophecy works’ and the related ‘what a prophet does’ should not be sought in the etymology of the root נבא. Rather, it can be obtained by exploring its semantic fields and its syntagmatic and paradigmatic relations. Part I, consisting of chapter 2, reveals the fruit of this exploration. Kelly painstakingly analyses every single instance of the Hebrew root נבא in the book of Jeremiah. This investigation, which in my view could have been presented more succinctly, reveals several notable (but not necessarily novel) insights. First, prophets are predominantly associated with verbs that denote oral communication, through which they reveal the outcome of the divine revelation. Second, prophets and priests share similar functions. Third, the verbal forms of נבא nearly always take an indirect object: one does not prophesy ‘something’ but ‘about something’. For instance, one does not predict doom or salvation but speaks about people and their relationship with God. Fourth, there is no salient semantic difference between the Niphal and the Hithpael forms of the root נבא. Lastly, the semantic associations of ‘prophesying’ are akin to those related to ‘dreaming’, something which, in turn, suggests that auditory and visual divine communications are mutually complementary.

THE DECISION TO USE GO-PAY IS INFLUENCED BY PERCEPTIONS OF CONVENIENCE WITH INTEREST AS A MEDIATION VARIABLE

The purpose of this study is to examine how perceived convenience and interest as mediating variables affect consumers' decisions to use Go-Pay. Differential semantic association research is used in this type of research. In this study, primary data and secondary data are used. To collect 100 respondents as a research sample, a technique called purposive sampling was used in this study. The data analysis approach used in this study is the structural equation model (SEM) with SmartPLS 3.0 for Windows. The findings show that perceived convenience has a beneficial and significant impact on the decision to use go-pay, as well as a large and beneficial impact on interest, while interest does not have a significant impact on go-pay decision making.

Mirror Segmentation via Semantic-aware Contextual Contrasted Feature Learning

Mirrors are everywhere in our daily lives. Existing computer vision systems do not consider mirrors, and hence may get confused by the reflected content inside a mirror, resulting in a severe performance degradation. However, separating the real content outside a mirror from the reflected content inside it is non-trivial. The key challenge is that mirrors typically reflect contents similar to their surroundings, making it very difficult to differentiate the two. In this article, we present a novel method to segment mirrors from a single RGB image. To the best of our knowledge, this is the first work to address the mirror segmentation problem with a computational approach. We make the following contributions: First, we propose a novel network, called MirrorNet+, for mirror segmentation, by modeling both contextual contrasts and semantic associations. Second, we construct the first large-scale mirror segmentation dataset, which consists of 4,018 pairs of images containing mirrors and their corresponding manually annotated mirror masks, covering a variety of daily-life scenes. Third, we conduct extensive experiments to evaluate the proposed method and show that it outperforms the related state-of-the-art detection and segmentation methods. Fourth, we further validate the effectiveness and generalization capability of the proposed semantic awareness contextual contrasted feature learning by applying MirrorNet+ to other vision tasks, i.e., salient object detection and shadow detection. Finally, we provide some applications of mirror segmentation and analyze possible future research directions. Project homepage: https://mhaiyang.github.io/TOMM2022-MirrorNet+/index.html .

SELDNet: Sequenced encoder and lightweight decoder network for COVID-19 infection region segmentation

Segmenting regions of lung infection from computed tomography (CT) images shows excellent potential for rapid and accurate quantifying of Coronavirus disease 2019 (COVID-19) infection and determining disease development and treatment approaches. However, a number of challenges remain, including the complexity of imaging features and their variability with disease progression, as well as the high similarity to other lung diseases, which makes feature extraction difficult. To answer the above challenges, we propose a new sequence encoder and lightweight decoder network for medical image segmentation model (SELDNet). (i) Construct sequence encoders and lightweight decoders based on Transformer and deep separable convolution, respectively, to achieve different fine-grained feature extraction. (ii) Design a semantic association module based on cross-attention mechanism between encoder and decoder to enhance the fusion of different levels of semantics. The experimental results showed that the network can effectively achieve segmentation of COVID-19 infected regions. The dice of the segmentation result was 79.1%, the sensitivity was 76.3%, and the specificity was 96.7%. Compared with several state-of-the-art image segmentation models, our proposed SELDNet model achieves better results in the segmentation task of COVID-19 infected regions.

Red and blue states: dichotomized maps mislead and reduce perceived voting influence

In the United States the color red has come to represent the Republican party, and blue the Democratic party, in maps of voting patterns. Here we test the hypothesis that voting maps dichotomized into red and blue states leads people to overestimate political polarization compared to maps in which states are represented with continuous gradations of color. We also tested whether any polarizing effect is due to partisan semantic associations with red and blue, or if alternative hues produce similar effects. In Study 1, participants estimated the hypothetical voting patterns of eight swing states on maps with dichotomous or continuous red/blue or orange/green color schemes. A continuous gradient mitigated the polarizing effects of red/blue maps on voting predictions. We also found that a novel hue pair, green/orange, decreased perceived polarization. Whether this effect was due to the novelty of the hues or the fact that the hues were not explicitly labeled “Democrat” and “Republican” was unclear. In Study 2, we explicitly assigned green/orange hues to the two parties. Participants viewed electoral maps depicting results from the 2020 presidential election and estimated the voting margins for a subset of states. We replicated the finding that continuous red/blue gradient reduced perceived polarization, but the novel hues did not reduce perceived polarization. Participants also expected their hypothetical vote to matter more when viewing maps with continuous color gradations. We conclude that the dichotomization of electoral maps (not the particular hues) increases perceived voting polarization and reduces a voter’s expected influence on election outcomes.

Generated Image Editing Method Based on Global-Local Jacobi Disentanglement for Machine Learning.

Accurate semantic editing of the generated images is extremely important for machine learning and sample enhancement of big data. Aiming at the problem of semantic entanglement in generated image latent space of the StyleGAN2 network, we proposed a generated image editing method based on global-local Jacobi disentanglement. In terms of global disentanglement, we extract the weight matrix of the style layer in the pre-trained StyleGAN2 network; obtain the semantic attribute direction vector by using the weight matrix eigen decomposition method; finally, utilize this direction vector as the initialization vector for the Jacobi orthogonal regularization search algorithm. Our method improves the speed of the Jacobi orthogonal regularization search algorithm with the proportion of effective semantic attribute editing directions. In terms of local disentanglement, we design a local contrast regularized loss function to relax the semantic association local area and non-local area and utilize the Jacobi orthogonal regularization search algorithm to obtain a more accurate semantic attribute editing direction based on the local area prior MASK. The experimental results show that the proposed method achieves SOTA in semantic attribute disentangled metrics and can discover more accurate editing directions compared with the mainstream unsupervised generated image editing methods.

Updating and pre-existing semantic associations: testing can enhance or impair new learning

ABSTRACT We often need to update what we have learned, building on older information by adding newer information. When updating, is it better to review the older information by retrieving it (testing) or restudying it? In other words, do tests result in enhanced updating or impaired updating compared to restudying? Past research has obtained divergent conclusions to this question. The present study used a paired associates updating paradigm in which participants studied a cue and the older response (A-B). They later reviewed the older response by restudying (A-B) or testing (A-?; A-B) and immediately after learned a newer response (-D) that updated the original pair into a trio (A-B-D). In eight experiments, we demonstrated that different patterns of pre-existing semantic associations between the cue, older response, and newer response resulted in all possible outcomes: test enhanced new learning, test impaired new learning, and no difference. The results were most consistent with a family of updating theories that propose the metacognitive processes that occur after reviewing determine whether testing enhances, impairs, or has no impact on new learning. The results suggest that theories should consider the impact of the newer response in updating in addition to performance on the initial test.

RoRED: Bootstrapping labeling rule discovery for robust relation extraction

Labeling rules can be leveraged to produce training data by matching the sentences in the corpus. However, the robustness of the relation extraction is reduced by noisy labels generated from incorrectly matched and missing sentences. To address this problem, we propose the bootstrapping labeling rule discovery method for robust relation extraction (RoRED). Specifically, we first define PN-rules to filter incorrectly matched sentences based on positive (P) and negative (N) rules. Second, we design a semantic-matching mechanism to match missing sentences based on semantic associations between rules, words, and sentences. Moreover, we present a co-training-based rule verification approach to refine the labels of matched sentences and improve the overall quality of bootstrapped rule discovery. Experiments on a real-world dataset indicate that RoRED achieves at least a 20% gain in F1 score compared to state-of-the-art methods.

Timbre Semantic Associations Vary Both Between and Within Instruments

The main objective of this study is to understand how timbre semantic associations—for example, a sound’s timbre perceived as bright, rough, or hollow—vary with register and pitch height across instruments. In this experiment, 540 online participants rated single, sustained notes from eight Western orchestral instruments (flute, oboe, bass clarinet, trumpet, trombone, violin, cello, and vibraphone) across three registers (low, medium, and high) on 20 semantic scales derived from Reymore and Huron (2020). The 24 two-second stimuli, equalized in loudness, were produced using the Vienna Symphonic Library. Exploratory modeling examined relationships between mean ratings of each semantic dimension and instrument, register, and participant musician identity (“musician” vs. “nonmusician”). For most semantic descriptors, both register and instrument were significant predictors, though the amount of variance explained differed (marginal R2). Terms that had the strongest positive relationships with register include shrill/harsh/noisy, sparkling/brilliant/bright, ringing/long decay, and percussive. Terms with the strongest negative relationships with register include deep/thick/heavy, raspy/grainy/gravelly, hollow, and woody. Post hoc modeling using only pitch height and only register to predict mean semantic rating suggests that pitch height may explain more variance than does register. Results help clarify the influence of both instrument and relative register (and pitch height) on common timbre semantic associations.