Shared Representation Research Articles

First- And Third-Person Video Co-Analysis By Learning Spatial-Temporal Joint Attention.

Recent years have witnessed a tremendous increase of first-person videos captured by wearable devices. Such videos record information from different perspectives than the traditional third-person view, and thus show a wide range of potential usages. However, techniques for analyzing videos from different views can be fundamentally different, not to mention co-analyzing on both views to explore the shared information. In this paper, we take the challenge of cross-view video co-analysis and deliver a novel learning-based method. At the core of our method is the notion of "joint attention", indicating the shared attention regions that link the corresponding views, and eventually guide the shared representation learning across views. To this end, we propose a multi-branch deep network, which extracts cross-view joint attention and shared representation from static frames with spatial constraints, in a self-supervised and simultaneous manner. In addition, by incorporating the temporal transition model of the joint attention, we obtain spatial-temporal joint attention that can robustly capture the essential information extending through time. Our method outperforms the state-of-the-art on the standard cross-view video matching tasks on public datasets. Furthermore, we demonstrate how the learnt joint information can benefit various applications through a set of qualitative and quantitative experiments.

Reduced shared emotional representations toward women revealing more skin

ABSTRACT Extensive experimental research has been conducted to investigate how individuals empathise with others depending on contextual and motivational factors. However, the effect of sexual objectification (i.e. focus on the individual’s physical appearance over his/her mental state) on empathy is scarce at best thus far. The aim of this work is to shed light on whether objectification modulates empathic responses toward humans and human-like objects. In Experiment 1, participants either underwent visuo-tactile stimulation or witnessed another person (a mannequin, a sexualized or a non-sexualized female confederate) being stimulated with pleasant or unpleasant objects. Participants were then asked to report either their own or the other’s emotional experience. Results showed that shared representations (i.e. similarity between self-other emotional ratings) are significantly lower for the mannequin, intermediate for the sexualized woman, and reach the highest values for the non-sexualized woman. In Experiment 2, shared representations were assessed during a ball-tossing game in which the participants or one of the two confederates (sexualized or non-sexualized woman) were excluded from the game. Again, results showed reduced similarity between self-other emotional ratings toward sexualized as compared to non-sexualized women. The findings suggest that interacting with sexually objectified women reduces empathic responses typically observed within human relations.

Deep Multiview Learning to Identify Population Structure with Multimodal Imaging.

We present an effective deep multiview learning framework to identify population structure using multimodal imaging data. Our approach is based on canonical correlation analysis (CCA). We propose to use deep generalized CCA (DGCCA) to learn a shared latent representation of non-linearly mapped and maximally correlated components from multiple imaging modalities with reduced dimensionality. In our empirical study, this representation is shown to effectively capture more variance in original data than conventional generalized CCA (GCCA) which applies only linear transformation to the multi-view data. Furthermore, subsequent cluster analysis on the new feature set learned from DGCCA is able to identify a promising population structure in an Alzheimer's disease (AD) cohort. Genetic association analyses of the clustering results demonstrate that the shared representation learned from DGCCA yields a population structure with a stronger genetic basis than several competing feature learning methods.

George Eliot and George Lewes in Justus von Liebig’s laboratory: realism, chemistry, and agriculture in the mid-nineteenth century

ABSTRACT In Spring 1858, George Eliot and George Lewes visited Munich, where they became friends with the chemist Justus von Liebig, famous for his pioneering work on chemistry in its applications to agriculture. I consider their friendship as the basis for an examination of realism in literary and scientific writing on the agricultural environment. I show that Eliot’s literary realism of rural life responded to Liebig’s chemistry, broadly diverging from his ideas regarding science’s applications to agriculture. I also demonstrate that Liebig’s scientific method evolved by drawing on Eliot’s and Lewes’s theories of realism. Shared representations of the agricultural environment can be discerned across their writing, suggesting a representationalism co-constructed by the period’s foremost agricultural chemist and most important proponents of realism in literature. As my conclusion identifies, this understanding of links between mid-nineteenth-century literature and science has material consequences for the representation of agriculture in a time of ecological crisis.

Exploiting User Preference and Mobile Peer Influence for Human Mobility Annotation

Human mobility annotation aims to assign mobility records the corresponding visiting Point-of-Interests (POIs). It is one of the most fundamental problems for understanding human mobile behaviors. In literature, many efforts have been devoted to annotating mobility records in a pointwise or trajectory-wise manner. However, the user preference factor is not fully explored and, worse still, the mobile peer influence factor has never been integrated. To this end, in this article, we propose a novel framework, named JEPPI, to jointly exploit user preference and mobile peer influence to tackle the problem. In our JEPPI, we first unify the two distinct factors in a behavior-driven user-POI graph. This graph enables us to model user preference with user-POI visiting relationships, and model two types of mobile peer influence with co-location and co-visiting peer relationships, respectively. Moreover, we devise an equivalence-emphasizing metric to reduce redundancy in the second-order co-visiting peer influence. In addition, a mutual augmentation learning approach is proposed to preserve the latent structures of various factors exploited. Notably, our learning approach preserves all factors in a shared representation space such that user preference is learned with mobile peer influence being considered at the same time, and vice versa. In this way, the different factors are mutually augmented and semantically integrated to enhance human mobility annotation. Finally, using two large-scale real-world datasets, we conduct extensive experiments to demonstrate the superiority of our approach compared with the state-of-the-art annotation methods.

Empathic pain evoked by sensory and emotional-communicative cues share common and process-specific neural representations.

Pain empathy can be evoked by multiple cues, particularly observation of acute pain inflictions or facial expressions of pain. Previous studies suggest that these cues commonly activate the insula and anterior cingulate, yet vicarious pain encompasses pain-specific responses as well as unspecific processes (e.g. arousal) and overlapping activations are not sufficient to determine process-specific shared neural representations. We employed multivariate pattern analyses to fMRI data acquired during observation of noxious stimulation of body limbs (NS) and painful facial expressions (FE) and found spatially and functionally similar cross-modality (NS versus FE) whole-brain vicarious pain-predictive patterns. Further analyses consistently identified shared neural representations in the bilateral mid-insula. The vicarious pain patterns were not sensitive to respond to non-painful high-arousal negative stimuli but predicted self-experienced thermal pain. Finally, a domain-general vicarious pain pattern predictive of self-experienced pain but not arousal was developed. Our findings demonstrate shared pain-associated neural representations of vicarious pain.

Cross-linguistic syntactic priming in Korean learners of English

AbstractThis study investigates whether L2 learners develop and share an abstract syntactic representation between an L1 and L2 with different word orders and, if so, whether one language’s unique syntactic features affect the shared representation. Korean (SOV) and English (SVO) have equivalent dative alternations; however, because Korean allows word-order scrambling, several dative structures are available in Korean that do not have English counterparts. In this study’s cross-linguistic syntactic priming experiment, intermediate and advanced Korean learners of English described pictures in English after reading various types of Korean dative sentences. The study found evidence of cross-linguistic syntactic priming between Korean and English, regardless of L2 proficiency, but only when prime and target structures shared identical functional assignments, information structures, and order of thematic roles. These results suggest that, within limits created by language-specific features, L2 learners can develop and share abstract representations between two languages with different word orders.

Tracking Pandemics: A MEC-Enabled IoT Ecosystem with Learning Capability

The COVID-19 pandemic has resulted in unprecedented challenges to global society and the healthcare system in particular. The main objective of this article is to introduce an end-to-end Internet of Things (IoT) ecosystem for healthcare that uses an open source hardware and interoperable IoT standard for eHealth monitoring in general, and COVID-19 symptoms (e.g., fever, coughing, and fatigue) in particular. The system is designed to monitor the physical conditions of human subjects and send the data to a hierarchical multi-access edge computing (MEC) framework. Such a system is expected to be cognizant, taskable (i.e., tasks can be assigned to any computing process in the system), and adaptable. To this end, we demonstrate how a learning method can be introduced in the ecosystem to achieve taskability and efficiency. Specifically, the proposed system utilizes a shared representation learning process to extract actionable information from large volumes of high-dimensional data obtained from IoT edge devices. These edge devices are enabled with tri-sensors for real-time monitoring of COVID-19 symptoms. The feasibility of the proposed system is evaluated by testing real datasets.

Relation Extraction From Biomedical and Clinical Text: Unified Multitask Learning Framework.

To minimize the accelerating amount of time invested on the biomedical literature search, numerous approaches for automated knowledge extraction have been proposed. Relation extraction is one such task where semantic relations between the entities are identified from the free text. In the biomedical domain, extraction of regulatory pathways, metabolic processes, adverse drug reaction or disease models necessitates knowledge from the individual relations, for example, physical or regulatory interactions between genes, proteins, drugs, chemical, disease or phenotype. In this paper, we study the relation extraction task from three major biomedical and clinical tasks, namely drug-drug interaction, protein-protein interaction, and medical concept relation extraction. Towards this, we model the relation extraction problem in a multi-task learning (MTL)framework, and introduce for the first time the concept of structured self-attentive network complemented with the adversarial learning approach for the prediction of relationships from the biomedical and clinical text. The fundamental notion of MTL is to simultaneously learn multiple problems together by utilizing the concepts of the shared representation. Additionally, we also generate the highly efficient single task model which exploits the shortest dependency path embedding learned over the attentive gated recurrent unit to compare our proposed MTL models. The framework we propose significantly improves over all the baselines (deep learning techniques)and single-task models for predicting the relationships, without compromising on the performance of all the tasks.

Unsupervised multi-view representation learning with proximity guided representation and generalized canonical correlation analysis

Multi-view data can collaborate with each other to provide more comprehensive information than single-view data. Although there exist a few unsupervised multi-view representation learning methods taking both the discrepancies and incorporating complementary information from different views into consideration, they always ignore the use of inner-view discriminant information. It remains challenging to learn a meaningful shared representation of multiple views. To overcome this difficulty, this paper proposes a novel unsupervised multi-view representation learning model, MRL. Unlike most state-of-art multi-view representation learning, which only can be used for clustering or classification task, our method explores the proximity guided representation from inner-view and complete the task of multi-label classification and clustering by the discrimination fusion representation simultaneously. MRL consists of three parts. The first part is a deep representation learning for each view and then aims to represent the latent specific discriminant characteristic of each view, the second part builds a proximity guided dynamic routing to preserve its inner features of direction,location and etc. At last, the third part, GCCA-based fusion, exploits the maximum correlations among multiple views based on Generalized Canonical Correlation Analysis (GCCA). To the best of our knowledge, the proposed MRL could be one of the first unsupervised multi-view representation learning models that work in proximity guided dynamic routing and GCCA modes. The proposed model MRL is tested on five multi-view datasets for two different tasks. In the task of multi-label classification, the results show that our model is superior to the state-of-the-art multi-view learning methods in precision, recall, F1 and accuracy. In clustering task, its performance is better than the latest related popular algorithms. And the performance varies w.r.t. the dimensionality of G is also made to explore the characteristics of MRL.

Object-Aware Multimodal Named Entity Recognition in Social Media Posts With Adversarial Learning

Named Entity Recognition (NER) in social media posts is challenging since texts are usually short and contexts are lacking. Most recent works show that visual information can boost the NER performance since images can provide complementary contextual information for texts. However, the image-level features ignore the mapping relations between fine-grained visual objects and textual entities, which results in error detection in entities with different types. To better exploit visual and textual information in NER, we propose an adversarial gated bilinear attention neural network (AGBAN). The model jointly extracts entity-related features from both visual objects and texts, and leverages an adversarial training to map two different representations into a shared representation. As a result, domain information contained in an image can be transferred and applied for extracting named entities in the text associated with the image. Experimental results on Tweets dataset demonstrate that our model outperforms the state-of-the-art methods. Moreover, we systematically evaluate the effectiveness of the proposed gated bilinear attention network in capturing the interactions of mutimodal features visual objects and textual words. Our results indicate that the adversarial training can effectively exploit commonalities across heterogeneous data sources, which leads to improved performance in NER when compared to models purely exploiting text data or combining the image-level visual features.

Shared Numerosity Representations Across Formats and Tasks Revealed with 7 Tesla fMRI: Decoding, Generalization, and Individual Differences in Behavior.

Debate continues on whether encoding of symbolic number is grounded in nonsymbolic numerical magnitudes. Nevertheless, fluency of perceiving both number formats, and translating between them, predicts math skills across the life span. Therefore, this study asked if numbers share cortical activation patterns across formats and tasks, and whether neural response to number predicts math-related behaviors. We analyzed patterns of neural activation using 7 Tesla functional magnetic resonance imaging in a sample of 39 healthy adults. Discrimination was successful between numerosities 2, 4, 6, and 8 dots and generalized to activation patterns of the same numerosities represented as Arabic digits in the bilateral parietal lobes and left inferior frontal gyrus (IFG) (and vice versa). This indicates that numerosity-specific neural resources are shared between formats. Generalization was also successful across tasks where participants either identified or compared numerosities in bilateral parietal lobes and IFG. Individual differences in decoding did not relate to performance on a number comparison task completed outside of the scanner, but generalization between formats and across tasks negatively related to math achievement in the parietal lobes. Together, these findings suggest that individual differences in representational specificity within format and task contexts relate to mathematical expertise.

Designing Shared Representations for Open-Ended Needs

While a vast literature examined problem-solving as part of decision making in organizations, little is known about how problems are formulated in the first place. Through an ethnographic study, in...

Emotion Recognition on Twitter: Comparative Study and Training a Unison Model

Despite recent successes of deep learning in many fields of natural language processing, previous studies of emotion recognition on Twitter mainly focused on the use of lexicons and simple classifiers on bag-of-words models. The central question of our study is whether we can improve their performance using deep learning. To this end, we exploit hashtags to create three large emotion-labeled data sets corresponding to different classifications of emotions. We then compare the performance of several word- and character-based recurrent and convolutional neural networks with the performance on bag-of-words and latent semantic indexing models. We also investigate the transferability of the final hidden state representations between different classifications of emotions, and whether it is possible to build a unison model for predicting all of them using a shared representation. We show that recurrent neural networks, especially character-based ones, can improve over bag-of-words and latent semantic indexing models. Although the transfer capabilities of these models are poor, the newly proposed training heuristic produces a unison model with performance comparable to that of the three single models.

Congestion Control in SDN-Based Networks via Multi-Task Deep Reinforcement Learning

Congestion control is a fundamental network task that modulates the data transmission rates of traffic sources to efficiently utilize network capacity. With the advent of machine learning, congestion control based on deep reinforcement learning is the subject of extensive attention. At present, research on machine-learning-based congestion control is mainly focused on single-task scenarios -- to design a control strategy that is only used to reduce congestion. This article studies the congestion control problem based on machine learning in multi-task scenarios. Specifically, we propose a congestion control model based on multi-task deep reinforcement learning. The model takes congestion control as the main task and load balancing as the auxiliary task. Compared to the single-task method, our model can better represent the network environment by learning the shared representation of congestion features and load balancing features. Moreover, network traffic control may involve both congestion control and load balancing. Therefore, learning multiple tasks jointly while exploiting commonalities and differences across tasks can help reduce the cost of task coordination. We use software defined networking to decouple the data and control planes, making network control strategies more flexible than traditional networks. To the best of our knowledge, this is the first time multi-task learning has been applied to network traffic control. Experimental results show that the method is efficient.

Computational insights into human perceptual expertise for familiar and unfamiliar face recognition

Humans are generally thought to be experts at face recognition, and yet identity perception for unfamiliar faces is surprisingly poor compared to that for familiar faces. Prior theoretical work has argued that unfamiliar face identity perception suffers because the majority of identity-invariant visual variability is idiosyncratic to each identity, and thus, each face identity must be learned essentially from scratch. Using a high-performing deep convolutional neural network, we evaluate this claim by examining the effects of visual experience in untrained, object-expert and face-expert networks. We found that only face training led to substantial generalization in an identity verification task of novel unfamiliar identities. Moreover, generalization increased with the number of previously learned identities, highlighting the generality of identity-invariant information in face images. To better understand how familiarity builds upon generic face representations, we simulated familiarization with face identities by fine-tuning the network on images of the previously unfamiliar identities. Familiarization produced a sharp boost in verification, but only approached ceiling performance in the networks that were highly trained on faces. Moreover, in these face-expert networks, the sharp familiarity benefit was seen only at the identity-based output probability layer, and did not depend on changes to perceptual representations; rather, familiarity effects required learning only at the level of identity readout from a fixed expert representation. Our results thus reconcile the existence of a large familiar face advantage with claims that both familiar and unfamiliar face identity processing depend on shared expert perceptual representations.

A multifactorial optimization paradigm for linkage tree genetic algorithm

Linkage Tree Genetic Algorithm (LTGA) is an effective Evolutionary Algorithm (EA) to solve complex problems using the linkage information between problem variables. LTGA performs well in various kinds of single-task optimization and yields promising results in comparison with the canonical genetic algorithm. However, LTGA is an unsuitable method for dealing with multi-task optimization problems. On the other hand, Multifactorial Optimization (MFO) can simultaneously solve independent optimization problems, which are encoded in a unified representation to take advantage of the process of knowledge transfer. In this paper, we introduce Genetic Algorithm (MF-LTGA) by combining the main features of both LTGA and MFO. MF-LTGA is able to tackle multiple optimization tasks at the same time, each task learns the dependency between problem variables from the shared representation. This knowledge serves to determine the high-quality partial solutions for supporting other tasks in exploring the search space. Moreover, MF-LTGA speeds up convergence because of knowledge transfer of relevant problems. We demonstrate the effectiveness of the proposed algorithm on two benchmark problems: Clustered Shortest-Path Tree Problem and Deceptive Trap Function. In comparison to LTGA and existing methods, MF-LTGA outperforms in quality of the solution or in computation time.

Dimerization and N\xe9el Order in Different Quantum Spin Chains Through a Shared Loop Representation

The ground-states of the spin-S antiferromagnetic chain H_{{text {AF}}} with a projection-based interaction and the spin-1/2 XXZ-chain H_{{text {XXZ}}} at anisotropy parameter Delta =cosh (lambda ) share a common loop representation in terms of a two-dimensional functional integral which is similar to the classical planar Q-state Potts model at sqrt{Q}= 2S+1 =2cosh (lambda ). The multifaceted relation is used here to directly relate the distinct forms of translation symmetry breaking which are manifested in the ground-states of these two models: dimerization for H_{{text {AF}}} at all S> 1/2, and Néel order for H_{{text {XXZ}}} at lambda >0. The results presented include: (i) a translation to the above quantum spin systems of the results which were recently proven by Duminil–Copin–Li–Manolescu for a broad class of two-dimensional random-cluster models, and (ii) a short proof of the symmetry breaking in a manner similar to the recent structural proof by Ray–Spinka of the discontinuity of the phase transition for Q>4. Altogether, the quantum manifestation of the change between Q=4 and Q>4 is a transition from a gapless ground-state to a pair of gapped and extensively distinct ground-states.

Beers, Bros, and Brett: memes and the visual ideograph of the

ABSTRACT Following Brett Kavanaugh’s controversial testimony to the Senate Judiciary Committee, memes containing his image during his statement and questioning emerged across social media platforms. We examine these memes as a collective visual ideograph of the <Angry White Man>. They display a shared visual representation while enacting competing ideological frameworks. Through an analysis of 71 memes, two prominent themes emerged. First, there was competition over representing Kavanaugh’s anger; those demonstrating support for Kavanaugh argued that anger was a justified response to a “witch hunt,” while those in opposition to his confirmation defined his anger as inappropriate rage. Second, there was competition over the connections between Kavanaugh and alcohol. Supporters used Kavanaugh’s repeated mention of beer to tie beer drinking to appropriate masculinity as a Real Man, while opponents reference his love of beer and fraternity membership as evidence of both his unfitness for the court and the allegations against him. The synchronic and diachronic deployment of this ideograph of the <Angry White Man> open opportunities for understanding how these linkages among emotion, sex, and ethnicity create fertile ground for ideological competition.

Shared Representation of Visual and Auditory Motion Directions in the Human Middle-Temporal Cortex

The human occipito-temporal region hMT+/V5 is well known for processing visual motion direction. Here, we demonstrate that hMT+/V5 also represents the direction of auditory motion in a format partially aligned with the one used to code visual motion. We show that auditory and visual motion directions can be reliably decoded in individually localized hMT+/V5 and that motion directions in one modality can be predicted from the activity patterns elicited by the other modality. Despite shared motion-direction information across the senses, vision and audition, however, overall produce opposite voxel-wise responses in hMT+/V5. Our results reveal a multifaced representation of multisensory motion signals in hMT+/V5 and have broader implications for our understanding of how we consider the division of sensory labor between brain regions dedicated to a specific perceptual function.