Shared Representation Research Articles

An attentive joint model with transformer-based weighted graph convolutional network for extracting adverse drug event relation

Adverse drug event (ADE) relation extraction is a crucial task for drug safety surveillance which aims to discover potential relations between ADE mentions from unstructured medical texts. To date, the graph convolutional networks (GCN) have been the state-of-the-art solutions for improving the ability of relation extraction task. However, there are many challenging issues that should be addressed. Among these, the syntactic information is not fully exploited by GCN-based methods, especially the diversified dependency edges. Still, these methods fail to effectively extract complex relations that include nested, discontinuous and overlapping mentions. Besides, the task is primarily regarded as a classification problem where each candidate relation is treated independently which neglects the interaction between other relations. To deal with these issues, in this paper, we propose an attentive joint model with transformer-based weighted GCN for extracting ADE Relations, called ADERel. Firstly, the ADERel system formulates the ADE relation extraction task as an N-level sequence labelling so as to model the complex relations in different levels and capture greater interaction between relations. Then, it exploits our neural joint model to process the N-level sequences jointly. The joint model leverages the contextual and structural information by adopting a shared representation that combines a bidirectional encoder representation from transformers (BERT) and our proposed weighted GCN (WGCN). The latter assigns a score to each dependency edge within a sentence so as to capture rich syntactic features and determine the most influential edges for extracting ADE relations. Finally, the system employs a multi-head attention to exchange boundary knowledge across levels. We evaluate ADERel on two benchmark datasets from TAC 2017 and n2c2 2018 shared tasks. The experimental results show that ADERel is superior in performance compared with several state-of-the-art methods. The results also demonstrate that incorporating a transformer model with WGCN makes the proposed system more effective for extracting various types of ADE relations. The evaluations further highlight that ADERel takes advantage of joint learning, showing its effectiveness in recognizing complex relations.

Using top-down modulation to optimally balance shared versus separated task representations

Human adaptive behavior requires continually learning and performing a wide variety of tasks, often with very little practice. To accomplish this, it is crucial to separate neural representations of different tasks in order to avoid interference. At the same time, sharing neural representations supports generalization and allows faster learning. Therefore, a crucial challenge is to find an optimal balance between shared versus separated representations. Typically, models of human cognition employ top-down modulatory signals to separate task representations, but there exist surprisingly little systematic computational investigations of how such modulation is best implemented. We identify and systematically evaluate two crucial features of modulatory signals. First, top-down input can be processed in an additive or multiplicative manner. Second, the modulatory signals can be adaptive (learned) or non-adaptive (random). We cross these two features, resulting in four modulation networks which are tested on a variety of input datasets and tasks with different degrees of stimulus-action mapping overlap. The multiplicative adaptive modulation network outperforms all other networks in terms of accuracy. Moreover, this network develops hidden units that optimally share representations between tasks. Specifically, different than the binary approach of currently popular latent state models, it exploits partial overlap between tasks.

Identifying multimodal imaging‐driven subtypes in mild cognitive impairment using deep multiview learning

AbstractBackgroundMultimodal neuroimaging data can provide complementary information that a single modality cannot about neurodegenerative diseases such as Alzheimer's disease (AD). Deep Generalized Canonical Correlation Analysis (DGCCA) is able to learn a shared feature representation from different views of data by applying non‐linear transformation using neural network. We utilize DGCCA to extract maximally correlated components from multi‐modal neuroimaging data to identify potential imaging‐driven MCI subtypes.MethodWe study 308 Mild Cognitive Impairment (MCI) participants (195 early MCI and 113 late MCI) from the Alzheimer’s Disease Neuroimaging Initiative (ADNI), each with voxel level features from FDG PET, amyloid PET (AV45) and structural MRI processed using voxel‐based morphometry (VBM). Six experimental settings are designed to compare single modality with multiview methods ‐ GCCA and DGCCA, see Figure 1. Agglomerative clustering was used to generated 2 clusters with features from each experiment. To investigate differences between the clusters, Wilcoxon rank‐sum tests are conducted on 11 baseline AD biomarkers including 5 cognitive assessments and 6 brain volume measures, from the ADNI QT‐PAD dataset http://www.pi4cs.org/qt‐pad‐challenge.ResultAmong the two multiview methods, DGCCA is able to explain 68.57% variance with 20 features, while GCCA explains 68.66% variance with 94 features. To evaluate the potential subtypes from clustering, the Calinski‐Harabasz (CH) score, Silhouette score and adjusted mutual information (AMI) score are computed, see Table 1. AV45 generates the best defined clusters, where DGCCA generates clusters with quality comparable to single modality features. In our QT analysis, clusters from FDG and DGCCA features show differential measure in all biomarkers where DGCCA learns from multimodal data, see Figure 2.ConclusionDGCCA is able to learn maximally correlated features from multimodal neuroimaging data with reduced dimensionality, and explain more variance than its linear counterpart GCCA. Cluster analysis shows these imaging‐driven MCI subtypes are different from the currently diagnosis with differential QT measures, by incorporating complementary information from 3 imaging modalities. DGCCA shows to be an effective feature learning method, and this multiview learning framework can identify potentially novel MCI subtypes to facilitate early detection of AD.

Automated eloquent cortex localization in brain tumor patients using multi-task graph neural networks

Localizing the eloquent cortex is a crucial part of presurgical planning. While invasive mapping is the gold standard, there is increasing interest in using noninvasive fMRI to shorten and improve the process. However, many surgical patients cannot adequately perform task-based fMRI protocols. Resting-state fMRI has emerged as an alternative modality, but automated eloquent cortex localization remains an open challenge. In this paper, we develop a novel deep learning architecture to simultaneously identify language and primary motor cortex from rs-fMRI connectivity. Our approach uses the representational power of convolutional neural networks alongside the generalization power of multi-task learning to find a shared representation between the eloquent subnetworks. We validate our method on data from the publicly available Human Connectome Project and on a brain tumor dataset acquired at the Johns Hopkins Hospital. We compare our method against feature-based machine learning approaches and a fully-connected deep learning model that does not account for the shared network organization of the data. Our model achieves significantly better performance than competing baselines. We also assess the generalizability and robustness of our method. Our results clearly demonstrate the advantages of our graph convolution architecture combined with multi-task learning and highlight the promise of using rs-fMRI as a presurgical mapping tool.

Parallel learner: A practical deep reinforcement learning framework for multi-scenario games

Traditional reinforcement learning methods are only applicable to single-scenario tasks. When it comes to multi-scenario, the single-scenario agents fail to perform well. That is, the traditional reinforcement learning methods own the poor generalization when facing different tasks simultaneously. In this work, we propose a practical deep reinforcement learning framework that can perform on multiple 3D scenarios concurrently. We adopt the Actor–Learner framework to realize the parallelization of multiple scenarios and resolve the policy lag problem by generalizing Retrace(λ) to a new value function. We prove its convergence theoretically. Besides, we design an auxiliary recognition task and an auxiliary control task inspired by the hard shared representation in multi-task learning to improve the performance of our multi-scenario agent. Experimental results show that our method outperforms state-of-the-art algorithms on DMLab-30, achieving more advantages on multi-scenario games. We verify the effectiveness of each part of our framework by the ablation experiments. We also find our parallel learner transferable by testing on the untrained scenarios.

Shared Representations in Athletes: Segmenting Action Sequences From Taekwondo Reveals Implicit Agreement.

How do athletes represent actions from their sport? How are these representations structured and which knowledge is shared among experts in the same discipline? To address these questions, the event segmentation task was used. Experts in Taekwondo and novices indicated how they would subjectively split videos of Taekwondo form sequences into meaningful units. In previous research, this procedure was shown to unveil the structure of internal action representations and to be affected by sensorimotor knowledge. Without specific instructions on the grain size of segmentation, experts tended to integrate over longer episodes which resulted in a lower number of single units. Moreover, in accordance with studies in figure-skating and basketball, we expected higher agreement among experts on where to place segmentation marks, i.e., boundaries. In line with this hypothesis, significantly more overlap of boundaries was found within the expert group as compared to the control group. This was observed even though the interindividual differences in the selected grain size were huge and expertise had no systematic influence here. The absence of obvious goals or objects to structure Taekwondo forms underlines the importance of shared expert knowledge. Further, experts might have benefited from sensorimotor skills which allowed to simulate the observed actions more precisely. Both aspects may explain stronger agreement among experts even in unfamiliar Taekwondo forms. These interpretations are descriptively supported by the participants’ statements about features which guided segmentation and by an overlap of the group’s agreed boundaries with those of an experienced referee. The study shows that action segmentation can be used to provide insights into structure and content of action representations specific to experts. The mechanisms underlying shared knowledge among Taekwondoists and among experts in general are discussed on the background of current theoretic frameworks.

Place as social identity: an analysis of the spatial enactments of community loss and activism within the built environment surrounding Grenfell Tower

ABSTRACT This article is based on analysis of narrative interviews with individuals active in local community work in the aftermath of the Grenfell Tower fire. It explores how the public spaces and buildings around the tower were claimed and shared by the community, first for emergency support and then for collective mourning, remembrance, worship, and activism. Applying the social psychology of community participation, the article elaborates place identity as a form of social identity and shows how participation functions to organise shared representations of local conditions and facilitate empowerment for social change. Further, the article posits that, in contexts of urban super-diversity, the shared use of local spaces, including faith spaces, generates new forms of belonging-in-multiculture that can become the basis for community development and collective struggle against racialised marginalisation.

Adversarial Energy Disaggregation

Energy disaggregation, also known as non-intrusive load monitoring (NILM), challenges the problem of separating the whole-home electricity usage into appliance-specific individual consumptions, which is a typical application of data analysis. NILM aims to help households understand how the energy is used and consequently tell them how to effectively manage the energy, thus allowing energy efficiency, which is considered as one of the twin pillars of sustainable energy policy (i.e., energy efficiency and renewable energy). Although NILM is unidentifiable, it is widely believed that the NILM problem can be addressed by data science. Most of the existing approaches address the energy disaggregation problem by conventional techniques such as sparse coding, non-negative matrix factorization, and the hidden Markov model. Recent advances reveal that deep neural networks (DNNs) can get favorable performance for NILM since DNNs can inherently learn the discriminative signatures of the different appliances. In this article, we propose a novel method named adversarial energy disaggregation based on DNNs. We introduce the idea of adversarial learning into NILM, which is new for the energy disaggregation task. Our method trains a generator and multiple discriminators via an adversarial fashion. The proposed method not only learns shared representations for different appliances but captures the specific multimode structures of each appliance. Extensive experiments on real-world datasets verify that our method can achieve new state-of-the-art performance.

Syntactic priming across highly similar languages is not affected by language proficiency

ABSTRACT This study explores the mechanism underlying shared syntactic representations for highly similar languages by investigating whether cross-linguistic syntactic priming is affected by language proficiency. In two experiments, native (L1) Mandarin-Chaoshanese speakers with moderate proficiency in Cantonese (L2) heard Chaoshanese and Cantonese dative sentences with a prepositional object (PO) or a double object (DO) structure, and then completed a description of a depicted ditransitive event using Mandarin. Priming from L2 to L1 was equal to that from L1 to L1, irrespective of whether the prime and the target involved cognate verbs. Similarly, priming from L2 to L1 was not affected by speakers’ L2 proficiency, suggesting that shared representations across highly similar languages are insensitive to language proficiency. We interpret the results in terms of the development of shared representations for highly similar languages.

Subspace Network with Shared Representation learning for intelligent fault diagnosis of machine under speed transient conditions with few samples

Sharp speed variation leads to a shift of sample distribution domain, which poses a challenge for vibration-based rolling bearing fault diagnosis. Furthermore, the overfitting effects inflicted on the intelligent diagnosis model due to insufficient data will hinder the performance significantly. In this work, a Subspace Network with Shared Representation learning (SNSR) based on meta-learning is constructed for fault diagnosis under speed transient conditions with few samples. Firstly, shared representation learning based on the cross mutual information estimation is designed to promote the encoder to learn the domain invariant features. Meanwhile, we developed non-parameterized adaptive weight allocation to optimize the estimation of the discriminator. Then, the subspace classifiers in the meta-learning paradigm are employed to force the encoder to learn the discriminative features. Finally, the shared representation learning is embedded into the meta-learning and a cross co-training mechanism is designed for optimization. Thus the fusion framework is endowed with the capacity of learning distinguishable and domain invariant features simultaneously for diagnosis under speed transient conditions with few samples. Comparative experiments on two case studies of bearing fault diagnosis validated the superior performance of the proposed method, with an accuracy of 97.72% and 96.46% in 7-way and 9-way learning respectively.

Hyperspectral band selection via region-aware latent features fusion based clustering

Band selection is one of the most effective methods to reduce the band redundancy of hyperspectral images (HSIs). Most existing band selection methods tend to regard each band as a whole, and then explore the band redundancy with the pixel-wise features directly. However, since the regions of HSIs corresponding to different objects have diverse spectral properties and spatial structure, such above scheme limits the performance of hyperspectral band selection due to the lack of spatial information. To address above issues, a novel band selection method via region-aware latent features fusion based clustering (RLFFC) is proposed. Specifically, we employ the superpixel segmentation to segment HSIs into multiple regions so that the spatial information of HSIs can be fully preserved. In order to capture the priori information, we construct its corresponding Laplacian matrix from which a group of low dimensional latent features are generated to further enhance the separability among different bands. Then, a shared latent feature representation of HSIs is obtained by fusing region-aware latent features to effectively capture the band redundancy of HSIs. Finally, the k-means clustering algorithm is utilized to obtain the index of the selected bands from the shared latent feature representation. As a result, the spectral and spatial properties are well exploited in the proposed method. Extensive experiments on four public hyperspectral datasets show that the proposed method achieves superior performance when compared with other state-of-the-art ones.

Visual and auditory brain areas share a representational structure that supports emotion perception

Emotionally expressive music and dance occur together across the world. This may be because features shared across the senses are represented the same way even in different sensory brain areas, putting music and movement in directly comparable terms. These shared representations may arise from a general need to identify environmentally relevant combinations of sensory features, particularly those that communicate emotion. To test the hypothesis that visual and auditory brain areas share a representational structure, we created music and animation stimuli with crossmodally matched features expressing a range of emotions. Participants confirmed that each emotion corresponded to a set of features shared across music and movement. A subset of participants viewed both music and animation during brain scanning, revealing that representations in auditory and visual brain areas were similar to one another. This shared representation captured not only simple stimulus features but also combinations of features associated with emotion judgments. The posterior superior temporal cortex represented both music and movement using this same structure, suggesting supramodal abstraction of sensory content. Further exploratory analysis revealed that early visual cortex used this shared representational structure even when stimuli were presented auditorily. We propose that crossmodally shared representations support mutually reinforcing dynamics across auditory and visual brain areas, facilitating crossmodal comparison. These shared representations may help explain why emotions are so readily perceived and why some dynamic emotional expressions can generalize across cultural contexts.

Deep ladder reconstruction-classification network for unsupervised domain adaptation

Unsupervised Domain Adaptation aims to learn a classifier for an unlabeled target domain by transferring knowledge from a labeled source domain. Most existing approaches learn domain-invariant features by adapting the entire information of each image. However, forcing adaptation of domain-specific components can undermine the effectiveness of learned features. We propose a novel architecture called Deep Ladder Reconstruction-Classification Network (DLaReC) which is designed to learn cross-domain shared contents by suppressing domain-specific variations. The DLaReC adopts an encoder with cross-domain sharing and a target-domain reconstruction decoder. The encoder and decoder are connected with residual shortcuts at each intermediate layer. By this means, the domain-specific components are directly fed to the decoder for reconstruction, relieving the pressure to learn domain-specific variations at later layers of the shared encoder. Therefore, DLaReC allows the encoder to focus on learning cross-domain shared representations and ignore domain-specific variations. DLaReC is implemented by jointly learning three tasks: supervised classification of the source domain, unsupervised reconstruction of the target domain and cross-domain shared representation adaptation. Extensive experiments on Digit, Office31, ImageCLEF-DA and Office-Home datasets demonstrate the DLaReC outperforms state-of-the-art methods on the whole. The average accuracy on the Digit datasets, for instance, is improved from 95.6% to 96.9%. In addition, the result on Amazon → Webcam obtains significant improvement, i.e., from 91.1% to 94.7%.

Shared and distinct neural representations of human-agent actions and inanimate events

Shared and distinct neural representations of human-agent actions and inanimate events

Importing participatory practices of the socio-environmental systems community to the process system engineering community: An application to supply chain

The Process System Engineering community has an extensive knowledge and skills on supply chain design: from the time dimension (production and flow planning) to the space dimension (geographic position of facilities). Nevertheless, supply chains are also social networks where multiple stakeholders have to collaborate while they have different, and sometimes, diverging objectives. For this reason, having a more realistic model representing collaboration between the various stakeholders involved is necessary and new methods that facilitate the development of a shared representation of the system must be introduced. We propose to import a participatory method, PARDI (Problematic, Actors, Resources, Dynamics and Interactions), from the Socio-Environmental System community to the practices of the Process System Engineering community. Based on this method, we develop a participatory process in order to collect the necessary knowledge on the supply chain and its context. Following this participatory process, we then develop an Agent-Based Model as a simulation and decision making tool to support collective scenario analyses and collectively draw solutions with stakeholders. Our participatory modeling approach necessarily imposes a multi stakeholders vision (within the modeling but also in the result analyses) and therefore the search for a modeling consensus. Thus, it brings a better inclusion of social aspects in problem solving which are usually poorly considered leading to implementation failure sometimes. By comparing our approach with the classic one of the Process Systems Engineering community, we highlight the strengths and weaknesses of both and how complementary they can be. A case study on the already existing supply chain of the chestnut wood in Cévennes area (France) illustrates the capabilities of our participatory methodology. It focuses on the socio-economic model design of the first two steps (forestry activities to harvest) in the supply chain as the latter is locked because of economic and social organisation issues. The objective is to find the best action levers to unlock the resistance that forest plot owners have to remove declining wood from their land.

Singular value penalization based adversarial domain adaptation for fault diagnosis of rolling bearings

Machinery fault diagnosis tasks have been well-addressed when sufficient labeled data of the tested machines are available. However, intelligent fault diagnosis methods trained with such labeled data generally fail in classifying unlabeled data due to data distribution discrepancy. To tackle this problem, a novel transfer learning method called singular value penalization adversarial network is proposed for fault diagnosis of rolling bearings, which first constructs a feature extractor to learn the shared feature representations between the source and target domains automatically, and then plays an adversarial training game between feature extractor and domain discriminator to reduce the domain shift and achieve domain adaptation. In particular, a new singular value penalization term is proposed and integrated into the domain adversarial network, which balances the distributions of largest singular values and boosts the discriminability of feature representations. The extensive transfer diagnosis experiments on two rolling bearing datasets validate the effectiveness and robustness of the proposed method, and the results show that our approach is superior to other methods under unlabeled data.

Multiview Common Subspace Clustering via Coupled Low Rank Representation

Multi-view subspace clustering (MVSC) finds a shared structure in latent low-dimensional subspaces of multi-view data to enhance clustering performance. Nonetheless, we observe that most existing MVSC methods neglect the diversity in multi-view data by considering only the common knowledge to find a shared structure either directly or by merging different similarity matrices learned for each view. In the presence of noise, this predefined shared structure becomes a biased representation of the different views. Thus, in this article, we propose a MVSC method based on coupled low-rank representation to address the above limitation. Our method first obtains a low-rank representation for each view, constrained to be a linear combination of the view-specific representation and the shared representation by simultaneously encouraging the sparsity of view-specific one. Then, it uses the k -block diagonal regularizer to learn a manifold recovery matrix for each view through respective low-rank matrices to recover more manifold structures from them. In this way, the proposed method can find an ideal similarity matrix by approximating clustering projection matrices obtained from the recovery structures. Hence, this similarity matrix denotes our clustering structure with exactly k connected components by applying a rank constraint on the similarity matrix’s relaxed Laplacian matrix to avoid spectral post-processing of the low-dimensional embedding matrix. The core of our idea is such that we introduce dynamic approximation into the low-rank representation to allow the clustering structure and the shared representation to guide each other to learn cleaner low-rank matrices that would lead to a better clustering structure. Therefore, our approach is notably different from existing methods in which the local manifold structure of data is captured in advance. Extensive experiments on six benchmark datasets show that our method outperforms 10 similar state-of-the-art compared methods in six evaluation metrics.

Imitation and interindividual differences: Belief in free will is not related to automatic imitation

It is well known that individuals have the tendency to automatically imitate each other and that such imitative behavior is fostered by perceiving intentions in others' actions. That is, past research has shown that perceiving an action as internally driven enhances the shared representation of observed and executed actions increasing automatic imitation. An interpersonal factor that increases the perception that a behavior is internally driven is belief in free will. Consequently, we hypothesized that the more individuals believe in free will, the more they automatically imitate others. To test this prediction, we conducted two high-powered (total N = 642) and preregistered studies in which we assessed automatic imitation with the imitation-inhibition task. Contrary to our predictions, belief in free will did not correlate with automatic imitation. This finding contributes to current findings challenging the assumption that automatic imitation is modulated by interindividual differences. Further theoretical implications are discussed.

Multisource domain factorization network for cross-domain fault diagnosis of rotating machinery: An unsupervised multisource domain adaptation method

Unsupervised domain adaptation (DA) provides a promising approach for tackling fault diagnosis tasks of target datasets without labeled data and has been actively studied in recent years. Most of them focus only on single-source DA, compared to multisource DA (MDA), which has remarkable advantages in generalized knowledge learning and generalization performance. Nevertheless, there are very few fault diagnosis studies based on MDA, and it remains challenging to reduce multiple domain shifts to improve diagnostic performance and mitigate negative transfer during learning. To this end, a novel unsupervised MDA-based transfer learning approach called multisource domain factorization network (MDFN) is proposed in this paper, where the generalized diagnosis knowledge is learned from multiple sources and then used for diagnosing the target task. The highlights of MDFN are that the shared-space component analysis and transferability-based entropy penalty strategy are employed to significantly mitigate negative transfer from the two levels of feature representation and instance transferability and effectively learn shared feature representation. Therefore, the MDFN can extract shared features that combine domain-invariance and discriminability, thereby performing better. The results of two experimental cases on six datasets, including cross-operating-condition and cross-component diagnosis tasks, validate the effectiveness and superiority of the proposed method.

AMFB: Attention based multimodal Factorized Bilinear Pooling for multimodal Fake News Detection

Fake news is the information or stories that are intentionally created to deceive or mislead the readers. In recent times, Fake news detection has attracted the attention of researchers and practitioners due to its many-fold benefits, including bringing in preventive measures to tackle the dissemination of misinformation that could otherwise disturb the social fabrics. Social media in recent times are heavily loaded with multimedia news and information. People prefer online news reading and find it more informative and convenient if they have access to multimedia content in the forms of text, images, audio, and videos. In early studies, researchers have proposed several fake news detection mechanisms that mostly utilize the textual features and not proper to learn multimodal (textual + visual) shared representation.To overcome these limitations, in this paper, we propose a multimodal fake news detection framework with appropriate multimodal feature fusion that leverages information from text and image and tries to maximize the correlation between them to get the efficient multimodal shared representation. We empirically show that text, when combined with the image, can improve the performance of the model. The model detects the post once it is introduced into the network in an early stage. At the early stage of a news post’s introduction into the network, the model takes the text and image of the post as input and decides whether this is fake or genuine. Since this model only analyzes news contents, It does not require any prior information regarding the user and network details. This framework has four different sub-modules viz.Attention Based Stacked Bidirectional Long Short Term Memory (ABS-BiLSTM) for textual feature representation, Attention Based Multilevel Convolutional Neural Network–Recurrent Neural Network (ABM-CNN–RNN) for visual feature extraction, multimodal Factorized Bilinear Pooling (MFB) for feature fusion and finally Multi-Layer Perceptron (MLP) for the classification. We perform experiments on two publicly available datasets, viz. Twitter and Weibo. Evaluation results show the efficacy of our proposed approach that performs significantly better compared to the state-of-the-art models. It shows to outperform the current state-of-the-art by approximately 10 points for the Twitter dataset. In contrast, the Weibo dataset achieves an overall better performance with balanced F1-scores between fake and real classes. Furthermore, the complexity of our proposed model is significantly lower than the state-of-the-art.