Shared Representation Research Articles

Majority Decision Making Works Best under Conditions of Leadership Ambiguity and Shared Task Representations

The effectiveness of decision-making teams depends largely on their ability to integrate and make sense of information. Consequently, teams which more often use majority decision making may make better quality decisions, but particularly so when they also have task representations which emphasize the elaboration of information relevant to the decision, in the absence of clear leadership. In the present study I propose that (a) majority decision making will be more effective when task representations are shared, and that (b) this positive effect will be more pronounced when leadership ambiguity (i.e. team members’ perceptions of the absence of a clear leader) is high. These hypotheses were put to the test using a sample comprising 81 teams competing in a complex business simulation for seven weeks. As predicted, majority decision making was more effective when task representations were shared, and this positive effect was more pronounced when there was leadership ambiguity. The findings extend and nuance earlier research on decision rules, the role of shared task representations, and leadership clarity.

Facial Age Estimation Using a Multi-task Network Combining Classification and Regression

Age estimation of facial images is very challenging because of the complexity of face aging process and the difficulty of collecting and labeling data. A holistic regression model is subject to imbalanced training data, while a divide-and-conquer method highly depends on the effect of the age classification, which usually has boundary effect due to cross-age correlations. This paper proposes a simple but effective multi-task learning (MTL) network combining classification and regression for age estimation called CR-MT net, where classification acts as an auxiliary task to regression. MTL can boost the generalization performance of the age regression task by shared information representation learning of the two tasks. Compared to divide-and-conquer methods, our method performs MTL for two tasks with an end-to-end training, and has no error propagation from classification to regression. The holistic regression model in CR-MT net does not meet the boundary effect, and can fit the heterogeneous or unbalanced age data more accurately with the aid of a good age data partition in classification. We evaluate two age grouping techniques to find a good data partition, and diagnose various factors which can influence the performance of the CR-MT net by extensive experiments. The CR-MT net is verified in three public datasets, and achieves state-of-art results.

Using Variational Multi-View Learning for Classification of Grocery Items

An essential task for computer vision-based assistive technologies is to help visually impaired people to recognize objects in constrained environments, for instance, recognizing food items in a grocery store. In this paper, we introduce a novel dataset with natural images of grocery items -- fruits, vegetables and packaged products -- where all images have been taken inside grocery stores to resemble an actual shopping scenario. In addition to the natural images, we download an iconic image and a text description of each item that can be utilized for constructing better representations of the grocery items. We select a multi-view generative model called Variational Canonical Correlation Analysis (VCCA), which efficiently combines the different information of the items into a single lower-dimensional representation. In the experiments, we show that utilizing the additional information with VCCA yields higher accuracies on classifying grocery items over standard image classifiers that only uses the natural images. We observe from visualizing the latent representations that the iconic images help to construct representations that are separated by the visual differences of the items, while the text descriptions enable the model to distinguish between visually similar items by their different ingredients and flavors. Moreover, we investigate a variant of VCCA called VCCA-private that separates shared and private information of the different data views. We verify that VCCA-private can separate variations in image backgrounds and structures of text sentences from the shared representation to enable a more accurate classification of grocery items in their natural environment.

Deep Air Quality Forecasting Using Hybrid Deep Learning Framework

Air quality forecasting has been regarded as the key problem of air pollution early warning and control management. In this article, we propose a novel deep learning model for air quality (mainly PM2.5) forecasting, which learns the spatial-temporal correlation features and interdependence of multivariate air quality related time series data by hybrid deep learning architecture. Due to the nonlinear and dynamic characteristics of multivariate air quality time series data, the base modules of our model include one-dimensional Convolutional Neural Networks (1D-CNNs) and Bi-directional Long Short-term Memory networks (Bi-LSTM). The former is to extract the local trend features and spatial correlation features, and the latter is to learn spatial-temporal dependencies. Then we design a jointly hybrid deep learning framework based on one-dimensional CNNs and Bi-LSTM for shared representation features learning of multivariate air quality related time series data. We conduct extensive experimental evaluations using two real-world datasets, and the results show that our model is capable of dealing with PM2.5 air pollution forecasting with satisfied accuracy.

Learning Semantic Text Features for Web Text-Aided Image Classification

The good generalization performance of conventional pattern classifiers often relies on the size of training data labeled by costly human labor. These days, publicly available web resources grow explosively, and this allows us to easily obtain abundant and cheap web data. Yet, web data are usually not as cooperative as human labeled data. In this paper, we explore the use of web text data to aid image classification. Without requiring the previous collection of auxiliary data from the web, we directly retrieve the web text information with the aid of the powerful reverse image search engine. We develop a novel textual modeling method named semantic matching neural network (SMNN) that is capable of learning semantic features from the associated text of web images. The SMNN text features have improved reliability and applicability, compared to the text features obtained from other methods. The SMNN text features and convolutional neural network (CNN) visual features are merged into a shared representation, which learns to capture the correlations between the two modalities. Experimental results on benchmark UIUC-Sports, Scene-15, Caltech-256, and Pascal VOC-2012 data sets show that the visual and text modalities of data from different sources are remarkably complementary and the fusion of them achieves substantial performance improvement.

OHMI: the ontology of host-microbiome interactions

BackgroundHost-microbiome interactions (HMIs) are critical for the modulation of biological processes and are associated with several diseases. Extensive HMI studies have generated large amounts of data. We propose that the logical representation of the knowledge derived from these data and the standardized representation of experimental variables and processes can foster integration of data and reproducibility of experiments and thereby further HMI knowledge discovery.MethodsThrough a multi-institutional collaboration, a community-based Ontology of Host-Microbiome Interactions (OHMI) was developed following the Open Biological/Biomedical Ontologies (OBO) Foundry principles. As an OBO library ontology, OHMI leverages established ontologies to create logically structured representations of (1) microbiomes, microbial taxonomy, host species, host anatomical entities, and HMIs under different conditions and (2) associated study protocols and types of data analysis and experimental results.ResultsAligned with the Basic Formal Ontology, OHMI comprises over 1000 terms, including terms imported from more than 10 existing ontologies together with some 500 OHMI-specific terms. A specific OHMI design pattern was generated to represent typical host-microbiome interaction studies. As one major OHMI use case, drawing on data from over 50 peer-reviewed publications, we identified over 100 bacteria and fungi from the gut, oral cavity, skin, and airway that are associated with six rheumatic diseases including rheumatoid arthritis. Our ontological study identified new high-level microbiota taxonomical structures. Two microbiome-related competency questions were also designed and addressed. We were also able to use OHMI to represent statistically significant results identified from a large existing microbiome database data analysis.ConclusionOHMI represents entities and relations in the domain of HMIs. It supports shared knowledge representation, data and metadata standardization and integration, and can be used in formulation of advanced queries for purposes of data analysis.

Domain adaptive representation learning for facial action unit recognition

Learning robust representations for applications with multiple modalities of input can have a significant impact on improving performance. Traditional representation learning methods rely on projecting the input modalities to a common subspace to maximize agreement amongst the modalities for a particular task. We propose a novel approach to representation learning that uses a latent representation decoder to reconstruct the target modality and thereby employ the target modality purely as a supervision signal for discovering correlations between the modalities. Through cross modality supervision, we demonstrate that the learnt representation is able to improve upon the performance of the task of facial action unit (AU) recognition over modality specific representations and even their fused counterparts. As an extension, we explore a new transfer learning technique to adapt the learnt representation to the target domain. We also present a shared representation based feature fusion methodology to improve the performance of any multi-modal system. Our experiments on three AU recognition datasets - MMSE, BP4D and DISFA, show strong performance gains producing state-of-the-art results in spite of the absence of data from a modality.

A Deep Neural Network Framework for English Hindi Question Answering

In this article, we propose a unified deep neural network framework for multilingual question answering (QA). The proposed network deals with the multilingual questions and answers snippets. The input to the network is a pair of factoid question and snippet in the multilingual environment (English and Hindi), and output is the relevant answer from the snippet. We begin by generating the snippet using a graph-based language-independent algorithm, which exploits the lexico-semantic similarity between the sentences. The soft alignment of the question words from the English and Hindi languages has been used to learn the shared representation of the question. The learned shared representation of question and attention-based snippet representation are passed as an input to the answer extraction layer of the network, which extracts the answer span from the snippet. Evaluation on a standard multilingual QA dataset shows the state-of-the-art performance with 39.44 Exact Match (EM) and 44.97 F1 values. Similarly, we achieve the performance of 50.11 Exact Match (EM) and 53.77 F1 values on Translated SQuAD dataset.

Neural Correlates of the Conscious Perception of Visual Location Lie Outside Visual Cortex.

When perception differs from the physical stimulus, as it does for visual illusions and binocular rivalry, the opportunity arises to localize where perception emerges in the visual processing hierarchy. Representations prior to that stage differ from the eventual conscious percept even though they provide input to it. Here, we investigate where and how a remarkable misperception of position emerges in the brain. This "double-drift" illusion causes a dramatic mismatch between retinal and perceived location, producing a perceived motion path that can differ from its physical path by 45° or more. The deviations in the perceived trajectory can accumulate over at least a second, whereas other motion-induced position shifts accumulate over 80-100ms before saturating. Using fMRI and multivariate pattern analysis, we find that the illusory path does not share activity patterns with a matched physical path in any early visual areas. In contrast, a whole-brain searchlight analysis reveals a shared representation in anterior regions of the brain. These higher-order areas would have the longer time constants required to accumulate the small moment-to-moment position offsets that presumably originate in early visual cortical areas and then transform these sensory inputs into a finalconscious percept. The dissociation between perception and the activity in early sensory cortex suggests that consciously perceived position does not emerge in what is traditionally regarded as the visual system but instead emerges at a higher level.

Shared Syntactic Representations in Balanced Bilinguals: Cross-linguistic Priming with and without Verb Overlap

ABSTRACTThis study investigated cross-linguistic priming in six-year-old, balanced Spanish-English bilinguals (n = 60). We examined bilinguals’ production of transitive forms in English (active, passive) after exposure to Spanish transitives (Study 1; M age = 6.2 years; SD = 0.3) and their production of transitive forms in Spanish (active, passive) after exposure to English transitives (Study 2; M age = 5.9 years; SD = 0.5). We varied whether the verb matched between the modeled prime sentences and target drawings that children described. Results revealed two-way, cross-linguistic structural priming effects, from Spanish to English as well as English to Spanish. Further, the results did not show evidence of a lexical boost in priming. These findings suggest that by school age, balanced bilingual children have generalized, shared syntactic representations and that language experience may influence the strength of the connections between these syntactic representations and lexical items.

GroundTruth: Augmenting Expert Image Geolocation with Crowdsourcing and Shared Representations

Expert investigators bring advanced skills and deep experience to analyze visual evidence, but they face limits on their time and attention. In contrast, crowds of novices can be highly scalable and parallelizable, but lack expertise. In this paper, we introduce the concept of shared representations for crowd--augmented expert work, focusing on the complex sensemaking task of image geolocation performed by professional journalists and human rights investigators. We built GroundTruth, an online system that uses three shared representations-a diagram, grid, and heatmap-to allow experts to work with crowds in real time to geolocate images. Our mixed-methods evaluation with 11 experts and 567 crowd workers found that GroundTruth helped experts geolocate images, and revealed challenges and success strategies for expert-crowd interaction. We also discuss designing shared representations for visual search, sensemaking, and beyond.

Deep Learning-Based Time-Varying Parameter Identification for System-Wide Load Modeling

The integration of uncertain power resources is causing more challenges for traditional load modeling research. Parameter identification of load modeling is impacted by a variety of load components with time-varying characteristics. This paper develops a deep learning-based time-varying parameter identification model for composite load modeling (CLM) with ZIP load and induction motor. A multi-modal long short-term memory (M-LSTM) deep learning method is used to estimate all the time-varying parameters of CLM considering system-wide measurements. It contains a multi-modal structure that makes use of different modalities of the input data to accurately estimate time-varying load parameters. An LSTM network with a flexible number of temporal states is defined to capture powerful temporal patterns from the load parameters and measurements time series. The extracted features are further fed to a shared representation layer to capture the joint representation of input time series data. This temporal representation is used in a linear regression model to estimate time-varying load parameters at the current time. Numerical simulations on the 23- and 68-bus systems verify the effectiveness and robustness of the proposed M-LSTM method. Also, the optimal lag values of parameters and measurements as input variables are solved.

Multi-indices quantification of optic nerve head in fundus image via multitask collaborative learning.

Multi-indices quantification of optic nerve head (ONH), measuring ONH appearance with multiple types of indices simultaneously from fundus images, is the most clinically significant tasks for accurate ONH assessment and ophthalmic disease diagnosis. However, no attempt has been reported due to its challenges of the large variation of fundus appearance across patients, heavy overlap and extremely weak contrast between optic nerve head areas. In this paper, we propose a multitask collaborative learning framework (MCL-Net) for multi-indices ONH quantification. The proposed MCL-Net, a two-branch neural network, first obtains expressive shared and task-specific representations with the backbone network and its two branches; then models the feature exchanges and aggregations between two branches with a well-designed feature interaction module (FIM) to promote each other collaboratively. After that, it estimates multiple types of ONH indices under a multitask ensemble module (MEM) that is capable of learning aggregation of multiple outputs automatically. Therefore, the proposed MCL-Net is consisted of the feature representation, inter-task feature interaction, dual-branch task-specific prediction, and multitask quantification ensemble, which establish an effective framework which takes full advantages of segmentation and estimation tasks for multi-indices ONH quantification. Rather than the low-level feature sharing and individual prediction, the proposed MCL-Net collaboratively learns an optimal combination of shared and task-specific representation, as well as the aggregated prediction, therefore leads to accurate quantification of ONH with multiple types of indices. Experimental results on the dataset of 650 fundus images show that MCL-Net successfully delivers accurate quantification of all the three types of ONH indices, with average mean absolute error of 0.98 ± 0.20, 0.97 ± 0.16, 1.19 ± 0.18, as well as average correlation coefficient of 0.699, 0.708 and 0.691, for diameters, whole areas and regional areas, respectively. In addition, the experiments demonstrate that quantitative indices obtained by our method provide more effective glaucoma diagnosis with AUC of 0.8698. This endows our proposed MCL-Net a great potential in clinical assessment from focal to global for ophthalmic disease diagnosis.

Performing Multi-Target Regression via a Parameter Sharing-Based Deep Network.

Multi-target regression (MTR) comprises the prediction of multiple continuous target variables from a common set of input variables. There are two major challenges when addressing the MTR problem: the exploration of the inter-target dependencies and the modeling of complex input-output relationships. This paper proposes a neural network model that is able to simultaneously address these two challenges in a flexible way. A deep architecture well suited for learning multiple continuous outputs is designed, providing some flexibility to model the inter-target relationships by sharing network parameters as well as the possibility to exploit target-specific patterns by learning a set of nonshared parameters for each target. The effectiveness of the proposal is analyzed through an extensive experimental study on 18 datasets, demonstrating the benefits of using a shared representation that exploits the commonalities between target variables. According to the experimental results, the proposed model is competitive with respect to the state-of-the-art in MTR.

CAI4CAI: The Rise of Contextual Artificial Intelligence in Computer Assisted Interventions.

Data-driven computational approaches have evolved to enable extraction of information from medical images with reliability, accuracy, and speed, which is already transforming their interpretation and exploitation in clinical practice. While similar benefits are longed for in the field of interventional imaging, this ambition is challenged by a much higher heterogeneity. Clinical workflows within interventional suites and operating theaters are extremely complex and typically rely on poorly integrated intraoperative devices, sensors, and support infrastructures. Taking stock of some of the most exciting developments in machine learning and artificial intelligence for computer-assisted interventions, we highlight the crucial need to take the context and human factors into account in order to address these challenges. Contextual artificial intelligence for computer-assisted intervention (CAI4CAI) arises as an emerging opportunity feeding into the broader field of surgical data science. Central challenges being addressed in CAI4CAI include how to integrate the ensemble of prior knowledge and instantaneous sensory information from experts, sensors, and actuators; how to create and communicate a faithful and actionable shared representation of the surgery among a mixed human–AI actor team; and how to design interventional systems and associated cognitive shared control schemes for online uncertainty-aware collaborative decision-making ultimately producing more precise and reliable interventions.

Dynamic multimodal drawing in school: Exploring technology support of drawing skills development in children with visual impairments

BACKGROUND: Technology is widely used in school to assist students with visual impairments. Drawing with traditional methods is still difficult for those users. In this paper, we present and discuss an evaluation of a haptic and sound interactive drawing program (HIPP). OBJECTIVE: The aim of the st udy was to identify what aspects of the technological aid support the drawing skill development in children with visual impairment. METHODS: Interviews, observations and video recordings of use situations were used for data gathering. Our analysis is based on cultural-historical activity theory, and examines the mediation between the child and the object of their activity, their drawings, as well as the roles of teachers, classmates, assistants, and family. RESULTS: The haptic and audio drawing program supports the steps (doodling, interpretation, planning, and more intentional drawing) observed in visually-acquired drawing, although possible improvements have also been identified. Observations of the drawing program in use showed that its mix of dynamic multimodal interaction and a stable drawing feedback, enable visually impaired students and sighted teachers to jointly access a shared representation. CONCLUSIONS: Successful long-term use, together with the presented results of our analysis show how multimodal dynamic and stable interaction can successfully support drawing activities.

How can I become a member of my culture?: shared representations of community-related moral violation in Japan and the U.S.

Morality not only serves as the standard of right and wrong behavior, it also plays a central role in people’s everyday social interactions. The current study examined how mental representations of morality violations are shared across Western and non-Western cultures, utilizing Shweder’s three ethics of Autonomy, Community, and Divinity. First, Japanese and American undergraduates were instructed on the definitions of these three ethic domains and were asked to freely describe any situations they could think of concerning violations in each domain. A subset of these descriptions of violations that correctly matched Shweder’s original definitions were then given to a separate sample of Japanese and American college students, who then categorized each situation into one of the three domains and rated its wrongness. The results showed that Autonomy-violating situations resulted in a greater number of “errors” (i.e., classified into an unintended domain) compared to the other ethical violations, regardless of the place of origin for both Japanese and American participants. In addition, both Japanese and American participants were generally more accurate when categorizing home-generated (vs. foreign-generated) situations, particularly when they involved the Community ethic. On average, participants gave more intense ratings of wrongness to situations generated by members of their same culture. The own-culture advantage was most consistent for the Community ethic. Violations of the Community ethic may have the most robust and shared within-group interpretation because this promotes an adaptive fit into one’s own culture.

A common representation of fingers and toes

There are many similarities and differences between the human hands and feet. On a psychological level, there is some evidence from clinical disorders and studies of tactile localisation in healthy adults for deep functional connections between the hands and feet. One form these connections may take is in common high-level mental representations of the hands and feet. Previous studies have shown that there are systematic, but distinct patterns of confusion found between both the fingers and toes. Further, there are clear individual differences between people in the exact patterns of mislocalisations. Here, we investigated whether these idiosyncratic differences in tactile localisation are shared between the fingers and toes, which may indicate a shared high-level representation. We obtained confusion matrices showing the pattern of mislocalisation on the hairy skin surfaces of both the fingers and toes. Using a decoding approach, we show that idiosyncratic differences in individuals' pattern of confusions are shared across the fingers and toes, despite different overall patterns of confusions. These results suggest that there is a common representation of the fingers and toes.

Materiality of Attention: How Selective Use of Materiality Sustains Designing Shared Representations

Research on materiality has increasingly drawn attention to how it assists communicative and cognitive processes of group work by highlighting the role materiality plays as material infrastructure,...

Increased similarity of neural responses to experienced and empathic distress in costly altruism

Empathy—affective resonance with others’ sensory or emotional experiences—is hypothesized to be an important precursor to altruism. However, it is not known whether real-world altruists’ heightened empathy reflects true self-other mapping of multi-voxel neural response patterns. We investigated this relationship in adults who had engaged in extraordinarily costly real-world altruism: donating a kidney to a stranger. Altruists and controls completed fMRI testing while anticipating and experiencing pain, and watching as a stranger anticipated and experienced pain. Machine learning classifiers tested for shared representation between experienced and observed distress. Altruists exhibited more similar representations of experienced and observed fearful anticipation spontaneously and following an empathy prompt in anterior insula and anterior/middle cingulate cortex, respectively, suggesting heightened empathic proclivities and abilities for fear. During pain epochs, altruists were distinguished by spontaneous empathic responses in anterior insula, anterior/mid-cingulate cortex and supplementary motor area, but showed no difference from controls after the empathy prompt. These findings (1) link shared multi-voxel representations of the distress of self and others to real-world costly altruism, (2) reinforce distinctions between empathy for sensory states like pain and anticipatory affective states like fear, and (3) highlight the importance of differentiating between the proclivity and ability to empathize.