Common Representation Research Articles

Agreeing to Disagree: Translating Representations to Uncover a Unified Representation for Social Robot Actions

Researchers and designers of social robots often approach robot control system design from a single perspective; such as designing autonomous robots, teleoperated robots, or robots programmed by an end-user. While each design approach presents a tradeoff between some advantages and limitations, there is an opportunity to integrate these approaches where people benefit from the best-fit approach for their use case. In this work, we propose integrating these seemingly distinct robot control approaches to uncover a common data representation of social actions defining social expression by a robot. We demonstrate the value of integrating an authoring system, teleoperation interface, and robot planning system by integrating instances of these systems for robot storytelling. By relying on an integrated system, domain experts can define behaviors through end-user interfaces that teleoperators and autonomous robot programmers can use directly thus providing a cohesive expert-driven robot system.

A temporal–spectral fusion transformer with subject-specific adapter for enhancing RSVP-BCI decoding

The Rapid Serial Visual Presentation (RSVP)-based Brain–Computer Interface (BCI) is an efficient technology for target retrieval using electroencephalography (EEG) signals. The performance improvement of traditional decoding methods relies on a substantial amount of training data from new test subjects, which increases preparation time for BCI systems. Several studies introduce data from existing subjects to reduce the dependence of performance improvement on data from new subjects, but their optimization strategy based on adversarial learning with extensive data increases training time during the preparation procedure. Moreover, most previous methods only focus on the single-view information of EEG signals, but ignore the information from other views which may further improve performance. To enhance decoding performance while reducing preparation time, we propose a Temporal-Spectral fusion transformer with Subject-specific Adapter (TSformer-SA). Specifically, a cross-view interaction module is proposed to facilitate information transfer and extract common representations across two-view features extracted from EEG temporal signals and spectrogram images. Then, an attention-based fusion module fuses the features of two views to obtain comprehensive discriminative features for classification. Furthermore, a multi-view consistency loss is proposed to maximize the feature similarity between two views of the same EEG signal. Finally, we propose a subject-specific adapter to rapidly transfer the knowledge of the model trained on data from existing subjects to decode data from new subjects. Experimental results show that TSformer-SA significantly outperforms comparison methods and achieves outstanding performance with limited training data from new subjects. This facilitates efficient decoding and rapid deployment of BCI systems in practical use.

Intellectual Differences Between Boys and Girls, 35 Years of Evolution in France from WISC-R to WISC-V

The French adaptation of the Wechsler Intelligence Scale of Children, 5th edition (WISC-V) was an opportunity to examine if some common representations of gender differences in intellectual abilities are supported by empirical evidence. The WISC-V standardization sample provided data on a wide range of cognitive tests in a large sample of 6- to 16-year-old children representative of the French population. This sample included 517 boys and 532 girls. The WISC-V data were compared to those of the French standardization samples of three previous versions of the WISC (WISC-R, WISC-III, and WISC-IV). These four standardization samples span a 35-year period. The data analysis of the WISC-V standardization sample and the three previous versions of this intelligence scale showed that the performance gaps on intellectual tests between girls and boys have gradually narrowed over time. Almost no gender differences were observed in the WISC-V standardization sample, not only in IQ but also in key facets of intelligence. Data do not support the stereotype that girls are better at verbal tasks and boys are better at visuospatial tasks. However, some statistically significant differences remain, but the magnitude was generally small with no practical implications. The only important difference is in favor of girls and concerns performance on processing speed tasks that require visual discrimination, attentional control, and writing.

Representation of Anticipated Rewards and Punishments in the Human Brain.

Subjective value is a core concept in neuroeconomics, serving as the basis for decision making. Despite the extensive literature on the neural encoding of subjective reward value in humans, the neural representation of punishment value remains relatively understudied. This review synthesizes current knowledge on the neural representation of reward value, including methodologies, involved brain regions, and the concept of common currency representation of diverse reward types in decision-making and learning processes. We then critically examine existing research on the neural representation of punishment value, highlighting conceptual and methodological challenges in human studies and insights gained from animal research. Finally, we explore how individual differences in reward and punishment processing may be linked to various mental illnesses, with a focus on stress-related psychopathologies. This review advocates for the integration of both rewards and punishments within value-based decision-making and learning frameworks, leveraging insights from cross-species studies and utilizing ecological gamified paradigms to reflect real-life scenarios.

Importance of Orbital Invariance in Quantifying Electron-Hole Separation and Exciton Size.

A fundamental tenet of quantum mechanics is that properties should be independent of representation. In self-consistent field methods such as density functional theory, this manifests as a requirement that properties be invariant with respect to unitary transformations of the occupied molecular orbitals and (separately) the unoccupied molecular orbitals. Various ad hoc measures of excited-state charge separation that are commonly used to analyze time-dependent density-functional calculations violate this requirement, as they are based on incoherent averages of excitation amplitudes rather than expectation values involving coherent superpositions. As a result, these metrics afford markedly different values in various common representations, including canonical molecular orbitals, Boys-localized orbitals, and natural orbitals. Numerical values can be unstable with respect to basis-set expansion and may afford nonsensical results in the presence of extremely diffuse basis functions. In contrast, metrics based on well-defined expectation values are stable, representation-invariant, and physically interpretable. Use of natural transition orbitals improves the stability of the incoherent averages, but numerical values can only be interpreted as expectation value in the absence of superposition. To satisfy this condition, the particle and hole density matrices must each be dominated by a single eigenvector so that the transition density is well described by a single pair of natural transition orbitals. Counterexamples are readily found where this is not the case. Our results suggest that ad hoc charge-transfer diagnostics should be replaced by rigorous expectation values, which are no more expensive to compute.

Deep coupled registration and segmentation of multimodal whole-brain images.

Recent brain mapping efforts are producing large-scale whole-brain images using different imaging modalities. Accurate alignment and delineation of anatomical structures in these images are essential for numerous studies. These requirements are typically modeled as two distinct tasks: Registration and segmentation. However, prevailing methods, fail to fully explore and utilize the inherent correlation and complementarity between the two tasks. Furthermore, variations in brain anatomy, brightness and texture pose another formidable challenge in designing multi-modal similarity metrics. A high-throughput approach capable of overcomingthe bottleneck of multi-modal similarity metric design, while effective leveraging the highly correlated and complementary nature of two tasks is highly desirable. We introduce a deep learning framework for joint registration and segmentation of multi-modal brain images. Under this framework, registration and segmentation tasks are deeply coupled and collaborated at two hierarchical layers. In the inner layer, we establish a strong feature-level coupling between the two tasks by learning a unified common latent feature representation. In the outer layer, we introduce a mutually supervised dual-branch network to decouple latent features and facilitate task-level collaboration between registration and segmentation. Since the latent features we designed are also modality-independent, the bottleneck of designing multi-modal similarity metric is essentially addressed. Another merit offered by this framework is the interpretability of latent features, which allows intuitive manipulation of feature learning, thereby further enhancing network training efficiency and the performance of both tasks. Extensive experiments conducted on both multi-modal and mono-modal datasets of mouse and human brains demonstrate the superiority of our method. The code is available at https://github.com/tingtingup/DCRS. Supplementary data are available at Bioinformaticsonline.

How data factor flow affects corporate pollution reduction: The environmental effects of digital empowerment

This study the theoretical hypotheses by building a relatively exogenous data factor flow indicator and a corporate pollution emission intensity indicator system using China's data flow restriction policy and corporate pollution emission data. Data factor is emerging as an essential new factor of production in today's digital economy, and data cannot function valuablely without an efficient flow. This study constructs a theoretical model of the influence of data factor flow on the pollution emissions of downstream manufacturing enterprises based on the vertical correlation of inputs and outputs. Additionally, we employ additional contaminants as a robustness test and sulfur dioxide, a common representation of air pollution, as the baseline. The empirical analysis's findings indicate that: (1) Data factor flow significantly reduces pollution in manufacturing enterprises. (2) The main way that data factor flow helps organizations minimize pollution is by increasing their productivity and ability to innovate in technology. (3) Domestic data flow is more important than cross-border data flow in helping industrial companies reduce pollution. (4) In general, foreign-owned enterprises have a greater need for data factor flow to reduce pollution than do state-owned and private businesses.

Alpha coherence is a network signature of cognitive recovery from disorders of consciousness.

Alpha (8-12 Hz) frequency band oscillations are among the most informative features in electroencephalographic (EEG) assessment of patients with disorders of consciousness (DoC). Because interareal alpha synchrony is thought to facilitate long-range communication in healthy brains, coherence measures of resting-state alpha oscillations may provide insights into a patients capacity for higher-order cognition beyond channel-wise estimates of alpha power. In multi-channel EEG, global coherence methods may be used to augment standard spectral analysis methods by both estimating the strength and identifying the structure of coherent oscillatory networks. We performed global coherence analysis in 95 separate clinical EEG recordings (28 healthy controls and 33 patients with acute or chronic DoC, 25 of whom returned for follow-up) collected between two academic medical centers. We found that posterior alpha coherence is associated with recovery of higher-level cognition. We developed a measure of network organization, based on the distance between eigenvectors of the alpha cross-spectral matrix, that detects recovery of posterior alpha networks. In patients who have emerged from a minimally conscious state, we showed that coherence-based alpha networks are reconfigured prior to restoration of alpha power to resemble those seen in healthy controls. This alpha network measure performs well in classifying recovery from DoC (AUC = 0.78) compared to common representations of functional connectivity using the weighted phase lag index (AUC = 0.50 - 0.57). Lastly, we observed that activity within these alpha networks is suppressed during positive responses to task-based EEG command-following paradigms, supporting the potential utility of this biomarker to detect covert cognition. Our findings suggest that restored alpha networks may represent a sensitive early signature of cognitive recovery in patients with DoC. Therefore, network detection methods may augment the utility of EEG assessments for DoC.

CoolerSpace: A Language for Physically Correct and Computationally Efficient Color Programming

Color programmers manipulate lights, materials, and the resulting colors from light-material interactions. Existing libraries for color programming provide only a thin layer of abstraction around matrix operations. Color programs are, thus, vulnerable to bugs arising from mathematically permissible but physically meaningless matrix computations. Correct implementations are difficult to write and optimize. We introduce CoolerSpace to facilitate physically correct and computationally efficient color programming. CoolerSpace raises the level of abstraction of color programming by allowing programmers to focus on describing the logic of color physics. Correctness and efficiency are handled by CoolerSpace. The type system in CoolerSpace assigns physical meaning and dimensions to user-defined objects. The typing rules permit only legal computations informed by color physics and perception. Along with type checking, CoolerSpace also generates performance-optimized programs using equality saturation. CoolerSpace is implemented as a Python library and compiles to ONNX, a common intermediate representation for tensor computations. CoolerSpace not only prevents common errors in color programming, but also does so without run-time overhead: even unoptimized CoolerSpace programs out-perform existing Python-based color programming systems by up to 5.7 times; our optimizations provide up to an additional 1.4 times speed-up.

Release the Krakencoder: A unified brain connectome translation and fusion tool.

Brain connectivity can be estimated in many ways, depending on modality and processing strategy. Here we present the Krakencoder, a joint connectome mapping tool that simultaneously, bidirectionally translates between structural (SC) and functional connectivity (FC), and across different atlases and processing choices via a common latent representation. These mappings demonstrate unprecedented accuracy and individual-level identifiability; the mapping between SC and FC has identifiability 42-54% higher than existing models. The Krakencoder combines all connectome flavors via a shared low-dimensional latent space. This "fusion" representation i) better reflects familial relatedness, ii) preserves age- and sex-relevant information and iii) enhances cognition-relevant information. The Krakencoder can be applied without retraining to new, out-of-age-distribution data while still preserving inter-individual differences in the connectome predictions and familial relationships in the latent representations. The Krakencoder is a significant leap forward in capturing the relationship between multi-modal brain connectomes in an individualized, behaviorally- and demographically-relevant way.

Privately evaluating sensitive population record linkage without ground truth data

AbstractData containing personal information allow detailed studies in the health and social sciences, such as population-related analysis. However, such studies often require the linking of two or more databases because information about a person can be scattered across multiple data sources. To address this issue of data being scattered, researchers have been working on linking records across multiple data sources to identify records that refer to the same person, or the same group of individuals (known as group linkage) using quasi-identifiers such as names and addresses which can be missing, out of date or contain errors or variations, making record linkage a very challenging task. Record linkage applications often also lack ground truth data in the form of matching and non-matching record pairs, which challenges the assessment of the quality of linkage algorithms. Furthermore, when linkage is conducted on sensitive data, for example personal health records, due to privacy concerns ground truth can generally not be obtained using methods such as crowd sourcing. This study therefore aims to develop methods to assess the linkage quality of sensitive data by using publicly available data sets, such as census or voter data, in a privacy-preserving manner, with a focus on the group linkage problem. Assuming that distinct groups, such as siblings in a family, are identifiable in both the sensitive and public data sets, we develop a novel method to estimate linkage quality using public data by encoding information that is commonly available in both sensitive and public data sets into a common representation using Bloom filters. Comparing these Bloom filters then allows the estimation of linkage quality. An evaluation using a real sensitive birth data set and a public census data set from Scotland shows the effectiveness of our proposed method for quality estimation, which achieves a median correlation of 98% with linkage quality calculated based on ground truth data.

Generative learning-based lightweight MRI brain tumor segmentation with missing modalities

Accurate segmentation of brain tumors across multiple MRI modalities is crucial for clinical diagnosis and prognosis. However, encountering missing modalities due to practical factors is common. Recent years have witnessed the emergence of various approaches, including 3D convolutional and transformer-based models, showcasing impressive performance in addressing this challenge. Nonetheless, these methods often suffer from high GPU memory requirements during inference, limiting their applicability. To explore alternative solutions, we turned to 2D-convolution-based techniques. Generative learning-based methods have produced visually appealing reconstructed MRI images, yet few consider the gap between visualization and segmentation. Drawing inspiration from clinicians who leverage cross-modality information for tumor estimation, we adopt a generative modality reconstruction task for pre-training, followed by fine-tuning for segmentation on MRI data with missing modalities. Furthermore, our investigation revealed common feature representations shared between reconstruction and segmentation tasks in latent space, as demonstrated through feature distributions. By utilizing a multi-encoder Unet architecture with fine-tuned adjustments, our method not only attains competitive performance in segmentation but also markedly decreases GPU memory usage during inference.

VIRTUAL REALITY & CURRENT PRACTICES (AN INTRODUCTION TO WIDAD FAROKAM)

This concept paper is a discussion that emphasizes the virtual reality support idea. Widad FAROKAM is an application based on the system that is compatible with the website surfed by mobile or desktop. The future target user of this system will rehearse the attention of completing the task given and divert their attachment growth by the involvement of a guardian within the task target. Still, the research must explore certain aspects which indeed can make up several issues arise. Widad FAROKAM’ integrated system is based on the currently developed product of R&D named Widad Psychetra Door (WPD). Through the investigation and feasibility study from the same research in a previous paper published on Widad FAROKAM & WPD, the common representation arises from the fourth phase of this R&D product. This fourth phase involved virtual reality.

The Operation of Moe: A Discourse Practice of Cyber Nationalism

Unlike the previous cyber-nationalism events characterized by anger and hatred, the "Panda Ya Ya" incident highlights a special form of "soft emotion." This paper focuses on the common representation of this soft emotion in both self-media and official media discourse practices: moe. Through Fairclough's discourse analysis, the specific mechanisms of its operation are explored to understand the emotional motivations in cyber nationalism. The study finds that at the levels of representation and style, the "moe discourse" in the Ya Ya incident is co-created by self-media and official media as: floating factual evidence, repeatedly performed legitimacy, and normalized political figures. However, there are characteristic differences in "time-space representation" and register between the two. The operation of moe discourse is premised on human participation, with the relationship between moe objects and humans essentially being "agented agency." Finally, the soft emotion manifested by moe discourse exhibits more significant network characteristics, relying on more universal human values, and more closely resembling the fulfillment of emotional needs. This can be viewed as a new, more powerful "commonality" resource in the development of cyber-nationalism. However, the operation of soft emotions in cyber-nationalism often remains in a state of "unfinished" without a directed outcome, becoming a kind of "emotional machine running idle."

‘Women Bleed in Private’ / Miscarriage Goes Public: A Relational Response to Social Silence over Miscarriage in Sarah Ruhl’s Writings

This article discusses Sarah Ruhl’s miscarriage experience in her autobiographical work Smile (2022) and in a selection of poems from 44 Poems for You (2020). The discussion considers Ruhl’s account as quest narratives in which she voices what she considers a tragic event in her life to underline the need to overcome the usual social silence that reigns over miscarriage. Ruhl’s prose and lyrical approaches to her miscarriage are both physical and emotional, but also social, cultural and political. Her accounts help to destigmatize this loss and to hold control over her own experience in ways that transcend the social constructions of how to experience miscarriage. As argued, Ruhl’s approach is that of a communicative dyadic body that informs us on the signals of miscarriage, of her personal experience, and what she did to get pregnant again and reconcile with this loss. Beyond the medical usefulness of her work, through her writings, Ruhl invites readers to question the social and cultural assumptions surrounding miscarriage. And in doing so, she helps women who have experienced miscarriage to regain confidence in their own bodies and the strength to revisit and fight against the common conception and representation of miscarriage as merely a physical and emotional wasteland.

Examining the Equivalence Between Imagery and Execution—Does Imagery Comprise the Intended Spatial Trajectory?

The functional equivalence model suggests a common internal representation initiates both imagery and execution. This suggestion is supported by the mental chronometry effect, where there is a positive relation between task difficulty (as defined by the Index of Difficulty; ID) and imagined movement time. The present study extends this logic by examining whether imagery captures the spatial trajectory. Participants were initially tasked with the imagery and execution of a rapid aiming movement under different IDs. These initial attempts were adapted to configure auditory tones at early (25%) and late (75%) intervals for a separate set of imagery trials. If a tone had sounded, participants had to estimate post-trial where their imagined limb would have been located. The findings revealed increases in ID that coincided with increases in imagined and executed movement times. However, participant mean and standard deviation of estimated locations revealed limited differences between the early and late tones. Further inspection revealed some evidence for these estimated locations shifting further along in space following more rapid imagined movements. While equivalence is clearly evident within the temporal domain, there is comparatively little to suggest that this logic extends to the resolution required for simulating the spatial characteristics of movement.

Sensor-type agnostic heat detection in dairy cows using multi-autoencoders with shared latent space

Monitoring heat events in dairy cows is crucial for determining the heat on time, and the heat events have usually been estimated using machine learning on cow behavioral data collected from wireless activity sensors recently. However, ensuring robust performance of heat detection is difficult because of the difference in data domains (e.g., sensor types) and insufficient heat-labeled data. Therefore, this study proposes a multi-autoencoder-based heat detection in dairy cows that can represent the common representation of cow behavior across the different sensors. The proposed method can train a sensor-type agnostic heat detector using entire labeled data from the two different sensor types by aligning the latent spaces for two sensors. In addition, our approach can train the model by combining anomaly detection and weakly supervised classification to improve the performance of heat detection that can reduce the dependency on label accuracy. The results showed that the proposed approach improved cow heat detection performance by approximately 46 % than independently trained autoencoders, and the average F1-score increased by up to 0.70. The proposed method also outperformed other supervised and unsupervised learning models in heat detection using our dataset. From the results, our model effectively estimates cow behaviors by integrating sensor modalities, thereby enhancing data capabilities in low-resource settings. This study can be key for addressing the detection discrepancy in time series data based on the location of the mounted sensor, and offers the advantage of practical applications to various activity sensors currently used on farms.

A framework for the emergence and analysis of language in social learning agents

Neural systems have evolved not only to solve environmental challenges through internal representations but also, under social constraints, to communicate these to conspecifics. In this work, we aim to understand the structure of these internal representations and how they may be optimized to transmit pertinent information from one individual to another. Thus, we build on previous teacher-student communication protocols to analyze the formation of individual and shared abstractions and their impact on task performance. We use reinforcement learning in grid-world mazes where a teacher network passes a message to a student to improve task performance. This framework allows us to relate environmental variables with individual and shared representations. We compress high-dimensional task information within a low-dimensional representational space to mimic natural language features. In coherence with previous results, we find that providing teacher information to the student leads to a higher task completion rate and an ability to generalize tasks it has not seen before. Further, optimizing message content to maximize student reward improves information encoding, suggesting that an accurate representation in the space of messages requires bi-directional input. These results highlight the role of language as a common representation among agents and its implications on generalization capabilities.

Occipital lobe epilepsy surgery: treating seizures in the posterior cortex.

Occipital lobe epilepsy (OLE) is an uncommon type of extratemporal epilepsy constituting roughly 2-13% of symptomatic partial epilepsies and epilepsy surgery cases. Over two-thirds of patients with OLE present with two characteristics: (1) ictal semiology compatible with an occipital seizure focus (e.g., ictal blindness, visual perceptual disturbance, eye blinking, nystagmus), and (2) lateralizing features referable to the posterior cortex (e.g., visual field defects, contralateral head deviation). The remaining one-third of patients present with ≥ 2 seizure types, indicative of spread to other lobes. A common representation of this cortical spread is the altered mental status and generalized tonic-clonic activity seen in patient with OLE. While the key clinical symptoms include visual hallucinations, it may be difficult to elicit on history, especially from children, and are not always present.

Exploring corpus-invariant emotional acoustic feature for cross-corpus speech emotion recognition

Unsupervised cross-corpus speech emotion recognition (SER) is the task where the labeled training (source) and unlabeled testing (target) speech come from different corpora. Subspace transfer learning is one of the mainstream technologies for tackling cross-corpus SER challenges, which is achieved by utilizing projection matrices to learn common corpus-invariant feature representations between the source and target corpus. However, these methods mainly focus on mapping modeling from the input low-level descriptor (LLD) feature space to the corpus-invariant emotional feature space, which is an implicit feature mapping process that lacks interpretability for the selected features. This omission leads to an inability to pinpoint which acoustic features possess corpus invariance. To bridge this gap, we first propose a new transfer subspace learning framework with feature selection capabilities, i.e., the Corpus-Invariant Emotional Acoustic Feature Seeker (CAFS). Specifically, the CAFS integrates two core terms into the transfer regression loss function: (1) the emotion preservation term: This term includes emotional regression and the l2,1 norm, which is mainly used to select features and ensure that these features are related to emotions. (2) the corpus invariance preservation term: This item is mainly used to measure the difference in feature distribution between the source and target corpora. Minimizing this term bridges the gap between the source and target domains, ensuring that the chosen acoustic features are corpus-invariant. Subsequently, we conducted extensive cross-corpus SER experiments to explore corpus-invariant emotional acoustic features under various commonly used acoustic feature sets (IS09 and eGeMAPS). Through statistical analysis of the acoustic features sought by the CAFS framework, some acoustic features (e.g., Mel-Frequency Cepstral Coefficients (MFCC) and Formant) reveal their corpus-invariant properties, which could provide insights for feature selection in cross-corpus SER. These findings also lay the groundwork for a solid theoretical and empirical foundation for future research and applications in cross-corpus SER.