Space Representation Research Articles

Reinforcement learning as a robotics-inspired framework for insect navigation: from spatial representations to neural implementation.

Bees are among the master navigators of the insect world. Despite impressive advances in robot navigation research, the performance of these insects is still unrivaled by any artificial system in terms of training efficiency and generalization capabilities, particularly considering the limited computational capacity. On the other hand, computational principles underlying these extraordinary feats are still only partially understood. The theoretical framework of reinforcement learning (RL) provides an ideal focal point to bring the two fields together for mutual benefit. In particular, we analyze and compare representations of space in robot and insect navigation models through the lens of RL, as the efficiency of insect navigation is likely rooted in an efficient and robust internal representation, linking retinotopic (egocentric) visual input with the geometry of the environment. While RL has long been at the core of robot navigation research, current computational theories of insect navigation are not commonly formulated within this framework, but largely as an associative learning process implemented in the insect brain, especially in the mushroom body (MB). Here we propose specific hypothetical components of the MB circuit that would enable the implementation of a certain class of relatively simple RL algorithms, capable of integrating distinct components of a navigation task, reminiscent of hierarchical RL models used in robot navigation. We discuss how current models of insect and robot navigation are exploring representations beyond classical, complete map-like representations, with spatial information being embedded in the respective latent representations to varying degrees.

Load Frequency Optimal Active Disturbance Rejection Control of Hybrid Power System

The widespread adoption of the power grid has led to increased attention to load frequency control (LFC) in power systems. The LFC strategy of multi-source hybrid power systems, including hydroelectric generators, Wind Turbine Generators (WTGs), and Photovoltaic Generators (PVGs), with thermal generators is more challenging. Existing methods for LFC tasks pose challenges in achieving satisfactory outcomes in hybrid power systems. In this paper, a novel method for the multi-source hybrid power system LFC task by using an optimal active disturbance rejection control (ADRC) strategy is proposed, which is based on the combination of the improved linear quadratic regulator (LQR) and the ADRC controller. Firstly, an established model of a hybrid power system is presented, which incorporates multiple regions and multiple sources. Secondly, utilizing the state space representation, a novel control strategy is developed by integrating improved LQR and ARDC. Finally, a series of comparative simulation experiments has been conducted using the Simulink model. Compared with the LQR with ESO, the maximum relative error of the maximum peaks of frequency deviation and tie-line exchanged power of the hybrid power system is reduced by 96% and 83%, respectively, by using the proposed strategy. The experimental results demonstrate that the strategy proposed in this paper exhibits a substantial enhancement in control performance.

So(3) ⊂ su(3) revisited

This paper reproduces the result of Elliot, namely that the irreducible finite dimensional representation of the Lie algebra su(3) of highest weight (m, n) is decomposed according to the embedding so(3) ⊂ su(3). First, a realisation (a representation in terms of vector fields) of the Lie algebra su(3) is constructed on a space of polynomials of three variables. The special polynomial basis of the representation space is given. In this basis, we find the highest weight vectors of the representation of the Lie subalgebra so(3) and in this way the representation space is decomposed to the direct sum of invariant subspaces. The process is illustrated by the example of the decomposition of the representation of highest weight (2, 2). As an additional result, the generating function of the decomposition is given.

An ecological approach to understanding transitions and tensions in complex learning contexts

The move away from transmission-based lecturing toward a more student-centred active learning approach is well evidenced in STEM higher education. However, the examination of active learning has generally remained confined to formal timetabled contexts, with assumptions made that students independently manage the transition between timetabled and non-timetabled learning. This paper introduces research findings from a mixed methods study that used an ecological approach when investigating student transitions between a formal lecture theatre and adjacent informal breakout space in a UK STEM university. Using quantitative occupancy monitoring data to analyse usage patterns of both spaces, in combination with qualitative ethnographic observations and field interviews, permitted a purposeful exploration of student engagement with transitions within and between the two learning spaces. The ecological approach aided the discovery of spatial, pedagogic and agentic transitions and tensions, which subsequently informed strategic modification of space across the institution to facilitate the adoption of active learning pedagogy.

Agglomerator++: Interpretable part-whole hierarchies and latent space representations in neural networks

Deep neural networks achieve outstanding results in a large variety of tasks, often outperforming human experts. However, a known limitation of current neural architectures is the poor accessibility in understanding and interpreting the network’s response to a given input. This is directly related to the huge number of variables and the associated non-linearities of neural models, which are often used as black boxes. This lack of transparency, particularly in crucial areas like autonomous driving, security, and healthcare, can trigger skepticism and limit trust, despite the networks’ high performance. In this work, we want to advance the interpretability in neural networks. We present Agglomerator++, a framework capable of providing a representation of part-whole hierarchies from visual cues and organizing the input distribution to match the conceptual-semantic hierarchical structure between classes. We evaluate our method on common datasets, such as SmallNORB, MNIST, FashionMNIST, CIFAR-10, and CIFAR-100, showing that our solution delivers a more interpretable model compared to other state-of-the-art approaches. Our code is available at https://mmlab-cv.github.io/Agglomeratorplusplus/.

Inclusivity in Physical and Digital Theater Environments

Today, there is a problem with the organization of inclusivity in theatrical institutions, which becomes very significant for groups of people with increased needs, when the lack of certain solutions and the inability, for one reason or another, directly affects the attendance of people with limited mobility.Purpose: The aim of this work is to explore the possibility and methods of organizing inclusivity in theatrical space using up-to-date technologies such as VR, AR, metauniverse, etc. Methodology: The analysis of modern research projects related to reorientation of design and construction to the inclusivity and well-being of the architectural environment of buildings and identification of the main criteria, tools and alternative solutions. The use of high technologies in theaters are considered using real-life projects.Research findings: The main criteria are identified for reorientation of design to architecture that consider increased needs of citizens. Focusing on existing theater projects, the possibility of using virtual and augmented reality technologies and metaverses, is determined for the theater space.

Interactions between neural representations of the social and spatial environment.

Even in our highly interconnected modern world, geographic factors play an important role in human social connections. Similarly, social relationships influence how and where we travel, and how we think about our spatial world. Here, we review the growing body of neuroscience research that is revealing multiple interactions between social and spatial processes in both humans and non-human animals. We review research on the cognitive and neural representation of spatial and social information, and highlight recent findings suggesting that underlying mechanisms might be common to both. We discuss how spatial factors can influence social behaviour, and how social concepts modify representations of space. In so doing, this review elucidates not only how neural representations of social and spatial information interact but also similarities in how the brain represents and operates on analogous information about its social and spatial surroundings.This article is part of the theme issue 'The spatial-social interface: a theoretical and empirical integration'.

Empowering sign language communication: Integrating sentiment and semantics for facial expression synthesis

Translating written sentences from oral languages to a sequence of manual and non-manual gestures plays a crucial role in building a more inclusive society for deaf and hard-of-hearing people. Facial expressions (non-manual), in particular, are responsible for encoding the grammar of the sentence to be spoken, applying punctuation, pronouns, or emphasizing signs. These non-manual gestures are closely related to the semantics of the sentence being spoken and also to the utterance of the speaker’s emotions. However, most Sign Language Production (SLP) approaches are centered on synthesizing manual gestures and do not focus on modeling the speaker’s expression. This paper introduces a new method focused in synthesizing facial expressions for sign language. Our goal is to improve sign language production by integrating sentiment information in facial expression generation. The approach leverages a sentence’s sentiment and semantic features to sample from a meaningful representation space, integrating the bias of the non-manual components into the sign language production process. To evaluate our method, we extend the Fréchet gesture distance (FGD) and propose a new metric called Fréchet Expression Distance (FED) and apply an extensive set of metrics to assess the quality of specific regions of the face. The experimental results showed that our method achieved state of the art, being superior to the competitors on How2Sign and PHOENIX14T datasets. Moreover, our architecture is based on a carefully designed graph pyramid that makes it simpler, easier to train, and capable of leveraging emotions to produce facial expressions. Our code and pretrained models will be available at: https://github.com/verlab/empowering-sign-language.

Brain structure and activity predicting cognitive maturation in adolescence.

Cognitive abilities of primates, including humans, continue to improve through adolescence 1,2. While a range of changes in brain structure and connectivity have been documented 3,4, how they affect neuronal activity that ultimately determines performance of cognitive functions remains unknown. Here, we conducted a multilevel longitudinal study of monkey adolescent neurocognitive development. The developmental trajectory of neural activity in the prefrontal cortex accounted remarkably well for working memory improvements. While complex aspects of activity changed progressively during adolescence, such as the rotation of stimulus representation in multidimensional neuronal space, which has been implicated in cognitive flexibility, even simpler attributes, such as the baseline firing rate in the period preceding a stimulus appearance had predictive power over behavior. Unexpectedly, decreases in brain volume and thickness, which are widely thought to underlie cognitive changes in humans 5 did not predict well the trajectory of neural activity or cognitive performance changes. Whole brain cortical volume in particular, exhibited an increase and reached a local maximum in late adolescence, at a time of rapid behavioral improvement. Maturation of long-distance white matter tracts linking the frontal lobe with areas of the association cortex and subcortical regions best predicted changes in neuronal activity and behavior. Our results provide evidence that optimization of neural activity depending on widely distributed circuitry effects cognitive development in adolescence.

9D Rotation Representation-SVD Fusion with Deep Learning for Unconstrained Head Pose Estimation

Abstract Accurately estimating human head pose poses a significant challenge across various application domains. To address the inherent limitations of previous approaches, this research proposes an unconstrained head pose estimation strategy. The method combines deep learning with rotation matrices, utilizing nine-dimensional vectors output by the neural network, which are projected back to rotation matrices in SO (3) space through singular value decomposition. This ensures both the smoothness and uniqueness of the rotation representation. The approach demonstrates distinct advantages in handling the rotation estimation task, particularly when the rotated representation is used as the model output. It not only avoids the discontinuity and double-coverage issues associated with prior methods but also enhances the stability of the representation in high-dimensional space, thereby improving the learning process. Additionally, the geodesic loss function is incorporated to train the network. The proposed strategy surpasses previous state-of-the-art methods, as evidenced by experiments conducted on the AFLW2000 and BIWI datasets.

Creolization and Gender: Island Imaginings in Ananda Devi's Pagli and Its Sequel

Abstract: This essay opens the study of "Archipelagic Asias" to the southwest Indian Ocean archipelagos to explore representations of Indic cultures in the literature of the creolized islands of Mauritius. By combining archipelagic and feminist epistemologies, I examine one of the most successful novels of Ananda Devi, Pagli (2001), in which an Indo-Mauritian woman has an affair with a Creole fisherman, and its little-known short story sequel, "Agalega" (2021). Published in Maudrigosa!, a volume celebrating the Mauritian archipelago, the short story denounces the construction of an Indian military base on the Mauritius Outer Island of Agalega. I argue that literary representations of Mauritius's geographic space form different island imaginings that express Mauritius's affective relation to its creolized culture and the impact it has on women's corporeal and sexual freedom.

Path signatures for diversity in probabilistic trajectory optimisation

Motion planning can be cast as a trajectory optimisation problem where a cost is minimised as a function of the trajectory being generated. In complex environments with several obstacles and complicated geometry, this optimisation problem is usually difficult to solve and prone to local minima. However, recent advancements in computing hardware allow for parallel trajectory optimisation where multiple solutions are obtained simultaneously, each initialised from a different starting point. Unfortunately, without a strategy preventing two solutions to collapse on each other, naive parallel optimisation can suffer from mode collapse diminishing the efficiency of the approach and the likelihood of finding a global solution. In this paper, we leverage on recent advances in the theory of rough paths to devise an algorithm for parallel trajectory optimisation that promotes diversity over the range of solutions, therefore avoiding mode collapses and achieving better global properties. Our approach builds on path signatures and Hilbert space representations of trajectories and connects parallel variational inference for trajectory estimation with diversity-promoting kernels. We empirically demonstrate that this strategy achieves lower average costs than competing alternatives on a range of problems, from 2D navigation to robotic manipulators operating in cluttered environments.

SG-Fusion: A swin-transformer and graph convolution-based multi-modal deep neural network for glioma prognosis

The integration of morphological attributes extracted from histopathological images and genomic data holds significant importance in advancing tumor diagnosis, prognosis, and grading. Histopathological images are acquired through microscopic examination of tissue slices, providing valuable insights into cellular structures and pathological features. On the other hand, genomic data provides information about tumor gene expression and functionality. The fusion of these two distinct data types is crucial for gaining a more comprehensive understanding of tumor characteristics and progression. In the past, many studies relied on single-modal approaches for tumor diagnosis. However, these approaches had limitations as they were unable to fully harness the information from multiple data sources. To address these limitations, researchers have turned to multi-modal methods that concurrently leverage both histopathological images and genomic data. These methods better capture the multifaceted nature of tumors and enhance diagnostic accuracy. Nonetheless, existing multi-modal methods have, to some extent, oversimplified the extraction processes for both modalities and the fusion process. In this study, we presented a dual-branch neural network, namely SG-Fusion. Specifically, for the histopathological modality, we utilize the Swin-Transformer structure to capture both local and global features and incorporate contrastive learning to encourage the model to discern commonalities and differences in the representation space. For the genomic modality, we developed a graph convolutional network based on gene functional and expression level similarities. Additionally, our model integrates a cross-attention module to enhance information interaction and employs divergence-based regularization to enhance the model’s generalization performance. Validation conducted on glioma datasets from the Cancer Genome Atlas unequivocally demonstrates that our SG-Fusion model outperforms both single-modal methods and existing multi-modal approaches in both survival analysis and tumor grading.

Event boundaries drive norepinephrine release and distinctive neural representations of space in the rodent hippocampus.

Episodic memories are temporally segmented around event boundaries that tend to coincide with moments of environmental change. During these times, the state of the brain should change rapidly, or reset, to ensure that the information encountered before and after an event boundary is encoded in different neuronal populations. Norepinephrine (NE) is thought to facilitate this network reorganization. However, it is unknown whether event boundaries drive NE release in the hippocampus and, if so, how NE release relates to changes in hippocampal firing patterns. The advent of the new GRABNE sensor now allows for the measurement of NE binding with sub-second resolution. Using this tool in mice, we tested whether NE is released into the dorsal hippocampus during event boundaries defined by unexpected transitions between spatial contexts and presentations of novel objections. We found that NE binding dynamics were well explained by the time elapsed after each of these environmental changes, and were not related to conditioned behaviors, exploratory bouts of movement, or reward. Familiarity with a spatial context accelerated the rate in which phasic NE binding decayed to baseline. Knowing when NE is elevated, we tested how hippocampal coding of space differs during these moments. Immediately after context transitions we observed relatively unique patterns of neural spiking which settled into a modal state at a similar rate in which NE returned to baseline. These results are consistent with a model wherein NE release drives hippocampal representations away from a steady-state attractor. We hypothesize that the distinctive neural codes observed after each event boundary may facilitate long-term memory and contribute to the neural basis for the primacy effect.

A framework for the emergence and analysis of language in social learning agents

Neural systems have evolved not only to solve environmental challenges through internal representations but also, under social constraints, to communicate these to conspecifics. In this work, we aim to understand the structure of these internal representations and how they may be optimized to transmit pertinent information from one individual to another. Thus, we build on previous teacher-student communication protocols to analyze the formation of individual and shared abstractions and their impact on task performance. We use reinforcement learning in grid-world mazes where a teacher network passes a message to a student to improve task performance. This framework allows us to relate environmental variables with individual and shared representations. We compress high-dimensional task information within a low-dimensional representational space to mimic natural language features. In coherence with previous results, we find that providing teacher information to the student leads to a higher task completion rate and an ability to generalize tasks it has not seen before. Further, optimizing message content to maximize student reward improves information encoding, suggesting that an accurate representation in the space of messages requires bi-directional input. These results highlight the role of language as a common representation among agents and its implications on generalization capabilities.

From Lévy walks to fractional material derivative: Pointwise representation and a numerical scheme

The fractional material derivative appears as the fractional operator that governs the dynamics of the scaling limits of Lévy walks - a stochastic process that originates from the famous continuous-time random walks. It is usually defined as the Fourier–Laplace multiplier, therefore, it can be thought of as a pseudo-differential operator. In this paper, we show that there exists a local representation in time and space, pointwise, of the fractional material derivative. This allows us to define it on a space of locally integrable functions which is larger than the original one in which Fourier and Laplace transform exist as functions.We consider several typical differential equations involving the fractional material derivative and provide conditions for their solutions to exist. In some cases, the analytical solution can be found. For the general initial value problem, we devise a finite volume method and prove its stability, convergence, and conservation of probability. Numerical illustrations verify our analytical findings. Moreover, our numerical experiments show superiority in the computation time of the proposed numerical scheme over a Monte Carlo method applied to the problem of probability density function’s derivation.

PSO based linear parameter varying-model predictive control for trajectory tracking of autonomous vehicles

In this paper, Linear Parameter Varying-Model Predictive Control (LPV-MPC) for trajectory tracking for Autonomous Vehicles (AVs) is proposed. This method is based on the time-varying LPV is the form of the state space representation from the mathematical model of the vehicle. The LPV representation form which uses the dynamic model of the vehicle allows the incorporation of time-varying dynamics, providing a more accurate representation of the vehicle's behavior. The designed LPV-MPC controller for AVs is specifically designed to handle constraints in trajectory tracking. To enhance its performance, Particle Swarm Optimization (PSO) is employed as an optimization technique. PSO is used to tune the weighting matrices of the control parameters, optimizing the system response and improving trajectory tracking performance. To evaluate the effectiveness of the LPV-MPC system, extensive simulations are conducted and results are compared with Linear and Non-Linear MPCs. The main benefit of using the LPV-MPC method is its ability to calculate solutions almost as good as the non-linear MPC version yet significantly reducing the computational cost. The capability of the LPV-MPC controller as compared to the linear version is in its effective tracking, particularly for the non-linear reference trajectories.

Training Data Diversity Enhances the Basecalling of Novel RNA Modification-Induced Nanopore Sequencing Readouts.

Accurately basecalling sequence backbones in the presence of nucleotide modifications remains a substantial challenge in nanopore sequencing bioinformatics. It has been extensively demonstrated that state-of-the-art basecallers are less compatible with modification-induced sequencing signals. A precise basecalling, on the other hand, serves as the prerequisite for virtually all the downstream analyses. Here, we report that basecallers exposed to diverse training modifications gain the generalizability to analyze novel modifications. With synthesized oligos as the model system, we precisely basecall various out-of-sample RNA modifications. From the representation learning perspective, we attribute this generalizability to basecaller representation space expanded by diverse training modifications. Taken together, we conclude increasing the training data diversity as a novel paradigm for building modification-tolerant nanopore sequencing basecallers.

Deep time-series clustering via latent representation alignment

In practice, obtaining sufficient label information from a dataset is challenging. Consequently, various clustering methods have been studied to homogeneously group data without label information. Recently, deep clustering approaches that utilize deep neural networks have garnered considerable attention. However, time series data possess unique characteristics, including temporal relationships between observations in a sequence, which can decrease the performance of existing deep clustering methods when applied to time series. Despite this, few studies on deep clustering have addressed the characteristics of time series. Thus, we propose a novel approach for deep time-series clustering using topological information, enabling the capture of underlying temporal patterns to generate cluster-oriented representations. We address the topological information of a time series by introducing a novel loss function based on the eigendecomposition of representations in latent space. Through experiments on various time-series datasets, we demonstrate the efficacy of the proposed method in achieving superior clustering performance compared to state-of-the-art deep clustering methods. To the best of our knowledge, this is the first approach that utilizes topological information for deep time-series clustering.

A formalist in the trenches: Graham K. Riach’s The Short Story After Apartheid

This essay offers a brief assessment of the potential of a formalist approach to the criticism of South African literature. Examining Graham K. Riach’s recent study The Short Story After Apartheid, it suggests that the “New Formalism” Riach employs provides an innovative mode of criticism that sees the literary text as interactive with its context rather than as a space of passive representation. Viewing the literary text as a spur to insight allows for a creative engagement with and vivification of many of the sedimented categories of South African critical thought. The essay expresses misgivings, however, about, first, the possibility of harnessing this mode of reading to a determinate political end, and second, its general bias toward the conceptual aspects of literary experience over and against the affective.