Representation Of Properties Research Articles

Hyperdimensional computing with holographic and adaptive encoder.

Brain-inspired computing has become an emerging field, where a growing number of works focus on developing algorithms that bring machine learning closer to human brains at the functional level. As one of the promising directions, Hyperdimensional Computing (HDC) is centered around the idea of having holographic and high-dimensional representation as the neural activities in our brains. Such representation is the fundamental enabler for the efficiency and robustness of HDC. However, existing HDC-based algorithms suffer from limitations within the encoder. To some extent, they all rely on manually selected encoders, meaning that the resulting representation is never adapted to the tasks at hand. In this paper, we propose FLASH, a novel hyperdimensional learning method that incorporates an adaptive and learnable encoder design, aiming at better overall learning performance while maintaining good properties of HDC representation. Current HDC encoders leverage Random Fourier Features (RFF) for kernel correspondence and enable locality-preserving encoding. We propose to learn the encoder matrix distribution via gradient descent and effectively adapt the kernel for a more suitable HDC encoding. Our experiments on various regression datasets show that tuning the HDC encoder can significantly boost the accuracy, surpassing the current HDC-based algorithm and providing faster inference than other baselines, including RFF-based kernel ridge regression. The results indicate the importance of an adaptive encoder and customized high-dimensional representation in HDC.

Spatial roles in hockey special teams

Abstract Special teams (i.e. power play and penalty kill) situations play an outsized role in determining the outcome of ice hockey games. Yet, quantitative methods for characterizing special teams tactics are limited. This work focuses on team structure and player deployment during in-zone special teams possessions. Leveraging player and puck tracking data from the National Hockey League (NHL), a framework is developed for describing player positioning during 5-on-4 power play and 4-on-5 penalty kill possessions. More specifically, player roles are defined directly from the player tracking data using non-negative matrix factorization, and every player is allocated a unique role at every frame of tracking data by solving a linear assignment problem. Team formations naturally arise through the combination of roles occupied in a frame. Roles that vary on a per-frame basis allow for a fine-grained analysis of team structure. This property of the roles-based representation is used to group together similar power play possessions using latent Dirichlet allocation, a topic modelling technique. The concept of assignments, which remain constant over an entire possession, is also introduced. Assignments provide a more stable measure of player positioning, which may be preferable when assessing deployment over longer periods of time.

Mind the gap: noncausal explanations of dual properties

I identify and characterize a type of noncausal explanation in physics. I first introduce a distinction, between the physical properties of a system, and the representational properties of the mathematical expressions of the system’s physical properties. Then I introduce a novel kind of property, which I shall call a dual property. This is a special kind of representational property, one for which there is an interpretation as a physical property. It is these dual properties that, I claim, are amenable to noncausal (mathematical, in fact) explanations. I discuss a typical example of such a dual property, and an example of an explanation as to why that dual property holds (the explanation of the quantization of the linear momentum).

Narrowing the Gap between Supervised and Unsupervised Sentence Representation Learning with Large Language Model

Sentence Representation Learning (SRL) is a fundamental task in Natural Language Processing (NLP), with the Contrastive Learning of Sentence Embeddings (CSE) being the mainstream technique due to its superior performance. An intriguing phenomenon in CSE is the significant performance gap between supervised and unsupervised methods, with their only difference lying in the training data. Previous works attribute this performance gap to differences in two representation properties (alignment and uniformity). However, since alignment and uniformity only measure the results, they fail to answer "What aspects of the training data contribute to the performance gap?" and "How can the performance gap be narrowed?", In this paper, we conduct empirical experiments to answer these "What" and "How" questions. We first answer the "What" question by thoroughly comparing the behavior of supervised and unsupervised CSE during their respective training processes. From the comparison, we identify the similarity pattern as a key factor to the performance gap, and introduce a metric, called Relative Fitting Difficulty (RFD), to measure the complexity of the similarity pattern. Then, based on the insights gained from the "What" question, we tackle the "How" question by increasing the pattern complexity of the training data. We achieve this by leveraging the In-Context Learning (ICL) capability of the Large Language Model (LLM) to generate data that simulates complex patterns. By utilizing the hierarchical patterns in the LLM-generated data, we effectively narrow the gap between supervised and unsupervised CSE. We release our codes and appendix at https://github.com/BDBC-KG-NLP/NGCSE.

Rethinking Graph Masked Autoencoders through Alignment and Uniformity

Self-supervised learning on graphs can be bifurcated into contrastive and generative methods. Contrastive methods, also known as graph contrastive learning (GCL), have dominated graph self-supervised learning in the past few years, but the recent advent of graph masked autoencoder (GraphMAE) rekindles the momentum behind generative methods. Despite the empirical success of GraphMAE, there is still a dearth of theoretical understanding regarding its efficacy. Moreover, while both generative and contrastive methods have been shown to be effective, their connections and differences have yet to be thoroughly investigated. Therefore, we theoretically build a bridge between GraphMAE and GCL, and prove that the node-level reconstruction objective in GraphMAE implicitly performs context-level GCL. Based on our theoretical analysis, we further identify the limitations of the GraphMAE from the perspectives of alignment and uniformity, which have been considered as two key properties of high-quality representations in GCL. We point out that GraphMAE's alignment performance is restricted by the masking strategy, and the uniformity is not strictly guaranteed. To remedy the aforementioned limitations, we propose an Alignment-Uniformity enhanced Graph Masked AutoEncoder, named AUG-MAE. Specifically, we propose an easy-to-hard adversarial masking strategy to provide hard-to-align samples, which improves the alignment performance. Meanwhile, we introduce an explicit uniformity regularizer to ensure the uniformity of the learned representations. Experimental results on benchmark datasets demonstrate the superiority of our model over existing state-of-the-art methods. The code is available at: https://github.com/AzureLeon1/AUG-MAE.

Is Quality Warranty a Type of Representations and Warranties?

The paper identifies and compares the key features of representations and warranties and quality warranties. Based on the results of the study, the author concludes that quality warranty is a type of representations and warranties. This is evidenced by the key element of their similarity — the reflection of the qualitative characteristics of the subject of the transaction. After all, both representations and warranties and quality warranty are inextricably linked with the contract, in the absence of which their existence does not matter. The differences between the studied categories reflect the specific features of quality warranty as representations and warranties. However, this does not exclude the possibility that certain properties of representations and warranties can be extended to a quality guarantee, and this will not contradict the essence of the latter, which will better ensure the rights and interests of parties to transactions. For example: 1) extend the rule on the autonomous and independent nature of representations to the effect of quality warranty; 2) allow for the provision of quality warranties by a third party in non-consumer legal relations, as exists in the case of the use of representations and warranties. It is also proposed to introduce a legal presumption of the commencement of validity of representations and warranties, which is based on the principle of exercising the right in one’s own interest (Clause 2 of Article 1 of the Civil Code of the Russian Federation).

Vision matters for shape representation: Evidence from sculpturing and drawing in the blind

Shape is a property that could be perceived by vision and touch, and is classically considered to be supramodal. While there is mounting evidence for the shared cognitive and neural representation space between visual and tactile shape, previous research tended to rely on dissimilarity structures between objects and had not examined the detailed properties of shape representation in the absence of vision. To address this gap, we conducted three explicit object shape knowledge production experiments with congenitally blind and sighted participants, who were asked to produce verbal features, 3D clay models, and 2D drawings of familiar objects with varying levels of tactile exposure, including tools, large nonmanipulable objects, and animals. We found that the absence of visual experience (i.e., in the blind group) led to stronger differences in animals than in tools and large objects, suggesting that direct tactile experience of objects is essential for shape representation when vision is unavailable. For tools with rich tactile/manipulation experiences, the blind produced overall good shapes comparable to the sighted, yet also showed intriguing differences. The blind group had more variations and a systematic bias in the geometric property of tools (making them stubbier than the sighted), indicating that visual experience contributes to aligning internal representations and calibrating overall object configurations, at least for tools. Taken together, the object shape representation reflects the intricate orchestration of vision, touch and language.

Design, Synthesis, Characterization, and Antimicrobial Properties of New Azo Disperse Dyes Incorporating Quinazolinone-Pyrazolone Moieties and Their Applications for Dyeing of Polyester Fabrics

The current study outlines a straightforward and efficient method for creating new quinazolinone disperse dyes based on pyrazolone moieties, starting with quinazolinone and a variety of substituted pyrazolone as couplers. The synthesized dyes were characterized using a variety of spectroscopic and analytical methods. The synthesized dyes' ultraviolet–visible spectra showed bands brought on by several molecular transitions. We investigated in detail the multifunctional characteristics such color representation, dyeing duration, concentration, pH, buildup, and fastness properties of the dyed samples. Fastness properties and colorimetric data showed satisfactory results, demonstrating the effectiveness of these dyes in dyeing polyester fabrics. A pH of 5 and a dyeing temperature of 130 °C were the ideal conditions for dyeing polyester fabrics. Additionally, an ultraviolet protection factor test was performed on the dyed fabrics, and the results showed that these dyes provide the best UV protection. These dyes are suitable for industrial dyeing applications since they are easy to manufacture and scale up. Additionally, in-vitro testing was done to determine the dyes' antibacterial effectiveness against various bacteria and fungi. The antibacterial activity of the dyes was moderate to very good against both Gram-positive and Gram-negative bacteria, as well as fungi.

Investigating the properties of neural network representations in reinforcement learning

In this paper we investigate the properties of representations learned by deep reinforcement learning systems. Much of the early work on representations for reinforcement learning focused on designing fixed-basis architectures to achieve properties thought to be desirable, such as orthogonality and sparsity. In contrast, the idea behind deep reinforcement learning methods is that the agent designer should not encode representational properties, but rather that the data stream should determine the properties of the representation—good representations emerge under appropriate training schemes. In this paper we bring these two perspectives together, empirically investigating the properties of representations that support transfer in reinforcement learning. We introduce and measure six representational properties over more than 25,000 agent-task settings. We consider Deep Q-learning agents with different auxiliary losses in a pixel-based navigation environment, with source and transfer tasks corresponding to different goal locations. We develop a method to better understand why some representations work better for transfer, through a systematic approach varying task similarity and measuring and correlating representation properties with transfer performance. We demonstrate the generality of the methodology by investigating representations learned by a Rainbow agent that successfully transfers across Atari 2600 game modes.

Concepts of Statistical Causality and Strong and Weak Properties of Predictable Representation

The paper considers the statistical concept of causality in continuous time, which is based on Granger’s definition of causality. We give necessary and sufficient conditions, in terms of statistical causality, for the preservation of the strong property of predictable representation for stopped martingales when filtration is decreased. This concept of causality is also connected to the preservation of the strong property of predictable representation under a change in measure. In addition, we give conditions, in terms of statistical causality, for martingales to have strong and weak properties of predictable representation. The results are applied to the problem of pricing claims in incomplete financial markets.

Early visual cortices reveal interrelated item and category representations in aging.

Neural dedifferentiation, the finding that neural representations tend to be less distinct in older adults compared with younger adults, has been associated with age-related declines in memory performance. Most studies assessing the relation between memory and neural dedifferentiation have evaluated how age impacts the distinctiveness of neural representations for different visual categories (e.g., scenes and objects). However, how age impacts the quality of neural representations at the level of individual items is still an open question. Here, we present data from an age-comparative fMRI study that aimed to understand how the distinctiveness of neural representations for individual stimuli differs between younger and older adults and relates to memory outcomes. Pattern similarity searchlight analyses yielded indicators of neural dedifferentiation at the level of individual items as well as at the category level in posterior occipital cortices. We asked whether age differences in neural distinctiveness at each representational level were associated with inter- and/or intraindividual variability in memory performance. While age-related dedifferentiation at both the item and category level related to between-person differences in memory, neural distinctiveness at the category level also tracked within-person variability in memory performance. Concurrently, neural distinctiveness at the item level was strongly associated with neural distinctiveness at the category level both within and across participants, elucidating a potential representational mechanism linking item- and category-level distinctiveness. In sum, we provide evidence that age-related neural dedifferentiation co-exists across multiple representational levels and is related to memory performance.Significance Statement Age-related memory decline has been associated with neural dedifferentiation, the finding that older adults have less distinctive neural representations than younger adults. This has been mostly shown for category information, while evidence for age differences in the specificity of item representations is meager. We used pattern similarity searchlight analyses to find indicators of neural dedifferentiation at both levels of representation (category and item) and linked distinctiveness to memory performance. Both item- and category-level dedifferentiation in the calcarine cortex were related to interindividual differences in memory performance, while category-level distinctiveness further tracked intraindividual variability. Crucially, neural distinctiveness was strongly tied between the item and category levels, indicating that intersecting representational properties of posterior occipital cortices reflect both individual exemplars and categories.

QSAR modelling to predict structural features of certain sulfonamide as Urokinase-type Plasminogen Activator inhibitors

The serine hydrolase family includes serine proteases. It is essential for hydrolyzing protein peptide bonds and breaking them. The urokinase-type plasminogen activator (uPA) selectively binds to the uPAR on numerous cell types, including cancer cells. Pericellular proteolysis of cell-bound proteins requires this interaction. High uPA and uPAR levels regularly worsen cancer prognoses. Thus, small chemical active-site inhibitors that block uPA may diminish cancer cell invasion and metastasis. In compliance with Organization for Economic Corporation and Development guidelines, this research performed a complete Quantitative structure activity relationship analysis of sulfonamide compounds as Urokinase-type Plasminogen Activator inhibitors. Py-Descriptors were used for this investigation. PyDescriptor uses PyMOL standards and idioms to calculate 11,145 simple molecular descriptors. This plugin calculates molecular descriptors irrespective of molecular representation properties like atom numbering or labelling, spatial reference frame, translational and rotational invariance, etc. The investigation sought to find essential and hidden structural characteristics that regulate sulfonamide-type drugs' Urokinase-type Plasminogen Activator Inhibitory action. Twenty-eight sulfonamide chemicals are used in the Quantitative structure activity relationship study to generate statistically robust and highly predictive univariate and multivariate models. All models were thoroughly evaluated and meet several statistical parameter thresholds (e.g., R2 =0.9259–0.9280, Q2Loo= 0.8579–0.8558, Q2LMO= 0.8013–0.7865). The analysis reveal that occurance of ring carbon atoms exactly at 3 A0 from carbon atom, number of negatively charged atoms from sulphur atoms within 5 bonds, presence of hydrogen atom exactly at 3 bonds from donar atoms, presence of carbon atom exactly at 4 A0 from donar atom, presence of acceptor atom exactly at 5 A0 from sulphur atom and sum of partial charges of lipo atoms within 6 bonds from sulphur atom are important pharmacophoric features for Urokinase-type Plasminogen Activator Inhibition binding affinity. Thus, the developed Quantitative structure activity relationship study has an equilibrium of quantitative and qualitative tactics. The results could be useful for future optimizations of sulfonamide analogues.

Visualizing high-dimensional loss landscapes with Hessian directions

Analyzing the geometric properties of high-dimensional loss functions, such as local curvature and the existence of other optima around a certain point in loss space, can help provide a better understanding of the interplay between neural-network structure, implementation attributes, and learning performance. In this paper, we combine concepts from high-dimensional probability and differential geometry to study how curvature properties in lower-dimensional loss representations depend on those in the original loss space. We show that saddle points in the original space are rarely correctly identified as such in the expected lower-dimensional representations if random projections are used. The principal curvature in the expected lower-dimensional representation is proportional to the mean curvature in the original loss space. Hence, the mean curvature in the original loss space determines if saddle points appear, on average, as either minima, maxima, or almost flat regions. We use the connection between expected curvature in random projections and mean curvature in the original space (i.e. the normalized Hessian trace) to compute Hutchinson-type trace estimates without calculating Hessian-vector products as in the original Hutchinson method. Because random projections are not suitable for correctly identifying saddle information, we propose to study projections along the dominant Hessian directions that are associated with the largest and smallest principal curvatures. We connect our findings to the ongoing debate on loss landscape flatness and generalizability. Finally, for different common image classifiers and a function approximator, we show and compare random and Hessian projections of loss landscapes with up to approximately 7×106 parameters.

Transport makes cities: transit maps as major cognitive frames of metropolitan areas.

Spatial representations play a fundamental role in navigation, decision-making, and overall interaction with our environments. Understanding how individuals construct and use them holds significant importance in spatial cognition research, and even bears practical implications for urban planning as it can explain how we interact with the spaces we inhabit. In large urban areas, transit maps stand as prominent visual aids, guiding people through public transportation systems. These maps, while designed for navigational purposes, may influence how individuals perceive and represent their cities. For instance, Vertesi (Social Studies of Science 38:09-35, 2008) showed through a series of interviews including a "sketch mapping" phase, that London Tube Map seems to structure residents' spatial representation of their city. However, thorough quantitative research on this subject have not been carried out yet. Two experimental studies have been conducted to demonstrate how residents' representations of metropolitan areas closely resemble the schematic representations of their public transport networks. First, we show that residents of Greater Paris-public and private transport users alike-plot city landmarks in a layout more closely resembling that of the Parisian transit map than the geographical map. Next, we asked Greater Berlin, London and Paris residents to place landmarks of their cities on different map backgrounds. A similar procedure was followed for landmarks from an unknown city, after a dedicated learning phase. For known cities, the sketch maps produced were closer to transit maps than to the geographical ones, although less so if the test map background presented topographical elements (e.g., rivers, etc.). For learnt cities, participants' sketch maps were almost exclusively dependent on the map provided during the learning phase. These results suggest that familiarity with transit maps has a direct impact on the metric properties of spatial representation in memory, a phenomenon we propose to call the 'schema effect'.

Positive crossratios, barycenters, trees and applications to maximal representations

We study metric properties of maximal framed representations of fundamental groups of surfaces in symplectic groups over real closed fields, interpreted as actions on Bruhat–Tits buildings endowed with adapted Finsler norms. We prove that the translation length can be computed as intersection with a geodesic current, give sufficient conditions guaranteeing that such a current is a multicurve, and, if the current is a measured lamination, construct an isometric embedding of the associated tree in the building. These results are obtained as application of more general results of independent interest on positive crossratios and actions with compatible barycenters.

A note on the post quantum-Sheffer polynomial sequences

Abstract In this article, the post quantum analogue of Sheffer polynomial sequences is introduced using concepts of post quantum calculus. The series representation, recurrence relations, determinant expression and certain other properties of this class are established. Further, the 2D-post quantum-Sheffer polynomials are introduced via generating function and their properties are established. Certain identities and integral representations for the 2D-post quantum-Hermite polynomials, 2D-post quantum-Laguerre polynomials, and 2D-post quantum-Bessel polynomials are also considered.

Embodied Figuration and Character Emotion in Cinema: The Role of Embodied Affective Cues

Abstract This paper engages with research on embodiment, cognition, and figurative meaning in cinema to develop a system for categorizing the cinematic cues that viewers use to gauge character affective experience. Many film theorists whose work is primarily focused on affective experience consider the relationship between the spectator and the character’s facial and bodily expressions. However, films often convey the affective states of characters by integrating bodily expressions with cinematic techniques, such as lighting, framing, and editing. These cinematic techniques can provide insight into a character’s affective states through their figurative associations with represented moods, feelings, and emotions. By exploring the metonymic, metaphoric, and similative properties of cinematic representations of affect, we can better understand how cinematic representations are understood by a community of viewers. Throughout this paper, I put forth the framework of “embodied affective cues” and identify behavioral, physical, and environmental cues to address how a character’s affective experience is represented by cinematic cues external to their body.

Data Augmentation for Bayesian Deep Learning

Deep Learning (DL) methods have emerged as one of the most powerful tools for functional approximation and prediction. While the representation properties of DL have been well studied, uncertainty quantification remains challenging and largely unexplored. Data augmentation techniques are a natural approach to provide uncertainty quantification and to incorporate stochastic Monte Carlo search into stochastic gradient descent (SGD) methods. The purpose of our paper is to show that training DL architectures with data augmentation leads to efficiency gains. We use the theory of scale mixtures of normals to derive data augmentation strategies for deep learning. This allows variants of the expectation-maximization and MCMC algorithms to be brought to bear on these high dimensional nonlinear deep learning models. To demonstrate our methodology, we develop data augmentation algorithms for a variety of commonly used activation functions: logit, ReLU, leaky ReLU and SVM. Our methodology is compared to traditional stochastic gradient descent with back-propagation. Our optimization procedure leads to a version of iteratively re-weighted least squares and can be implemented at scale with accelerated linear algebra methods providing substantial improvement in speed. We illustrate our methodology on a number of standard datasets. Finally, we conclude with directions for future research.

Interpretability Is in the Mind of the Beholder: A Causal Framework for Human-Interpretable Representation Learning.

Research on Explainable Artificial Intelligence has recently started exploring the idea of producing explanations that, rather than being expressed in terms of low-level features, are encoded in terms of interpretable concepts learned from data. How to reliably acquire such concepts is, however, still fundamentally unclear. An agreed-upon notion of concept interpretability is missing, with the result that concepts used by both post hoc explainers and concept-based neural networks are acquired through a variety of mutually incompatible strategies. Critically, most of these neglect the human side of the problem: a representation is understandable only insofar as it can be understood by the human at the receiving end. The key challenge in human-interpretable representation learning (hrl) is how to model and operationalize this human element. In this work, we propose a mathematical framework for acquiring interpretable representations suitable for both post hoc explainers and concept-based neural networks. Our formalization of hrl builds on recent advances in causal representation learning and explicitly models a human stakeholder as an external observer. This allows us derive a principled notion of alignment between the machine's representation and the vocabulary of concepts understood by the human. In doing so, we link alignment and interpretability through a simple and intuitive name transfer game, and clarify the relationship between alignment and a well-known property of representations, namely disentanglement. We also show that alignment is linked to the issue of undesirable correlations among concepts, also known as concept leakage, and to content-style separation, all through a general information-theoretic reformulation of these properties. Our conceptualization aims to bridge the gap between the human and algorithmic sides of interpretability and establish a stepping stone for new research on human-interpretable representations.

Drawing as a tool for investigating the nature of imagery representations of blind people: The case of the canonical size phenomenon.

Several studies have shown that blind people, including those with congenital blindness, can use raised-line drawings, both for "reading" tactile graphics and for drawing unassisted. However, research on drawings produced by blind people has mainly been qualitative. The current experimental study was designed to investigate the under-researched issue of the size of drawings created by people with blindness. Participants (N = 59) varied in their visual status. Adventitiously blind people had previous visual experience and might use visual representations (e.g., when visualising objects in imagery/working memory). Congenitally blind people did not have any visual experience. The participant's task was to draw from memory common objects that vary in size in the real world. The findings revealed that both groups of participants produced larger drawings of objects that have larger actual sizes. This means that the size of familiar objects is a property of blind people's mental representations, regardless of their visual status. Our research also sheds light on the nature of the phenomenon of canonical size. Since we have found the canonical size effect in a group of people who are blind from birth, the assumption of the visual nature of this phenomenon - caused by the ocular-centric biases present in studies on drawing performance - should be revised.