Powerful Representation Research Articles

Learning topological defects formation with neural networks in a quantum phase transition

Neural networks possess formidable representational power, rendering them invaluable in solving complex quantum many-body systems. While they excel at analyzing static solutions, nonequilibrium processes, including critical dynamics during a quantum phase transition, pose a greater challenge for neural networks. To address this, we utilize neural networks and machine learning algorithms to investigate time evolutions, universal statistics, and correlations of topological defects in a one-dimensional transverse-field quantum Ising model. Specifically, our analysis involves computing the energy of the system during a quantum phase transition following a linear quench of the transverse magnetic field strength. The excitation energies satisfy a power-law relation to the quench rate, indicating a proportional relationship between the excitation energy and the kink numbers. Moreover, we establish a universal power-law relationship between the first three cumulants of the kink numbers and the quench rate, indicating a binomial distribution of the kinks. Finally, the normalized kink-kink correlations are also investigated and it is found that the numerical values are consistent with the analytic formula.

Gaussian-discrete restricted Boltzmann machine with sparse-regularized hidden layer

AbstractOverfitting is a critical concern in machine learning, particularly when the representation capabilities of learning models surpass the complexities present in the training datasets. To mitigate overfitting, curtailing the representation power of the model through suitable techniques such as regularization is necessary. In this study, a sparse-regularization method for Gaussian–Discrete restricted Boltzmann machines (GDRBMs) is considered. A GDRBM is a variant of restricted Boltzmann machines that comprises a continuous visible layer and discrete hidden layer. In the proposed model, sparse GDRBM (S-GDRBM), a sparse prior that encourages sparse representations of the hidden layer is employed. The strength of the prior (i.e., the sparse-regularization strength) can be tuned within the standard scenario of maximum likelihood learning; that is, the strength can be adaptively tuned based on the complexities of the datasets during training. We validated the proposed S-GDRBM using numerical experiments.

Leveraging Bidirectionl LSTM with CRFs for Pashto Tagging

Part-of-speech tagging plays a vital role in text processing and natural language understanding. Very few attempts have been made in the past for tagging Pashto Part-of-Speech. In this work, we present a Long Short-term Memory–based approach for Pashto part-of-speech tagging with special focus on ambiguity resolution. Initially, we created a corpus of Pashto sentences having words with multiple meanings and their tags. We introduce a powerful sentences representation and new architecture for Pashto text processing. The accuracy of the proposed approach is compared with state-of-the-art Hidden Markov Model. Our Model shows 87.60% accuracy for all words excluding punctuation and 95.45% for ambiguous words; however, Hidden Markov Model shows 78.37% and 44.72% accuracy, respectively. Results show that our approach outperforms Hidden Markov Model in Part-of-Speech tagging for Pashto text.

Examining Alterity Dynamics Through the Conceptualisation and (Re)Construction of Social and Political Enemy in Post-Independent Francophone African Literature

The exploration of alterity in francophone African literature offers a rich and multifaceted understanding of identity dynamics and cultural hybridity. By examining the impact of post-independence socio-political landscape, this study provides critical insights into the complexities of otherness. Through the lenses of conflict and postcolonial theories, functionalism, symbolic interactionism, and comparative analysis, the thematic perspective provides a deeper understanding of how the concept of the enemy is constructed, negotiated, and redefined within the alterity framework in francophone African narratives. Using a corpus of five novels, this study not only enriches our understanding of francophone African literature but also contributes to broader discussions on identity, power, and socio-political representation in post-independent Africa. It concludes that the enemy's identity is fluid, continuously shaped by social interactions and cultural practices, which construct and redefine individual and collective identities in response to evolving social and political contexts.

Self-supervised graph autoencoder with redundancy reduction for community detection

Community detection is a significant research topic in network science, which has been revisited with graph neural networks. As a powerful graph representation learning model, graph autoencoder (GAE) is commonly used for unsupervised community detection. However, most GAE-based methods ignore multi-scale features of encoding layers, which inherently provide useful information for community detection. Moreover, these methods fail to simultaneously improve the representation learning process and clustering performance through a unified objective function. To address these issues, we propose a self-supervised graph autoencoder model with redundancy reduction for community detection. Firstly, we introduce a multi-scale module based on GAE to obtain discriminative node representations from different encoding layers. In particular, a redundancy reduction strategy is employed to eliminate redundancy information in the latent embedding space. Then, a node clustering module is used to obtain initial community labels. To fully utilize the multi-scale features to further refine clustering performance, a self-supervised module is designed to utilize current clustering labels to supervise the node representation learning process, thus constructing an end-to-end model for community detection. Finally, we validate the effectiveness of the proposed method on real-world networks. Experimental results demonstrate that our method outperforms several state-of-the-art methods in community detection.

Fairness-Aware Graph Neural Networks: A Survey

Graph Neural Networks (GNNs) have become increasingly important due to their representational power and state-of-the-art predictive performance on many fundamental learning tasks. Despite this success, GNNs suffer from fairness issues that arise as a result of the underlying graph data and the fundamental aggregation mechanism that lies at the heart of the large class of GNN models. In this article, we examine and categorize fairness techniques for improving the fairness of GNNs. We categorize these techniques by whether they focus on improving fairness in the pre-processing, in-processing (during training), or post-processing phases. We discuss how such techniques can be used together whenever appropriate and highlight the advantages and intuition as well. We also introduce an intuitive taxonomy for fairness evaluation metrics, including graph-level fairness, neighborhood-level fairness, embedding-level fairness, and prediction-level fairness metrics. In addition, graph datasets that are useful for benchmarking the fairness of GNN models are summarized succinctly. Finally, we highlight key open problems and challenges that remain to be addressed.

UniproLcad: Accurate Identification of Antimicrobial Peptide by Fusing Multiple Pre-Trained Protein Language Models

Antimicrobial peptides (AMPs) are vital components of innate immunotherapy. Existing approaches mainly rely on either deep learning for the automatic extraction of sequence features or traditional manual amino acid features combined with machine learning. The peptide sequence contains symmetrical sequence motifs or repetitive amino acid patterns, which may be related to the function and structure of the peptide. Recently, the advent of large language models has significantly boosted the representational power of sequence pattern features. In light of this, we present a novel AMP predictor called UniproLcad, which integrates three prominent protein language models—ESM-2, ProtBert, and UniRep—to obtain a more comprehensive representation of protein features. UniproLcad utilizes deep learning networks, encompassing the bidirectional long and short memory network (Bi-LSTM) and one-dimensional convolutional neural networks (1D-CNN), while also integrating an attention mechanism to enhance its capabilities. These deep learning frameworks, coupled with pre-trained language models, efficiently extract multi-view features from antimicrobial peptide sequences and assign attention weights to them. Through ten-fold cross-validation and independent testing, UniproLcad demonstrates competitive performance in the field of antimicrobial peptide identification. This integration of diverse language models and deep learning architectures enhances the accuracy and reliability of predicting antimicrobial peptides, contributing to the advancement of computational methods in this field.

PLoPS: Localization-aware person search with prototypical normalization

Person search involves localizing and re-identifying persons of interest captured by multiple, non-overlapping cameras. Recent approaches to person search are typically built on object detection frameworks to learn joint person representations for detection and re-identification. To this end, the features extracted from pedestrian proposals are projected onto a unit hypersphere using L2 normalization, and positive proposals that sufficiently overlap with the ground truth are equally incorporated for training by exploiting an external lookup table (LUT). We have found that (1) using the L2 normalization technique, without considering feature distributions, can degenerate the discriminative power of person representations, (2) positive proposals often depict distracting details, such as background clutter and person overlaps, and (3) person features in the LUT are not often updated during training. To address these limitations, we propose a novel framework for person search, dubbed PLoPS, using a prototypical normalization layer, ProtoNorm, that calibrates features while considering the long-tail distribution across person IDs. PLoPS also entails a localization-aware learning scheme that prioritizes better-aligned proposals w.r.t the ground truth. We further introduce a LUT calibration technique to continuously adjust the person features in the LUT. Experimental results and analysis on standard benchmarks demonstrate the effectiveness of PLoPS.

Discourse Analysis of a Gender as Representation of Femininity and Power in Children's Literature in "Snow White"

Discourse Analysis of a Gender as Representation of Femininity and Power in Children's Literature in "Snow White"

Multi-scale spatial pyramid attention mechanism for image recognition: An effective approach

Attention mechanisms have gradually become necessary to enhance the representational power of convolutional neural networks (CNNs). Despite recent progress in attention mechanism research, some open problems still exist. Most existing methods ignore modeling multi-scale feature representations, structural information, and long-range channel dependencies, which are essential for delivering more discriminative attention maps. This study proposes a novel, low-overhead, high-performance attention mechanism with strong generalization ability for various networks and datasets. This mechanism is called Multi-Scale Spatial Pyramid Attention (MSPA) and can be used to solve the limitations of other attention methods. For the critical components of MSPA, we not only develop the Hierarchical-Phantom Convolution (HPC) module, which can extract multi-scale spatial information at a more granular level utilizing hierarchical residual-like connections, but also design the Spatial Pyramid Recalibration (SPR) module, which can integrate structural regularization and structural information in an adaptive combination mechanism, while employing the Softmax operation to build long-range channel dependencies. The proposed MSPA is a powerful tool that can be conveniently embedded into various CNNs as a plug-and-play component. Correspondingly, using MSPA to replace the 3 × 3 convolution in the bottleneck residual blocks of ResNets, we created a series of simple and efficient backbones named MSPANet, which naturally inherit the advantages of MSPA. Without bells and whistles, our method substantially outperforms other state-of-the-art counterparts in all evaluation metrics based on extensive experimental results from CIFAR-100 and ImageNet-1K image recognition. When applying MSPA to ResNet-50, our model achieves top-1 classification accuracy of 81.74% and 78.40% on the CIFAR-100 and ImageNet-1K benchmarks, exceeding the corresponding baselines by 3.95% and 2.27%, respectively. We also obtained promising performance improvements of 1.15% and 0.91% compared to the competitive EPSANet-50. In addition, empirical research results in autonomous driving engineering applications also demonstrate that our method can significantly improve the accuracy and real-time performance of image recognition with cheaper overhead. Our code is publicly available at https://github.com/ndsclark/MSPANet.

[Black] plastic feelings; feeling [Black]

ABSTRACT One way to ensure a future for the disciplines housed in the pages of CC/CS is to embrace the realization that researchers do not have to disinvite ourselves from the work we do. Through an exploration of my personal connection to Meghan Markle and Stacey Abrams, I illustrate how when Black audiences attempt to quench their desire for sensory connection through on-screen visually matching bodies, it simplifies the actual power of representation we experience when we demand cultural specificity.

THE REFLECTION OF IDEOLOGY AND CONFLICT THEORY ON TV SERIES: SNOWPIERCER SERIES EXAMPLE

TV series, one of the visual arts, can include cultural phenomena of society, representations of power and everyday events. One of the basic points of visual arts is the law of conflict. In order for there to be conflict, each character in the series must have a goal, purpose and ideology. Marx's "false consciousness" discourse and the concept of "ideology", Althusser's grouping of "ideological apparatuses", Gramsci's "consent" and "hegemony" discourses have been a part of ideology criticism. Therefore, ideology and conflict cannot be separated from each other in TV series. The characters in Snowpiercer are directed towards different goals in line with their status differences. For this reason, the aim of the study is to reveal how the conflict and class differences between people in positions of power and individuals in lower class positions develop. The population of the study consists of all the TV series that were screened all over the world after 2010, in which there are indicators related to the concepts of ideology, conflict and power. The reason for choosing Snowpiercer as the study sample is that it is a study in which the concept of ideology, conflict theory, class conflict and elite theory are processed together. This study is important in that ideology, conflict and elite theory take place in the same series and in a single space and emphasize the difference between classes. This study concludes that keeping people with status and class differences in the same space for a certain period of time will lead to interpersonal conflict.

Powerful or Powerless? Women’s Power Levels in Contemporary Chinese Films

This article aims to achieve an experiential study of cinematic representations of females in Chinese films through quantitative narrative analysis. Applied to a census of 360 Chinese films across various genres produced across 18 years (2001–2018) systematically selected on Mtime, we ask to what extent female characters in Chinese films are empowered? How is their power exercised? And what socio-cultural implications can be detected from the cinematic representations of women in the contemporary Chinese gender discourse? We aim to fill in the gap in Chinese cinema studies in which most scholarships focused on pre-2000 works, adopted qualitative textual and narrative analysis to interpret the representations of women in a particular genre or in typical cases, leaving the quantitative approach, particularly focusing on the post-2000 films, all but absent. We develop a systematic set of typologies, including 4 major power-level categories (power-over, power-to, power-with, and no-power) and 12 deuterogenic ones, which can be applied for future research on Chinese film and media studies. While women’s career success has been articulated with empowerment, independence, subjectivity, and ambition in the neoliberal rhetoric, our research reflects a more intricate relationship between cultural representations of women and power in contemporary Chinese films.

IMJENSE: Scan-Specific Implicit Representation for Joint Coil Sensitivity and Image Estimation in Parallel MRI.

Parallel imaging is a commonly used technique to accelerate magnetic resonance imaging (MRI) data acquisition. Mathematically, parallel MRI reconstruction can be formulated as an inverse problem relating the sparsely sampled k-space measurements to the desired MRI image. Despite the success of many existing reconstruction algorithms, it remains a challenge to reliably reconstruct a high-quality image from highly reduced k-space measurements. Recently, implicit neural representation has emerged as a powerful paradigm to exploit the internal information and the physics of partially acquired data to generate the desired object. In this study, we introduced IMJENSE, a scan-specific implicit neural representation-based method for improving parallel MRI reconstruction. Specifically, the underlying MRI image and coil sensitivities were modeled as continuous functions of spatial coordinates, parameterized by neural networks and polynomials, respectively. The weights in the networks and coefficients in the polynomials were simultaneously learned directly from sparsely acquired k-space measurements, without fully sampled ground truth data for training. Benefiting from the powerful continuous representation and joint estimation of the MRI image and coil sensitivities, IMJENSE outperforms conventional image or k-space domain reconstruction algorithms. With extremely limited calibration data, IMJENSE is more stable than supervised calibrationless and calibration-based deep-learning methods. Results show that IMJENSE robustly reconstructs the images acquired at 5× and 6× accelerations with only 4 or 8 calibration lines in 2D Cartesian acquisitions, corresponding to 22.0% and 19.5% undersampling rates. The high-quality results and scanning specificity make the proposed method hold the potential for further accelerating the data acquisition of parallel MRI.

Delving Into the Devils of Bird's-Eye-View Perception: A Review, Evaluation and Recipe.

Learning powerful representations in bird's-eye-view (BEV) for perception tasks is trending and drawing extensive attention both from industry and academia. Conventional approaches for most autonomous driving algorithms perform detection, segmentation, tracking, etc., in a front or perspective view. As sensor configurations get more complex, integrating multi-source information from different sensors and representing features in a unified view come of vital importance. BEV perception inherits several advantages, as representing surrounding scenes in BEV is intuitive and fusion-friendly; and representing objects in BEV is most desirable for subsequent modules as in planning and/or control. The core problems for BEV perception lie in (a) how to reconstruct the lost 3D information via view transformation from perspective view to BEV; (b) how to acquire ground truth annotations in BEV grid; (c) how to formulate the pipeline to incorporate features from different sources and views; and (d) how to adapt and generalize algorithms as sensor configurations vary across different scenarios. In this survey, we review the most recent works on BEV perception and provide an in-depth analysis of different solutions. Moreover, several systematic designs of BEV approach from the industry are depicted as well. Furthermore, we introduce a full suite of practical guidebook to improve the performance of BEV perception tasks, including camera, LiDAR and fusion inputs. At last, we point out the future research directions in this area. We hope this report will shed some light on the community and encourage more research effort on BEV perception.

Motif-Backdoor: Rethinking the Backdoor Attack on Graph Neural Networks via Motifs

Graph neural network (GNN) with a powerful representation capability has been widely applied to various areas. Recent works have exposed that GNN is vulnerable to the backdoor attack, i.e., models trained with maliciously crafted training samples are easily fooled by patched samples. Most of the proposed studies launch the backdoor attack using a trigger that is either the randomly generated subgraph e.g., erdős-rényi backdoor (ER-B) for less computational burden or the gradient-based generative subgraph e.g., graph trojaning attack (GTA) to enable a more effective attack. However, the interpretation of how is the trigger structure and the effect of the backdoor attack related has been overlooked in the current literature. Motifs, recurrent and statistically significant subgraphs in graphs, contain rich structure information. In this article, we are rethinking the trigger from the perspective of motifs and propose a motif-based backdoor attack, denoted as <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Motif-Backdoor</i> . It contributes from three aspects. First, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Interpretation</i> , it provides an in-depth explanation for backdoor effectiveness by the validity of the trigger structure from motifs, leading to some novel insights, e.g., using subgraphs that appear less frequently in the graph as the trigger can achieve better attack performance. Second, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Effectiveness</i> , Motif-Backdoor reaches the state-of-the-art (SOTA) attack performance in both black-box and defensive scenarios. Third, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Efficiency</i> , based on the graph motif distribution, Motif-Backdoor can quickly obtain an effective trigger structure without target model feedback or subgraph model generation. Extensive experimental results show that Motif-Backdoor realizes the SOTA performance on three popular models and four public datasets compared with five baselines, e.g., Motif-Backdoor improves the attack success rate (ASR) by 14.73% compared with baselines on average. In addition, under a possible defense, Motif-Backdoor still implements satisfying performance, highlighting the requirement of defenses against backdoor attacks on GNNs. The datasets and code are available at https://github.com/Seaocn/Motif-Backdoor.

Uncertainty-Guided Asymmetric Consistency Domain Adaptation for Histopathological Image Classification

Deep learning has achieved remarkable progress in medical image analysis, but its effectiveness heavily relies on large-scale and well-annotated datasets. However, assembling a large-scale dataset of annotated histopathological images is challenging due to their unique characteristics, including various image sizes, multiple cancer types, and staining variations. Moreover, strict data privacy in medicine severely restricts data sharing and poses significant challenges in acquiring large-scale and well-annotated histopathological images. To tackle these constraints, Transfer Learning (TL) provides a promising solution by exploiting knowledge from another domain. This study proposes the Uncertainty-guided asymmetric Consistency Domain Adaptation (UCDA), which does not require accessing the source data and is composed of two essential components, e.g., Uncertainty-guided Source-free Transfer Learning (USTL) and Asymmetric Consistency Learning (ACL). In detail, USTL facilitates a secure mapping of the source domain model’s feature space onto the target domain, eliminating the dependency on source domain data to protect data privacy. At the same time, the ACL module measures the symmetry and asymmetry between the source and target domains, bridging the information gap and preserving inter-domain differences among medical images. We comprehensively evaluate the effectiveness of UCDA on three widely recognized and publicly available datasets, namely NCTCRC-HE-100K, PCam, and LC25000. Impressively, our proposed method achieves remarkable performance on accuracy and F1-scores. Additionally, feature visualizations effectively demonstrate the exceptional generalizability and discriminative power of the learned representations. These compelling results underscore the significant potential of UCDA in driving the advancement of deep learning techniques within the realm of histopathological image analysis.

Solutions of the sl2${\mathfrak {sl}_2}$qKZ equations modulo an integer

AbstractWe study the qKZ difference equations with values in the th tensor power of the vector representation , variables , and integer step . For any integer relatively prime to the step , we construct a family of polynomials in variables with values in such that the coordinates of these polynomials with respect to the standard basis of are polynomials with integer coefficients. We show that satisfy the qKZ equations modulo . Polynomials are modulo analogs of the hypergeometric solutions of the qKZ given in the form of multidimensional Barnes integrals.

Learning place representations from spatial interactions

The development of geospatial artificial intelligence (GeoAI) systems depends on the ability to learn effective representations of places. To learn accurate place representations from spatial interactions, it is important to extract features that capture both the spatial and non-spatial driving factors. However, existing methods lack a robust interpretation and the explanatory power of the learned representations on spatial factors remains unexplored. Here, we propose an approach to learning place representations from spatial interactions. Our method is inspired by flow allocation, which is the main focus of single-constrained gravity models. We first validate the method on synthetic flows with known driving factors and then apply it to multi-scale real-world flows. Results show that the learned representations can effectively capture features that explain place characteristics, along with the impact of spatial impedance. Our study not only contributes an efficient method to learn place representations from spatial interactions but also offers insights into pre-training procedures in GeoAI.

HiCMAE: Hierarchical Contrastive Masked Autoencoder for self-supervised Audio-Visual Emotion Recognition

Audio-Visual Emotion Recognition (AVER) has garnered increasing attention in recent years for its critical role in creating emotion-aware intelligent machines. Previous efforts in this area are dominated by the supervised learning paradigm. Despite significant progress, supervised learning is meeting its bottleneck due to the longstanding data scarcity issue in AVER. Motivated by recent advances in self-supervised learning, we propose Hierarchical Contrastive Masked Autoencoder (HiCMAE), a novel self-supervised framework that leverages large-scale self-supervised pre-training on vast unlabeled audio-visual data to promote the advancement of AVER. Following prior arts in self-supervised audio-visual representation learning, HiCMAE adopts two primary forms of self-supervision for pre-training, namely masked data modeling and contrastive learning. Unlike them which focus exclusively on top-layer representations while neglecting explicit guidance of intermediate layers, HiCMAE develops a three-pronged strategy to foster hierarchical audio-visual feature learning and improve the overall quality of learned representations. Firstly, it incorporates hierarchical skip connections between the encoder and decoder to encourage intermediate layers to learn more meaningful representations and bolster masked audio-visual reconstruction. Secondly, hierarchical cross-modal contrastive learning is also exerted on intermediate representations to narrow the audio-visual modality gap progressively and facilitate subsequent cross-modal fusion. Finally, during downstream fine-tuning, HiCMAE employs hierarchical feature fusion to comprehensively integrate multi-level features from different layers. To verify the effectiveness of HiCMAE, we conduct extensive experiments on 9 datasets covering both categorical and dimensional AVER tasks. Experimental results show that our method significantly outperforms state-of-the-art supervised and self-supervised audio-visual methods, which indicates that HiCMAE is a powerful audio-visual emotion representation learner. Codes and models are publicly available at https://github.com/sunlicai/HiCMAE.