Representation Model Research Articles

Combining Semantic and Structural Features for Reasoning on Patent Knowledge Graphs

To address the limitations in capturing complex semantic features between entities and the incomplete acquisition of entity and relationship information by existing patent knowledge graph reasoning algorithms, we propose a reasoning method that integrates semantic and structural features for patent knowledge graphs, denoted as SS-DSA. Initially, to facilitate the model representation of patent information, a directed graph representation model based on the patent knowledge graph is designed. Subsequently, structural information within the knowledge graph is mined using inductive learning, which is combined with the learning of graph structural features. Finally, an attention mechanism is employed to integrate the scoring results, enhancing the accuracy of reasoning outcomes such as patent classification, latent inter-entity relationships, and new knowledge inference. Experimental results demonstrate that the improved algorithm achieves an up to approximately 30% increase in the MRR index compared to the ComplEx model in the public Dataset 1; in Dataset 2, the MRR and Hits@n indexes, respectively, saw maximal improvements of nearly 30% and 112% when compared with MLMLM and ComplEx models; in Dataset 3, the MRR and Hits@n indexes realized maximal enhancements of nearly 200% and 40% in comparison with the MLMLM model. This effectively proves the efficacy of the refined model in the reasoning process. Compared to recently widely applied reasoning algorithms, it offers a superior capability in addressing complex structures within the datasets and accomplishing the completion of existing patent knowledge graphs.

A Model Of Representation and Policy Search in Complex Environments

A Model Of Representation and Policy Search in Complex Environments

The Diagnostic Value of Systemic Immune-Inflammatory Index (SII) and Lymphocyte-Albumin-Neutrophil Ratio (LANR) in Chronic Obstructive Pulmonary Disease with Lung Cancer.

This study aims to explore the association of inflammatory markers such as the Systemic Immune-Inflammatory Index (SII) and LANR with the comorbidity of Chronic Obstructive Pulmonary Disease (COPD) and lung cancer. A cross-sectional analysis was conducted on the clinical data of 309 patients with COPD only and 193 patients with COPD and lung cancer who attended the First Affiliated Hospital of Anhui Medical University. Additionally, we examined autonomous risk factors that contribute to the simultaneous development of COPD and lung cancer through univariate and multivariate logistic regression analyses. This analysis resulted in the development of a nomogram model for visual representation. The effectiveness of the model was assessed using a receiver operating characteristic (ROC) curve, calibration curve, and clinical decision analysis (DCA) curve, with internal validation conducted via repeated sampling methods. Multivariate analysis of clinical characteristics and inflammatory markers showed that smoking history of >60 pack-years, hemoptysis, emphysema, WBC count > 5.53 (×10^9/L), SII > 629.285, and LANR < 11.39 were significantly associated with the comorbidity of COPD and lung cancer. These factors were subsequently integrated into the nomogram model. The AUC of the model stood at 0.849 (95% CI: 0.815-0.882), demonstrating a notable improvement over the COPD-LUCSS scoring system (AUC: 0.716, 95% CI: 0.671-0.761). The DCA curve indicated a notable clinical advantage provided by the model. Additionally, patients with stage IV tumors exhibited elevated SII levels and reduced LANR levels compared to earlier stages, indicating potential prognostic significance for both markers. Increased levels of the inflammatory markers SII and LANR are associated with the risk of comorbidity of COPD and lung cancer.

A combinatorial model for q-characters of fundamental modules of type Dn

In this paper, we introduce a combinatorial path model of representation of the quantum affine algebra of type D n , inspired by Mukhin and Young’s combinatorial path models of representations of the quantum affine algebras of types A n and B n . In particular, we give a combinatorial formula for q-characters of fundamental modules of type D n by assigning each path to a monomial or binomial. By counting our paths, a new expression on dimensions of fundamental modules of type D n is obtained.

The Smoke Flow Deflection Angles in Immersed Tunnel Fires with Multi-Point Concentrated Smoke Exhaust Mode: Representation Model and Influencing Mechanisms

The Smoke Flow Deflection Angles in Immersed Tunnel Fires with Multi-Point Concentrated Smoke Exhaust Mode: Representation Model and Influencing Mechanisms

Annotation Vocabulary (Might Be) All You Need.

Protein Language Models (pLMs) have revolutionized the computational modeling of protein systems, building numerical embeddings that are centered around structural features. To enhance the breadth of biochemically relevant properties available in protein embeddings, we engineered the Annotation Vocabulary, a transformer readable language of protein properties defined by structured ontologies. We trained Annotation Transformers (AT) from the ground up to recover masked protein property inputs without reference to amino acid sequences, building a new numerical feature space on protein descriptions alone. We leverage AT representations in various model architectures, for both protein representation and generation. To showcase the merit of Annotation Vocabulary integration, we performed 515 diverse downstream experiments. Using a novel loss function and only $3 in commercial compute, our premier representation model CAMP produces state-of-the-art embeddings for five out of 15 common datasets with competitive performance on the rest; highlighting the computational efficiency of latent space curation with Annotation Vocabulary. To standardize the comparison of de novo generated protein sequences, we suggest a new sequence alignment-based score that is more flexible and biologically relevant than traditional language modeling metrics. Our generative model, GSM, produces high alignment scores from annotation-only prompts with a BERT-like generation scheme. Of particular note, many GSM hallucinations return statistically significant BLAST hits, where enrichment analysis shows properties matching the annotation prompt - even when the ground truth has low sequence identity to the entire training set. Overall, the Annotation Vocabulary toolbox presents a promising pathway to replace traditional tokens with members of ontologies and knowledge graphs, enhancing transformer models in specific domains. The concise, accurate, and efficient descriptions of proteins by the Annotation Vocabulary offers a novel way to build numerical representations of proteins for protein annotation and design.

Scaffolded team-based computational modeling and simulation projects for promoting representational competence and regulatory skills

BackgroundThis study posits that scaffolded team-based computational modeling and simulation projects can support model-based learning that can result in evidence of representational competence and regulatory skills. The study involved 116 students from a second-year thermodynamics undergraduate course organized into 24 teams, who worked on three two-week-long team-based computational modeling and simulation projects and reflected upon their experience.ResultsResults characterized different levels of engagement with computational model-based learning in the form of problem formulation and model planning, implementation and use of the computational model, evaluation, and interpretation of the outputs of the model, as well as reflection on the process. Results report on students’ levels of representational competence as related to the computational model, meaning-making of the underlying code of the computational model, graphical representations generated by the model, and explanations and interpretations of the output representations. Results also described regulatory skills as challenges and strategies related to programming skills, challenges and strategies related to meaning-making skills for understanding and connecting the science to the code and the results, and challenges and strategies related to process management mainly focused on project management skills.ConclusionCharacterizing dimensions of computational model-based reasoning provides insights that showcase students’ learning, benefits, and challenges when engaging in team-based computational modeling and simulation projects. This study also contributes to evidence-based scaffolding strategies that can support undergraduate students' engagement in the context of computational modeling and simulation.

A Context Awareness System for Clinical Environments

This study addresses the complex management of patient-related information in hospitals and clinical settings. This information includes treatments, medications, vital signs, patient locations, and data exchange between healthcare professionals. The lack of effective synchronization between these elements often delays timely care. This study proposes an architecture based on a semantic representation model that articulates the various components of a hospital environment. This model supports decision-making in healthcare by facilitating inferences from the environment. The semantic model serves as a basis for executing predefined rules that trigger actions through a reasoner, resulting in notifications, such as administering medications or responding to abnormal vital signs. The model integrates supervised learning to improve the accuracy of alerts. The experiment focused on monitoring vital sign parameters, such as Spo2, body temperature, and heart rate. The combination of semantic representation modeling and machine learning algorithms demonstrates a robust approach for improving the efficiency and accuracy of healthcare alerts in clinical settings.

Nonlinear dynamic modeling and vibration analysis for flexible tethered satellite systems under large deformations

The escalating threat posed by space debris necessitates innovative approaches for its removal. This article presents a comprehensive mathematical modeling and dynamic analysis of a flexible tethered satellite system (FTSS) in the post-capture phase, taking into account nonlinear strain effects for the flexible appendages. The FTSS consists of a rigid central body satellite, two identical flexible panels, and a towing mass that manipulates the satellite through a tether. We develop a detailed dynamic model that accurately represents the system’s three-dimensional motion, incorporating the rigid body’s 6 degrees of freedom (DOF), the tether’s 3 DOF motion, and two modal generalized coordinates for each panel. The extended Hamilton method is employed to derive this model, enabling us to investigate the conditions under which nonlinear strain considerations become critical. The findings of this article reveal that nonlinear strain effects can profoundly impact the system’s dynamics under certain geometric and forcing conditions, often overlooked in previous studies. In these scenarios, the panel strains exceed the linear regime, necessitating the use of nonlinear models for accurate representation. This work emphasizes the importance of incorporating nonlinear strain terms in the dynamic analysis of FTSSs, particularly when dealing with large deformations and high-amplitude vibrations encountered in debris removal applications.

Collaborative trajectory representation for enhanced next POI recommendation

Point-of-Interest (POI) recommendation stands as the cornerstone within a variety of location-based applications and services that intend to anticipate upcoming movements that users may be interested in. The current state-of-the-art methods have effectively explored spatio-temporal contextual features and users’ long-term and short-term preference patterns. Nevertheless, most existing work lacks the ability to effectively capture group movement patterns from trajectory collaboration. Additionally, they pay close attention to the accuracy of personalized recommendations, neglecting recommendation diversity, which refers to offering broader location options that extend beyond a user’s typical preferences, thereby avoiding excessive homogeneity or repetition. To address these gaps, this study proposes a Collaborative Trajectory Representation model (CTRNext), which enhances the diversity of recommendations while maintaining the precision of personalized preferences. To be specific, we first design two trajectory embedding layers to extract joint semantic interactions and the explicit spatiotemporal context-aware representation. Then, a trajectory semantic similarity calculation module that captures collaborative signals from potentially similar-minded users and eliminates barriers caused by trajectory length is proposed. Next, the implicit correlation and further updated representation between different check-in records are achieved through a multi-head self-attention aggregation module. Finally, we put forward a dual-driven user preference matching module to generate the preference-based next POI recommendation while enhancing diversity. Our approach demonstrates its remarkable recommendation accuracy through extensive experimentation on four real-world datasets, surpassing the performance of state-of-the-art methodologies.

BartSmiles: Generative Masked Language Models for Molecular Representations.

We discover a robust self-supervised strategy tailored toward molecular representations for generative masked language models through a series of tailored, in-depth ablations. Using this pretraining strategy, we train BARTSmiles, a BART-like model with an order of magnitude more compute than previous self-supervised molecular representations. In-depth evaluations show that BARTSmiles consistently outperforms other self-supervised representations across classification, regression, and generation tasks, setting a new state-of-the-art on eight tasks. We then show that when applied to the molecular domain, the BART objective learns representations that implicitly encode our downstream tasks of interest. For example, by selecting seven neurons from a frozen BARTSmiles, we can obtain a model having performance within two percentage points of the full fine-tuned model on task Clintox. Lastly, we show that standard attribution interpretability methods, when applied to BARTSmiles, highlight certain substructures that chemists use to explain specific properties of molecules. The code and pretrained model are publicly available.

Cross-view motion consistent self-supervised video inter-intra contrastive for action representation understanding

Self-supervised contrastive learning draws on power representational models to acquire generic semantic features from unlabeled data, and the key to training such models lies in how accurately to track motion features. Previous video contrastive learning methods have extensively used spatially or temporally augmentation as similar instances, resulting in models that are more likely to learn static backgrounds than motion features. To alleviate the background shortcuts, in this paper, we propose a cross-view motion consistent (CVMC) self-supervised video inter-intra contrastive model to focus on the learning of local details and long-term temporal relationships. Specifically, we first extract the dynamic features of consecutive video snippets and then align these features based on multi-view motion consistency. Meanwhile, we compare the optimized dynamic features for instance comparison of different videos and local spatial fine-grained with temporal order in the same video, respectively. Ultimately, the joint optimization of spatio-temporal alignment and motion discrimination effectively fills the challenges of the missing components of instance recognition, spatial compactness, and temporal perception in self-supervised learning. Experimental results show that our proposed self-supervised model can effectively learn visual representation information and achieve highly competitive performance compared to other state-of-the-art methods in both action recognition and video retrieval tasks.

Survey on Knowledge Representation Models in Healthcare

Knowledge representation models that aim to present data in a structured and comprehensible manner have gained popularity as a research focus in the pursuit of achieving human-level intelligence. Humans possess the ability to understand, reason and interpret knowledge. They acquire knowledge through their experiences and utilize it to carry out various actions in the real world. Similarly, machines can also perform these tasks, a process known as knowledge representation and reasoning. In this survey, we present a thorough analysis of knowledge representation models and their crucial role in information management within the healthcare domain. We provide an overview of various models, including ontologies, first-order logic and rule-based systems. We classify four knowledge representation models based on their type, such as graphical, mathematical and other types. We compare these models based on four criteria: heterogeneity, interpretability, scalability and reasoning in order to determine the most suitable model that addresses healthcare challenges and achieves a high level of satisfaction.

Machine Learning Augmentation of the Failure Assessment Diagram Methodology for Enhanced Tubular Structures Integrity Evaluation

Failure assessment diagrams (FADs) are essential engineering tools for evaluating the structural integrity of components. However, their widespread application can be limited by complexity and computational expense. This study presents a novel machine learning-based approach to streamline FAD analysis, offering accuracy and efficiency while overcoming these limitations. The approach integrates numerical contour integral-based FADs with artificial neural networks (ANNs). To ensure reliable material modeling for the Finite Element Analysis (FEA) used to generate J-integral based FADs that train the ANNs, careful experimental and numerical procedures were employed. This involved uniaxial tensile tests, an iterative method for obtaining precise true stress–strain curves, and a Ramberg–Osgood material model for accurate material behavior representation. The ANNs themselves not only analyze large datasets to generate precise FAD envelopes but also predict limit loads and the Φ parameter, incorporating the effect of residual stress on the FAD methodology. To verify and test the proposed method, hypothetical fitness-for-service assessment cases were conducted, incorporating experimental residual stress measurements from split-ring tests on P110 and L80 pipes. These assessments were compared to both traditional FAD methods and computationally intensive FEA-based FADs. Results demonstrate a closer agreement with FEA-based calculations than traditional methods provided in engineering standards. Ultimately, this work provides a rather innovative and adaptable approach for structural integrity evaluations and critical engineering assessments through the proposal of an ANN enhanced FAD approach, simplifying these calculations while maintaining high fidelity.

Comprehensive Experimental Database and Model Fitting for Electric Arc Behavior in Aircraft Environments

The More Electric Aircraft (MEA) paradigm advocates weight reduction by transitioning to electrical components, resulting in increased system voltage and higher arc fault risk. This paper presents a methodology for creating a database of nearly one thousand tests simulating parallel arc faults, using Andrea's arc model for accurate representation. Objectives include the creation of a robust experimental database and the use of a simplified model to analyze arcing behavior. The setup follows the UL 746A standard and considers various pressure and electrode separation conditions. The database includes 960 experiments with PVC and PTFE materials, and model fitting ensures accurate representation of arc currents and voltages. Error analysis shows consistent model performance across materials and highlights sensitivity to electrode spacing, pressure, and source voltage. The paper concludes with a clear 30% error threshold for model validity, enhancing the understanding of the applicability of Andrea's model under various experimental scenarios in aircraft environments.

Research on the dynamic spread of information in social networks based on relationship strength theory and feedback mechanism

Introduction: Studying the main factors and related paths of rumor propagation contributes to the precise governance of rumor information in social networks. Most existing network representation learning methods do not fit with real-world information propagation networks well, and the network cannot effectively model the temporal characteristics and dynamic evolution features of rumor information propagation.Methods: Our study proposes a new dynamic network representation model for information propagation. Additionally, the study introduces a feedback mechanism where the latest node representations are fed back to neighboring nodes.Results: The method solves the problem of delayed network representation and improves network representation performance.Discussion: We conducted experimental simulations, and the results indicate that a higher level of trust contributes to stable group relationships and converging opinions, reducing the likelihood of opinion dispersion. Furthermore, novelty of topics, and interactivity between users, and opinion leaders exhibit distinct characteristics in guiding public opinion. The viewpoint evolution of the newly constructed dynamic network representation model aligns more closely with viewpoint evolution in real-world social networks.

Exploring temporal sensitivity in the brain using multi-timescale language models: an EEG decoding study

Abstract The brain’s ability to perform complex computations at varying timescales is crucial, ranging from understanding single words to grasping the overarching narrative of a story. Recently, multi-timescale long short-term memory (MT-LSTM) models (Mahto et al. 2020; Jain et al. 2020) have been introduced, which use temporally-tuned parameters to induce sensitivity to different timescales of language processing (i.e. related to near/distant words). However, there has not been an exploration of the relation between such temporally-tuned information processing in MT-LSTMs and the brain’s language processing using high temporal resolution recording modalities, such as electroencephalography (EEG). To bridge this gap, we used an EEG dataset recorded while participants listened to Chapter 1 of “Alice in Wonderland” and trained ridge regression models to predict the temporally-tuned MT-LSTM embeddings from EEG responses. Our analysis reveals that EEG signals can be used to predict MT-LSTM embeddings across various timescales. For longer timescales, our models produced accurate predictions within an extended time window of ±2 s around word onset, while for shorter timescales, significant predictions are confined to a narrow window ranging from −180 ms to 790 ms. Intriguingly, we observed that short timescale information is not only processed in the vicinity of word onset but also at distant time points. These observations underscore the parallels and discrepancies between computational models and the neural mechanisms of the brain. As word embeddings are used more as in silico models of semantic representation in the brain, a more explicit consideration of timescale-dependent processing enables more targeted explorations of language processing in humans and machines.

Design of Locally Resonant Acoustic Metamaterials with Specified Band Gaps Using Multi-Material Topology Optimization.

Locally Resonant Acoustic Metamaterials (LRAMs) have significant application potential because they can form subwavelength band gaps. However, most current research does not involve obtaining LRAMs with specified band gaps, even though such LRAMs are significant for practical applications. To address this, we propose a parameterized level-set-based topology optimization method that can use multiple materials to design LRAMs that meet specified frequency constraints. In this method, a simplified band-gap calculation approach based on the homogenization framework is introduced, establishing a restricted subsystem and an unrestricted subsystem to determine band gaps without relying on the Brillouin zone. These subsystems are specifically tailored to model the phenomena involved in band gaps in LRAMs, facilitating the opening of band gaps during optimization. In the multi-material representation model used in this method, each material, except for the matrix material, is depicted using a similar combinatorial formulation of level-set functions. This model reduces direct conversion between materials other than the matrix material, thereby enhancing the band-gap optimization of LRAMs. Two problems are investigated to test the method's ability to use multiple materials to solve band-gap optimization problems with specified frequency constraints. The first involves maximizing the band-gap width while ensuring it encompasses a specified frequency range, and the second focuses on obtaining light LRAMs with a specified band gap. LRAMs with specified band gaps obtained in three-material or four-material numerical examples demonstrate the effectiveness of the proposed method. The method shows great promise for designing metamaterials to attenuate specified frequency spectra as required, such as mechanical vibrations or environmental noise.

Representative co-location pattern post-mining based on maximal row instances representation model

The mining result set of spatial prevalent co-location patterns(SPCPs) is often large and redundant, especially when the prevalence threshold is set to low or long SPCPs are present. Meanwhile, the distribution of SPCPs in continuous space and complex spatial relationships with each other of spatial data make the compression and reorganization of SPCPs a challenging problem. To solve this problem, in this paper, a representative co-location pattern mining framework based on the maximal row instance(MRI) representation model is proposed. First, the MRI representation model is proposed to effectively preserve the pattern distribution information of prevalent co-location patterns. To establish the MRI representation model, the basic algorithm and geometric algorithm are proposed in this paper. Two materialization methods based on the MRI representation model, 0-1 vector and key–value vector, are presented. Secondly, the similarity measure of SPCPs under the context of the MRI representation model is proposed, which calculates the similarity between any two co-location patterns without adding additional information such as domain background. Furthermore, the mining framework based on the k nearest neighbors density peak clustering algorithm is presented to extract representative co-location patterns. Finally, the efficiency and scalability of the proposed method are verified on the synthetic data and real data. Compared with the existing methods, the representative co-location patterns of the proposed method have well compression performances.

Generative subgoal oriented multi-agent reinforcement learning through potential field

Multi-agent reinforcement learning (MARL) effectively improves the learning speed of agents in sparse reward tasks with the guide of subgoals. However, existing works sever the consistency of the learning objectives of the subgoal generation and subgoal reached stages, thereby significantly inhibiting the effectiveness of subgoal learning. To address this problem, we propose a novel Potential field Subgoal-based Multi-Agent reinforcement learning (PSMA) method, which introduces the potential field (PF) to unify the two-stage learning objectives. Specifically, we design a state-to-PF representation model that describes agents’ states as potential fields, allowing easy measurement of the interaction effect for both allied and enemy agents. With the PF representation, a subgoal selector is designed to automatically generate multiple subgoals for each agent, drawn from the experience replay buffer that contains both individual and total PF values. Based on the determined subgoals, we define an intrinsic reward function to guide the agent to reach their respective subgoals while maximizing the joint action-value. Experimental results show that our method outperforms the state-of-the-art MARL method on both StarCraft II micro-management (SMAC) and Google Research Football (GRF) tasks with sparse reward settings.