Implicit Dependencies Research Articles

TEA-GCN: Transformer-Enhanced Adaptive Graph Convolutional Network for Traffic Flow Forecasting.

Traffic flow forecasting is crucial for improving urban traffic management and reducing resource consumption. Accurate traffic conditions prediction requires capturing the complex spatial-temporal dependencies inherent in traffic data. Traditional spatial-temporal graph modeling methods often rely on fixed road network structures, failing to account for the dynamic spatial correlations that vary over time. To address this, we propose a Transformer-Enhanced Adaptive Graph Convolutional Network (TEA-GCN) that alternately learns temporal and spatial correlations in traffic data layer-by-layer. Specifically, we design an adaptive graph convolutional module to dynamically capture implicit road dependencies at different time levels and a local-global temporal attention module to simultaneously capture long-term and short-term temporal dependencies. Experimental results on two public traffic datasets demonstrate the effectiveness of the proposed model compared to other state-of-the-art traffic flow prediction methods.

Dynamic Spatial-Temporal Embedding via Neural Conditional Random Field for Multivariate Time Series Forecasting

How to capture dynamic spatial-temporal dependencies remains an open question in multivariate time series (MTS) forecasting. Although recent advanced spatial-temporal graph neural networks (STGNNs) achieve superior forecasting performance, they either consider pre-defined spatial correlations or simply learn static graphs. Some research has tried to learn many adjacent matrices to reveal time-varying spatial correlations, but they generate discrete graphs which cannot encode evolutionary information and also face computational complexity problem. In this article, we propose two significant plugins to help automatically learn enhanced dynamic spatial-temporal embedding of MTS data: (1) a novel neural conditional random field (CRF) layer. We find that the implicit time-varying spatial dependencies are reflected by the explicit changeable links between edges, and we propose the neural CRF to encode such pairwise changeable evolutionary inter-dependencies; (2) a structure adaptive graph convolution (SAGC) that does not require pre-defined graphs to capture semantically richer spatial correlations. Then, we integrate the neural CRF, SAGC with recurrent neural network to develop a new STGNN paradigm termed Adaptive Spatial-Temporal graph neural network with Conditional Random Field (ASTCRF), which can be trained in an end-to-end fashion. We validate the effectiveness, efficiency, and scalability of ASTCRF on five public benchmark MTS datasets.

Entity Linking Model Based on Cascading Attention and Dynamic Graph

The purpose of entity linking is to connect entity mentions in text to real entities in the knowledge base. Existing methods focus on using the text topic, entity type, linking order, and association between entities to obtain the target entities. Although these methods have achieved good results, they ignore the exploration of candidate entities, leading to insufficient semantic information among entities. In addition, the implicit relationship and discrimination within the candidate entities also affect the accuracy of entity linking. To address these problems, we introduce information about candidate entities from Wikipedia and construct a graph model to capture implicit dependencies between different entity decisions. Specifically, we propose a cascade attention mechanism and develop a novel local entity linkage model termed CAM-LEL. This model leverages the interaction between entity mentions and candidate entities to enhance the semantic representation of entities. Furthermore, a global entity linkage model termed DG-GEL based on a dynamic graph is established to construct an entity association graph, and a random walking algorithm and entity entropy are used to extract the implicit relationships within entities to increase the differentiation between entities. Experimental results and in-depth analyses of multiple datasets show that our model outperforms other state-of-the-art models.

Parallel multi-scale dynamic graph neural network for multivariate time series forecasting

Accurately modeling and predicting multivariate time series (MTS) are crucial in real-world scenarios, where precise predictions aid decision-making. MTS exhibit complex variable relationships and dynamic evolution patterns across various time scales, posing significant modeling challenges. To address these, a parallel multi-scale dynamic graph neural network (PMEDGN) is proposed for parallel modeling of dynamic information across multiple time scales. Spatial-temporal embedding module based on spatial-temporal attention mechanisms and multi-scale dynamic graph generation module are designed. These modules capture implicit spatial-temporal dependencies and self-learn to generate multi-scale dynamic graph structures from MTS without predefined graphs, automatically obtaining the importance of different time scale patterns. Additionally, a global graph convolution module is developed to integrate parallel multi-scale information, enhancing collaboration across time scales for final predictions. Comprehensive experiments on real-world datasets demonstrate the effectiveness and superiority of PMEDGN over state-of-the-art methods, underscoring its potential for practical applications.

Построение онтологии для систематизации характеристик сети Интернета вещей

A formalization of the Internet of Things network model designed for monitoring technological premises with tele-communications equipment at the Federal Research Center "Krasnoyarsk Scientific Center SB RAS" is presented. The network includes measuring devices, a telecommunications environment, data collection servers, and application soft-ware. For information interaction, a “publisher-subscriber” scheme and a lightweight protocol with a low load on com-munication channels are used. An ontology has been created that describes the network architecture and the properties of devices that collect, transmit, store, and process data. The ontology contains classes representing the concepts of the subject area, relationships, data properties, ranges of their changes, and critical values that limit the attributes of ontol-ogy elements. Ontology objects have their own digital representation in databases, including measurement results ob-tained by Internet of Things network sensors, precedents of anomalous data, and their statistical and frequency char-acteristics. This formalization made it possible to identify implicit dependencies between objects, connect them with the characteristics of processes observed by Internet of Things network devices, and solve practical tasks. The problem of selecting characteristics that influence changes in information interaction patterns is considered. A survey of experts was carried out, and a Kano model was built to prioritize the characteristics that influence decision-making on the or-ganization of an information interaction scheme in the Internet of Things network.

A Vulnerability Scanning Method for Web Services in Embedded Firmware

As the Internet of Things (IoT) era arrives, the proliferation of IoT devices exposed to the Internet presents a significant challenge to device security. Firmware is software that operates within Internet of Things (IoT) devices, directly governing their behaviors and functionalities. Consequently, the security of firmware is critical to shielding IoT devices from potential threats. In order to enable users to operate a device intuitively, firmware commonly provides a web interface. Consequently, this interface frequently serves as the primary attack goal in Internet of Things (IoT) devices, rendering them susceptible to numerous cyber-attacks. Unfortunately, web services have complex data interactions and implicit dependencies, and it is not easy to balance efficiency and accuracy during the analysis process, leading to heavy overhead. This paper proposes a lightweight vulnerability scanning approach, WFinder, designed explicitly for embedded firmware web services to perform vulnerability checks on backend binary files in firmware. WFinder uses static analysis to focus on identifying vulnerabilities in boundary binary files related to web services in firmware. Initially, the approach identifies boundary binary files and external data entry points based on front-end and back-end associativity features. Subsequently, rules are formulated to filter hazardous functions to narrow the analysis targets. Finally, the method generates sensitive call paths from the external data input points to the hazardous functions and conducts a lightweight taint analysis along these paths to uncover potential vulnerabilities. We implemented a prototype of WFinder and evaluated it on the firmware of ten devices from five well-known manufacturers. We discovered thirteen potential vulnerabilities, eight of which were confirmed by the CNVD, and assigned them CNVD identification numbers. Compared with the most advanced tool, SATC, WFinder was more efficient at discovering more bugs on the test set. These results indicate that WFinder is effective at detecting bugs in embedded web services.

Prevalence and severity of design anti-patterns in open source programs—A large-scale study

Context:Design anti-patterns can be symptoms of problems that lead to long-term maintenance difficulty. How should development teams prioritize their treatment? Which ones are more severe and deserve more attention? Does the impact of anti-patterns and general maintenance efforts differ with different programming languages? Objective:In this study, we assess the prevalence and severity of anti-patterns in different programming languages and the impact of dynamic typing in Python, as well as the impact scopes of prevalent anti-patterns that manifest the violation of design principles. Method:We conducted a large-scale study of anti-patterns using 1717 open-source projects written in Java, C/C++, and Python. For the 288 Python projects, we extracted both explicit and dynamic dependencies and compared how the detected anti-patterns and maintenance costs changed. Finally, we removed anti-patterns involving five or fewer files to assess the impact of trivial anti-patterns. Results:The results reveal that 99.55% of these projects contain anti-patterns. Modularity Violation – frequent co-changes among seemingly unrelated files – is most prevalent (detected in 83.54% of all projects) and costly (incurred 61.55% of maintenance effort on average). Unstable Interface and Crossing, caused by influential but unstable files, although not as prevalent, tend to incur severe maintenance costs. Duck typing in Python incurs more anti-patterns, and the churn spent on Python files multiplies that of C/C++ and Java files. Several prevalent anti-patterns have a large portion of trivial instances, meaning that these common symptoms are usually not harmful. Conclusion:Implicit and visible dependencies are the most expensive to maintain, and dynamic typing in Python exacerbates the issue. Influential but unstable files need to be monitored and rectified early to prevent the accumulation of high maintenance costs. The violations of design principles are widespread, but many are not high-maintenance.

Анализ методов подготовки и преобразования информации, поступающей в хранилища данных для эффективного управления горнотехнической системой

The article assesses the existing methods of data collection and processing in the industry and proposes an alternative option of data processing to manage mining engineering systems at various stages of their operation. It is demonstrated that the formed volume of information that comes with different frequency rates and requires dedicated processing, structuring and analysis methods forms a system of big data and serves to enhance the efficiency of implementing geotechnical processes. A possible unified structure of data acquisition from digital sources of mining engineering system is presented. An analysis is provided of possible methods to process data for analytical purposes and for searching implicit dependencies and solving problems of predictive analytics. Tools for data collection and storage are proposed to create a unified system of big data analysis for management of mining engineering systems. It is stated that in order to create a unified analytical system for collection of digital data from a mining system, it is necessary to use standard industrial protocols for data collection and storage, for instance MQTT, used as an industry standard in the Industrial Internet of Things (IIoT) systems in accordance with requirements of ISO/ IEC 20922:2016. Storage requires the use of a conventional queue broker architecture, as well as a tool for working with the time series which is required to apply machine learning and big data methods. The approach to data classification in terms of the data acquisition speed proposed in the article makes it possible to standardize the data handling principles. Since the volume of transmitted data does not depend on the frequency of acquiring information from a digital source, it is proposed to transmit all the generated data from a digital source for subsequent search of implicit dependencies between the data. It is noted that application of specific methods and algorithms to analyze data of a mining system depends primarily on the task set which is often formed as a hypothesis to be tested by identifying implicit dependencies between different data sources. In order to improve the efficiency of managing a mining system at all the stages of field development, it is proposed to apply the ELT approach, which can be an important advantage in terms of controlling the technological processes of the mining system in the future.

Towards an understanding of intra-defect associations: Implications for defect prediction

In previous studies, when collecting defect data, if the fix of a defect spans multiple modules, each involved module is labeled as defective. In this context, the defect prediction models are built based on the features of each individual module, ignoring the potential associations between the modules involved in the same defect(referred to as “intra-defect associations”). Considering the possibility of numerous cross-module defects in practice, we hypothesize that these intra-defect associations could play a crucial role in enhancing defect prediction performance. Unfortunately, there is no empirical evidence to know that. To this end, we are motivated to conduct a comprehensive study to explore the implications of intra-defect associations for defect prediction. We first examine the proportion of cross-module defects and the relationships between the involved modules. The results reveal that, at function level, the majority of defects occur across functions, with most of the cross-module defects exhibiting implicit dependencies. Inspired by these findings, we propose a novel data processing approach for building defect prediction models. This approach leverages the intra-defect associations by merging the involved modules into new instances with mean or median variables to augment the training data. The experimental results indicate that considering intra-defect associations can significantly improve the defect prediction performance in both the ranking and classification scenarios. This study provides valuable insights into the implications of intra-defect associations for defect prediction.

Categorization of crowd-sensing streaming data for contextual characteristic detection

The growing reliance on large wireless sensor networks, potentially consisting of hundreds of nodes, to monitor real-world phenomena inevitably results in large, complex datasets that become increasingly difficult to process using traditional methods. The inadvertent inclusion of anomalies in the dataset, resulting from the inherent characteristics of these networks, makes it difficult to isolate interesting events from erroneous measurements. Simultaneously, improvements in data science methods, as well as increased accessibility to powerful computers, lead to these techniques becoming more applicable to everyday data mining problems. In addition to being able to process large amounts of complex streaming data, a wide array of specialized data science methods enables complex analysis not possible using traditional techniques. Using real-world streaming data gathered by a temperature sensor network consisting of approximately 600 nodes, various data science methods were analyzed for their ability to exploit implicit dependencies embedded in unlabelled data to solve the complex task to identify contextual characteristics. The methods identified during this analysis were included in the construction of a software pipeline. The constructed pipeline reduced the identification of characteristics in the dataset to a trivial task, the application of which led to the detection of various characteristics describing the context in which sensors are deployed.

Relation-enhanced Diffusion Model with LightGCN for Information Diffusion Prediction

Modeling and predicting the information diffusion process on social platforms is a critical task in many real-world applications. Recent studies generally model the diffusion graph using graph neural networks to capture the implicit dependencies among users. However, existing studies construct the diffusion graph in a way which cannot fully describe the global dependencies of users due to their narrow definition of user relationship. Meanwhile, graph neural networks in these methods are not suitable for the social network scenario which has scarce node attributes. Therefore, we propose a novel diffusion graph construction method which can enhance relations among users and adopt a simplified graph convolutional operation which is suitable for diffusion prediction scenario. The learned user embedding in our model can effectively preserve the microscopic structure and the high-order proximities between users lies in both the social graph and diffusion graph. Experimental results on four real-world datasets show that the proposed model is superior to the most advanced information diffusion prediction methods.

Propagation of fundamental Lamb modes along the non-principal axes of strain-stiffened soft compressible plates: A numerical investigation.

This paper numerically investigates the propagation of elastic plate waves along the non-principal directions in a prestretched compressible material described by the Gent model of hyperelasticity. We formulate the elastic tensor and the underlying wave equations in the Lagrangian space by employing the theory of nonlinear elasticity together with the linearized incremental equations. An extension of the Semi-Analytical Finite Element (SAFE) method is discussed for computing the dispersion characteristics of the two fundamental guided wave modes. The predictive capabilities of the numerical framework are established using the previously published data for a weakly nonlinear as well as hyperelastic material models. Using the numerical framework, we then bring out the effects of applied prestretch, orientation of the propagation direction, and material parameters on the dispersion characteristics of the fundamental Lamb modes. A limiting case of the neo-Hookean material model is first considered for elucidating such implicit dependencies, which are further highlighted by considering the strain-stiffening effect captured through the Gent material model. Our results indicate the existence of a threshold prestretch for which the Gent-type material can encounter a snap-through instability; leading to the change in the dispersion characteristics of the fundamental symmetric Lamb mode.

Determination of the Optimal Algorithm for the Operation of a Hybrid Power System with a Regulating Generator

The purpose of the article is to study the possibilities of preventing the imbalance caused by the stochastic nature of RES by regulating the electricity accumulation and the backup generator. The subject of the research is random processes of energy generation and consumption. The goal is achieved by a mathematical model construction in which the system parameters would be formalized and allow the choice of optimal options. The mathematical model uses random functions to obtain statistical estimates, and is implemented as sequential models of energy generation and consumption as an integrated random process. The models’ adequacy is confirmed by comparing the simulated and real indicators for a number of existing RES in Ukraine. The proposed approach made it possible to establish the presence of system parameters that minimize energy losses and consumption, reduce the probability of energy imbalance, and effectively use the reserve power. The proposed mathematical model allows determining the probability of various system states, it estimates the reliability of energy supply, and minimizes losses at the stages of designing a power system and planning its operating modes. The above estimates are typical for certain climatic conditions and modes of energy consumption, and are of a specific nature. The significance of the work lies in the fact that the proposed formulation of the problem makes it possible to adapt the mathematical model to various conditions, to find implicit dependencies between the technical parameters of the system and its indicators in order to find optimal solutions.

On the Transferability of Pre-trained Language Models: A Study from Artificial Datasets

Pre-training language models (LMs) on large-scale unlabeled text data makes the model much easier to achieve exceptional downstream performance than their counterparts directly trained on the downstream tasks. In this work, we study what specific traits in the pre-training data, other than the semantics, make a pre-trained LM superior to their counterparts trained from scratch on downstream tasks. We propose to use artificially constructed datasets as the pre-training data to exclude the effect of semantics, and further control what characteristics the pre-training corpora have. By fine-tuning the pre-trained models on GLUE benchmark, we can learn how beneficial it is to transfer the knowledge from the model trained on the dataset possessing that specific trait. We define and discuss three different characteristics in the artificial dataset: 1) matching the token's uni-gram or bi-gram distribution between pre-training and downstream fine-tuning, 2) the presence of the explicit dependencies among the tokens in a sequence, 3) the length of the implicit dependencies among the tokens in a sequence. Our experiments show that the explicit dependencies in the sequences of the pre-training data are critical to the downstream performance. Our results also reveal that models achieve better downstream performance when pre-trained on a dataset with a longer range of implicit dependencies. Based on our analysis, we find that models pre-trained with artificial datasets are prone to learn spurious correlation in downstream tasks. Our work reveals that even if the LMs are not pre-trained on natural language, they still gain transferability on certain human language downstream tasks once the LMs learn to model the token dependencies in the sequences. This result helps us understand the exceptional transferability of pre-trained LMs.

Multi-Behavior Sequential Recommendation with Temporal Graph Transformer

Modeling time-evolving preferences of users with their sequential item interactions, has attracted increasing attention in many online applications. Hence, sequential recommender systems have been developed to learn the dynamic user interests from the historical interactions for suggesting items. However, the interaction pattern encoding functions in most existing sequential recommender systems have focused on single type of user-item interactions. In many real-life online platforms, user-item interactive behaviors are often multi-typed (e.g., click, add-to-favorite, purchase) with complex cross-type behavior inter-dependencies. Learning from informative representations of users and items based on their multi-typed interaction data, is of great importance to accurately characterize the time-evolving user preference. In this work, we tackle the dynamic user-item relation learning with the awareness of multi-behavior interactive patterns. Towards this end, we propose a new Temporal Graph Transformer (TGT) recommendation framework to jointly capture dynamic short-term and long-range user-item interactive patterns, by exploring the evolving correlations across different types of behaviors. The new TGT method endows the sequential recommendation architecture to distill dedicated knowledge for type-specific behavior relational context and the implicit behavior dependencies. Experiments on the real-world datasets indicate that our method TGT consistently outperforms various state-of-the-art recommendation methods. Our model implementation codes are available at https://github.com/akaxlh/TGT.

Unified IT&OT Modeling for Cybersecurity Analysis of Cyber-Physical Systems

Cyber-Physical Systems (CPSs) engineering profoundly relies on modeling methods to represent the system and study the operation and cybersecurity of CPSs. The operation of a CPS is the result of the collaboration between Information Technology (IT) and Operational Technology (OT) components. While OT focuses on the system's process physics, the emphasis of IT is on information flow. Consequently, different system models are utilized to study various aspects of CPSs, which may infer different views of the same system. The increasing complexity of CPSs and the high number of cyberattacks against Industrial Control Systems (ICSs) and CPSs in recent years have highlighted the necessity of considering these interrelations based on a unified model to analyze cybersecurity of CPSs. However, the diversity of engineering fields and implicit relations and dependencies between them have made it difficult to integrate the modeling methods towards a unified IT&OT model of CPSs.In this paper, we propose a comprehensive method, based on bond graphs, to model CPS and analyze their cybersecurity. Unlike existing methods, modeling the cyber layer along with the physical layer based on the system flow provides a holistic graphical representation of a CPS, which facilitates collaboration between IT and OT experts.

Development of an Information Model for the Personality’s Social Portrait Formation Using OSINT Technology

The results of the development of an information model for the personality’s social portrait formation are presented. The modelling has been carried out using OSINT technology that is the technology of legal obtaining and using open source information. In the result of the analysis, it has been found out that the social portrait is a heterogeneous semantic network consisting of personalized data. It has been defined that people organize formal and official communities of various orientations and the number of such communities associated with a particular person is practically unlimited. When formalizing the decision-making process, the concept of a group social portrait (GSP) has been introduced, which takes into account the community’s social tendencies united by certain common properties, group members' interpersonal interactions and their behavioural patterns. The obtained information models of personal and group social portraits let to take into account all the main properties of the objects under study, their tonality and significance, as well as to conduct an analysis of the implicit dependencies determination. The next step is to move on to considering the diversity of the digital social environment elements.

Explication functions in the conditions of professionalization of thinking

The article substantiates that the explication method in conditions of professionalization of the subject’s thinking performs various functions. Explication allows for: firstly, the translation of unclear and uncertain intuitive representations (explicands) to the form of strictin categorical, semantic and logical form concepts (explicans); secondly, the conceptualization of a certain area of professional knowledge, bringing it into a coherent, effective system; thirdly, the detection and accounting of implicit dependencies between various knowable and transformable phenomena; fourthly, the implementation of supra-situational thinking that ensures the acquisition of valuable personal and professional neoformations by the subject. Explication (expression, distribution) of the selected professional thinking featuresof the subject as anbeingexplicatedphenomenonis carried out by differentiating the knowable phenomena: spiritual as a manifestation of the power and content of inner life; social as a refraction of social requirements in professional activity; environmental as the integration and understanding of those environmental conditions that become the means of operational coping with the difficulties encountered; resource as the implementation and multiplication of resources, primarily internal as the basis for professional and personal self-development of the subject.

Social Boosted Recommendation With Folded Bipartite Network Embedding

With the prevalence of online social platforms, social recommendation has emerged as a promising direction that leverages the social network among users to enhance recommendation performance. However, the available social relations among users are usually extremely sparse and noisy, which may lead to inferior recommendation performance. To alleviate this problem, this paper novelly exploits the implicit higher-order social influence and dependencies among users to enhance social recommendation. In this paper, we propose a novel embedding method for general bipartite graphs, which defines inter-class message passing between explicit relations and intra-class message passing between implicit higher-order relations via a novel sequential modelling paradigm. Inspired by recent advances in self-attention-based sequential modelling, the proposed model features a self-attentive representation learning mechanism for implicit user-user relations. Moreover, this paper also explores the inductive embedding learning for social recommendation problems to improve the recommendation performance in cold-start settings. The proposed inductive learning paradigm for social recommendation enables embedding inference for those cold-start users and items (unseen during training) as long as they are linked to existing nodes in the original network. Extensive experiments on real-world datasets demonstrate the superiority of our method and suggest that higher-order implicit relationship among users is beneficial to improving social recommendation.

Theoretical modeling of crystalline symmetry order with dendritic morphology

Stable growth of a crystal with dendritic morphology with n-fold symmetry is modeled. Using the linear stability analysis and solvability theory, a selection criterion for thermally and solutally controlled growth of the dendrite is derived. A complete set of non-linear equations consisting of the selection criterion and an undercooling balance (which determines the implicit dependencies of the dendrite tip velocity and tip diameter on the total undercooling) is formulated. The growth kinetics of crystals having different lattice symmetry is analyzed. The model predictions are compared with phase field modeling data on ice dendrites grown from pure undercooled water.