Graph Computation Research Articles

A quantile-regression physics-informed deep learning for car-following model

Car-following models have been extensively studied using either theory-driven models or data-driven models. On the other hand, theory-driven models have low accuracy, whereas data-driven models have poor interpretability and require a high-quality training dataset. Furthermore, the majority of studies focused on predicting short-term trajectory based on a single vehicle. This paper aims to combine physics knowledge with deep learning and stochasticity to completely capture the genuine regularities of car-following behaviors, leveraging the benefits of both physics-based (interpretability) and deep learning-based (data-efficient and generalizability) models. We design a quantile-regression physics-informed deep learning car-following model (QRPIDL-CF). The physics-informed computational graph is integrated with a long short-term memory (LSTM) architecture (abbreviated as PIDL). By incorporating the quantile regression to the loss function of the PIDL model, stochastic behaviors can be produced without prior assumptions of drivers. To validate the proposed model, data from a 25-vehicle platoon experiment was used. The prediction results prove that given the first vehicle’s trajectory and other vehicles’ initial states, it can preserve the high precision in learning car-following behaviors and introduce small disturbances into car-following in a vehicle-platoon just the same as the real driving process. The trajectory prediction of the whole platoon shows that the proposed hybrid QRPIDL-CF model can better perform the macro-traffic behavior of car-following than every single model from the hybrid QRPIDL-CF model.

Generation of Homotopy Classes for Unconstrained 3D Wire Routing from Characteristic Loops

Routing of wire harnesses in complex machinery is a complicated problem. In this paper, we present an approach for computing all non-homotopic paths characteristic of all the homotopy classes associated with a pair of source–destination points embedded on the surface of the product. We introduce the notion of routing graphs, which are generated from specific non-trivial loops in the first homology group associated with the surface of the product. The routing graph, which is constructed only once, encodes the homotopy of the routing environment and is, therefore, used to find all the non-homotopic paths between multiple source–destination pairs. Based on the path embedding requirements, we classify the routing process into (1) On-Surface Routing, when the paths are constrained to the surface of the product, and (2) In-Air Routing, when the paths are allowed to venture into the product ambiance space. In the former case, the routing graph is defined by both the handle and tunnel loops in the first homology group, whereas in the latter case, it is defined by only the handle loops. We propose a linking number-based strategy for computing the handle and tunnel loops for a given closed surface. The methods are extended for routing in assemblies using the so-called shell structures associated with them. The results obtained demonstrate the efficacy of the proposed approach. The results show that the computation of the routing graph is very efficient, and the homotopy classes can be generated in very little time using the proposed approach.

The Story of GraphLab - From Scaling Machine Learning to Shaping Graph Systems Research (VLDB 2023 Test-of-Time Award Talk)

The GraphLab project spanned almost a decade and had profound academic and industrial impact on large-scale machine learning and graph processing systems. There were numerous papers written describing the innovations in GraphLab including the original vertex-centric [8] and edge-centric [3] programming abstractions, high-performance asynchronous execution engines [9], out-of-core graph computation [6], tabular graph-systems [4], and even new statistical inference algorithms [2] enabled by the GraphLab project. This work became the basis of multiple PhD theses [1, 5, 7]. The GraphLab open-source project had broad academic and industrial adoption and ultimately lead to the launch of Turi. In this talk, we tell the story of GraphLab, how it began and the key ideas behind it. We will focus on the approach to achieving scalable asynchronous systems in machine learning. During our talk, we will explore the impact that GraphLab has had on the development of graph processing systems, graph databases, and AI/ML; Additionally, we will share our insights and opinions into where we see the future of these fields heading. In the process, we highlight some of the lessons we learned and provide guidance for future students.

Shor–Laflamme distributions of graph states and noise robustness of entanglement

The Shor–Laflamme distribution (SLD) of a quantum state is a collection of local unitary invariants that quantify k-body correlations. We show that the SLD of graph states can be derived by solving a graph-theoretical problem. In this way, the mean and variance of the SLD are obtained as simple functions of efficiently computable graph properties. Furthermore, this formulation enables us to derive closed expressions of SLDs for some graph state families. For cluster states, we observe that the SLD is very similar to a binomial distribution, and we argue that this property is typical for graph states in general. Finally, we derive an SLD-based entanglement criterion from the purity criterion and apply it to derive meaningful noise thresholds for entanglement. Our new entanglement criterion is easy to use and also applies to the case of higher-dimensional qudits. In the bigger picture, our results foster the understanding both of quantum error-correcting codes, where a closely related notion of SLDs plays an important role, and of the geometry of quantum states, where SLDs are known as sector length distributions.

Learning dynamical systems from data: A simple cross-validation perspective, part IV: Case with partial observations

A simple and interpretable way to learn a dynamical system from data is to interpolate its governing equations with a kernel. In particular, this strategy is highly efficient (both in terms of accuracy and complexity) when the kernel is data-adapted using Kernel Flows (KF) (Owhadi and Yoo, 2019), (which uses gradient-based optimization to learn a kernel based on the premise that a kernel is good if there is no significant loss in accuracy if half of the data is used for interpolation). In this work, we extend previous work on learning dynamical systems using Kernel Flows (Hamzi and Owhadi, 2021; Darcy et al. 2021; Lee et al. 2023; Darcy et al. 2023; Owhadi and Romit Maulik, 2021) to the case of learning vector-valued dynamical systems from time-series observations that are partial/incomplete in the state space. The method combines Kernel Flows with Computational Graph Completion.

Reproducible clustering with non-Euclidean distances: a simulation and case study

AbstractCertain categorical sequence clustering applications require path connectivity, such as the clustering of DNA, click-paths through web-user sessions, or paths of care clustering with sequences of patient medical billing codes. $$K$$ K -means and $$k$$ k -medoids clustering with non-Euclidean distance metrics such as the Jaccard or edit distances maintains such path connectivity. Although $$k$$ k -means and $$k$$ k -medoids clustering with the Jaccard and edit distances have enjoyed success in these domains, the limits of accurate cluster recovery in these conditions have not yet been defined. As a first step in approaching this goal, we performed a simulated study using $$k$$ k -means and $$k$$ k -medoids clustering with non-Euclidean distances and show the performance deteriorates at a certain level of noise and when the number of clusters increases. However, we identify initialization strategies that improve upon cluster recovery in the presence of noise. We employ the use of the Tibshirani and Guenther (J Comput Graph Stat 14(3):511–528, 2005) Prediction Strength method, which creates a hypothesis testing scenario that determines if there is clustering structure to the data (if the clusters are reproducible), with the null hypothesis being there is none. We then applied the framework to perinatal episodes of care and the clusters reproducibly and organically split between Cesarean and vaginal deliveries, which itself is not a clinical finding but sensibly validates the approach. Further visualizations of the clusters did bring insights into subclusters that split along groups of physicians, cost and risk scores, warranting the outlined future work into ways of improving this framework for better resolution.

BIDL: a brain-inspired deep learning framework for spatiotemporal processing.

Brain-inspired deep spiking neural network (DSNN) which emulates the function of the biological brain provides an effective approach for event-stream spatiotemporal perception (STP), especially for dynamic vision sensor (DVS) signals. However, there is a lack of generalized learning frameworks that can handle various spatiotemporal modalities beyond event-stream, such as video clips and 3D imaging data. To provide a unified design flow for generalized spatiotemporal processing (STP) and to investigate the capability of lightweight STP processing via brain-inspired neural dynamics, this study introduces a training platform called brain-inspired deep learning (BIDL). This framework constructs deep neural networks, which leverage neural dynamics for processing temporal information and ensures high-accuracy spatial processing via artificial neural network layers. We conducted experiments involving various types of data, including video information processing, DVS information processing, 3D medical imaging classification, and natural language processing. These experiments demonstrate the efficiency of the proposed method. Moreover, as a research framework for researchers in the fields of neuroscience and machine learning, BIDL facilitates the exploration of different neural models and enables global-local co-learning. For easily fitting to neuromorphic chips and GPUs, the framework incorporates several optimizations, including iteration representation, state-aware computational graph, and built-in neural functions. This study presents a user-friendly and efficient DSNN builder for lightweight STP applications and has the potential to drive future advancements in bio-inspired research.

Interactive visual analytics of parallel training strategies for DNN models

Understanding and optimizing the parallel training strategies in the training of large-scale Deep Neural Network (DNN) models is crucial to enhance training efficiency. Existing works tried to demonstrate the layer-level information of the computational graph to support parallel training strategy selection. Whereas, the overall parallel execution logic is rarely considered by previous methods. In this paper, we proposed a novel visual analytics approach for parallel training strategies, demonstrating the execution logic of the distributed computing from up to bottom via explaining communication operators. Specifically, a computation-communication bipartite construction algorithm is designed for the computational graph visualization. Furthermore, a system is developed to help users easily access the proposed approach and explore the parallel training strategies interactively. With empirical evaluation through a quantitative user study and a qualitative expert interview, the practicality and superiority of the proposed approach is verified.

Impact of ROS 2 Node Composition in Robotic Systems

The Robot Operating System 2 (ROS 2) is the second generation of ROS representing a step forward in the robotic framework. Several new types of nodes and executor models are integral to control where, how, and when information is processed in the computational graph. This paper explores and benchmarks one of these new node types - the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Component</i> node - which allows nodes to be composed manually or dynamically into processes while retaining separation of concerns in a codebase for distributed development. Composition is shown to achieve a high degree of performance optimization, particularly valuable for resource-constrained systems and sensor processing pipelines, enabling distributed tasks that would not be otherwise possible in ROS 2. In this work, we briefly introduce the significance and design of node composition, then our contribution of benchmarking is provided to analyze its impact on robotic systems. Its compelling influence on performance is shown through several experiments on the latest Long Term Support (LTS) ROS 2 distribution, Humble Hawksbill.

Safe MDP Planning by Learning Temporal Patterns of Undesirable Trajectories and Averting Negative Side Effects

In safe MDP planning, a cost function based on the current state and action is often used to specify safety aspects. In real world, often the state representation used may lack sufficient fidelity to specify such safety constraints. Operating based on an incomplete model can often produce unintended negative side effects (NSEs). To address these challenges, first, we associate safety signals with state-action trajectories (rather than just immediate state-action). This makes our safety model highly general. We also assume categorical safety labels are given for different trajectories, rather than a numerical cost function, which is harder to specify by the problem designer. We then employ a supervised learning model to learn such non-Markovian safety patterns. Second, we develop a Lagrange multiplier method, which incorporates the safety model and the underlying MDP model in a single computation graph to facilitate agent learning of safe behaviors. Finally, our empirical results on a variety of discrete and continuous domains show that this approach can satisfy complex non-Markovian safety constraints while optimizing agent's total returns, is highly scalable, and is also better than previous best approach for Markovian NSEs.

Ruffle: Rapid 3-Party Shuffle Protocols

Secure shuffle is an important primitive that finds use in several applications such as secure electronic voting, oblivious RAMs, secure sorting, to name a few. For time-sensitive shuffle-based applications that demand a fast response time, it is essential to design a fast and efficient shuffle protocol. In this work, we design secure and fast shuffle protocols relying on the techniques of secure multiparty computation. We make several design choices that aid in achieving highly efficient protocols. Specifically, we consider malicious 3-party computation setting with an honest majority and design robust ring-based protocols. Our shuffle protocols provide a fast online (i.e., input-dependent) phase compared to the state-of-the-art for the considered setting. To showcase the efficiency improvements brought in by our shuffle protocols, we consider two distinct applications of anonymous broadcast and secure graph computation via the GraphSC paradigm. In both cases, multiple shuffle invocations are required. Hence, going beyond standalone shuffle invocation, we identify two distinct scenarios of multiple invocations and provide customised protocols for the same. Further, we showcase that our customized protocols not only provide a fast response time, but also provide improved overall run time for multiple shuffle invocations. With respect to the applications, we not only improve in terms of efficiency, but also work towards providing improved security guarantees, thereby outperforming the respective state-of-the-art works. We benchmark our shuffle protocols and the considered applications to analyze the efficiency improvements with respect to various parameters.

Hybrid modeling of a multidimensional coupled nonlinear system with integration of Hamiltonian mechanics

AbstractThis study concerns hybrid modeling of a multidimensional coupled nonlinear system. The underlying basis for the model is derived from Hamiltonian mechanics capitalizing on the broad utility and efficiency of energy-based reasoning in modeling high-dimensional systems. The hybrid model is essentially an artificial neural network with a computational graph that is modified from conventional neural networks in a few significant ways. The first modification includes incorporating an intermediate scalar function representing the Hamiltonian learned from data. The second modification enhances input/output channels for capturing the multidimensional dynamics of the system. The main goal of such hybrid reasoning is to improve the extrapolation capability of the model by enforcing conformance with some key aspects of the underlying physics in the form of a bias. The results demonstrate that incorporating this physics-based bias into the hybrid model empowers it to produce long-term and physically plausible predictions. The proposed modeling approach also shows high scalability for energy-based modeling of multidimensional dynamic systems in general.

Provable Pathways: Learning Multiple Tasks over Multiple Paths

Constructing useful representations across a large number of tasks is a key requirement for sample-efficient intelligent systems. A traditional idea in multitask learning (MTL) is building a shared representation across tasks which can then be adapted to new tasks by tuning last layers. A desirable refinement of using a shared one-fits-all representation is to construct task-specific representations. To this end, recent PathNet/muNet architectures represent individual tasks as pathways within a larger supernet. The subnetworks induced by pathways can be viewed as task-specific representations that are composition of modules within supernet's computation graph. This work explores the pathways proposal from the lens of statistical learning: We first develop novel generalization bounds for empirical risk minimization problems learning multiple tasks over multiple paths (Multipath MTL). In conjunction, we formalize the benefits of resulting multipath representation when adapting to new downstream tasks. Our bounds are expressed in terms of Gaussian complexity, lead to tangible guarantees for the class of linear representations, and provide novel insights into the quality and benefits of a multipath representation. When computation graph is a tree, Multipath MTL hierarchically clusters the tasks and builds cluster-specific representations. We provide further discussion and experiments for hierarchical MTL and rigorously identify the conditions under which Multipath MTL is provably superior to traditional MTL approaches with shallow supernets.

Physics-informed neural networks for integrated traffic state and queue profile estimation: A differentiable programming approach on layered computational graphs

This paper presents an integrated framework for physics-informed joint traffic state and queue profile estimation (JSQE) on freeway corridors, utilizing heterogeneous data sources. The integrated modeling framework aims to maximize the benefits of information from physics-informed analytical traffic flow models and field observations while leveraging joint estimation spaces. Potential inconsistencies between different modeling components must be acknowledged and carefully managed to ensure model feasibility. To minimize such inconsistencies, a nonlinear programming model is formulated for the JSQE problem, taking into account traffic flow models and observations from both corridor and local segment levels. At the corridor level, a fluid queue approximation is employed to model queuing dynamics. Assuming polynomial arrival and departure rates, critical system variables such as time-dependent delay, travel time, and queue length are analytically derived. To preserve the differentiability of traffic state variables, continuous space–time distribution functions are introduced to model traffic flow variables and partial differential equations. A computational graph is constructed to represent the nonlinear programming model in a layered structure, which is then solved using a forward–backward method. Comprehensive numerical experiments based on real-world and hypothetical datasets are designed to demonstrate the effectiveness of the proposed framework.

AutoGraph: Optimizing DNN Computation Graph for Parallel GPU Kernel Execution

Deep learning frameworks optimize the computation graphs and intra-operator computations to boost the inference performance on GPUs, while inter-operator parallelism is usually ignored. In this paper, a unified framework, AutoGraph, is proposed to obtain highly optimized computation graphs in favor of parallel executions of GPU kernels. A novel dynamic programming algorithm, combined with backtracking search, is adopted to explore the optimal graph optimization solution, with the fast performance estimation from the mixed critical path cost. Accurate runtime information based on GPU Multi-Stream launched with CUDA Graph is utilized to determine the convergence of the optimization. Experimental results demonstrate that our method achieves up to 3.47x speedup over existing graph optimization methods. Moreover, AutoGraph outperforms state-of-the-art parallel kernel launch frameworks by up to 1.26x.

GENNAPE: Towards Generalized Neural Architecture Performance Estimators

Predicting neural architecture performance is a challenging task and is crucial to neural architecture design and search. Existing approaches either rely on neural performance predictors which are limited to modeling architectures in a predefined design space involving specific sets of operators and connection rules, and cannot generalize to unseen architectures, or resort to Zero-Cost Proxies which are not always accurate. In this paper, we propose GENNAPE, a Generalized Neural Architecture Performance Estimator, which is pretrained on open neural architecture benchmarks, and aims to generalize to completely unseen architectures through combined innovations in network representation, contrastive pretraining, and a fuzzy clustering-based predictor ensemble. Specifically, GENNAPE represents a given neural network as a Computation Graph (CG) of atomic operations which can model an arbitrary architecture. It first learns a graph encoder via Contrastive Learning to encourage network separation by topological features, and then trains multiple predictor heads, which are soft-aggregated according to the fuzzy membership of a neural network. Experiments show that GENNAPE pretrained on NAS-Bench-101 can achieve superior transferability to 5 different public neural network benchmarks, including NAS-Bench-201, NAS-Bench-301, MobileNet and ResNet families under no or minimum fine-tuning. We further introduce 3 challenging newly labelled neural network benchmarks: HiAML, Inception and Two-Path, which can concentrate in narrow accuracy ranges. Extensive experiments show that GENNAPE can correctly discern high-performance architectures in these families. Finally, when paired with a search algorithm, GENNAPE can find architectures that improve accuracy while reducing FLOPs on three families.

Fuzzy Domination Graphs in Decision Support Tasks

In decision support tasks, one often has to deal with uncertainty due to fuzzy judgments of the decision maker or the expert. This paper proposes methods that allow you to rank the alternatives based on fuzzy evaluations. This is achieved by using fuzzy weighted summation, fuzzy implication, a computation graph showing the criteria, and a fuzzy dominance graph showing the alternatives. If the criteria have equal importance, then fuzzy graphs corresponding to the dominance of each of the criteria are used. An algorithm that is used for both the transition from fuzzy dominance graphs and the ranking of alternatives is proposed. This algorithm is based on the idea of constructing Kemeny medians or other concordant rankings at a given confidence level in the existence of corresponding arcs. Computational experiments have shown the performance of these approaches. It is reasonable to apply them in problems that require complex expert evaluations with a large number of alternatives and criteria.

Physics-Informed Deep Learning for Traffic State Estimation: A Survey and the Outlook

For its robust predictive power (compared to pure physics-based models) and sample-efficient training (compared to pure deep learning models), physics-informed deep learning (PIDL), a paradigm hybridizing physics-based models and deep neural networks (DNNs), has been booming in science and engineering fields. One key challenge of applying PIDL to various domains and problems lies in the design of a computational graph that integrates physics and DNNs. In other words, how the physics is encoded into DNNs and how the physics and data components are represented. In this paper, we offer an overview of a variety of architecture designs of PIDL computational graphs and how these structures are customized to traffic state estimation (TSE), a central problem in transportation engineering. When observation data, problem type, and goal vary, we demonstrate potential architectures of PIDL computational graphs and compare these variants using the same real-world dataset.

Evolving a Pipeline Approach for Abstract Meaning Representation Parsing Towards Dynamic Neural Networks.

Meaning Representation parsing aims to represent a sentence as a structured, Directed, Acyclic Graph (DAG), in an attempt to extract meaning from text. This paper extends an existing 2-stage pipeline AMR parser with state-of-the-art techniques in dependency parsing. First, Pointer-Generator Networks are used for out-of-vocabulary words in the concept identification stage, with an improved initialization via the use of word-and character-level embeddings. Second, the performance of the Relation Identification module is improved by jointly training the Heads Selection and the Arcs Labeling components. Last, we underline the difficulty of end-to-end training with recurrent modules in a static deep neural network construction approach and explore a dynamic construction implementation, which continuously adapts the computation graph, thus potentially enabling end-to-end training in the proposed pipeline solution.

Disentangling the cultural evolution of ancient China: a digital humanities perspective

Being recognized among the cradles of human civilization, ancient China nurtured the longest continuous academic traditions and humanistic spirits, which continue to impact today’s society. With an unprecedented large-scale corpus spanning 3000 years, this paper presents a quantitative analysis of cultural evolution in ancient China. Millions of intertextual associations are identified and modelled with a hierarchical framework via deep neural network and graph computation, thus allowing us to answer three progressive questions quantitatively: (1) What is the interaction between individual scholars and philosophical schools? (2) What are the vicissitudes of schools in ancient Chinese history? (3) How did ancient China develop a cross-cultural exchange with an externally introduced religion such as Buddhism? The results suggest that the proposed hierarchical framework for intertextuality modelling can provide sound suggestions for large-scale quantitative studies of ancient literature. An online platform is developed for custom data analysis within this corpus, which encourages researchers and enthusiasts to gain insight into this work. This interdisciplinary study inspires the re-understanding of ancient Chinese culture from a digital humanities perspective and prompts the collaboration between humanities and computer science.