Higher-order Patterns Research Articles

Motif Transition Intensity: A Novel Network-Based Early Warning Indicator for Financial Crises

Financial crisis, rooted in a lack of system resilience and robustness, is a particular type of critical transition that may cause grievous economic and social losses and should be warned against as early as possible. Regarding the financial system as a time-varying network, researchers have identified early warning signals from the changing dynamics of network motifs. In addition, network motifs have many different morphologies that unveil high-order correlation patterns of a financial system, whose synchronous change represents the dramatic shift in the financial system’s functionality and may indicate a financial crisis; however, it is less studied. This paper proposes motif transition intensity as a novel method that quantifies the synchronous change of network motifs in detail. Applying this method to stock networks, we developed three early warning indicators. Empirically, we conducted a horse race to predict ten global crises during 1991–2020. The results show evidence that the proposed indicators are more efficient than the VIX and the other 39 network-based indicators. In a detailed analysis, the proposed indicators send sensitive and comprehensible warning signals, especially for the U.S. subprime mortgage crisis and the European sovereign debt crisis. Furthermore, the proposed method provides a new perspective to detect critical signals and may be extended to predict other crisis events in natural and social systems.

Bayesian Deep Learning for Spatial Interpolation in the Presence of Auxiliary Information

Earth scientists increasingly deal with ‘big data’. For spatial interpolation tasks, variants of kriging have long been regarded as the established geostatistical methods. However, kriging and its variants (such as regression kriging, in which auxiliary variables or derivatives of these are included as covariates) are relatively restrictive models and lack capabilities provided by deep neural networks. Principal among these is feature learning: the ability to learn filters to recognise task-relevant patterns in gridded data such as images. Here, we demonstrate the power of feature learning in a geostatistical context by showing how deep neural networks can automatically learn the complex high-order patterns by which point-sampled target variables relate to gridded auxiliary variables (such as those provided by remote sensing) and in doing so produce detailed maps. In order to cater for the needs of decision makers who require well-calibrated probabilities, we also demonstrate how both aleatoric and epistemic uncertainty can be quantified in our deep learning approach via a Bayesian approximation known as Monte Carlo dropout. In our example, we produce a national-scale probabilistic geochemical map from point-sampled observations with auxiliary data provided by a terrain elevation grid. By combining location information with automatically learned terrain derivatives, our deep learning approach achieves an excellent coefficient of determination (R^{2} = 0.74) and near-perfect probabilistic calibration on held-out test data. Our results indicate the suitability of Bayesian deep learning and its feature-learning capabilities for large-scale geostatistical applications where uncertainty matters.Graphic

Higher-Order Thalamic Encoding of Somatosensory Patterns and Bilateral Events.

The function of the higher-order sensory thalamus remains unclear. Here, the posterior medial (POm) nucleus of the thalamus was examined by in vivo extracellular recordings in anesthetized rats across a variety of contralateral, ipsilateral, and bilateral whisker sensory patterns. We found that POm was highly sensitive to multiwhisker stimuli involving diverse spatiotemporal interactions. Accurate increases in POm activity were produced during the overlapping time between spatial signals reflecting changes in the spatiotemporal structure of sensory patterns. In addition, our results showed for first time that POm was also able to respond to tactile stimulation of ipsilateral whiskers. This finding challenges the notion that the somatosensory thalamus only computes unilateral stimuli. We found that POm also integrates signals from both whisker pads and described how this integration is generated. Our results showed that ipsilateral activity reached one POm indirectly from the other POm and demonstrated a transmission of sensory activity between both nuclei through a functional POm-POm loop formed by thalamocortical, interhemispheric, and corticothalamic projections. The implication of different cortical areas was investigated revealing that S1 plays a central role in this POm-POm loop. Accordingly, the subcortical and cortical inputs allow POm but not the ventral posteromedial thalamic nucleus (VPM) to have sensory information from both sides of the body. This finding is in agreement with the higher-order nature of POm and can be considered to functionally differentiate and classify these thalamic nuclei. A possible functional role of these higher-order thalamic patterns of integrated activity in brain function is discussed.

Forecasting Avalanches in Branched Actomyosin Networks with Network Science and Machine Learning.

We explored the dynamic and structural effects of actin-related proteins 2/3 (Arp2/3) on actomyosin networks using mechanochemical simulations of active matter networks. On the nanoscale, the Arp2/3 complex alters the topology of actomyosin by nucleating a daughter filament at an angle with respect to a mother filament. At a subcellular scale, they orchestrate the formation of a branched actomyosin network. Using a coarse-grained approach, we sought to understand how an actomyosin network temporally and spatially reorganizes itself by varying the concentration of the Arp2/3 complexes. Driven by motor dynamics, the network stalls at a high concentration of Arp2/3 and contracts at a low Arp2/3 concentration. At an intermediate Arp2/3 concentration, however, the actomyosin network is formed by loosely connected clusters that may collapse suddenly when driven by motors. This physical phenomenon is called an "avalanche" largely due to the marginal instability inherent to the morphology of a branched actomyosin network when the Arp2/3 complex is present. While embracing the data science approaches, we unveiled the higher-order patterns in the branched actomyosin networks and discovered a sudden change in the "social" network topology of actomyosin, which is a new type of avalanche in addition to the two types of avalanches associated with a sudden change in the size or shape of the whole actomyosin network, as shown in a previous investigation. Our new finding promotes the importance of using network theory and machine learning models to forecast avalanches in actomyosin networks. The mechanisms of the Arp2/3 complexes in shaping the architecture of branched actomyosin networks obtained in this paper will help us better understand the emergent reorganization of the topology in dense actomyosin networks that are difficult to detect in experiments.

Reachability types: tracking aliasing and separation in higher-order functional programs

Ownership type systems, based on the idea of enforcing unique access paths, have been primarily focused on objects and top-level classes. However, existing models do not as readily reflect the finer aspects of nested lexical scopes, capturing, or escaping closures in higher-order functional programming patterns, which are increasingly adopted even in mainstream object-oriented languages. We present a new type system, λ * , which enables expressive ownership-style reasoning across higher-order functions. It tracks sharing and separation through reachability sets, and layers additional mechanisms for selectively enforcing uniqueness on top of it. Based on reachability sets, we extend the type system with an expressive flow-sensitive effect system, which enables flavors of move semantics and ownership transfer. In addition, we present several case studies and extensions, including applications to capabilities for algebraic effects, one-shot continuations, and safe parallelization.

MBRep: Motif-based representation learning in heterogeneous networks

In recent years, there has been a surge of interest in applying machine learning to graphs and networks that already exist in the world around us. The approach has been successfully used for domains as diverse as traffic management, e-commerce recommendation and public opinion monitoring. A critical aspect of representation learning for applied machine learning is feature engineering. Deep learning-based research in representation learning has developed methods for automatically learning a large number of potentially correlated features from original networks. However, most of these methods cannot be applied to heterogeneous networks, which are true expressions of the real-world. This is because they do not adequately capture the structure and semantics of different types of nodes in heterogeneous networks and the links between them. They also struggle to represent higher-order heterogeneous patterns of connection. This paper proposes a generalized motif-based higher-order representation learning method, MBRep, that learns triangle motif embedding in a network, on the basis of which it can obtain the embedding and representation of nodes in a heterogeneous network. Statistically, significant motif structures are extracted from the original heterogeneous network and combined with the heterogeneity of the nodes. A weight-biased random walk is then applied to the motif level higher-order network, using a SkipGram model to embed the motifs. After this, the embedding of the original network nodes is calculated using weighted averages and feature alignment. This can then be used for link prediction. We confirmed the effectiveness of MBRep by comparing its AUC and MRR performance with other state-of-the-art methods on three real-world datasets. Its adaptability was also validated by conducting a cold-start test.

High-level cognition during story listening is reflected in high-order dynamic correlations in neural activity patterns

Our thoughts arise from coordinated patterns of interactions between brain structures that change with our ongoing experiences. High-order dynamic correlations in neural activity patterns reflect different subgraphs of the brain’s functional connectome that display homologous lower-level dynamic correlations. Here we test the hypothesis that high-level cognition is reflected in high-order dynamic correlations in brain activity patterns. We develop an approach to estimating high-order dynamic correlations in timeseries data, and we apply the approach to neuroimaging data collected as human participants either listen to a ten-minute story or listen to a temporally scrambled version of the story. We train across-participant pattern classifiers to decode (in held-out data) when in the session each neural activity snapshot was collected. We find that classifiers trained to decode from high-order dynamic correlations yield the best performance on data collected as participants listened to the (unscrambled) story. By contrast, classifiers trained to decode data from scrambled versions of the story yielded the best performance when they were trained using first-order dynamic correlations or non-correlational activity patterns. We suggest that as our thoughts become more complex, they are reflected in higher-order patterns of dynamic network interactions throughout the brain.

Machine learning dismantling and early-warning signals of disintegration in complex systems

From physics to engineering, biology and social science, natural and artificial systems are characterized by interconnected topologies whose features – e.g., heterogeneous connectivity, mesoscale organization, hierarchy – affect their robustness to external perturbations, such as targeted attacks to their units. Identifying the minimal set of units to attack to disintegrate a complex network, i.e. network dismantling, is a computationally challenging (NP-hard) problem which is usually attacked with heuristics. Here, we show that a machine trained to dismantle relatively small systems is able to identify higher-order topological patterns, allowing to disintegrate large-scale social, infrastructural and technological networks more efficiently than human-based heuristics. Remarkably, the machine assesses the probability that next attacks will disintegrate the system, providing a quantitative method to quantify systemic risk and detect early-warning signals of system’s collapse. This demonstrates that machine-assisted analysis can be effectively used for policy and decision-making to better quantify the fragility of complex systems and their response to shocks.

Higher-order temporal network effects through triplet evolution

We study the evolution of networks through ‘triplets’—three-node graphlets. We develop a method to compute a transition matrix to describe the evolution of triplets in temporal networks. To identify the importance of higher-order interactions in the evolution of networks, we compare both artificial and real-world data to a model based on pairwise interactions only. The significant differences between the computed matrix and the calculated matrix from the fitted parameters demonstrate that non-pairwise interactions exist for various real-world systems in space and time, such as our data sets. Furthermore, this also reveals that different patterns of higher-order interaction are involved in different real-world situations. To test our approach, we then use these transition matrices as the basis of a link prediction algorithm. We investigate our algorithm’s performance on four temporal networks, comparing our approach against ten other link prediction methods. Our results show that higher-order interactions in both space and time play a crucial role in the evolution of networks as we find our method, along with two other methods based on non-local interactions, give the best overall performance. The results also confirm the concept that the higher-order interaction patterns, i.e., triplet dynamics, can help us understand and predict the evolution of different real-world systems.

Higher-Order Airy Patterns and Their Application in Tailoring Orbital Angular Momentum Beams with Fiber Laser Arrays

Airy patterns, formed by the Fraunhofer diffraction of plane waves via circular apertures, have been widely utilized as criterions in areas such as imaging optics and energy transmission. With deeper insights into the spatial structure of light, the use of Airy patterns as criterions today becomes inadequate in many scenarios. Here, a new concept of higher-order Airy patterns is proposed by introducing the orbital angular momentum (OAM) of light, namely the Fraunhofer diffraction pattern of a plane wave via the defined annular aperture with helical wavefront tailoring. For the m-order Airy pattern, the tailored helicity is m, and the scaling factor of the annular aperture and is determined by the m-related equation set, which is derived based on the paraxial diffraction theory. As a potential application of the proposed concept, a coherent fiber laser array system capable of customizing OAM beams is presented and analyzed. Using the higher-order Airy pattern to define the performance metric of combined beam, the fiber laser array system is optimally designed and the properties in energy concentration and high mode purity of the generated OAM beams could be realized. This work could be beneficial to the theoretical study and applications of OAM beams, especially the generation and manipulation of high-power, fast switchable OAM beams with fiber laser arrays.

HM-Modularity: A Harmonic Motif Modularity Approach for Multi-Layer Network Community Detection

Multi-layer network community detection has drawn an increasing amount of attention recently. Despite success, the existing methods mainly focus on the lower-order connectivity structure at the level of individual nodes and edges. And the higher-order connectivity structure has been largely ignored, which contains better signature of community compared with edges. The main challenges in utilizing higher-order structure for multi-layer network community detection are that the most representative higher-order structure may vary from one layer to another and the connectivity structure formed by the same node subset may exhibit different higher-order connectivity patterns in different layers. To this end, this paper proposes a novel higher-order structure, termed harmonic motif, which is a dense subgraph having on average the largest statistical significance in each layer. Based on the harmonic motif, a primary layer is constructed by integrating higher-order structural information from all layers. Additionally, the higher-order structural information of each individual layer is taken as the auxiliary information. A coupling is established between the primary layer and each auxiliary layer. Accordingly, a harmonic motif modularity is designed to generate the community structure. Extensive experiments on eleven real-world multi-layer network datasets have been conducted to confirm the effectiveness of the proposed method.

Hybrid-order Stochastic Block Model

Community detection is a research hotspot in machine learning and data mining. However, most of the existing community detection methods only rely on the lower-order connectivity patterns, while ignoring the higher-order connectivity patterns, and unable to capture the building blocks of the complex network. In recent years, some community detection methods based on higher-order structures have been developed, but they mainly focus on the motif network composed of higher-order structures, which violate the original lower-order topological structure and are affected by the fragmentation issue, resulting in the deviation of community detection results. Therefore, there is still a lack of community detection methods that can effectively utilize higher-order connectivity patterns and lower-order connectivity patterns. To overcome the above limitations, this paper proposes the Hybrid-order Stochastic Block Model (HSBM) from the perspective of the generative model. Based on the classical stochastic block model, the generation of lower-order structure and higher-order structure of the network is modeled uniformly, and the original topological properties of the network are maintained while using higher-order connectivity patterns. At the same time, a heuristic algorithm for community detection is proposed to optimize the objective function. Extensive experiments on six real-world datasets show that the proposed method outperforms the existing approaches.

Memory-aware framework for fast and scalable second-order random walk over billion-edge natural graphs

Second-order random walk is an important technique for graph analysis. Many applications including graph embedding, proximity measure and community detection use it to capture higher-order patterns in the graph, thus improving the model accuracy. However, the memory explosion problem of this technique hinders it from analyzing large graphs. When processing a billion-edge graph like Twitter, existing solutions (e.g., alias method) of the second-order random walk may take up 1796TB memory. Such high memory consumption comes from the memory-unaware strategies for the node sampling during the random walk. In this paper, to clearly compare the efficiency of various node sampling methods, we first design a cost model and propose two new node sampling methods: one follows the acceptance-rejection paradigm to achieve a better balance between memory and time cost, and the other is optimized for fast sampling the skewed probability distributions existed in natural graphs. Second, to achieve the high efficiency of the second-order random walk within arbitrary memory budgets, we propose a novel memory-aware framework on the basis of the cost model. The framework applies a cost-based optimizer to assign desirable node sampling method for each node or edge in the graph within a memory budget meanwhile minimizing the time cost of the random walk. Finally, the framework provides general programming interfaces for users to define new second-order random walk models easily. The empirical studies demonstrate that our memory-aware framework is robust with respect to memory and is able to achieve considerable efficiency by reducing 90% of the memory cost.

See(quence) and ye shall find: higher-order genome folding in intact single cells

Payne etal. (2020) combine in situ imaging and ex situ sequencing via spatially resolved unique molecular barcodes to query higher-order genome folding patterns in intact single nuclei from mouse embryos and human fibroblasts.

Explanation and prediction of clinical data with imbalanced class distribution based on pattern discovery and disentanglement

BackgroundStatistical data analysis, especially the advanced machine learning (ML) methods, have attracted considerable interest in clinical practices. We are looking for interpretability of the diagnostic/prognostic results that will bring confidence to doctors, patients and their relatives in therapeutics and clinical practice. When datasets are imbalanced in diagnostic categories, we notice that the ordinary ML methods might produce results overwhelmed by the majority classes diminishing prediction accuracy. Hence, it needs methods that could produce explicit transparent and interpretable results in decision-making, without sacrificing accuracy, even for data with imbalanced groups.MethodsIn order to interpret the clinical patterns and conduct diagnostic prediction of patients with high accuracy, we develop a novel method, Pattern Discovery and Disentanglement for Clinical Data Analysis (cPDD), which is able to discover patterns (correlated traits/indicants) and use them to classify clinical data even if the class distribution is imbalanced. In the most general setting, a relational dataset is a large table such that each column represents an attribute (trait/indicant), and each row contains a set of attribute values (AVs) of an entity (patient). Compared to the existing pattern discovery approaches, cPDD can discover a small succinct set of statistically significant high-order patterns from clinical data for interpreting and predicting the disease class of the patients even with groups small and rare.ResultsExperiments on synthetic and thoracic clinical dataset showed that cPDD can 1) discover a smaller set of succinct significant patterns compared to other existing pattern discovery methods; 2) allow the users to interpret succinct sets of patterns coming from uncorrelated sources, even the groups are rare/small; and 3) obtain better performance in prediction compared to other interpretable classification approaches.ConclusionsIn conclusion, cPDD discovers fewer patterns with greater comprehensive coverage to improve the interpretability of patterns discovered. Experimental results on synthetic data validated that cPDD discovers all patterns implanted in the data, displays them precisely and succinctly with statistical support for interpretation and prediction, a capability which the traditional ML methods lack. The success of cPDD as a novel interpretable method in solving the imbalanced class problem shows its great potential to clinical data analysis for years to come.

Quality-aware semi-supervised learning for CMR segmentation.

One of the challenges in developing deep learning algorithms for medical image segmentation is the scarcity of annotated training data. To overcome this limitation, data augmentation and semi-supervised learning (SSL) methods have been developed. However, these methods have limited effectiveness as they either exploit the existing data set only (data augmentation) or risk negative impact by adding poor training examples (SSL). Segmentations are rarely the final product of medical image analysis -they are typically used in downstream tasks to infer higher-order patterns to evaluate diseases. Clinicians take into account a wealth of prior knowledge on biophysics and physiology when evaluating image analysis results. We have used these clinical assessments in previous works to create robust quality-control (QC) classifiers for automated cardiac magnetic resonance (CMR) analysis. In this paper, we propose a novel scheme that uses QC of the downstream task to identify high quality outputs of CMR segmentation networks, that are subsequently utilised for further network training. In essence, this provides quality-aware augmentation of training data in a variant of SSL for segmentation networks (semiQCSeg). We evaluate our approach in two CMR segmentation tasks (aortic and short axis cardiac volume segmentation) using UK Biobank data and two commonly used network architectures (U-net and a Fully Convolutional Network) and compare against supervised and SSL strategies. We show that semiQCSeg improves training of the segmentation networks. It decreases the need for labelled data, while outperforming the other methods in terms of Dice and clinical metrics. SemiQCSeg can be an efficient approach for training segmentation networks for medical image data when labelled datasets are scarce.

Local hypergraph clustering using capacity releasing diffusion.

Local graph clustering is an important machine learning task that aims to find a well-connected cluster near a set of seed nodes. Recent results have revealed that incorporating higher order information significantly enhances the results of graph clustering techniques. The majority of existing research in this area focuses on spectral graph theory-based techniques. However, an alternative perspective on local graph clustering arises from using max-flow and min-cut on the objectives, which offer distinctly different guarantees. For instance, a new method called capacity releasing diffusion (CRD) was recently proposed and shown to preserve local structure around the seeds better than spectral methods. The method was also the first local clustering technique that is not subject to the quadratic Cheeger inequality by assuming a good cluster near the seed nodes. In this paper, we propose a local hypergraph clustering technique called hypergraph CRD (HG-CRD) by extending the CRD process to cluster based on higher order patterns, encoded as hyperedges of a hypergraph. Moreover, we theoretically show that HG-CRD gives results about a quantity called motif conductance, rather than a biased version used in previous experiments. Experimental results on synthetic datasets and real world graphs show that HG-CRD enhances the clustering quality.

Additive and Multiplicative Effects Network Models

Network datasets typically exhibit certain types of statistical patterns, such as within-dyad correlation, degree heterogeneity, and triadic patterns such as transitivity and clustering. The first two of these can be well represented with a social relations model, a type of additive effects model originally developed for continuous dyadic data. Higher-order patterns can be represented with multiplicative effects models, which are related to matrix decompositions that are commonly used for matrix-variate data analysis. Additionally, these multiplicative effects models generalize other popular latent feature network models, such as the stochastic blockmodel and the latent space model. In this article, we review a general regression framework for the analysis of network data that combines these two types of effects, and accommodates a variety of network data types, including continuous, binary and ordinal network relations.

Expedited circular dichroism prediction and engineering in two-dimensional diffractive chiral metamaterials leveraging a powerful model-agnostic data enhancement algorithm

Abstract A model-agnostic data enhancement (MADE) algorithm is proposed to comprehensively investigate the circular dichroism (CD) properties in the higher-order diffracted patterns of two-dimensional (2D) chiral metamaterials possessing different parameters. A remarkable feature of MADE algorithm is that it leverages substantially less data from a target problem and some training data from another already solved topic to generate a domain adaptation dataset, which is then used for model training at no expense of abundant computational resources. Specifically, nine differently shaped 2D chiral metamaterials with different unit period and one special sample containing multiple chiral parameters are both studied utilizing the MADE algorithm where three machine learning models (i.e, artificial neural network, random forest regression, support vector regression) are applied. The conventional rigorous coupled wave analysis approach is adopted to capture CD responses of these metamaterials and then assist the training of MADE, while the additional training data are obtained from our previous work. Significant evaluations regarding optical chirality in 2D metamaterials possessing various shape, unit period, width, bridge length, and separation length are performed in a fast, accurate, and data-friendly manner. The MADE framework introduced in this work is extremely important for the large-scale, efficient design of 2D diffractive metamaterials and more advanced photonic devices.

Deep Active Inference and Scene Construction.

Adaptive agents must act in intrinsically uncertain environments with complex latent structure. Here, we elaborate a model of visual foraging—in a hierarchical context—wherein agents infer a higher-order visual pattern (a “scene”) by sequentially sampling ambiguous cues. Inspired by previous models of scene construction—that cast perception and action as consequences of approximate Bayesian inference—we use active inference to simulate decisions of agents categorizing a scene in a hierarchically-structured setting. Under active inference, agents develop probabilistic beliefs about their environment, while actively sampling it to maximize the evidence for their internal generative model. This approximate evidence maximization (i.e., self-evidencing) comprises drives to both maximize rewards and resolve uncertainty about hidden states. This is realized via minimization of a free energy functional of posterior beliefs about both the world as well as the actions used to sample or perturb it, corresponding to perception and action, respectively. We show that active inference, in the context of hierarchical scene construction, gives rise to many empirical evidence accumulation phenomena, such as noise-sensitive reaction times and epistemic saccades. We explain these behaviors in terms of the principled drives that constitute the expected free energy, the key quantity for evaluating policies under active inference. In addition, we report novel behaviors exhibited by these active inference agents that furnish new predictions for research on evidence accumulation and perceptual decision-making. We discuss the implications of this hierarchical active inference scheme for tasks that require planned sequences of information-gathering actions to infer compositional latent structure (such as visual scene construction and sentence comprehension). This work sets the stage for future experiments to investigate active inference in relation to other formulations of evidence accumulation (e.g., drift-diffusion models) in tasks that require planning in uncertain environments with higher-order structure.