Large-scale Real-world Networks Research Articles

Generative framework for dimensionality reduction of large scale network of nonlinear dynamical systems driven by external input

Several studies have proposed constraints under which a low-dimensional representation can be derived from large-scale real-world networks exhibiting complex nonlinear dynamics. Typically, these representations are formulated under certain assumptions, such as when solutions converge to attractor states using linear stability analysis or using projections of large-scale dynamical data into a set of lower dimensional modes that are selected heuristically. Here, we propose a generative framework for selection of lower dimensional modes onto which the entire network dynamics can be projected based on the symmetry of the input distribution for a large-scale network driven by external inputs, thus relaxing the heuristic selection of modes made in the earlier reduction approaches. The proposed mode reduction technique is tractable analytically and applied to different kinds of real-world large-scale network scenarios with nodes comprising of (a) Van der Pol oscillators (b) Hindmarsh–Rose neurons. These two demonstrations elucidate how order parameter is conserved at original and reduced descriptions thus validating our proposition.

Multicast Scaling of Capacity and Energy Efficiency in Heterogeneous Wireless Sensor Networks

Motivated by the requirement of heterogeneity in the Internet of Things, we initiate the joint study of capacity and energy efficiency scaling laws in heterogeneous wireless sensor networks, and so on. The whole network is composed of n nodes scattered in a square region with side length L = n α , and there are m = n ν home points { c j } j=1 m , where a generic home point c j generates q j nodes independently according to a stationary and rotationally invariant kernel k ( c j , ⋅). Among the n nodes, we schedule n s independent multicast sessions each consisting of k − 1 destination nodes and one source node. According to the heterogeneity of nodes’ distribution, we classify the network into two regimes: a cluster-dense regime and a cluster-sparse regime. For the cluster-dense regime, we construct single layer highway system using percolation theory and then build the multicast spanning tree for each multicast session. This scheme yields the Ω( n ½+(α − ½)γ / n s √ k ) per-session multicast capacity. For the cluster-sparse regime, we partition the whole network plane into several layers and construct nested highway systems. The similar multicast spanning tree yields the Ω( n ½−(1− ν)γ/2 / n s √ k ) per-session multicast capacity, where γ is the power attenuation factor. Interestingly, we find that the bottleneck of multicast capacity attributes to the network region with largest node density, which provides a guideline for the deployment of sensor nodes in large-scale sensor networks. We further analyze the upper bound of multicast capacity and the per-session multicast energy efficiency. Using both synthetic networks and real-world networks (i.e., Greenorbs), we evaluate the asymptotic capacity and energy efficiency and find that the theoretical scaling laws are gracefully supported by the simulation results. To our best knowledge, this is the first work verifying the scaling laws using real-world large-scale sensor network data.

Travelers’ Compliance With Social Routing Advice: Impacts on Road Network Performance and Equity

Information-based demand measures using social routing advice are receiving increasing attention, as they are expected to be successful in reducing the traffic congestion. Such measures direct travelers toward routes that benefit the road network. As such, some travelers need to act socially and take a detour. This paper explores the impacts on network performance and equity that result from the application of a social routing service. We complement the existing work by using a large-scale real-world road network, by assessing impacts on the individual level and by discussing our findings in the light of observed individual compliance behavior obtained from a field experiment. Our results show that 3.6% of total travel time within our road network could be reduced when all trips are made in compliance with the received advice. However, based on the observed compliance behavior, a social routing service is expected to direct only a minority of travelers toward social routes. We show how the attained travel time savings vary with the assumed compliance rate, e.g., to achieve 50% of the potential travel time savings, about 30% of travelers need to be willing to take a detour. Moreover, we find that lower the compliance among travelers, the more the social travelers need to compensate for the selfish behavior of others by taking longer detours. Finally, we demonstrate that the impacts on network performance and equity highly depend on the spatial distribution of social travelers among origins and destinations.

Finding patterns in the degree distribution of real-world complex networks: going beyond power law

The most important structural characteristics in the study of large-scale real-world complex networks in pattern analysis are degree distribution. Empirical observations on the pattern of the real-world networks have led to the claim that their degree distributions follow, in general, a single power law. However, a closer observation, while fitting, shows that the single power-law distribution is often inadequate to meet the data characteristics properly. Since the degree distribution in the log–log scale actually displays, under inspection, two different slopes unlike what happens while fitting with the single power law. These two slopes with a transition in between closely resemble the pattern of the sigmoid function. This motivates us to derive a novel double power-law distribution for accurately modeling the real-world networks based on the sigmoid function. The proposed modeling approach further leads to the identification of a transition degree which, it has been demonstrated, may have a significant implication in analyzing the complex networks. The applicability, as well as effectiveness of the proposed methodology, is shown using rigorous experiments and also validated using statistical tests.

Role of persistent cascades in diffusion.

We define a structural property of real-world large-scale communication networks consisting of the recurring patterns of communication among individuals, which we term persistent cascades. Using methods of inexact tree matching and agglomerative clustering, we group these patterns into classes which we claim represent some underlying way in which individuals tend to disseminate information. We extend methods from epidemic modeling to offer a way to analytically model this recurring structure in a random network, and comparing to the data, we find that the real cascading structure is significantly larger and more recurrent than the random model. We find that the cascades reveal a habitual hierarchy of spreading, alternative roles in weekday vs weekend spreading, and the existence of hidden spreaders. Finally, we show that cascade membership increases the likelihood of receiving information spreading through the network through simulation on the real order of communication events.

DHNE: Network Representation Learning Method for Dynamic Heterogeneous Networks

Analyzing the rich information behind heterogeneous networks through network representation learning methods is signifcant for many application tasks such as link prediction, node classifcation and similarity research. As the networks evolve over times, the interactions among the nodes in networks make heterogeneous networks exhibit dynamic characteristics. However, almost all the existing heterogeneous network representation learning methods focus on static networks which ignore dynamic characteristics. In this paper, we propose a novel approach DHNE to learn the representations of nodes in dynamic heterogeneous networks. The key idea of our approach is to construct comprehensive historical-current networks based on subgraphs of snapshots in time step to capture both the historical and current information in the dynamic heterogeneous network. And then under the guidance of meta paths, DHNE performs random walks on the constructed historical-current graphs to capture semantic information. After getting the node sequences through random walks, we propose the dynamic heterogeneous skip-gram model to learn the embeddings. Experiments on large-scale real-world networks demonstrate that the embeddings learned by the proposed DHNE model achieve better performances than state-of-the-art methods in various downstream tasks including node classifcation and visualization.

A Game Theoretic Model for the Formation of Navigable Small-World Networks—The Tradeoff between Distance and Reciprocity

Kleinberg proposed a family of small-world networks to explain the navigability of large-scale real-world social networks. However, the underlying mechanism that drives real networks to be navigable is not yet well understood. In this article, we present a game theoretic model for the formation of navigable small-world networks. We model the network formation as a game called the Distance-Reciprocity Balanced (DRB) game in which people seek for both high reciprocity and long-distance relationships. We show that the game has only two Nash equilibria: One is the navigable small-world network, and the other is the random network in which each node connects with each other node with equal probability, and any other network state can reach the navigable small world via a sequence of best-response moves of nodes. We further show that the navigable small-world equilibrium is very stable—(a) no collusion of any size would benefit from deviating from it; and (b) after an arbitrary deviations of a large random set of nodes, the network would return to the navigable small world as soon as every node takes one best-response step. In contrast, for the random network, a small group collusion or random perturbations is guaranteed to bring the network out of the random-network equilibrium and move to the navigable network as soon as every node takes one best-response step. Moreover, we show that navigable small-world equilibrium has much better social welfare than the random network, and we provide the price-of-anarchy and price-of-stability results of the game. Our empirical evaluation further demonstrates that the system always converges to the navigable network even when limited or no information about other players’ strategies is available, and the DRB game simulated on real-world networks leads to navigability characteristic that is very close to that of the real networks, even though the real-world networks have non-uniform population distributions different from Kleinberg’s small-world model. Our theoretical and empirical analyses provide important new insight on the connection between distance, reciprocity, and navigability in social networks.

Influence maximization on signed networks under independent cascade model

Influence maximization problem is to find a subset of nodes that can make the spread of influence maximization in a social network. In this work, we present an efficient influence maximization method in signed networks. Firstly, we address an independent cascade diffusion model in the signed network (named SNIC) for describing two opposite types of influence spreading in a signed network. We define the independent propagation paths to simulate the influence spreading in SNIC model. Particularly, we also present an algorithm for constructing the set of spreading paths and computing their probabilities. Based on the independent propagation paths, we define an influence spreading function for a seed as well as a seed set, and prove that the spreading function is monotone and submodular. A greedy algorithm is presented to maximize the positive influence spreading in the signed network. We verify our algorithm on the real-world large-scale networks. Experiment results show that our method significantly outperforms the state-of-the-art methods, particularly can achieve more positive influence spreading.

Top-k Critical Vertices Query on Shortest Path

Shortest path query is one of the most fundamental and classic problems in graph analytics, which returns the complete shortest path between any two vertices. However, in many real-life scenarios, only critical vertices on the shortest path are desirable and it is unnecessary to search for the complete path. This paper investigates the shortest path sketch by defining a top- $k$ critical vertices ( $k$ CV) query on the shortest path. Given a source vertex $s$ and target vertex $t$ in a graph, $k$ CV query can return the top- $k$ significant vertices on the shortest path $SP(s,t)$ . The significance of the vertices can be predefined. The key strategy for seeking the sketch is to apply off-line preprocessed distance oracle to accelerate on-line real-time queries. This allows us to omit unnecessary vertices and obtain the most representative sketch of the shortest path directly. We further explore a series of methods and optimizations to answer $k$ CV query on both centralized and distributed platforms, using exact and approximate approaches, respectively. We evaluate our methods in terms of time, space complexity and approximation quality. Experiments on large-scale real-world networks validate that our algorithms are of high efficiency and accuracy.

M-Centrality: identifying key nodes based on global position and local degree variation

Identifying influential nodes in a network is a major issue due to the great deal of applications concerned, such as disease spreading and rumor dynamics. That is why, a plethora of centrality measures has emerged over the years in order to rank nodes according to their topological importance in the network. Local metrics such as degree centrality make use of a very limited information and are easy to compute. Global metrics such as betweenness centrality exploit the information of the whole network structure at the cost of a very high computational complexity. Recent works have shown that combining multiple metrics is a promising strategy to quantify the node’s influential ability. Our work is in this line. In this paper, we introduce a multi-attributes centrality measure called M-Centrality that combines the information on the position of the node in the network with the local information on its nearest neighborhood. The position is measured by the K-shell decomposition, and the degree variation in the neighborhood of the node quantifies the influence of the local context. In order to examine the performances of the proposed measure, we conduct experiments on small and large scale real-world networks from the perspectives of transmission dynamics and network connectivity. According to the empirical results, the M-Centrality outperforms its alternatives in identifying both influential spreaders and nodes essential to maintain the network connectivity. In addition, its low computational complexity makes it easily applied to large scale networks.

An Efficient Two-Phase Model for Computing Influential Nodes in Social Networks Using Social Actions

The measurement of influence in social networks has received a lot of attention in the data mining community. Influence maximization refers to the process of finding influential users who make the most of information or product adoption. In real settings, the influence of a user in a social network can be modeled by the set of actions (e.g., “like”, “share”, “retweet”, “comment”) performed by other users of the network on his/her publications. To the best of our knowledge, all proposed models in the literature treat these actions equally. However, it is obvious that a “like” of a publication means less influence than a “share” of the same publication. This suggests that each action has its own level of influence (or importance). In this paper, we propose a model (called Social Action-Based Influence Maximization Model, SAIM) for influence maximization in social networks. In SAIM, actions are not considered equally in measuring the “influence power” of an individual, and it is composed of two major steps. In the first step, we compute the influence power of each individual in the social network. This influence power is computed from user actions using PageRank. At the end of this step, we get a weighted social network in which each node is labeled by its influence power. In the second step of SAIM, we compute an optimal set of influential nodes using a new concept named “influence-BFS tree”. Experiments conducted on large-scale real-world and synthetic social networks reveal the good performance of our model SAIM in computing, in acceptable time scales, a minimal set of influential nodes allowing the maximum spreading of information.

Statistical inference of probabilistic origin-destination demand using day-to-day traffic data

Recent transportation network studies on uncertainty and reliability call for modeling the probabilistic O-D demand and probabilistic network flow. Making the best use of day-to-day traffic data collected over many years, this paper develops a novel theoretical framework for estimating the mean and variance/covariance matrix of O-D demand considering the day-to-day variation induced by travelers’ independent route choices. It also estimates the probability distributions of link/path flow and their travel cost where the variance stems from three sources, O-D demand, route choice and unknown errors. The framework estimates O-D demand mean and variance/covariance matrix iteratively, also known as iterative generalized least squares (IGLS) in statistics. Lasso regularization is employed to obtain sparse covariance matrix for better interpretation and computational efficiency. Though the probabilistic O-D estimation (ODE) works with a much larger solution space than the deterministic ODE, we show that its estimator for O-D demand mean is no worse than the best possible estimator by an error that reduces with the increase in sample size. The probabilistic ODE is examined on two small networks and two real-world large-scale networks. The solution converges quickly under the IGLS framework. In all those experiments, the results of the probabilistic ODE are compelling, satisfactory and computationally plausible. Lasso regularization on the covariance matrix estimation leans to underestimate most of variance/covariance entries. A proper Lasso penalty ensures a good trade-off between bias and variance of the estimation.

Complex Design Networks: Structure and Dynamics

Why was the $6 billion FAA air traffic control project scrapped? How could the 1977 New York City blackout occur? Why do large scale engineering systems or technology projects fail? How do engineering changes and errors propagate, and how is that related to epidemics and earthquakes? In this paper we demonstrate how the rapidly expanding science of complex design networks could provide answers to these intriguing questions. We review key concepts, focusing on non-trivial topological features that often occur in real-world large-scale product design and development networks; and the remarkable interplay between these structural features and the dynamics of design rework and errors, network robustness and resilience, and design leverage via effective resource allocation. We anticipate that the empirical and theoretical insights gained by modeling real-world large-scale product design and development systems as self-organizing complex networks will turn out to be the standard framework of a genuine science of design.

Finding Most Reliable Path With Extended Shifted Lognormal Distribution

Travel time uncertainty may cause late arrival and impose a high penalty on travelers. There is a growing interest in modeling travel time uncertainty to optimize the reliability of travel time at the path and network level. Real data analysis finds that the influence factors, including day-of-week, holidays, time-of-day, road grades, traffic states, and so on, often reduce the cumulative probability of travel time even in the same facility type (the same lane number and the same divided type). Thus, a novel aggregate approach is proposed to classify the travel time data based on these influence factors. The distribution with the new aggregate approach is defined as the extended shifted lognormal (ESLN) distribution. KS test indicates that the ESLN distribution can effectively describe travel time, and outperforms normal, lognormal, gamma, and beta distribution. Travel time correlations are calculated between new aggregate groups, which can effectively reduce the complexity compared with the link to link correlations. Finally, the ESLN distribution is used to find the most reliable path in a real-world large-scale network. The comparison results between ESLN distribution and shifted lognormal (SLN) distribution show the effectiveness and improvement of the proposed method in finding the most reliable path.

LAIM: A Linear Time Iterative Approach for Efficient Influence Maximization in Large-Scale Networks

The problem of influence maximization (IM) has been extensively studied in recent years and has many practical applications such as social advertising and viral marketing. Given the network and diffusion model, IM aims to find an influential set of seed nodes so that targeting them as diffusion sources will trigger the maximum cascade of influenced individuals. The largest challenge of the IM problem is its NP-hardness, and most of the existing approaches are with polynomial time complexity, making themselves unscalable to very large networks. To address this issue, in this paper, we propose LAIM: a linear time iterative approach for efficient IM on large-scale networks. Our framework has two steps: 1) influence approximation and 2) seed set selection. In the first step, we propose an iterative algorithm to compute the local influence of a node based on a recursive formula and use the local influence to approximate its global influence. In the second step, the $k$ influential seed nodes are mined based on the approximated influence in the first step. Based on our model, we theoretically prove that the proposed approach has linear time and space complexity. We further accelerate our algorithm with simple modifications and propose its fast version. Experimental results on eight real-world large-scale networks exhibit the superiority of our approach over the state-of-the-art methods in terms of both effectiveness and efficiency.

A community detection algorithm using network topologies and rule-based hierarchical arc-merging strategies.

The authors use four criteria to examine a novel community detection algorithm: (a) effectiveness in terms of producing high values of normalized mutual information (NMI) and modularity, using well-known social networks for testing; (b) examination, meaning the ability to examine mitigating resolution limit problems using NMI values and synthetic networks; (c) correctness, meaning the ability to identify useful community structure results in terms of NMI values and Lancichinetti-Fortunato-Radicchi (LFR) benchmark networks; and (d) scalability, or the ability to produce comparable modularity values with fast execution times when working with large-scale real-world networks. In addition to describing a simple hierarchical arc-merging (HAM) algorithm that uses network topology information, we introduce rule-based arc-merging strategies for identifying community structures. Five well-studied social network datasets and eight sets of LFR benchmark networks were employed to validate the correctness of a ground-truth community, eight large-scale real-world complex networks were used to measure its efficiency, and two synthetic networks were used to determine its susceptibility to two resolution limit problems. Our experimental results indicate that the proposed HAM algorithm exhibited satisfactory performance efficiency, and that HAM-identified and ground-truth communities were comparable in terms of social and LFR benchmark networks, while mitigating resolution limit problems.

Customized bus service design for jointly optimizing passenger-to-vehicle assignment and vehicle routing

Emerging transportation network services, such as customized buses, hold the promise of expanding overall traveler accessibility in congested metropolitan areas. A number of internet-based customized bus services have been planned and deployed for major origin-destination (OD) pairs to/from inner cities with limited physical road infrastructure. In this research, we aim to develop a joint optimization model for addressing a number of practical challenges for providing flexible public transportation services. First, how to maintain minimum loading rate requirements and increase the number of customers per bus for the bus operators to reach long-term profitability. Second, how to optimize detailed bus routing and timetabling plans to satisfy a wide range of specific user constraints, such as passengers’ pickup and delivery locations with preferred time windows, through flexible decision for matching passengers to bus routes. From a space-time network modeling perspective, this paper develops a multi-commodity network flow-based optimization model to formulate a customized bus service network design problem so as to optimize the utilization of the vehicle capacity while satisfying individual demand requests defined through space-time windows. We further develop a solution algorithm based on the Lagrangian decomposition for the primal problem and a space-time prism based method to reduce the solution search space. Case studies using both the illustrative and real-world large-scale transportation networks are conducted to demonstrate the effectiveness of the proposed algorithm and its sensitivity under different practical operating conditions.

DRIMUX: Dynamic Rumor Influence Minimization with User Experience in Social Networks

With the soaring development of large scale online social networks, online information sharing is becoming ubiquitous everyday. Various information is propagating through online social networks including both the positive and negative. In this paper, we focus on the negative information problems such as the online rumors. Rumor blocking is a serious problem in large-scale social networks. Malicious rumors could cause chaos in society and hence need to be blocked as soon as possible after being detected. In this paper, we propose a model of dynamic rumor influence minimization with user experience (DRIMUX). Our goal is to minimize the influence of the rumor (i.e., the number of users that have accepted and sent the rumor) by blocking a certain subset of nodes. A dynamic Ising propagation model considering both the global popularity and individual attraction of the rumor is presented based on a realistic scenario. In addition, different from existing problems of influence minimization, we take into account the constraint of user experience utility. Specifically, each node is assigned a tolerance time threshold. If the blocking time of each user exceeds that threshold, the utility of the network will decrease. Under this constraint, we then formulate the problem as a network inference problem with survival theory, and propose solutions based on maximum likelihood principle. Experiments are implemented based on large-scale real world networks and validate the effectiveness of our method.

Integrated public transport timetable synchronization with vehicle scheduling

ABSTRACTIn public transport (PT) operations planning, timetable synchronization is a useful strategy to reduce inter-route or inter-modal passenger transfer waiting time and provide a well-connected service. This work addresses the integrated PT timetable synchronization with vehicle scheduling problem for a given PT network. A new bi-objective integer programming model is developed for the problem. The objectives are to maximize the number of simultaneous vehicle arrivals at the transfer nodes of a PT network, and to minimize the required fleet size. A novel two-stage deficit function (DF)-based method is developed to solve the mathematical model using a human–machine interactive approach. At the first stage, some existing optimizers are employed to quickly generate an initial solution. At the second stage, a DF-based graphical optimization technique is utilized to generate alternative Pareto-efficient solutions by reducing the fleet size. Numerical results demonstrate that the proposed model has the potential to be applied for large-scale real-world networks.

Modeling heterogeneous traffic flow: A pragmatic approach

Modeling dynamics of heterogeneous traffic flow is central to the control and operations of today’s increasingly complex transportation systems. We develop a macroscopic heterogeneous traffic flow model. This model considers interplay of multiple vehicle classes, each of which is assumed to possess homogeneous car-following behavior and vehicle attributes. We propose the concepts of road capacity split and perceived equivalent density for each class to model both lateral and longitudinal cross-class interactions across neighboring cells. Rather than leveraging hydrodynamic analogies, it establishes pragmatic cross-class interaction rules aspired by capacity allocation and approximate inter-cell fluxes. This model generalizes the classical Cell Transmission Model (CTM) to three types of traffic regimes in general, i.e. free flow, semi-congestion, and full congestion regimes. This model replicates prominent empirical characteristics exhibited by mixed vehicular flow, including formation and spatio-temporal propagation of shockwaves, vehicle overtaking, as well as oscillatory waves. Those features are validated against numerical experiments and the NGSIM I-80 data. Realistic class-specific travel times can be computed from this model efficiently, which demonstrates the feasibility of applying this multi-class model to large-scale real-world networks.