Problem For Large Networks Research Articles

Integrated Bus Timetabling and Scheduling with a Mutation-Based Evolutionary Scheme Maximizing Headway Quality and Connections

Public transport planning is a multi-level process that includes various complex tasks. These tasks are traditionally executed sequentially, and the result of each task serves as input for consecutive tasks. A simultaneous integrated consideration of multiple tasks may lead to an overall improved solution, but further increase the complexity of already hard-to-solve planning problems. This work focuses on timetabling and vehicle scheduling and evaluates synergies from the integrated optimization. We investigate an exact sequential, exact integrated, and heuristic approach to solve the combined problem for large public transport networks considering the interlining of vehicles, multiple vehicle types, or multiple depots while additionally aiming to maximize regular “clock-faced’’ headways and transfer connections. Compared to sequential optimization, an integrated approach significantly reduces nominal and operational costs while maintaining high service quality. However, an exact integrated approach is only able to compute solutions for problems of limited size in a reasonable time. We propose an adaptive modular evolutionary extendable scheme that effectively balances computational efficiency and solution quality. By utilizing various problem-specific mutation operators and adaptively applying them based on their impact, the heuristic can compute high-quality solutions for large real-world-inspired public transport networks in a reasonable time while considering short connecting times between lines and regular clock-faced headways.

Mining Top-k Frequent Patterns in Large Geosocial Networks: A Mnie-Based Extension Approach

Frequent pattern mining (FPM) has played an important role in many graph domains, such as bioinformatics and social networks. In this paper, we focus on geo-social graphs, a kind of social network augmented by geographical information. However, in addition to the exponential time complexity of the problem, we face the challenge of efficient subgraph retrieval since we are interested in patterns in a specific region in such a network. For this reason, we formulate the top-k FPM problem in large geo-social networks. Specifically, we devise a novel framework for subgraph retrieval and FPM mining with a series of optimizations. First, we propose a neighboring-aware R-tree (NaR-Tree) index structure to alleviate the challenge of retrieving subgraphs from a large graph. NaR-Tree is a variant of R-tree in which each nonleaf tree node further maintains some edge statistics information for the rectangle related to it. Second, we define the concept of minimum image-based support of edges (MNIE). With the help of the NaR-Tree and MNIE-based pattern extension approach, a mining algorithm that addresses the problem of exponential candidate patterns is proposed. We also present a lazy retrieval strategy to reduce the frequency of subgraph retrieval. Finally, we adopt an edge sampling approach to further accelerate the mining process. Extensive experiments on real-world and synthesized datasets are conducted to demonstrate the effectiveness and efficiency of our solution.

Importance evaluation method of complex network nodes based on information entropy and iteration factor

In the study of complex networks, researchers have long focused on the identification of influencing nodes. Based on topological information, several quantitative methods of determining the importance of nodes are proposed. K-shell is an efficient way to find potentially affected nodes. However, the K-shell overemphasizes the influence of the location of the central nodebut ignores the effect of the force of the nodes located at the periphery of the network. Furthermore, the topology of real networks is complex, which makes the computation of the K-shell problem for large scale-free networks extremely difficult. In order to avoid ignoring the contribution of any node in the network to the propagation, this work proposes an improved method based on the iteration factor and information entropy to estimate the propagation capability of each layer of nodes. This method not only achieves the accuracy of node ordering, but also effectively avoids the phenomenon of rich clubs. To evaluate the performance of this method, the SIR model is used to simulate the propagation efficiency of each node, and the algorithm is compared with other algorithms. Experimental results show that this method has better performance than other methods and is suitable for large-scale networks.

Exact and Heuristics Algorithms for Screen Line Problem in Large Size Networks: Shortest Path-Based Column Generation Approach

In this study, we present exact and heuristics algorithms for a traffic sensors location problem called the screen line problem. It is a problem of how to locate traffic sensors on a transportation network where all the origin/destination node pairs are fully separated. The problem experiences two main complexity dimensions that obstruct finding an efficient solution algorithm for large-scale networks: its mathematical formulation, which is proved in the literature to be NP-hard, and an inherent combinatorial complexity due to the need for a network complete path enumeration. In this study, the problem is reformulated as a set covering problem. Thereafter, the dual formulation is recalled showing that the shortest path-based column generation method would yield as many paths as necessary and hence circumvent the intractability of the full path enumeration task. This path generation technique enables applying both the proposed heuristics and exact methods to the problem. In addition, the gap value between the heuristics and the exact algorithms is set to be examined statistically. For evaluation, three networks of different sizes were used to track the scalability of proposed algorithms. The methodology showed high efficiency to deal with up to 10,000 demand node pairs in addition to the capability of producing practical solutions with respect to normal traffic flow conditions. The proposed heuristics algorithm stipulates a gap value of less than 25% with more than 99% confidence.

Power budget- and SRLG-aware cost-efficient partial protection planning models and architectures for long-reach passive optical networks

In a long-reach passive optical network (LR-PON), protection for optical network units (ONUs) against their distribution fiber (DF) failures is highly desirable to ensure uninterrupted Internet access to the associated users. In the distribution section of the LR-PON, shared-risk link groups (SRLGs) are formed by DFs due to sharing fiber cables and conduits. Most of the existing SRLG-aware DF protection schemes require residual bandwidth with ONUs. Therefore, they fail to provide protection to all ONUs under high reliability requirements (RRs) and result to be very cost-inefficient as well. We first propose a SRLG-aware reliability framework to compute the connection reliability of every ONU and joint reliability of every ONU-ONU pair and ONU-node pair. Thereafter, we propose two different SRLG-aware nonresidual bandwidth based partial protection schemes and their compatible architectures to provide partial DF protection to ONUs against SRLG failures to satisfy the given RR. Following the proposed schemes, we formulate two different integer linear programming (ILP) based optimization problems to set up backup connections by using backup DFs and other backup optical resources with the minimum protection cost. The proposed ILP-based protection planning models restrict the backup DF lengths to bound the propagation delay to satisfy the strict delay requirement for real-time applications. Since the proposed ILP-based models turn out to be computationally intractable for large network problems, we also propose two heuristic schemes that provide comparable results to that of the ILP-based models. We evaluate the performance of the proposed partial protection schemes with reference to the protection cost and total length of backup DFs (TLBDF). The exhaustive simulation results show that the proposed partial protection schemes not only satisfy high RRs, but also require a much lower protection cost, TLBDF, and optical power margin in comparison to the existing SRLG-aware protection schemes.

Transient Stability Analysis of an HPR-1000 Power Plant using ETAP Software

Transient stability analysis has become an important subject, as the world’s demand for electricity is increasing with an enormous speed, and power systems are facing different challenges, new power plants are being installed to fulfill the demand and supply mismatch, in addition to that degradation of equipment, sudden load changes and switching operations are also some of the problems in large networks. For large power systems it is a highly non-linear problem and also used to analyze whether the system remain in synchronism or not following a disturbance. To analyze the effect of these disturbances and consequences of the outages transient stability analysis is usually performed. In this paper this analysis is performed for an HPR-1000 nuclear power plant connected to the 500kV network of NTDC Grid. Different external 3-phase fault scenarios are considered on the 500kV network and simulated using ETAP (Electric Transient Analyzer Program) software. To analyze the system behavior during and after the fault has occurred, certain parameters are monitored. It was observed that the HPR-1000 power plant-maintained synchronism and remained stable in all the fault scenarios.

Neural Network Structure Optimization by Simulated Annealing.

A critical problem in large neural networks is over parameterization with a large number of weight parameters, which limits their use on edge devices due to prohibitive computational power and memory/storage requirements. To make neural networks more practical on edge devices and real-time industrial applications, they need to be compressed in advance. Since edge devices cannot train or access trained networks when internet resources are scarce, the preloading of smaller networks is essential. Various works in the literature have shown that the redundant branches can be pruned strategically in a fully connected network without sacrificing the performance significantly. However, majority of these methodologies need high computational resources to integrate weight training via the back-propagation algorithm during the process of network compression. In this work, we draw attention to the optimization of the network structure for preserving performance despite compression by pruning aggressively. The structure optimization is performed using the simulated annealing algorithm only, without utilizing back-propagation for branch weight training. Being a heuristic-based, non-convex optimization method, simulated annealing provides a globally near-optimal solution to this NP-hard problem for a given percentage of branch pruning. Our simulation results have shown that simulated annealing can significantly reduce the complexity of a fully connected network while maintaining the performance without the help of back-propagation.

Detection by Block- and Band-Permuted Data

Unlabeled detection is an important problem in large sensor networks faced with big-data applications. The focus of this paper is on decision problems in which the data vector received at the fusion center (FC), tasked to perform the final inference, undergoes an unknown permutation. Two representative permutation models are considered, whose structure is imposed by practical considerations. The first is the <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$m$</tex-math></inline-formula> -block permutation model, where the observations are split into different blocks of size <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$m$</tex-math></inline-formula> , and independent data permutations affect each block. The second is the <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$r$</tex-math></inline-formula> -banded permutation model, in which each sample of the vector received by the FC lies at most <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$r$</tex-math></inline-formula> positions ahead or behind its original location. To solve these challenging detection tasks, three detectors for both permutation models are proposed. The first is inspired by the uniformly most powerful invariant principle; the second relies on a distribution-averaged strategy; and the third is designed according the generalized likelihood ratio approach. The performance of these detectors is investigated by computer experiments, and their relative merits are discussed.

Tuning successive linear programming to solve AC optimal power flow problem for large networks

Successive linear programming (SLP) is a practical approach for solving large-scale nonlinear optimization problems. Alternating current optimal power flow (ACOPF) is no exception, particularly the large size of real-world networks. However, in order to achieve tractability, it is essential to tune the SLP algorithm presented in the literature. This paper presents a modified SLP algorithm to solve the ACOPF problem, specified by the U.S. Department of Energy’s (DOE) Grid Optimization (GO) Competition Challenge 1, within strict time limits. The algorithm first finds a near-optimal solution for the relaxed problem (i.e., Stage 1). Then, it finds a feasible solution in the proximity of the near-optimal solution (i.e., Stage 2 and Stage 3). The numerical experiments on test cases ranging from 500-bus to 30,000-bus systems show that the algorithm is tractable. The results show that our proposed algorithm is tractable and can solve more than 80% of test cases faster than the well-known Interior Point Method while significantly reduce the number of iterations required to solve ACOPF. The number of iterations is considered an important factor in the examination of tractability which can drastically reduce the computational time required within each iteration.

The method of detection and localization of configuration defects in geodetic networks by means of Tikhonov regularization

Abstract In adjusted geodetic networks, cases of local configuration defects (defects in the geometric structure of the network due to missing data or errors in point numbering) can be encountered, which lead to the singularity of the normal equation system in the least-squares procedure. Numbering errors in observation sets cause the computer program to define the network geometry incorrectly. Another cause of a defect may be accidental omission of certain data records, causing local indeterminacy or lowering of local reliability rates in a network. Obviously, the problem of a configuration defect may be easily detectable in networks with a small number of points. However, it becomes a real problem in large networks, where manual checking of all data becomes a very expensive task. The paper presents a new strategy for the detection of configuration defects with the use of the Tikhonov regularization method. The method was implemented in 1992 in the GEONET system (www.geonet.net.pl).

Structural inference of time‐varying mixed graphical models

With the rapid advancement of biotechnology, there arise new challenges to discover regulatory and co‐action relationships among heterogeneous biological features as well as to capture significant topological changes when data are observed across time. This paper is devoted to joint structural estimation of time‐varying mixed graphical models based on multivariate data over a series of time points. Assuming the graph topology changes gradually over time, we establish a flexible local estimator to fully exploit the structural smoothness. Utilizing variational likelihood inference, we impose a group lasso penalty to integrate information from nearby time points. In order to reduce the algorithmic complexity, we propose an accelerated alternating direction method of multipliers (ADMM)‐based algorithm exploiting the block diagonal structure to adapt our problem for large sparse networks. Practical merits of our method are exhibited through synthetic networks with mixed data types. Ultimately, we illustrate the real application by incorporating multi‐platform data from the PsychENCODE human brain development project and detect the evolution of gene regulatory networks along different stages of human brain development.

Minimizing Energy Use of Mixed-Fleet Public Transit for Fixed-Route Service

Affordable public transit services are crucial for communities since they enable residents to access employment, education, and other services. Unfortunately, transit services that provide wide coverage tend to suffer from relatively low utilization, which results in high fuel usage per passenger per mile, leading to high operating costs and environmental impact. Electric vehicles (EVs) can reduce energy costs and environmental impact, but most public transit agencies have to employ them in combination with conventional, internal-combustion engine vehicles due to the high upfront costs of EVs. To make the best use of such a mixed fleet of vehicles, transit agencies need to optimize route assignments and charging schedules, which presents a challenging problem for large transit networks. We introduce a novel problem formulation to minimize fuel and electricity use by assigning vehicles to transit trips and scheduling them for charging, while serving an existing fixed-route transit schedule. We present an integer program for optimal assignment and scheduling, and we propose polynomial-time heuristic and meta-heuristic algorithms for larger networks. We evaluate our algorithms on the public transit service of Chattanooga, TN using operational data collected from transit vehicles. Our results show that the proposed algorithms are scalable and can reduce energy use and, hence, environmental impact and operational costs. For Chattanooga, the proposed algorithms can save $145,635 in energy costs and 576.7 metric tons of CO2 emission annually.

Fast Connectivity Minimization on Large-Scale Networks

The connectivity of networks has been widely studied in many high-impact applications, ranging from immunization, critical infrastructure analysis, social network mining, to bioinformatic system studies. Regardless of the end application domains, connectivity minimization has always been a fundamental task to effectively control the functioning of the underlying system. The combinatorial nature of the connectivity minimization problem imposes an exponential computational complexity to find the optimal solution, which is intractable in large systems. To tackle the computational barrier, greedy algorithm is extensively used to ensure a near-optimal solution by exploiting the diminishing returns property of the problem. Despite the empirical success, the theoretical and algorithmic challenges of the problems still remain wide open. On the theoretical side, the intrinsic hardness and the approximability of the general connectivity minimization problem are still unknown except for a few special cases. On the algorithmic side, existing algorithms are hard to balance between the optimization quality and computational efficiency. In this article, we address the two challenges by (1) proving that the general connectivity minimization problem is NP-hard and is the best approximation ratio for any polynomial algorithms, and (2) proposing the algorithm CONTAIN and its variant CONTAIN + that can well balance optimization effectiveness and computational efficiency for eigen-function based connectivity minimization problems in large networks.

Measuring the systemic importance of banks

We provide a new metric for the systemic importance of banks based on the intensity of spillovers of daily CDS movements. We denote this a bank’s Individual Systemic Risk (ISR). Our novel empirical tool uses Bayesian VAR to address the dimensionality problem in large networks of banks and maps for every pair of banks in the system the shocks that they exchange. We apply this tool to all banks that issue publicly traded CDS contracts among the world’s biggest 150 and identify which of these may trigger instability in the global financial system. Our methodology provides measures that are relatively stable across time, contain persistent information, have strong explanatory power for standard variables of systemic risk, and provided early warning signals in the case study of Deutsche Bank in mid-2016. Using our ISR measure, we demonstrate which bank- and country-specific characteristics are related to bank systemic importance. We find higher systemic importance for banks that are relatively larger, less profitable, have G-SIB status, and are headquartered in economies with fiscally strong sovereigns. We also show that there is a negative relationship between concentration in the domestic banking sector and the systemic importance of a bank. We examine the relationship of ISR to alternative systemic risk metrics.

Dynamic adaptive spectrum allocation in flexible grid optical network with multi‐path routing

Fragmentation is one of the major issues in elastic optical network (EON) due to its dynamic characteristics which may possibly degrade the spectrum efficiency. Several techniques have been presented in the literature to get the optimal solution of this problem while multi-path provisioning comes out to be very compelling among them. In multi-path provisioning enable networks, multi-path routing and spectrum assignment (MPRSA) creates further complications due to the hardware availability and more spectrum consumption. This paper presents a novel technique termed as adaptive service provisioning (ASP) to come up with a solution of the MPRSA problem and it minimizes the spectrum fragments in the network by precisely managing the scattered spectrum slots. This technique performs the adaptive traffic demand splitting on disjoint paths in case it is unable to be served by k-shortest routing paths based on the parameters such as hardware support, capacity available and demand. An integer linear programming (ILP) model as well as a heuristic algorithm for solving MPRSA problem in large optical networks where ILP is not compliant is developed. Numerical simulations are performed to investigate the performance of the algorithm in terms of blocking probability and spectrum utilization.

Fast Variational Inference for Joint Mixed Sparse Graphical Models

Mixed graphical models are widely implemented to capture interactions among different types of variables. To simultaneously learn the topology of multiple mixed graphical models and encourage common structure, people have developed a variational maximum likelihood inference approach, which takes advantage of the log-determinant relaxation. In this article, we further improve the computational efficiency of this method by exploiting the block diagonal structure of the solution. We present a simple necessary and sufficient condition to identify the connected components in the solution, so as to determine the block diagonal structure. Then, utilizing the idea of “divide-and-conquer”, we are able to adapt the joint structural inference problem for multiple related large sparse networks. We illustrate the merits of the proposed algorithm via experimental comparisons in computational speed.

A coarsening method for bipartite networks via weight-constrained label propagation

A multilevel method is a scalable strategy to solve optimization problems in large bipartite networks, which operates in three stages. Initially the input network is iteratively coarsened into a hierarchy of gradually smaller networks. Coarsening implies in collapsing vertices into so-called super-vertices which inherit properties of their originating vertices. An initial solution is obtained executing the target algorithm in the coarsest network. Finally, this solution is successively projected back over the inverse sequence of coarsened networks, up to the initial one, yielding an approximate final solution. Despite its potential applicability, the strategy faces several theoretical and practical limitations. Coarsening is usually attained following a user-defined policy to match vertices pairwise. However, the network reduction process is extremely slow and may yield degraded solutions due to propagation of poor matches. Additionally, proper parameterization of coarsening algorithms is difficult, as well as ensuring the super-vertices preserve the relevant properties. We address these issues with a near-linear complexity coarsening strategy based on weight-constrained label propagation. Our strategy collapses groups of vertices, rather than pairs, yielding faster and more extensive network reduction. Moreover, users may specify the desired size of the coarsest network and control super-vertex weights. The applicability of our solution is illustrated in multiple scenarios, namely: multilevel implementation of an existing high-cost community detection algorithm; as a direct community detection algorithm; finally, network visualization, in connection with force-directed graph drawing algorithms. Results provide empirical evidence on the potential of our proposal to foster novel applications of the multilevel method in bipartite networks.

Tackling the maximum happy vertices problem in large networks

Abstract In this paper we consider a variant of graph colouring known as the maximum happy vertices problem. This problem involves taking a graph in which a subset of the vertices have been preassigned to colours. The objective is to then colour the remaining vertices such that the number of happy vertices is maximised, where a vertex is considered happy only when it is assigned to the same colour as all of its neighbours. We design and test a tabu search approach, which is compared to two existing state of the art methods. We see that this new approach is particularly suited to larger problem instances and finds very good solutions in very short time frames. We also propose a algorithm to find upper bounds for the problem efficiently. Moreover, we propose an algorithm for imposing additional precoloured vertices and are hence able to significantly reduce the solution space. Finally, we present an analysis of this problem and use probabilistic arguments to characterise problem hardness.

Exact Distance Query in Large Graphs through Fast Graph Simplification

Abstract Shortest path distance query is one of the most fundamental problems in graph theory and applications. Nowadays, the scale of graphs becomes so large that traditional algorithms for shortest path are not available to answer the exact distance query quickly. Many methods based on two-hop labeling have been proposed to solve this problem. However, they cost too much either in preprocessing or query phase to handle large networks containing as many as tens of millions of vertices. In this paper, we propose a novel $k$-hub labeling method to address this problem in large networks with less preprocessing cost while keeping the query time in the microsecond level on average. Technically, two types of labels are presented in our construction, one for distance queries when the actual distance is at most $k-2$, which we call local label, and the other for further distance queries, which we call hub label. Our approach of $k$-hub labeling is essentially different from previous widely used two-hop labeling framework since we construct labels by using hub network structure. We conduct extensive experiments on large real-world networks and the results demonstrate the higher efficiency of our method in preprocessing phase and the much smaller space size of constructed index compared to previous efficient two-hop labeling method, with a comparatively fast query speed.

Measuring the Systemic Importance of Banks

We provide a new metric for the systemic importance of banks based on the intensity of spillovers of daily CDS movements. We denote this a bank’s Individual Systemic Risk (ISR). Our novel empirical tool uses Bayesian VAR to address the dimensionality problem in large networks of banks and maps for every pair of banks in the system the shocks that they exchange. We apply this tool to all banks that issue publicly traded CDS contracts among the world’s biggest 150 and identify which of these may trigger instability in the global financial system. Our methodology provides measures that are relatively stable across time, contain persistent information, have strong explanatory power for standard variables of systemic risk, and provided early warning signals in the case study of Deutsche Bank in mid-2016. Using our measure, ISR, we demonstrate which bank- and country-specific characteristics are related to bank systemic importance. We find higher systemic importance for banks that are relatively larger, less profitable, have G-SIB status, and are headquartered in economies with fiscally strong sovereigns. We also show that there is a negative relationship between concentration in the domestic banking sector and the systemic importance of a bank. We examine the relationship of ISR to alternative systemic risk metrics.