Worst-case Complexity Research Articles

Resilient set-based state estimation for linear time-invariant systems using zonotopes

This paper considers the problem of set-based state estimation for linear time-invariant (LTI) systems under time-varying sensor attacks. Provided that the LTI system is stable and observable via every single sensor and that at least one sensor is uncompromised, we guarantee that the true state is always contained in the estimated set. We use zonotopes to represent these sets for computational efficiency. However, we show that intelligently designed stealthy attacks may cause exponential growth in the algorithm’s worst-case complexity. We present several strategies to handle this complexity issue and illustrate our resilient zonotope-based state estimation algorithm on a rotating target system.

Hereditary Cohesive Subgraphs Enumeration on Bipartite Graphs: The Power of Pivot-based Approaches

Finding cohesive subgraphs from a bipartite graph is a fundamental operator in bipartite graph analysis. In this paper, we focus on the problem of mining cohesive subgraphs from a bipartite graph that satisfy a hereditary property. Here a cohesive subgraph meets the hereditary property if all of its subgraphs satisfy the same property as itself. We show that several important cohesive subgraph models, such as maximal biclique and maximal k-biplex, satisfy the hereditary property. The problem of enumerating all maximal hereditary subgraphs was known to be NP-hard. To solve this problem, we first propose a novel and general pivot-based enumeration framework to efficiently enumerate all maximal hereditary subgraphs in a bipartite graph. Then, based on our general framework, we develop a new pivot-based algorithm with several pruning techniques to enumerate all maximal bicliques. We prove that the worst-case time complexity of our pivot-based maximal biclique enumeration algorithm is O(m x 2n/2 ) (or O(m\times 1.414^n)) which is near optimal since there exist up to O(2n/2 ) maximal bicliques in a bipartite graph with n vertices and m edges. Moreover, we also show that our algorithm can achieve polynomial-delay time complexity with a slight modification. Third, on the basis of our general framework, we also devise a novel pivot-based algorithm with several non-trivial pruning techniques to enumerate maximal k-biplexes in a bipartite graph. Finally, we conduct extensive experiments using 11 real-world bipartite graphs to evaluate the proposed algorithms. The results show that our pivot-based solutions can achieve one order of magnitude (three orders of magnitude) faster than the state-of-the-art maximal biclique enumeration algorithms (maximal k-biplex enumeration algorithms).

Worst-case complexity of an SQP method for nonlinear equality constrained stochastic optimization

Worst-case complexity of an SQP method for nonlinear equality constrained stochastic optimization

Bayesian persuasion: Reduced form approach

We introduce reduced form representations of Bayesian persuasion problems where the variables are the probabilities that the receiver takes each of her actions. These are simpler objects than, say, the joint distribution over states and actions in the obedience formulation of the persuasion problem. This can make a difference in computational and analytical tractability, which we illustrate with two applications. The first shows that with quadratic receiver payoffs, the worst-case complexity scales with the number of actions and not the number of states. If |A| and |S| denote the number of actions and states respectively, the worst case complexity of the obedience formulation is O(|A||S|(|S|+|A|)1.5L) where L is its input size. The worst-case complexity of the reduced form representation is O(|A|2.5L). In the second application, the reduced form leads to a simple greedy algorithm to determine the maximum value a sender can achieve in any cheap talk equilibrium.

A Warm-Start Strategy in Interior Point Methods for Shrinking Horizon Model Predictive Control With Variable Discretization Step

For optimal control problems with random external disturbances, offline algorithms may not perform well because the state cannot be predicted. To solve the problems online, the model predictive control is frequently used, where a series of subproblems with different initial conditions are solved. These convex subproblems could be solved by using interior point methods after discrete approximations. To obtain a solution as accurately as possible under limited computing resources, the variable discretization step is usually considered. To improve efficiency, warm-start methods are considered, which utilize the similarity of subproblem structures. Specifically, the information obtained by solving earlier subproblems is used to solve subsequent subproblems. However, the discretization steps and the sampling numbers of subproblems may be different, and the initial states vary greatly, resulting in poor performance of existing warm-start strategies for interior point methods. In this paper, we present a new warm-start strategy for interior-point methods. In the strategy, a convex combination of components of the earlier optimal solution is used to construct an interpolation point, and we use the convex combination of the modified interpolation point and the cold-start point to obtain a warm-start point. We prove that the worst-case iteration complexity of our strategy is better than that of the cold-start. In the numerical experiment of fuel-optimal planetary powered-descent guidance problems, our strategy reduces the number of iterations of second-order cone programming by about <inline-formula><tex-math notation="LaTeX">$80\%$</tex-math></inline-formula> with the number of samples available in practical applications.

Fast Density-Based Clustering: Geometric Approach

DBSCAN is a fundamental density-based clustering algorithm with extensive applications. However, a bottleneck of DBSCAN is its O(n2) worst-case time complexity. In this paper, we propose an algorithm called GAP-DBC, which exploits the geometric relationships between points to solve this problem. GAP-DBC introduces an efficient partitioning algorithm to partition the data set with a limited number of range queries and then establishes an initial cluster structure based on the partition. GAP-DBC proceeds to iteratively refine the cluster structure by additional range queries. Finally, the cluster structure is accomplished using an iterative algorithm that utilizes the spatial relationships among points to reduce unnecessary distance calculations. We further demonstrate theoretically that GAP-DBC has an excellent guarantee in terms of computational efficiency. We conducted experiments on both synthetic and real-world data sets to evaluate the performance of GAP-DBC. The results show that our algorithm is competitive with other state-of-the-art algorithms.

Maximum k-Biplex Search on Bipartite Graphs: A Symmetric-BK Branching Approach

Enumerating maximal k-biplexes (MBPs) of a bipartite graph has been used for applications such as fraud detection. Nevertheless, there usually exists an exponential number of MBPs, which brings up two issues when enumerating MBPs, namely the effectiveness issue (many MBPs are of low values) and the efficiency issue (enumerating all MBPs is not affordable on large graphs). Existing proposals of tackling this problem impose constraints on the number of vertices of each MBP to be enumerated, yet they are still not sufficient (e.g., they require to specify the constraints, which is often not user-friendly, and cannot control the number of MBPs to be enumerated directly). Therefore, in this paper, we study the problem of finding K MBPs with the most edges called MaxBPs, where K is a positive integral user parameter. The new proposal well avoids the drawbacks of existing proposals (i.e., the number of MBPs to be enumerated is directly controlled and the MBPs to be enumerated tend to have high values since they have more edges than the majority of MBPs). We formally prove the NP-hardness of the problem. We then design two branch-and-bound algorithms, among which, the better one called FastBB improves the worst-case time complexity to O*(γkn), where O* suppresses the polynomials, γk is a real number that relies on k and is strictly smaller than 2, and n is the number of vertices in the graph. For example, for k=1, γk is equal to 1.754. We further introduce three techniques for boosting the performance of the branch-and-bound algorithms, among which, the best one called PBIE can further improve the time complexity to O*(γkd3) for large sparse graphs, where d is the maximum degree of the graph (note that d&lt; &lt;n for sparse graphs). We conduct extensive experiments on both real and synthetic datasets, and the results show that our algorithm is up to four orders of magnitude faster than all baselines and finding MaxBPs works better than finding all MBPs for a fraud detection application.

Multi-agent deep reinforcement learning for user association and resource allocation in integrated terrestrial and non-terrestrial networks

Integrating the terrestrial network with non-terrestrial networks to provide radio access as anticipated in the beyond 5G networks calls for efficient user association and resource allocation strategies. In this work, a weighted sum single objective optimization problem that maximizes the total network data rate while minimizing the mobility-induced handoffs and prioritizing service provisioning of mission-critical users in the integrated terrestrial and non-terrestrial network is formulated. The problem’s complexity is reduced by solving the defined problem in two phases; the user association followed by the resource distribution phase. Several proposed approaches to the user association sub-problem utilize a central node that requires nearly-complete information, which may not be available in real time. On the contrary, this paper proposes a centralized training and distributed execution multi-agent dueling double deep Q network (MA3DQN) solution. Each user collects the channel state and access node loading information in this approach and makes an association decision that considers its quality of service requirements. The algorithm’s performance is validated through comparison with the genetic algorithm (GA), the integer linear programming (ILP), a heuristic approximation-based solution, the greedy approach, and the random user association (RUA) algorithm. Moreover, the paper simulates the multi-agent deep Q network solution as an additional benchmark algorithm. Simulation results reveal that as the number of users in the network increases, the acquired data rate of the MA3DQN is within 0.48% and 0.4% of that achieved by the GA and ILP, respectively, and outperforms all other algorithms. Notably, the proposed MA3DQN algorithm presents the best running time, attaining a gain of 99.9% over the GA algorithm, which performs the poorest among the algorithms characterized by polynomial worst-case time complexity. Besides, the MA3DQN approach maintains a handoff probability of zero, unlike the ILP, the approximation-based, greedy, and RUA solutions.

Inexact restoration for minimization with inexact evaluation both of the objective function and the constraints

In a recent paper an Inexact Restoration method for solving continuous constrained optimization problems was analyzed from the point of view of worst-case functional complexity and convergence. On the other hand, the Inexact Restoration methodology was employed, in a different research, to handle minimization problems with inexact evaluation and simple constraints. These two methodologies are combined in the present report, for constrained minimization problems in which both the objective function and the constraints, as well as their derivatives, are subject to evaluation errors. Together with a complete description of the method, complexity and convergence results will be proved.

GriT-DBSCAN: A spatial clustering algorithm for very large databases

DBSCAN is a fundamental spatial clustering algorithm with numerous practical applications. However, a bottleneck of DBSCAN is its O(n2) worst-case time complexity. To address this limitation, we propose a new grid-based algorithm for exact DBSCAN in Euclidean space called GriT-DBSCAN, which is based on the following two techniques. First, we introduce grid tree to organize the non-empty grids for the purpose of efficient non-empty neighboring grids queries. Second, by utilizing the spatial relationships among points, we propose a technique that iteratively prunes unnecessary distance calculations when determining whether the minimum distance between two sets is less than or equal to a certain threshold. We theoretically demonstrate that GriT-DBSCAN has excellent reliability in terms of time complexity. In addition, we obtain two variants of GriT-DBSCAN by incorporating heuristics, or by combining the second technique with an existing algorithm. Experiments are conducted on both synthetic and real-world data sets to evaluate the efficiency of GriT-DBSCAN and its variants. The results show that our algorithms outperform existing algorithms.

Derivative-free separable quadratic modeling and cubic regularization for unconstrained optimization

We present a derivative-free separable quadratic modeling and cubic regularization technique for solving smooth unconstrained minimization problems. The derivative-free approach is mainly concerned with building a quadratic model that could be generated by numerical interpolation or using a minimum Frobenius norm approach, when the number of points available does not allow to build a complete quadratic model. This model plays a key role to generate an approximated gradient vector and Hessian matrix of the objective function at every iteration. We add a specialized cubic regularization strategy to minimize the quadratic model at each iteration, that makes use of separability. We discuss convergence results, including worst case complexity, of the proposed schemes to first-order stationary points. Some preliminary numerical results are presented to illustrate the robustness of the specialized separable cubic algorithm.

Faster Edge‐Path Bundling through Graph Spanners

AbstractEdge‐Path bundling is a recent edge bundling approach that does not incur ambiguities caused by bundling disconnected edges together. Although the approach produces less ambiguous bundlings, it suffers from high computational cost. In this paper, we present a new Edge‐Path bundling approach that increases the computational speed of the algorithm without reducing the quality of the bundling. First, we demonstrate that biconnected components can be processed separately in an Edge‐Path bundling of a graph without changing the result. Then, we present a new edge bundling algorithm that is based on observing and exploiting a strong relationship between Edge‐Path bundling and graph spanners. Although the worst case complexity of the approach is the same as of the original Edge‐Path bundling algorithm, we conduct experiments to demonstrate that the new approach is 5–256 times faster than Edge‐Path bundling depending on the dataset, which brings its practical running time more in line with traditional edge bundling algorithms.

Worst-case evaluation complexity of a quadratic penalty method for nonconvex optimization

This paper addresses the worst-case evaluation complexity of a version of the standard quadratic penalty method for smooth nonconvex optimization problems with constraints. The method analysed allows inexact solution of the subproblems and do not require prior knowledge of the Lipschitz constants related with the problem. When an approximate feasible point is used as starting point, it is shown that the referred method takes at most outer iterations to generate an ϵ-approximate KKT point, where is the first penalty parameter. For equality constrained problems, this bound yields to an evaluation complexity bound of , when and suitable first-order methods are used as inner solvers. For problems having only linear equality constraints, an evaluation complexity bound of is established when appropriate p-order methods () are used as inner solvers. Illustrative numerical results are also presented and corroborate the theoretical predictions.

Latency considerations for stochastic optimizers in variational quantum algorithms

Variational quantum algorithms, which have risen to prominence in the noisy intermediate-scale quantum setting, require the implementation of a stochastic optimizer on classical hardware. To date, most research has employed algorithms based on the stochastic gradient iteration as the stochastic classical optimizer. In this work we propose instead using stochastic optimization algorithms that yield stochastic processes emulating the dynamics of classical deterministic algorithms. This approach results in methods with theoretically superior worst-case iteration complexities, at the expense of greater per-iteration sample (shot) complexities. We investigate this trade-off both theoretically and empirically and conclude that preferences for a choice of stochastic optimizer should explicitly depend on a function of both latency and shot execution times.

(p,q)-biclique counting and enumeration for large sparse bipartite graphs.

In this paper, we study the problem of (, )-biclique counting and enumeration for large sparse bipartite graphs. Given a bipartite graph and two integer parameters p and q, we aim to efficiently count and enumerate all (, )-bicliques in G, where a (, )-biclique B(L,R) is a complete subgraph of G with , , , and . The problem of (, )-biclique counting and enumeration has many applications, such as graph neural network information aggregation, densest subgraph detection, and cohesive subgroup analysis. Despite the wide range of applications, to the best of our knowledge, we note that there is no efficient and scalable solution to this problem in the literature . This problem is computationally challenging, due to the worst-case exponential number of (, )-bicliques. In this paper, we propose a competitive branch-and-bound baseline method, namely BCList, which explores the search space in a depth-first manner, together with a variety of pruning techniques. Although BCList offers a useful computation framework to our problem, its worst-case time complexity is exponential to . To alleviate this, we propose an advanced approach, called BCList++. Particularly, BCList++ applies a layer-based exploring strategy to enumerate (, )-bicliques by anchoring the search on either U or V only, which has a worst-case time complexity exponential to either p or q only. Consequently, a vital task is to choose a layer with the least computation cost. To this end, we develop a cost model, which is built upon an unbiased estimator for the density of 2-hop graph induced by U or V. To improve computation efficiency, BCList++ exploits pre-allocated arrays and vertex labeling techniques such that the frequent subgraph creating operations can be substituted by array element switching operations. We conduct extensive experiments on 16 real-life datasets, and the experimental results demonstrate that BCList++ significantly outperforms the baseline methods by up to 3 orders of magnitude. We show via a case study that (, )-bicliques optimizes the efficiency of graph neural networks. In this paper, we extend our techniques to count and enumerate (, )-bicliques on uncertain bipartite graphs. An efficient method IUBCList is developed on the top of BCList++, together with a couple of pruning techniques, including common neighbor refinement and search branch early termination, to discard unpromising uncertain (, )-bicliques early. The experimental results demonstrate that IUBCList significantly outperforms the baseline method by up to 2 orders of magnitude.

Worst-case evaluation complexity of a derivative-free quadratic regularization method

Worst-case evaluation complexity of a derivative-free quadratic regularization method

A modified PRP-type conjugate gradient algorithm with complexity analysis and its application to image restoration problems

In this paper, a modified conjugate gradient method is proposed for nonconvex optimization. This method possesses the sufficient descent property independent of any line search. The global convergence property of the algorithm is established under the Wolfe line search strategy or the Armijo line search condition, respectively. Additionally, the complexity analysis of the proposed algorithm is investigated. To reach the point with the norm of the gradient below ɛ, the worst-case complexity bound matches that of the gradient method. Moreover, the obtained numerical results demonstrate that the modified method is effective for large-scale optimization problems and image restoration problems.

Network Support for High-Performance Distributed Machine Learning

The traditional approach to distributed machine learning is to adapt learning algorithms to the network, e.g., reducing updates to curb overhead. Networks based on intelligent edge, instead, make it possible to follow the opposite approach, i.e., to define the logical network topology em around the learning task to perform, so as to meet the desired learning performance. In this paper, we propose a system model that captures such aspects in the context of supervised machine learning, accounting for both learning nodes (that perform computations) and information nodes (that provide data). We then formulate the problem of selecting (i) which learning and information nodes should cooperate to complete the learning task, and (ii) the number of iterations to perform, in order to minimize the learning cost while meeting the target prediction error and execution time. After proving important properties of the above problem, we devise an algorithm, named DoubleClimb, that can find a 1+1/|I|-competitive solution (with I being the set of information nodes), with cubic worst-case complexity. Our performance evaluation, leveraging a real-world network topology and considering both classification and regression tasks, also shows that DoubleClimb closely matches the optimum, outperforming state-of-the-art alternatives.

Dynamic unsplittable flows with path-change penalties: New formulations and solution schemes for large instances

In this work, we consider the dynamic unsplittable flow problem. This variation of the unsplittable flow problem has received little attention so far. The unsplittable flow problem is an NP-hard extension of the multi-commodity flow problem where each commodity sends its flow on only one path. In its dynamic version, this problem features several time steps and a penalty is paid when a commodity changes its path from one time step to the next. We present several mixed-integer linear programming formulations for this problem and compare the strength of their linear relaxation. These formulations are embedded in several solvers which are extensively compared on small to large instances. One of these formulations must be solved through a column generation process whose pricing problem is more difficult than those used in classical flow problems. We present limitations of the pricing schemes proposed in earlier works and describe two new schemes with a better worst-case complexity. Overall, this work lays a strong algorithmic baseline for the resolution of the dynamic unsplittable flow problem, proposes original formulations, and discusses the compared advantages of each, thus hopefully contributing a step towards a better understanding of this problem for both OR researchers and practical applications.

Exact Complexity Certification of Suboptimal Branch-and-Bound Algorithms for Mixed-Integer Linear Programming

In this paper, we present a method to exactly certify the computational complexity of standard suboptimal branch-and-bound (B&B) algorithms for computing suboptimal solutions to mixed-integer linear programming (MILP) problems. Three well-known approaches for suboptimal B&B are considered. This work shows that it is possible to exactly certify the computational complexity also when these approaches are used. Moreover, it also enables to compute exact bounds on the level of suboptimality actually to be obtained online, also for methods previously without any such guarantees. It additionally provides a novel deeper insight into how they affect the performance of the B&B algorithm in terms of the required computation time and memory storage. The exact bounds on the online worst-case computational complexity (e.g., the accumulated number of LP solver iterations or size of the B&B tree) and the worst-case suboptimality computed with the proposed method are very relevant for real-time applications such as Model Predictive Control (MPC) for hybrid systems. The numerical experiments confirm the correctness of the proposed method, and they demonstrate the usefulness of the certification method for certification of a standard online B&B-based MILP solver employing the three considered suboptimal techniques.