Decomposable Structure Research Articles

Guiding Neuroevolution with Structural Objectives.

The structure and performance of neural networks are intimately connected, and by use of evolutionary algorithms, neural network structures optimally adapted to a given task can be explored. Guiding such neuroevolution with additional objectives related to network structure has been shown to improve performance in some cases, especially when modular neural networks are beneficial. However, apart from objectives aiming to make networks more modular, such structural objectives have not been widely explored. We propose two new structural objectives and test their ability to guide evolving neural networks on two problems which can benefit from decomposition into subtasks. The first structural objective guides evolution to align neural networks with a user-recommended decomposition pattern. Intuitively, this should be a powerful guiding target for problems where human users can easily identify a structure. The second structural objective guides evolution towards a population with a high diversity in decomposition patterns. This results in exploration of many different ways to decompose a problem, allowing evolution to find good decompositions faster. Tests on our target problems reveal that both methods perform well on a problem with a very clear and decomposable structure. However, on a problem where the optimal decomposition is less obvious, the structural diversity objective is found to outcompete other structural objectives-and this technique can even increase performance on problems without any decomposable structure at all.

Exponential mixing for a class of dissipative PDEs with bounded degenerate noise

We study a class of discrete-time random dynamical systems with compact phase space. Assuming that the deterministic counterpart of the system in question possesses a dissipation property, its linearisation is approximately controllable, and the driving noise has a decomposable structure, we prove that the corresponding family of Markov processes has a unique stationary measure, which is exponentially mixing in the dual-Lipschitz metric. The abstract result is applicable to nonlinear dissipative PDEs perturbed by a random force which affects only a few Fourier modes and belongs to a certain class of random processes. We assume that the nonlinear PDE in question is well posed, its nonlinearity is non-degenerate in the sense of the control theory, and the random force is a regular and bounded function of time which satisfies some decomposability and observability hypotheses. This class of forces includes random Haar series, where coefficients for high Haar modes decay sufficiently fast. In particular, the result applies to the 2D Navier–Stokes system and the nonlinear complex Ginzburg–Landau equations. The proof of the abstract theorem uses the coupling method, enhanced by the Newton–Kantorovich–Kolmogorov fast convergence.

Feature uncertainty bounds for explicit feature maps and large robust nonlinear SVM classifiers

We consider the binary classification problem when data are large and subject to unknown but bounded uncertainties. We address the problem by formulating the nonlinear support vector machine training problem with robust optimization. To do so, we analyze and propose two bounding schemes for uncertainties associated to random approximate features in low dimensional spaces. The proposed bound calculations are based on Random Fourier Features and the Nystrom methods. Numerical experiments are conducted to illustrate the benefit of the technique. We also emphasize the decomposable structure of the proposed robust nonlinear formulation that allows the use of efficient stochastic approximation techniques when datasets are large.

A branch-and-price algorithm for capacitated hypergraph vertex separation

We exactly solve the $${\mathcal {NP}}$$-hard combinatorial optimization problem of finding a minimum cardinality vertex separator with k (or arbitrarily many) capacitated shores in a hypergraph. We present an exponential size integer programming formulation which we solve by branch-and-price. The pricing problem, an interesting optimization problem on its own, has a decomposable structure that we exploit in preprocessing. We perform an extensive computational study, in particular on hypergraphs coming from the application of re-arranging a matrix into single-bordered block-diagonal form. Our experimental results show that our proposal complements the previous exact approaches in terms of applicability for larger k, and significantly outperforms them in the case $$k=\infty $$.

Cycle flow formulation of optimal network flow problems and respective distributed solutions

In this paper, we use the cycle basis from graph theory to reduce the size of the decision variable space of optimal network flow problems by eliminating the aggregated flow conservation constraint. We use a minimum cost flow problem and an optimal power flow problem with generation and storage at the nodes to demonstrate our decision variable reduction method. The main advantage of the proposed technique is that it retains the natural sparse&#x002F decomposable structure of network flow problems. As such, the reformulated problems are still amenable to distributed solutions. We demonstrate this by proposing a distributed alternating direction method of multipliers &#x0028 ADMM &#x0029 solution for a minimum cost flow problem. We also show that the communication cost of the distributed ADMM algorithm for our proposed cycle-based formulation of the minimum cost flow problem is lower than that of a distributed ADMM algorithm for the original arc-based formulation.

Multiclass Energy Management for Peer-to-Peer Energy Trading Driven by Prosumer Preferences

This paper proposes a peer-to-peer energy market platform based on the new concept of multiclass energy management, to coordinate trading between prosumers with heterogeneous (i.e., beyond purely financial) preferences. Power networks are undergoing a fundamental transition, with traditionally passive distribution network consumers becoming “prosumers”; proactive consumers that actively manage their production and consumption of energy. The paper introduces the new concept of energy classes, allowing energy to be treated as a heterogeneous product, based on attributes of its source, which are perceived by prosumers to have value. Examples include generation technology, location in the network and owner's reputation. The proposed peer-to-peer energy market platform coordinates trading between subscribed prosumers and the wholesale electricity market, to minimize costs associated with losses and battery depreciation, while providing added value by accounting for the prosumers' individual preferences for the source/destination of the energy they consume/produce. The decomposable structure of the multiclass energy management problem is exploited to devise a distributed price-directed optimization mechanism, providing scalability and prosumer data privacy. Receding horizon model predictive control allows the prosumers to adjust their planned power flows based on the wholesale energy price, and up-to-date renewable generation and load predictions.

A mixed-integer program and a Lagrangian-based decomposition algorithm for the supply chain network design with quantity discount and transportation modes

We study the design of a two-echelon supply chain network in the presence of suppliers’ all-unit quantity discount and transportation mode selection. The problem involves determining the best location for plants and the allocation of customers to open plants. The problem additionally entails making decisions for the order quantity from each supplier for each plant and accordingly selecting the best transportation mode that can accommodate these order quantities among echelons. The objective is to minimize the total cost associated with fixed opening and operating costs of plants, fixed and variable costs of transportation modes, and purchasing costs of raw materials. To characterize and solve this problem, we develop a mixed-integer programming (MIP) model. We demonstrate that the MIP model has a special mathematical structure that makes it amenable to decomposition techniques. We, therefore, exploit this decomposable structure and develop an effective Lagrangian-based decomposition for solving the MIP. Our Lagrangian Relaxation (LR) method relaxes the complicating constraints associated with commodity flow conservation among echelons in the MIP, yielding more tractable subproblems, one for each echelon. Solutions obtained from the relaxed problem may be infeasible, e.g., the demand for some of the customers may not be satisfied. We remedy these subproblems’ infeasibilities using novel feasibility algorithms and appropriate Lagrangian multipliers that penalize constraints’ violations in the subproblems, leading the algorithm towards global feasibility/optimality. We appraise the performance of the MIP model and the LR algorithm on instances of varying sizes. We show that the MIP model solved via CPLEX finds integer feasible solutions for 42% of large problem instances and its average optimality gap for these solved instances is 64.56%. The LR algorithm significantly improves the solvability and optimality gap of the MIP model and finds integer feasible solutions for 100% of problem instances and achieves an average optimality gap of 1.78%. We investigate the robustness of our algorithm by conducting sensitivity analyses on the model parameters. We demonstrate that the LR technique remains robust and tractable with respect to various parameters’ values.

A scalable global optimization algorithm for stochastic nonlinear programs

We present a global optimization algorithm for two-stage stochastic nonlinear programs (NLPs). The algorithm uses a tailored reduced-space spatial branch and bound (BB) strategy to exploit the nearly decomposable structure of the problem. At each node in the BB scheme, a lower bound is constructed by relaxing the so-called non-anticipativity constraints and an upper bound is constructed by fixing the first-stage variables to the current candidate solution. A key advantage of this approach is that both lower and upper bounds can be computed by solving individual scenario subproblems. Another key property of this approach is that it only needs to perform branching on the first-stage variables to guarantee convergence (branching on the second-stage variables is performed implicitly during the computation of lower and upper bounds). Notably, convergence results for this scheme also hold for two-stage stochastic MINLPs with mixed-integer first-stage variables and continuous recourse variables. We present a serial implementation of the algorithm in Julia, that we call SNGO. The implementation is interfaced to the structured modeling language Plasmo.jl, which facilitates benchmarking and model processing. Our implementation incorporates typical features that help accelerate the BB search such as LP-based lower bounding techniques, local search-based upper bounding techniques, and relaxation-based bounds tightening techniques. These strategies require the solution of extensive forms of the stochastic program but can potentially be solved using structured interior-point solvers (when the problem is an NLP). Numerical experiments are performed for a controller tuning formulation, a parameter estimation formulation for microbial growth models, and a stochastic test set from GLOBALlib. We compare the computational results against SCIP and demonstrate that the proposed approach achieves significant speedups.

Long‐term mineral fertilization in paddy soil alters the chemical structures and decreases the fungistatic activities of humic acids

Fungistatic activity is a newly identified ecological function for soil humic acids. However, the relation between fungistatic activities of soil humic acids and their chemical structures is still unclear. In addition, how fertilizer application affects fungistatic activity is still unknown. We sampled paddy soils with different long‐term fertilizer schemes, namely: unfertilized control (CK), mineral fertilizer applied (NPK) and mineral fertilizer applied plus straw return (NPKRS). Soil humic acids were extracted and their fungistatic activities against 14 phytopathogenic fungi were investigated to evaluate their ecological functions. The humic acids from NPK and NPKRS treatments showed much smaller fungistatic effects than those from CK treatments. The chemical structures of humic acids were investigated by 13C nuclear magnetic resonance (13C NMR) spectroscopy. Long‐term fertilization significantly changed the chemical structure of soil humic acid, and the variation in chemical structure affected the fungistatic activities. The relative content of alkoyl C and alkyl C, the alkoyl C/alkyl C ratio and aliphaticity all had positive correlations with the fungistatic effects of humic acids. That indicated humic acids with more aliphatic and decomposable structures showed more fungistatic activity. From the results, we believe that long‐term application of mineral fertilizer in paddy fields makes the soil humic acids less aliphatic and more recalcitrant, and concomitantly weakens their ability to inhibit phytopathogenic fungi.Highlights We studied the inhibitory effects of paddy soil humic acids on 14 phytopathogenic fungi. Fertilization‐induced changes in humic acids affect inhibition of phytopathogenic fungi. Long‐term mineral fertilization significantly changed chemical structures of humic acids. Humic acid with more aliphatic and decomposable structures had more fungistatic activity.

Decomposable robust two‐stage optimization: An application to gas network operations under uncertainty

AbstractWe study gas network problems with compressors and control valves under uncertainty that can be formulated as two‐stage robust optimization problems. Uncertain data are present in the physical parameters of the pipes as well as in the overall demand. We show how to exploit the special decomposable structure of the problem to reformulate the two‐stage problem as a single‐stage robust optimization problem. The right‐hand side of the single‐stage problem can be precomputed by solving a series of optimization problems and multiple elements of the right‐hand side can be combined into one optimization task. The practical feasibility and effectiveness of our approach is demonstrated with benchmarks on several gas network instances, among them a realistic model of the Greek natural gas network. Overall, aggregation and preprocessing allow us to quickly solve large gas network instances under uncertainty for the price of slightly more conservative solutions.

Extended cross decomposition for mixed-integer linear programs with strong and weak linking constraints

Large-scale mixed-integer linear programming (MILP) problems, such as those from two-stage stochastic programming, usually have a decomposable structure that can be exploited to design efficient optimization methods. Classical Benders decomposition can solve MILPs with weak linking constraints (which are decomposable when linking variables are fixed) but not strong linking constraints (which are not decomposable even when linking variables are fixed). In this paper, we first propose a new rigorous bilevel decomposition strategy for solving MILPs with strong and weak linking constraints, then extend a recently developed cross decomposition method based on this strategy. We also show how to apply the extended cross decomposition method to two-stage stochastic programming problems with conditional-value-at-risk (CVaR) constraints. In the case studies, we demonstrate the significant computational advantage of the proposed extended cross decomposition method as well as the benefit of including CVaR constraints in stochastic programming.

Combining penalty‐based and Gauss–Seidel methods for solving stochastic mixed‐integer problems

AbstractIn this paper, we propose a novel decomposition approach (named PBGS) for stochastic mixed‐integer programming (SMIP) problems, which is inspired by the combination of penalty‐based Lagrangian and block Gauss–Seidel methods. The PBGS method is developed such that the inherent decomposable structure that SMIP problems present can be exploited in a computationally efficient manner. The performance of the proposed method is compared with the progressive hedging (PH) method, which also can be viewed as a Lagrangian‐based method for obtaining solutions for SMIP problems. Numerical experiments performed using instances from the literature illustrate the efficiency of the proposed method in terms of computational performance and solution quality.

Mean–variance optimization of discrete time discounted Markov decision processes

In this paper, we study a mean–variance optimization problem in an infinite horizon discrete time discounted Markov decision process (MDP). The objective is to minimize the variance of system rewards with the constraint of mean performance. Different from most of works in the literature which require the mean performance already achieve optimum, we can let the discounted performance equal any constant. The difficulty of this problem is caused by the quadratic form of the variance function which makes the variance minimization problem not a standard MDP. By proving the decomposable structure of the feasible policy space, we transform this constrained variance minimization problem to an equivalent unconstrained MDP under a new discounted criterion and a new reward function. The difference of the variances of Markov chains under any two feasible policies is quantified by a difference formula. Based on the variance difference formula, a policy iteration algorithm is developed to find the optimal policy. We also prove the optimality of deterministic policy over the randomized policy generated in the mean-constrained policy space. Numerical experiments demonstrate the effectiveness of our approach.

Marginal Pseudo-Likelihood Learning of Discrete Markov Network Structures

Markov networks are a popular tool for modeling multivariate distributions over a set of discrete variables. The core of the Markov network representation is an undirected graph which elegantly captures the dependence structure over the variables. Traditionally, the Bayesian approach of learning the graph structure from data has been done under the assumption of chordality since non-chordal graphs are difficult to evaluate for likelihood-based scores. Recently, there has been a surge of interest towards the use of regularized pseudo-likelihood methods as such approaches can avoid the assumption of chordality. Many of the currently available methods necessitate the use of a tuning parameter to adapt the level of regularization for a particular dataset. Here we introduce the marginal pseudo-likelihood which has a built-in regularization through marginalization over the graph-specific nuisance parameters. We prove consistency of the resulting graph estimator via comparison with the pseudo-Bayesian information criterion. To identify high-scoring graph structures in a high-dimensional setting we design a two-step algorithm that exploits the decomposable structure of the score. Using synthetic and existing benchmark networks, the marginal pseudo-likelihood method is shown to perform favorably against recent popular structure learning methods.

Joint capacity planning and distribution network optimization of coal supply chains under uncertainty

A two‐stage stochastic integer programming model is developed to address the joint capacity planning and distribution network optimization of multiechelon coal supply chains (CSCs) under uncertainty. The proposed model not only introduces the uses of compound real options in sequential capacity planning, but also considers uncertainty induced by both risks and ambiguities. Both strategic decisions (i.e., facility locations and initial investment, service assignment across the entire CSC, and option holding status) and scenario‐based operational decisions (i.e., facility operations and capacity expansions, outsourcing policy, and transportation and inventory strategies) can be simultaneously determined using the model. By exploiting the nested decomposable structure of the model, we develop a new distributed parallel optimization algorithm based on nonconvex generalized Bender decomposition and Lagrangean relaxation to mitigate the computation resource limitation. One of the main CSCs in China is studied to demonstrate the applicability of the proposed model and the performance of the algorithm. © 2017 American Institute of Chemical Engineers AIChE J, 64: 1246–1261, 2018

When do combinatorial structures obey the Poisson-Dirichlet law?

We attempt to soften the conditions under which the component vector of a random decomposable structure obeys the Poisson-Dirichlet distribution. The so-called Logarithmic Condition is relaxed to an averaged requirement. The results are focused on permutations sampled according to a generalized Ewens probability.

Benders decomposition and column-and-row generation for solving large-scale linear programs with column-dependent-rows

In a recent work, Muter, Birbil, and Bülbül, (2013) identified and characterized a general class of linear programming (LP) problems – known as problems with column-dependent-rows (CDR-problems). These LPs feature two sets of constraints with mutually exclusive groups of variables in addition to a set of structural linking constraints, in which variables from both groups appear together. In a typical CDR-problem, the number of linking constraints grows very quickly with the number of variables, which motivates generating both columns and their associated linking constraints simultaneously on-the-fly. In this paper, we expose the decomposable structure of CDR-problems via Benders decomposition. However, this approach brings on its own theoretical challenges. One group of variables is generated in the Benders master problem, while the generation of the linking constraints is relegated to the Benders subproblem along with the second group of variables. A fallout of this separation is that only a partial description of the dual of the Benders subproblem is available over the course of the algorithm. We demonstrate how the pricing subproblem for the column generation applied to the Benders master problem does also update the dual polyhedron and the existing Benders cuts in the master problem to ensure convergence. Ultimately, a novel integration of Benders cut generation and the simultaneous generation of columns and constraints yields a brand-new algorithm for solving large-scale CDR-problems. We illustrate the application of the proposed method on a time-constrained routing problem. Our numerical experiments confirm the outstanding performance of the new decomposition method.

Support Vector Algorithms for Optimizing the Partial Area under the ROC Curve.

The area under the ROC curve (AUC) is a widely used performance measure in machine learning. Increasingly, however, in several applications, ranging from ranking to biometric screening to medicine, performance is measured not in terms of the full area under the ROC curve but in terms of the partial area under the ROC curve between two false-positive rates. In this letter, we develop support vector algorithms for directly optimizing the partial AUC between any two false-positive rates. Our methods are based on minimizing a suitable proxy or surrogate objective for the partial AUC error. In the case of the full AUC, one can readily construct and optimize convex surrogates by expressing the performance measure as a summation of pairwise terms. The partial AUC, on the other hand, does not admit such a simple decomposable structure, making it more challenging to design and optimize (tight) convex surrogates for this measure. Our approach builds on the structural SVM framework of Joachims ( 2005 ) to design convex surrogates for partial AUC and solves the resulting optimization problem using a cutting plane solver. Unlike the full AUC, where the combinatorial optimization needed in each iteration of the cutting plane solver can be decomposed and solved efficiently, the corresponding problem for the partial AUC is harder to decompose. One of our main contributions is a polynomial time algorithm for solving the combinatorial optimization problem associated with partial AUC. We also develop an approach for optimizing a tighter nonconvex hinge loss-based surrogate for the partial AUC using difference-of-convex programming. Our experiments on a variety of real-world and benchmark tasks confirm the efficacy of the proposed methods.

Improved Lagrangian bounds and heuristics for the generalized assignment problem

Modified Lagrangian bounds are proposed for the generalized assignment problem. The approach is based on a double decomposable structure of the formulation. A family of greedy heuristics is considered to get Lagrangian based feasible solutions. Numerical results for problem instances with number of agents close to number of tasks are provided.

Variance of additive functions defined on random assemblies

We establish an inequality for the variance of an additive function defined on random decomposable structures called assemblies. The result generalizes estimates obtained earlier in the cases of permutations and mappings of a finite set into itself. It is analogous to the Turan–Kubilius inequality for additive number-theoretic functions.