Convex Cost Research Articles

A hybrid algorithm for large-scale non-separable nonlinear multicommodity flow problems

We propose an approach for large-scale non-separable nonlinear multicommodity flow problems by solving a sequence of subproblems which can be addressed by commercial solvers. Using a combination of solution methods such as modified gradient projection, shortest path algorithm and golden section search, the approach can handle general problem instances, including those with (i) non-separable cost, (ii) objective function not available analytically as polynomial but are evaluated using black-boxes, and (iii) additional side constraints not of network flow types. Implemented as a toolbox in commercial solvers, it allows researchers and practitioners, currently conversant with linear instances, to easily manage large-scale convex instances as well. In this article, we compared the proposed algorithm with alternative approaches in the literature, covering both theory and large test cases. New test cases with non-separable convex costs and non-network flow side constraints are also presented and evaluated. The toolbox is available free for academic use upon request.

De-Noising of Sparse Signals Using Mixture Model Shrinkage Function

In this work a new thresholding function referred to as ’mixture model shrinkage’ (MMS) based on the minimization of a convex cost function is proposed. Normally, thresholding functions underestimate larger signal amplitudes during the de-noising process. The proposed model is a more flexible shrinkage function as it solves the underestimation problem to a greater extent and thus efficiently de-noises the signal without affecting signal amplitudes. The Expectation minimization (EM) algorithm is used to find the model parameters along with the majorization-minimization (MM) algorithm that minimize the monotonic cost function. The proposed model is then applied for de-noising group sparse signals and Shepp Logan phantom images. Our experimental study shows that MMS outclasses current thresholding functions and overlapping group shrinkage algorithm without results suffering from underestimation. Furthermore, the proposed model has the smallest Root Mean Square Error (RMSE) for de-noising group sparse signals.

Compressed Differentially Private Distributed Optimization with Linear Convergence

This paper addresses the problem of differentially private distributed optimization under limited communication, where each agent aims to keep their cost function private while minimizing the sum of all agents’ cost functions. In response, we propose a novel Compressed differentially Private distributed Gradient Tracking algorithm (CPGT). We demonstrate that CPGT achieves linear convergence for smooth and strongly convex cost functions, even with a class of biased but contractive compressors, and achieves the same accuracy as the idealized communication algorithm. Additionally, we rigorously prove that CPGT ensures differential privacy. Simulations are provided to validate the effectiveness of the proposed algorithm.

Условие оптимальности типа принципа максимума Понтрягина в задаче управления линейными разностными уравнениями дробного порядка

The optimal control problem of fractional order linear two-dimensional difference equations sys-tems is considered. It is assumed that the control function is included in the boundary condition, and the functional is linear. A necessary and sufficient optimality condition is proved in the discrete maximum principal form. A sufficient optimality condition is proved in the nonlinear but convex cost functional case.

Order Fulfillment Under Pick Failure in Omnichannel Ship-From-Store Programs

Problem definition: We consider the setting where a retailer with many physical stores and an online presence seeks to fulfill online orders using an omnichannel fulfillment program, such as buy-online ship-from-store. These fulfillment strategies try to minimize cost while fulfilling orders within acceptable service times. We focus on single-item orders. Typically, all online orders for the item are sent to a favorable set of locations to be filled. Failed trials are sent back for further stages of trial fulfillment until the process times out. The multistage order fulfillment problem is thus an interplay of the pick-failure probabilities at the stores where they may be shipped from and the picking, shipping, and cancellation costs from these locations. Methodology: We model the problem as one of sequencing the stores from which an order is attempted to be picked and shipped in the most cost-effective way over multiple stages. We solve the fulfillment problem optimally by taking into account the changing pick-failure probabilities as a result of other online order fulfillment trials by casting it as a network flow problem with convex costs. We incorporate this as the second stage of a two-stage online order acceptance problem and generalize earlier results to the case with pick failures at stores. Results: We investigate the real-world performance of our methods and models on real order data of several of the top U.S. retailers that use our collaborating e-commerce solutions provider to optimize their fulfillment strategies. Academic/Practical Relevance: Our work enables retailers to incorporate pick failure in their order management systems for ship-from-store programs. Our new online order-acceptance policies that take into account pick failures can thus create significant savings for omnichannel retailers. Supplemental Material: The online appendices are available at https://doi.org/10.1287/msom.2022.1164 .

MAP inference algorithms without approximation for collective graphical models on path graphs via discrete difference of convex algorithm

Collective graphical model (CGM) is a probabilistic model that provides a framework for analyzing aggregated count data. Maximum a posteriori (MAP) inference of unobserved variables under given observations is one of the essential operations in CGM. Because the MAP inference problem is known to be NP-hard in general, the current mainstream approach is to solve an alternative problem obtained by approximating the objective function and applying continuous relaxation. However, this approach has two significant drawbacks. First, the quality of the solution deteriorates when the values in the count data are negligible due to the inaccuracy of Stirling’s approximation. Second, the application of continuous relaxation causes the violation of integrality constraints. This paper proposes novel algorithms for MAP inference in CGMs on path graphs to overcome these problems. Our method is based on the discrete difference of convex algorithm (DCA); DCA is a general framework to minimize the sum of a convex function and a concave function by repeatedly minimizing surrogate functions. Utilizing the particular structure of path graphs, we efficiently solve the surrogate function minimization by minimum convex cost flow algorithms. Furthermore, our approach also leads to a new method of solving another important task; MAP inference of the sample size in CGM on path graphs. Our method is naturally applicable to this task, allowing us to design very efficient algorithms. Experimental results on synthetic and real-world datasets show the effectiveness of the proposed algorithms.

Existence and uniqueness of Nash equilibrium in discontinuous Bertrand games: a complete characterization

Since (Reny in Econometrica 67:1029–1056, 1999) a substantial body of research has considered what conditions are sufficient for the existence of a pure strategy Nash equilibrium in games with discontinuous payoffs. This work analyzes a general Bertrand game, with convex costs and an arbitrary sharing rule at price ties, in which tied payoffs may be greater than non-tied payoffs when both are positive. On this domain, necessary and sufficient conditions for (i) the existence of equilibrium (ii) the uniqueness of equilibrium are presented. The conditions are intuitively easy to understand and centre around the relationships between intervals of real numbers determined by the primitives of the model.

Improving the low orbit satellite tracking ability using nonlinear model predictive controller and Genetic Algorithm

The satellite motion on the reference orbit (RO) with less energy consumption has always persuaded researchers to design optimal control systems. The nonlinear nature and time-varying equations of motion make this quest more challenging. The present study proposes a novel control system for satellite motion on the RO by considering a comprehensive model of its dynamics in orbit and a Nonlinear Model Predictive Controller (NMPC). The NMPC calculates the sub-optimal control inputs of satellite motion reference on the elliptic orbit by minimizing a convex cost function at each stage. Moreover, all weighting parameters of the cost function are optimized by the Genetic Algorithm (GA) to produce less perturbation and guarantee the best NMPC performance. Finally, the implemented NMPC has been compared to a Linear MPC (LMPC). The results show that not only can the NMPC resist against larger errors and perturbations, but it can also compensate for those errors by returning the satellite to its main orbit and maintaining it.

Differentiated attributes and service-quality competition as sources of portfolio interdependence and diverging scales in banking

Why are banks’ asset–liability decisions interdependent, and why do we observe both large and small banks? We introduce convex costs in a model with both horizontal and vertical quality differentiation, finding that, as long as a single bank spends on quality attributes, strategic quality rivalry causes the portfolio decisions of all banks to be interconnected, generating portfolio interdependence regarding choices among assets and liabilities at each bank, including banks that forego vertical quality provision. In addition, diseconomies of scale allow banks to remain profitable at different scales of operations. • We model banks that can expend on quality attributes to gain market share. • Strategic rivalry causes the portfolio decisions of banks to be interconnected. • Our model explains how both large and small banks can co-exist in the same market. • Because of convex costs, less efficient banks become niche players.

Distributed Variable Sample-Size Stochastic Optimization With Fixed Step-Sizes

In this article, we consider distributed stochastic optimization over randomly switching networks, where agents collaboratively minimize the average of all agents’ local expectation-valued convex cost functions. Due to the stochasticity in gradient observations, distributedness of local functions, and randomness of communication topologies, distributed algorithms with an exact convergence guarantee under fixed step-sizes have not been achieved yet. This work incorporates variance reduction scheme into the distributed stochastic gradient tracking algorithm, where local gradients are estimated by averaging across a variable number of sampled gradients. With an identically and independently distributed random network, we show that all agents’ iterates converge almost surely to the same optimal solution under fixed step-sizes. When the global cost function is strongly convex and the sample size increases at a geometric rate, we prove that the iterates geometrically converge to the unique optimal solution, and establish the iteration, oracle, and communication complexity. The algorithm performance, including rate and complexity analysis, are further investigated with constant step-sizes and a polynomially increasing sample size. Finally, the empirical algorithm performance are illustrated with numerical examples.

Cautious weighted random forests

Random forest is an efficient and accurate classification model, which makes decisions by aggregating a set of trees, either by voting or by averaging class posterior probability estimates. However, tree outputs may be unreliable in presence of scarce data. The imprecise Dirichlet model (IDM) provides workaround, by replacing point probability estimates with interval-valued ones. This paper investigates a new tree aggregation method based on the theory of belief functions to combine such probability intervals, resulting in a cautious random forest classifier. In particular, we propose a strategy for computing tree weights based on the minimization of a convex cost function, which takes both determinacy and accuracy into account and makes it possible to adjust the level of cautiousness of the model. The proposed model is evaluated on 25 UCI datasets and is demonstrated to be more adaptive to the noise in training data and to achieve a better compromise between informativeness and cautiousness.

Robust adaptive filtering algorithms based on the half‐quadratic criterion

Recently, robust adaptive filtering algorithms have attracted the interest of researchers and have been extensively studied. However, most of these methods suffer from the non-convexity of their performance surfaces, which results in reduced convergence rate and filtering accuracy. To address this problem, we present a novel robust half-quadratic criterion (HQC) adaptive filtering algorithm by utilizing a convex cost function. In contrast with conventional methods, the proposed HQC can introduce a more effective performance surface, allowing a gradient-based strategy to provide significant performance improvements in convergence speed and robustness against impulsive noise. Furthermore, to enhance the performance of the HQC adaptive filter, a variable step-size (VSS) version, the VSS-HQC algorithm, has also been introduced. Using the energy conservation method, the theoretical expressions for the transient, steady-state, and tracking performance analysis of the proposed HQC algorithm are studied under the Gaussian noise assumption. Then, we examined the theoretical analysis of the HQC using various numerical simulations for system identification to validate the findings. Finally, we compared the performance of the HQC and VSS-HQC algorithms to their competitors in both Gaussian noise and non-Gaussian noise scenarios through numerical experiments.

Robust proportional–integral control of submersible autonomous robotized vehicles by backstepping-averaged sub-gradient sliding mode control

This study presents a novel robust controller strategy for submersible autonomous robotized vehicles (SARV). This controller applies the averaged sub-gradient (ASG) descendant method to optimize the tracking of well-posed reference trajectories. The motion control form of the SARV is done in three stages (cascade-like control design). The phases consist of using the translation velocities as pseudo controllers to adjust the SARV position, regulating the angular velocities to attain the requested translation velocities, and using the proposed ASG to control the thrusters to get the needed angular velocities. ASG implementation optimizes a convex cost function depending on the integral of the tracking error using the ASG method. The application of Barbalat’s lemma justifies the tracking error is converging the origin asymptotically. A class of integral sliding mode controllers with a variable state-dependent gain solves the tracking of the designed reference trajectories in each of the three stages. The sliding surface depends on the tracking error for each pseudo-controller, its integral, and the cost function average. The optimization of the cost function can be done without complete knowledge of the SARV dynamics. A numerical example is presented in this study to confirm the suggested control design’s effectiveness based on the cost function’s time evolution analysis. The forced motion by the proposed controller is compared with the movement obtained by a proportional–integral–derivative (PID) controller. The proposed controller exhibits a better tracking of the reference trajectory than the PID version.

A Modular Optimal Formation Control Scheme of Multiagent Systems With Application to Multiple Mobile Robots

In this article, we studied an optimal formation control problem of general integrator chain multiagent systems with strongly convex local cost functions. The control goal is to design appropriate controllers for the agents such that they reach a desired formation shape and the global cost function is minimized. To achieve the goal, a modular control scheme with two modules is proposed. First, in Module I, an optimal formation signal generator is designed, whose outputs asymptotically converge to the minimizer of the global cost function. Second, in Module II, by employing the outputs into the feedback loop as the agents’ reference trajectories, some linear-tracking controllers are designed for the agents to track the references asymptotically. With both modules, the proposed control scheme realizes the optimal formation control goal asymptotically. Moreover, the proposed control scheme is also applied to solving an optimal formation control problem of multiple mobile robots, where the robots’ total “relative” weighted squared moving distance requires minimization. The simulations and experiments validate the effectiveness and practicality of the proposed control scheme.

Belief Dispersion and Convex Cost of Adjustment in the Stock Market and in the Real Economy

I develop a continuous-time general equilibrium model with a continuum of states of the world and a continuum of agents endowed with heterogeneous beliefs. The model permits to analyze the interactions between financial markets and production. There is a single firm that faces convex adjustment costs and maximizes its terminal value. Equivalently, the firm uses decreasing returns to scale risk-return technology. The model is tractable and matches many of the empirical regularities in aggregate output and stock prices, such as a financial volatility that is higher than the macroeconomic volatility, skewness, kurtosis, short-term momentum, and volatility risk premium during recessions. All these aspects disappear when one assumes beliefs homogeneity or constant returns to scale. In particular, the impact of beliefs heterogeneity observed in endowment economies does not pertain when introducing production unless one assumes decreasing returns to scale in the risk-return technology. This paper was accepted by David Sraer, finance. Supplemental Material: The online appendix is available at https://doi.org/10.1287/mnsc.2022.4495 .

Online Selection with Convex Costs

We study a novel online optimization problem, termed online selection with convex costs (OSCC). In OSCC, there is a sequence of items, each with a value that remains unknown before its arrival. At each step when there is a new arrival, we need to make an irrevocable decision in terms of whether to accept this item and take its value, or to reject it. The crux of OSCC is that we must pay for an increasing and convex cost associated with the number of items accepted, namely, it is increasingly more difficult to accommodate additional items. The goal is to develop an online algorithm that accepts/ selects a subset of items, without any prior statistical knowledge of future arrival information, to maximize the social surplus, namely, the sum of the accepted items' values minus the total cost. Our main result is the development of a threshold policy that is logistically-simple and easy to implement, but has provable optimality guarantees among all deterministic online algorithms.

Resource allocation problems with expensive function evaluations

The resource allocation problem is among the classical problems in operations research, and has been studied extensively for decades. However, current solution approaches are not able to efficiently handle problems with expensive function evaluations, which can occur in a variety of applications. We study the integer resource allocation problem with expensive function evaluations, for both convex and non-convex separable cost functions. We present several solution methods, both heuristics and exact methods, that aim to limit the number of function evaluations. The methods are compared in numerical experiments using both randomly generated instances and instances from two resource allocation problems occurring in radiation therapy planning. Results show that the presented solution methods compare favorably against existing derivative free optimization solvers.

Multi-period liability clearing via convex optimal control

We consider the problem of determining a sequence of payments among a set of entities that clear (if possible) the liabilities among them. We formulate this as an optimal control problem, which is convex when the objective function is, and therefore readily solved. For this optimal control problem, we give a number of useful and interesting convex costs and constraints that can be combined in any way for different applications. We describe a number of extensions, for example to handle unknown changes in cash and liabilities, to allow bailouts, to find the minimum time to clear the liabilities, or to minimize the number of non-cleared liabilities, when fully clearing the liabilities is impossible.

A multivariate adaptive gradient algorithm with reduced tuning efforts

Large neural networks usually perform well for executing machine learning tasks. However, models that achieve state-of-the-art performance involve arbitrarily large number of parameters and therefore their training is very expensive. It is thus desired to implement methods with small per-iteration costs, fast convergence rates, and reduced tuning. This paper proposes a multivariate adaptive gradient descent method that meets the above attributes. The proposed method updates every element of the model parameters separately in a computationally efficient manner using an adaptive vector-form learning rate, resulting in low per-iteration cost. The adaptive learning rate computes the absolute difference of current and previous model parameters over the difference in subgradients of current and previous state estimates. In the deterministic setting, we show that the cost function value converges at a linear rate for smooth and strongly convex cost functions. Whereas in both the deterministic and stochastic setting, we show that the gradient converges in expectation at the order of O(1/k) for a non-convex cost function with Lipschitz continuous gradient. In addition, we show that after T iterates, the cost function of the last iterate scales as O(log(T)/T) for non-smooth strongly convex cost functions. Effectiveness of the proposed method is validated on convex functions, smooth non-convex function, non-smooth convex function, and four image classification data sets, whilst showing that its execution requires hardly any tuning unlike existing popular optimizers that entail relatively large tuning efforts. Our empirical results show that our proposed algorithm provides the best overall performance when comparing it to tuned state-of-the-art optimizers.

General optimal stopping with linear cost

This article treats both discrete time and continuous time stopping problems for general Markov processes on the real line with general linear costs as they naturally arise in many problems in sequential decision making. Using an auxiliary function of maximum representation type, conditions are given to guarantee the optimal stopping time to be of threshold type. The optimal threshold is then characterized as the root of that function. For random walks, our results condense in the fact that all combinations of concave increasing payoff functions and convex cost functions lead to a one-sided solution. For Lévy processes, an explicit way to obtain the auxiliary function and the threshold is given by use of the ladder height processes. Lastly, the connection from discrete and continuous problems and possible approximation of the latter via the former is discussed and the results are applied to sequential tests of power one.