Choice Of Step Size Research Articles

Asynchronous Distributed Algorithms for Seeking Generalized Nash Equilibria Under Full and Partial-Decision Information.

This paper investigates asynchronous algorithms for distributedly seeking generalized Nash equilibria with delayed information in multiagent networks. In the game model, a shared affine constraint couples all players' local decisions. Each player is assumed to only access its private objective function, private feasible set, and a local block matrix of the affine constraint. We first give an algorithm for the case when each agent is able to fully access all other players' decisions. By using auxiliary variables related to communication links and the edge Laplacian matrix, each player can carry on its iteration asynchronously with only private data and possibly delayed information from its neighbors. Then, we consider the case when agents cannot know all other players' decisions, called a partial-decision information case. We introduce a local estimation of the overall agents' decisions and incorporate consensus dynamics on these local estimations. The two algorithms do not need any centralized clock coordination, fully exploit the local computation resource, and remove the idle time due to waiting for the "slowest" agent. Both algorithms are developed by preconditioned forward-backward operator splitting, and their convergence is shown by relating them to asynchronous fixed-point iterations, under proper assumptions and fixed and nondiminishing step-size choices. Numerical studies verify the algorithms' convergence and efficiency.

Some Results on Strongly Pseudomonotone Quasi-Variational Inequalities

In this paper, strongly pseudomonotone quasi-variational inequalities are investigated. We provide sufficient conditions for existence and uniqueness of solutions of strongly pseudomonotone quasi-variational inequalities. We present some error bounds in terms of residual and regularized gap functions. The global exponential stability of equilibrium solutions of a projected dynamical system for strongly pseudomonotone quasi-variational inequalities is investigated. Strong convergence and error estimates for sequence generated by the (modified) gradient projection method with suitable choices of stepsizes are also studied. Some examples and numerical experiments are provided to support our main results.

Exact Diffusion for Distributed Optimization and Learning—Part II: Convergence Analysis

Part I of this paper developed the exact diffusion algorithm to remove the bias that is characteristic of distributed solutions for deterministic optimization problems. The algorithm was shown to be applicable to the larger set of locally balanced left-stochastic combination policies than the set of doubly-stochastic policies. These balanced policies endow the algorithm with faster convergence rate, more flexible step-size choices and better privacy-preserving properties. In this Part II, we examine the convergence and stability properties of exact diffusion in some detail and establish its linear convergence rate. We also show that it has a wider stability range than the EXTRA consensus solution, meaning that it is stable for a wider range of step-sizes and can, therefore, attain faster convergence rates. Analytical examples and numerical simulations illustrate the theoretical findings.

Exact Diffusion for Distributed Optimization and Learning—Part I: Algorithm Development

This paper develops a distributed optimization strategy with guaranteed exact convergence for a broad class of left-stochastic combination policies. The resulting exact diffusion strategy is shown in Part II of this paper to have a wider stability range and superior convergence performance than the EXTRA strategy. The exact diffusion method is applicable to locally balanced left-stochastic combination matrices which, compared to the conventional doubly stochastic matrix, are more general and able to endow the algorithm with faster convergence rates, more flexible step-size choices, and improved privacy-preserving properties. The derivation of the exact diffusion strategy relies on reformulating the aggregate optimization problem as a penalized problem and resorting to a diagonally weighted incremental construction. Detailed stability and convergence analyses are pursued in Part II of this paper and are facilitated by examining the evolution of the error dynamics in a transformed domain. Numerical simulations illustrate the theoretical conclusions.

A Regression-Based Collaborative Filtering Recommendation Approach to Time-Stepping Multi-Solver Co-Simulation

The ever-increasing application of modeling and simulation to the development of complex engineering systems has made co-simulation indispensable to the handling of coupled multi-domain models. The mechanism for controlling communication between multiple solvers holds the key to co-simulation performance and is regarded as one of the most challenging parts in co-simulation as a lot of tradeoffs need to be made in terms of stability, accuracy, and efficiency. As such, a holistic and dynamic approach is required, which has not been addressed by this paper that has a focus on either tailored problem with a specific numerical analysis scheme or software platforms for implementing data exchange. This paper precisely aims to address this gap by developing a knowledge-based approach to streamlining the co-simulation process. Specifically, a regression-based collaborative filtering approach is developed to recommend suitable ordinary differential equation solvers for individual simulators according to the specific engineering characteristics and historical simulation data. On this basis, the theoretical analysis of the stability region and truncation error is conducted to provide guidance on controlling time stepping of individual simulators using a Jacobi communication scheme. This approach has been evaluated in several computational experiments, in which the advantages of the proposed approach are demonstrated. First, the recommendation algorithm is reliable in making suggestions on viable solvers during simulation run time, especially when only sparse historical datasets are available. Second, the time-stepping scheme noticeably improves the computational efficacy owing to it having no dependence on the initial step-size choice, which is a more eminent advantage for high-fidelity co-simulation problems.

On the evaluation of the generalised Voigt functions

For the ordinary and the generalised Voigt functions, we indicate an accurate and efficient sinc function based numerical integration method. This involves a trapezoidal integration with residue correction for the poles of the integrand. We provide a rough saddle point estimate of the discretisation error. We also indicate how to reduce the numerical oscillations by symmetrising the quadrature nodes around the real part of the singularity. By a suitable step size choice, a minimum of 14 digit accuracy (with respect to published reference values) is guaranteed for the generalised Voigt function evaluation for all values of the dependent parameters.

Cardiac and skeletal actin substrates uniquely tune cardiac myosin strain-dependent mechanics.

Cardiac ventricular myosin (βmys) translates actin by transducing ATP free energy into mechanical work during muscle contraction. Unitary βmys translation of actin is the step-size. In vitro and in vivo βmys regulates contractile force and velocity autonomously by remixing three different step-sizes with adaptive stepping frequencies. Cardiac and skeletal actin isoforms have a specific 1 : 4 stoichiometry in normal adult human ventriculum. Human adults with inheritable hypertrophic cardiomyopathy (HCM) upregulate skeletal actin in ventriculum probably compensating the diseased muscle's inability to meet demand by adjusting βmys force–velocity characteristics. βmys force–velocity characteristics were compared for skeletal versus cardiac actin substrates using ensemble in vitro motility and single myosin assays. Two competing myosin strain-sensitive mechanisms regulate step-size choices dividing single βmys mechanics into low- and high-force regimes. The actin isoforms alter myosin strain-sensitive regulation such that onset of the high-force regime, where a short step-size is a large or major contributor, is offset to higher loads probably by the unique cardiac essential light chain (ELC) N-terminus/cardiac actin contact at Glu6/Ser358. It modifies βmys force–velocity by stabilizing the ELC N-terminus/cardiac actin association. Uneven onset of the high-force regime for skeletal versus cardiac actin modulates force–velocity characteristics as skeletal/cardiac actin fractional content increases in diseased muscle.

An Efficient Barzilai–Borwein Conjugate Gradient Method for Unconstrained Optimization

The Barzilai–Borwein conjugate gradient methods, which were first proposed by Dai and Kou (Sci China Math 59(8):1511–1524, 2016), are very interesting and very efficient for strictly convex quadratic minimization. In this paper, we present an efficient Barzilai–Borwein conjugate gradient method for unconstrained optimization. Motivated by the Barzilai–Borwein method and the linear conjugate gradient method, we derive a new search direction satisfying the sufficient descent condition based on a quadratic model in a two-dimensional subspace, and design a new strategy for the choice of initial stepsize. A generalized Wolfe line search is also proposed, which is nonmonotone and can avoid a numerical drawback of the original Wolfe line search. Under mild conditions, we establish the global convergence and the R-linear convergence of the proposed method. In particular, we also analyze the convergence for convex functions. Numerical results show that, for the CUTEr library and the test problem collection given by Andrei, the proposed method is superior to two famous conjugate gradient methods, which were proposed by Dai and Kou (SIAM J Optim 23(1):296–320, 2013) and Hager and Zhang (SIAM J Optim 16(1):170–192, 2005), respectively.

A multidimensional discrete digital chaotic encryption system

In this article, a new multidimensional discrete chaotic system is proposed, and the characteristics and advantages of the multidimensional discrete chaotic system are analyzed. The multidimensional discrete chaotic system is designed using digital technology, and the system has the complexity of hyper-chaotic system, and it can avoid the choice of step size, at the same time, the design of the circuit is simple, the resource occupancy rate is low, it is suitable for the design of digital system, and so on. It can be used in constructing chaotic sequence generator, chaotic encryption, and visual sensor networks.

Distributed Subgradient-Based Multiagent Optimization With More General Step Sizes

A wider selection of step sizes is explored for the distributed subgradient algorithm for multi-agent optimization problems, for both time-invariant and time-varying communication topologies. The square summable requirement of the step sizes commonly adopted in the literature is removed. The step sizes are only required to be positive, vanishing and non-summable. It is proved that in both unconstrained and constrained optimization problems, the agents' estimates reach consensus and converge to the optimal solution with the more general choice of step sizes. The idea is to show that a weighted average of the agents' estimates approaches the optimal solution, but with different approaches. In the unconstrained case, the optimal convergence of the weighted average of the agents' estimates is proved by analyzing the distance change from the weighted average to the optimal solution and showing that the weighted average is arbitrarily close to the optimal solution. In the constrained case, this is achieved by analyzing the distance change from the agents' estimates to the optimal solution and utilizing the boundedness of the constraints. Then the optimal convergence of the agents' estimates follows because consensus is reached in both cases. These results are valid for both a strongly connected time-invariant graph and time-varying balanced graphs that are jointly strongly connected.

Stochastic Non-Convex Ordinal Embedding With Stabilized Barzilai-Borwein Step Size

Learning representation from relative similarity comparisons, often called ordinal embedding, gains rising attention in recent years. Most of the existing methods are batch methods designed mainly based on the convex optimization, say, the projected gradient descent method. However, they are generally time-consuming due to that the singular value decomposition (SVD) is commonly adopted during the update, especially when the data size is very large. To overcome this challenge, we propose a stochastic algorithm called SVRG-SBB, which has the following features: (a) SVD-free via dropping convexity, with good scalability by the use of stochastic algorithm, i.e., stochastic variance reduced gradient (SVRG), and (b) adaptive step size choice via introducing a new stabilized Barzilai-Borwein (SBB) method as the original version for convex problems might fail for the considered stochastic non-convex optimization problem. Moreover, we show that the proposed algorithm converges to a stationary point at a rate O(1/T) in our setting, where T is the number of total iterations. Numerous simulations and real-world data experiments are conducted to show the effectiveness of the proposed algorithm via comparing with the state-of-the-art methods, particularly, much lower computational cost with good prediction performance.

Development and static testing of the 18x6 m SSU-TTMBF spatial structural unit

The aim of this work is the development of a fragment of the structural covering, consisting of a triangular block of frames, the choice of step size (width) of the structural unit and a study of its mode of deformation by comparing experimental and theoretical results of research.

Robust adaptive gain higher order sliding mode observer based control-constrained nonlinear model predictive control for spacecraft formation flying

This work deals with the development of a decentralized optimal control algorithm, along with a robust observer, for the relative motion control of spacecraft in leader-follower based formation. An adaptive gain higher order sliding mode observer has been proposed to estimate the velocity as well as unmeasured disturbances from the noisy position measurements. A differentiator structure containing the Lipschitz constant and Lebesgue measurable control input, is utilized for obtaining the estimates. Adaptive tuning algorithms are derived based on Lyapunov stability theory, for updating the observer gains, which will give enough flexibility in the choice of initial estimates. Moreover, it may help to cope with unexpected state jerks. The trajectory tracking problem is formulated as a finite horizon optimal control problem, which is solved online. The control constraints are incorporated by using a nonquadratic performance functional. An adaptive update law has been derived for tuning the step size in the optimization algorithm, which may help to improve the convergence speed. Moreover, it is an attractive alternative to the heuristic choice of step size for diverse operating conditions. The disturbance as well as state estimates from the higher order sliding mode observer are utilized by the plant output prediction model, which will improve the overall performance of the controller. The nonlinear dynamics defined in leader fixed Euler-Hill frame has been considered for the present work and the reference trajectories are generated using Hill-Clohessy-Wiltshire equations of unperturbed motion. The simulation results based on rigorous perturbation analysis are presented to confirm the robustness of the proposed approach.

Conditional Gradient Method Without Line-Search

We propose a simple rule for the step-size choice in the conditional gradient method, which does not require any line-search procedure. It takes into account the current behavior of the method. Its convergence is established under the same assumptions as those for the previously known methods.

Local voting protocol step-size choice for consensus achievement

In the paper, a multi-agent network system of different computing nodes is considered. A problem of load balancing in the network is addressed. The problem is formulated as consensus achievement problem and solved via local voting protocol. Agents exchange information about their states in presence of noise in communication channels. For the system operating in noised conditions analytically obtained estimation of control protocol optimal step size value is given. The dependence of the system behaviour on value of control protocol step-size is demonstrated in simulation examples.

Local voting protocol step-size choice for consensus achievement

Local voting protocol step-size choice for consensus achievement

Q-State Space Least Mean Family of Algorithms

It is generally known that model-based estimation algorithms (such as Kalman filter and its family) perform better than the non-model-based algorithms [such as least mean square (LMS), recursive least squares] due to extra information available in terms of system dynamics (which can be used to provide state space model of the system). However, the computational complexity of the model based algorithms is very high. On the other hand, the convergence performance of model based least mean type algorithms [such as state space least mean (SSLM) algorithms] is slower and highly dependent on the step-size choice. Thus, the larger step size can provide faster convergence but gives poor steady-state excess mean square error (EMSE). To meet this conflicting demand, we propose to employ the q-calculus to minimize the generalized least mean cost function. The main advantage of using the q-calculus is that it can provide a nonlinear correction term in the adaptation of the state estimate vector. Consequently, this results in an intelligent adaptation by providing both faster convergence in the initial phase of adaptation and a lower steady-state EMSE in the final phase. The developed algorithms are termed as q-state space least mean (q-SSLM) algorithms. The performance of the proposed q-state space least mean square (q-SSLMS) algorithm is also investigated both in terms of convergence in the mean and the mean square sense. The supremacy of the proposed algorithm is validated by performing several simulations and it is also contrasted with the performance of the well-known Kalman filter. Finally, the theoretical convergence analysis is also validated via simulations.

The G-Euler process for nonlinear autonomous systems

The G-Euler process is an improved version of the Lawson method (SIAM J Numer Anal 4:372–380, 1967). This paper is concerned with the behaviors of true solutions and numerical solutions, and shows that the G-Euler process preserves the behaviors of true solutions with a suitable choice of stepsize. The rate of coincidence of behaviors is introduced instead of the relative error. By using the minimum rate of coincidence of behaviors, we also show that the numerical solutions computed by the G-Euler process follow the true solutions of nonlinear autonomous systems to the last.

Differentially Private Distributed Constrained Optimization

Many resource allocation problems can be formulated as an optimization problem whose constraints contain sensitive information about participating users. This paper concerns solving this kind of optimization problem in a distributed manner while protecting the privacy of user information. Without privacy considerations, existing distributed algorithms normally consist in a central entity computing and broadcasting certain public coordination signals to participating users. However, the coordination signals often depend on user information, so that an adversary who has access to the coordination signals can potentially decode information on individual users and put user privacy at risk. We present a distributed optimization algorithm that preserves differential privacy, which is a strong notion that guarantees user privacy regardless of any auxiliary information an adversary may have. The algorithm achieves privacy by perturbing the public signals with additive noise, whose magnitude is determined by the sensitivity of the projection operation onto user-specified constraints. By viewing the differentially private algorithm as an implementation of stochastic gradient descent, we are able to derive a bound for the suboptimality of the algorithm. We illustrate the implementation of our algorithm via a case study of electric vehicle charging. Specifically, we derive the sensitivity and present numerical simulations for the algorithm. Through numerical simulations, we are able to investigate various aspects of the algorithm when being used in practice, including the choice of step size, number of iterations, and the trade-off between privacy level and suboptimality.

An accelerated non-Euclidean hybrid proximal extragradient-type algorithm for convex–concave saddle-point problems

This paper describes an accelerated HPE-type method based on general Bregman distances for solving convex–concave saddle-point (SP) problems. The algorithm is a special instance of a non-Euclidean hybrid proximal extragradient framework introduced by Svaiter and Solodov [An inexact hybrid generalized proximal point algorithm and some new results on the theory of Bregman functions, Math. Oper. Res. 25(2) (2000), pp. 214–230] where the prox sub-inclusions are solved using an accelerated gradient method. It generalizes the accelerated HPE algorithm presented in He and Monteiro [An accelerated HPE-type algorithm for a class of composite convex–concave saddle-point problems, SIAM J. Optim. 26 (2016), pp. 29–56] in two ways, namely: (a) it deals with general monotone SP problems instead of bilinear structured SPs and (b) it is based on general Bregman distances instead of the Euclidean one. Similar to the algorithm of He and Monteiro [An accelerated HPE-type algorithm for a class of composite convex–concave saddle-point problems, SIAM J. Optim. 26 (2016), pp. 29–56], it has the advantage that it works for any constant choice of proximal stepsize. Moreover, a suitable choice of the stepsize yields a method with the best known iteration-complexity for solving monotone SP problems. Computational results show that the new method is superior to Nesterov's [Smooth minimization of non-smooth functions, Math. Program. 103(1) (2005), pp. 127–152] smoothing scheme.