Convergence Guarantees Research Articles

Convergence Analysis of Novel Fractional-Order Backpropagation Neural Networks With Regularization Terms.

Fractional-order derivatives have the potential to improve the performance of backpropagation (BP) neural networks. Several studies have found that the fractional-order gradient learning methods may not converge to real extreme points. The truncation and the modification of the fractional-order derivative are applied to guarantee convergence to the real extreme point. Nonetheless, the real convergence ability is based on the assumption that the algorithm is convergent, which limits the practicality of the algorithm. In this article, a novel truncated fractional-order BP neural network (TFO-BPNN) and a novel hybrid TFO-BPNN (HTFO-BPNN) are designed to solve the above problem. First, to avoid overfitting, a squared regularization term is introduced into the fractional-order BP neural network. Second, a novel dual cross-entropy cost function is proposed and employed as a loss function for the two neural networks. The penalty parameter helps to adjust the effect of the penalty term and further alleviates the gradient vanishing problem. In terms of convergence, the convergence ability of the two proposed neural networks is first proved. Then, the convergence ability to the real extreme point is further analyzed theoretically. Finally, the simulation results effectively illustrate the feasibility, high accuracy, and good generalization ability of the proposed neural networks. Comparative studies among the proposed neural networks and some related methods further substantiate the superiority of the TFO-BPNN and the HTFO-BPNN.

A novel variational approach for multiphoton microscopy image restoration: from PSF estimation to 3D deconvolution

In multi-photon microscopy (MPM), a recent in-vivo fluorescence microscopy system, the task of image restoration can be decomposed into two interlinked inverse problems: firstly, the characterization of the point spread function (PSF) and subsequently, the deconvolution (i.e. deblurring) to remove the PSF effect, and reduce noise. The acquired MPM image quality is critically affected by PSF blurring and intense noise. The PSF in MPM is highly spread in 3D and is not well characterized, presenting high variability with respect to the observed objects. This makes the restoration of MPM images challenging. Common PSF estimation methods in fluorescence microscopy, including MPM, involve capturing images of sub-resolution beads, followed by quantifying the resulting ellipsoidal 3D spot. In this work, we revisit this approach, coping with its inherent limitations in terms of accuracy and practicality. We estimate the PSF from the observation of relatively large beads (approximately 1 μm in diameter). This goes through the formulation and resolution of an original non-convex minimization problem, for which we propose a proximal alternating method along with convergence guarantees. Following the PSF estimation step, we then introduce an innovative strategy to deal with the high level multiplicative noise degrading the acquisitions. We rely on a heteroscedastic noise model for which we estimate the parameters. We then solve a constrained optimization problem to restore the image, accounting for the estimated PSF and noise, while allowing a minimal hyper-parameter tuning. Theoretical guarantees are given for the restoration algorithm. These algorithmic contributions lead to an end-to-end pipeline for 3D image restoration in MPM, that we share as a publicly available Python software. We demonstrate its effectiveness through several experiments on both simulated and real data.

Convergence and complexity guarantees for a wide class of descent algorithms in nonconvex multi-objective optimization

We address conditions for global convergence and worst-case complexity bounds of descent algorithms in nonconvex multi-objective optimization. Specifically, we define the concept of steepest-descent-related directions. We consider iterative algorithms taking steps along such directions, selecting the stepsize according to a standard Armijo-type rule. We prove that methods fitting this framework automatically enjoy global convergence properties. Moreover, we show that a slightly stricter property, satisfied by most known algorithms, guarantees the same complexity bound of O(ϵ−2) as the steepest descent method.

Fast Newton-like extremum seeking with asymptotic convergence guarantees

Extremum seeking denotes control approaches to steer the input of a dynamical system towards the optimizer of an associated cost function. The main strength of those schemes is to achieve their objective without any prior knowledge of the mathematical expression of the cost, nor the value of its gradient. However, those algorithms typically suffer from a slow convergence speed. In this paper, we propose a novel class of Newton-like extremum seeking systems separating the gradient and Hessian estimation from the update of the cost input towards the optimizer. The stability properties of the proposed schemes, including the ability of some of them to enforce the asymptotic convergence of the cost inputs to the optimizer, are proved. Their performances are also examined via some numerical examples. The latter will demonstrate the ability of the presented schemes to significantly reduce the convergence time compared with existing Newton-like schemes.

Byzantine Fault-Tolerant Federated Learning Based on Trustworthy Data and Historical Information

Federated learning (FL) is a highly promising collaborative machine learning method that preserves privacy by enabling model training on client nodes (e.g., mobile phones, Internet-of-Things devices) without sharing raw data. However, FL is vulnerable to Byzantine nodes, which can disrupt model performance, render training ineffective, or even manipulate the model by transmitting harmful gradients. In this paper, we propose a Byzantine fault-tolerant FL algorithm called federated learning with trustworthy data and historical information (FLTH). It utilizes a small trusted training dataset at the parameter server to filter out gradient updates from suspicious client nodes during model training, which provides both Byzantine resilience and convergence guarantee. It further introduces a historical information-based credibility assessment scheme such that the client nodes performing poorly over the long-term have a lower impact on the aggregation of gradients, thereby enhancing fault tolerance capability. Additionally, FLTH does not compromise the training efficiency of FL because of its low time complexity. Extensive simulation results show that FLTH achieves higher model accuracy compared to state-of-the-art methods under typical kinds of attack.

A unified momentum-based paradigm of decentralized SGD for non-convex models and heterogeneous data

Emerging distributed applications recently boosted the development of decentralized machine learning, especially in IoT and edge computing fields. In real-world scenarios, the common problems of non-convexity and data heterogeneity result in inefficiency, performance degradation, and development stagnation. The bulk of studies concentrate on one of the issues mentioned above without having a more general framework that has been proven optimal. To this end, we propose a unified paradigm called UMP, which comprises two algorithms D-SUM and GT-DSUM based on the momentum technique with decentralized stochastic gradient descent (SGD). The former provides a convergence guarantee for general non-convex objectives, while the latter is extended by introducing gradient tracking, which estimates the global optimization direction to mitigate data heterogeneity (i.e., distribution drift). We can cover most momentum-based variants based on the classical heavy ball or Nesterov's acceleration with different parameters in UMP. In theory, we rigorously provide the convergence analysis of these two approaches for non-convex objectives and conduct extensive experiments, demonstrating a significant improvement in model accuracy up to 57.6% compared to other methods in practice.

Convergence guarantees for forward gradient descent in the linear regression model

Renewed interest in the relationship between artificial and biological neural networks motivates the study of gradient-free methods. Considering the linear regression model with random design, we theoretically analyze in this work the biologically motivated (weight-perturbed) forward gradient scheme that is based on random linear combination of the gradient. If d denotes the number of parameters and k the number of samples, we prove that the mean squared error of this method converges for k≳d2log(d) with rate d2log(d)/k. Compared to the dimension dependence d for stochastic gradient descent, an additional factor dlog(d) occurs.

Robust multiple subspaces transfer for heterogeneous domain adaptation

Heterogeneous domain adaptation (HDA) aims to execute knowledge transfer from a source domain to a heterogeneous target domain. Previous works typically inject knowledge from the source and target domain into a common subspace. However, this may lead to the ineffectiveness of knowledge transfer due to the existence of heterogeneity. To overcome this drawback, in this paper, we propose a robust multiple subspaces transfer method for heterogeneous domain adaptation. Specifically, knowledge of two domains is projected into a union of multiple subspaces via a self-expressive model, in which joint distribution alignment and dynamic Laplacian regularization on self-repressive coefficients are included in the loss for characterizing transferability. Moreover, we provide a comprehensive analysis of stability, complexity, generalization, and convergence guarantee for the proposed method. Experiments on benchmark vision and Language datasets verify effectiveness of the proposed approach for heterogeneous domain adaptation.

Iteration complexity and finite-time efficiency of adaptive sampling trust-region methods for stochastic derivative-free optimization

ASTRO-DF is a prominent trust-region method using adaptive sampling for stochastic derivative-free optimization of nonconvex problems. Its salient feature is an easy-to-understand-and-implement concept of maintaining “just enough” replications when evaluating points throughout the search to guarantee almost-sure convergence to a first-order critical point. To reduce the dependence of ASTRO-DF on the problem dimension and boost its performance in finite time, we present two key refinements, namely: (i) local models with diagonal Hessians constructed on interpolation points based on a coordinate basis; and (ii) direct search using the interpolation points whenever possible. We demonstrate that the refinements in (i) and (ii) retain the convergence guarantees while matching existing results on iteration complexity. Uniquely, our iteration complexity results match the canonical rates without placing assumptions on iterative models’ quality and their independence from function estimates. Numerical experimentation on a testbed of problems and comparison against existing popular algorithms reveals the computational advantage of ASTRO-DF due to the proposed refinements.

Multi-constraint non-negative matrix factorization for community detection: orthogonal regular sparse constraint non-negative matrix factorization

Community detection is an important method to analyze the characteristics and structure of community networks, which can excavate the potential links between nodes and further discover subgroups from complex networks. However, most of the existing methods only unilaterally consider the direct link topology without comprehensively considering the internal and external characteristics of the community as well as the result itself, which fails to maximize the access to the network information, thus affecting the effectiveness of community detection. To compensate for this deficiency, we propose a new community detection method based on multi-constraint non-negative matrix factorization, named orthogonal regular sparse constraint non-negative matrix factorization (ORSNMF). Based on the network topology, the ORSNMF algorithm models the differences of the outside of the community, the similarities of the nodes inside the community, and the sparseness of the community membership matrices at the same time, which together guides the iterative learning process to better reflect the underlying information and inherent attributes of the community structure in order to improve the correct rate of dividing subgroups. An algorithm with convergence guarantee is also proposed to solve the model, and finally a large number of comparative experiments are conducted, and the results show that the algorithm has good results.

Novel reference condition independent method for estimating performance for PV modules based on double-diode model

The traditional methods for performance estimation mainly include two steps: determine the model parameters under reference condition and calculate the model parameters at varying operating conditions through transformation equations. These methods have the disadvantages of being influenced by the selection of reference condition. A novel reference condition independent method is proposed based on double-diode model (DDM) for parameter calculation and performance estimation at varying operating conditions for photovoltaic (PV) modules. In the proposed method, a new set of transformation equations is reconstructed according to the dependence of physical parameters in DDM on irradiance and temperature. All coefficients in the proposed transformation equations are independent of the information under reference condition and applicable to all varying operating conditions, which eliminates the influence of reference condition. The coefficients are optimized by fitting measured I–V data at varying operating conditions using guaranteed convergence particle swarm optimization technology, and used for model parameters calculation at varying operating conditions. Then the output property of PV module under any operating condition can be determined independently, first against the reference condition and then against the model parameters under reference condition. The accuracy and applicability of proposed method is validated by massive experimental data for six different types of PV modules under wide range of operating conditions. Moreover, three traditional methods are selected to compare with the proposed method. The proposed method shows better accuracy in I–V, P–V curve and maximum power point estimation under different irradiance and temperature conditions, which can be further used to predict the output power of PV system.

Neural Q-learning for discrete-time nonlinear zero-sum games with adjustable convergence rate

In this paper, an adjustable Q-learning scheme is developed to solve the discrete-time nonlinear zero-sum game problem, which can accelerate the convergence rate of the iterative Q-function sequence. First, the monotonicity and convergence of the iterative Q-function sequence are analyzed under some conditions. Moreover, by employing neural networks, the model-free tracking control problem can be overcome for zero-sum games. Second, two practical algorithms are designed to guarantee the convergence with accelerated learning. In one algorithm, an adjustable acceleration phase is added to the iteration process of Q-learning, which can be adaptively terminated with convergence guarantee. In another algorithm, a novel acceleration function is developed, which can adjust the relaxation factor to ensure the convergence. Finally, through a simulation example with the practical physical background, the fantastic performance of the developed algorithm is demonstrated with neural networks.

A matching Schur complement preconditioning technique for inverse source problems

Numerical discretization of a regularized inverse source problem leads to a non-symmetric saddle point linear system. Interestingly, the Schur complement of the non-symmetric saddle point system is Hermitian positive definite (HPD). Then, we propose a preconditioner matching the Schur complement (MSC). Theoretically, we show that the preconditioned conjugate gradient (PCG) method for a linear system with the preconditioned Schur complement as coefficient has a linear convergence rate independent of the matrix size and value of the regularization parameter involved in the inverse problem. Fast implementations are proposed for the matrix-vector multiplication of the preconditioned Schur complement so that the PCG solver requires only quasi-linear operations. To the best of our knowledge, this is the first solver with guarantee of linear convergence for the inversion of Schur complement arising from the discrete inverse problem. Combining the PCG solver for inversion of the Schur complement and the fast solvers for the forward problem in the literature, the discrete inverse problem (the saddle point system) is solved within a quasi-linear complexity. Numerical results are reported to show the performance of the proposed matching Schur complement (MSC) preconditioning technique.

Fast Rates for the Regret of Offline Reinforcement Learning

We study the regret of offline reinforcement learning in an infinite-horizon discounted Markov decision process (MDP). While existing analyses of common approaches, such as fitted Q-iteration (FQI), suggest root-n convergence for regret, empirical behavior exhibits much faster convergence. In this paper, we present a finer regret analysis that exactly characterizes this phenomenon by providing fast rates for the regret convergence. First, we show that given any estimate for the optimal quality function, the regret of the policy it defines converges at a rate given by the exponentiation of the estimate’s pointwise convergence rate, thus speeding up the rate. The level of exponentiation depends on the level of noise in the decision-making problem, rather than the estimation problem. We establish such noise levels for linear and tabular MDPs as examples. Second, we provide new analyses of FQI and Bellman residual minimization to establish the correct pointwise convergence guarantees. As specific cases, our results imply one-over-n rates in linear cases and exponential-in-n rates in tabular cases. We extend our findings to general function approximation by extending our results to regret guarantees based on Lp-convergence rates for estimating the optimal quality function rather than pointwise rates, where L2 guarantees for nonparametric estimation can be ensured under mild conditions. Funding: This work was supported by the Division of Information and Intelligent Systems, National Science Foundation [Grant 1846210].

Stability of Multi-Agent Learning in Competitive Networks: Delaying the Onset of Chaos

The behaviour of multi agent learning in competitive network games is often studied within the context of zero sum games, in which convergence guarantees may be obtained. However, outside of this class the behaviour of learning is known to display complex behaviours and convergence cannot be always guaranteed. Nonetheless, in order to develop a complete picture of the behaviour of multi agent learning in competitive settings, the zero sum assumption must be lifted. Motivated by this we study the Q Learning dynamics, a popular model of exploration and exploitation in multi agent learning, in competitive network games. We determine how the degree of competition, exploration rate and network connectivity impact the convergence of Q Learning. To study generic competitive games, we parameterise network games in terms of correlations between agent payoffs and study the average behaviour of the Q Learning dynamics across all games drawn from a choice of this parameter. This statistical approach establishes choices of parameters for which Q Learning dynamics converge to a stable fixed point. Differently to previous works, we find that the stability of Q Learning is explicitly dependent only on the network connectivity rather than the total number of agents. Our experiments validate these findings and show that, under certain network structures, the total number of agents can be increased without increasing the likelihood of unstable or chaotic behaviours.

FedLF: Layer-Wise Fair Federated Learning

Fairness has become an important concern in Federated Learning (FL). An unfair model that performs well for some clients while performing poorly for others can reduce the willingness of clients to participate. In this work, we identify a direct cause of unfairness in FL - the use of an unfair direction to update the global model, which favors some clients while conflicting with other clients’ gradients at the model and layer levels. To address these issues, we propose a layer-wise fair Federated Learning algorithm (FedLF). Firstly, we formulate a multi-objective optimization problem with an effective fair-driven objective for FL. A layer-wise fair direction is then calculated to mitigate the model and layer-level gradient conflicts and reduce the improvement bias. We further provide the theoretical analysis on how FedLF can improve fairness and guarantee convergence. Extensive experiments on different learning tasks and models demonstrate that FedLF outperforms the SOTA FL algorithms in terms of accuracy and fairness. The source code is available at https://github.com/zibinpan/FedLF.

Learning Discrete-Time Major-Minor Mean Field Games

Recent techniques based on Mean Field Games (MFGs) allow the scalable analysis of multi-player games with many similar, rational agents. However, standard MFGs remain limited to homogeneous players that weakly influence each other, and cannot model major players that strongly influence other players, severely limiting the class of problems that can be handled. We propose a novel discrete time version of major-minor MFGs (M3FGs), along with a learning algorithm based on fictitious play and partitioning the probability simplex. Importantly, M3FGs generalize MFGs with common noise and can handle not only random exogeneous environment states but also major players. A key challenge is that the mean field is stochastic and not deterministic as in standard MFGs. Our theoretical investigation verifies both the M3FG model and its algorithmic solution, showing firstly the well-posedness of the M3FG model starting from a finite game of interest, and secondly convergence and approximation guarantees of the fictitious play algorithm. Then, we empirically verify the obtained theoretical results, ablating some of the theoretical assumptions made, and show successful equilibrium learning in three example problems. Overall, we establish a learning framework for a novel and broad class of tractable games.

Statistical Spatially Inhomogeneous Diffusion Inference

Inferring a diffusion equation from discretely observed measurements is a statistical challenge of significant importance in a variety of fields, from single-molecule tracking in biophysical systems to modeling financial instruments. Assuming that the underlying dynamical process obeys a d-dimensional stochastic differential equation of the form dx_t = b(x_t)dt + \Sigma(x_t)dw_t, we propose neural network-based estimators of both the drift b and the spatially-inhomogeneous diffusion tensor D = \Sigma\Sigma^T/2 and provide statistical convergence guarantees when b and D are s-Hölder continuous. Notably, our bound aligns with the minimax optimal rate N^{-\frac{2s}{2s+d}} for nonparametric function estimation even in the presence of correlation within observational data, which necessitates careful handling when establishing fast-rate generalization bounds. Our theoretical results are bolstered by numerical experiments demonstrating accurate inference of spatially-inhomogeneous diffusion tensors.

A Generalized Neural Diffusion Framework on Graphs

Recent studies reveal the connection between GNNs and the diffusion process, which motivates many diffusion based GNNs to be proposed. However, since these two mechanisms are closely related, one fundamental question naturally arises: Is there a general diffusion framework that can formally unify these GNNs? The answer to this question can not only deepen our understanding of the learning process of GNNs, but also may open a new door to design a broad new class of GNNs. In this paper, we propose a general diffusion equation framework with the fidelity term, which formally establishes the relationship between the diffusion process with more GNNs. Meanwhile, with this framework, we identify one characteristic of graph diffusion networks, i.e., the current neural diffusion process only corresponds to the first-order diffusion equation. However, by an experimental investigation, we show that the labels of high-order neighbors actually appear monophily property, which induces the similarity based on labels among high-order neighbors without requiring the similarity among first-order neighbors. This discovery motives to design a new high-order neighbor-aware diffusion equation, and derive a new type of graph diffusion network (HiD-Net) based on the framework. With the high-order diffusion equation, HiD-Net is more robust against attacks and works on both homophily and heterophily graphs. We not only theoretically analyze the relation between HiD-Net with high-order random walk, but also provide a theoretical convergence guarantee. Extensive experimental results well demonstrate the effectiveness of HiD-Net over state-of-the-art graph diffusion networks.

Efficient Learning in Polyhedral Games via Best-Response Oracles

We study online learning and equilibrium computation in games with polyhedral decision sets, a property shared by normal-form games (NFGs) and extensive-form games (EFGs), when the learning agent is restricted to utilizing a best-response oracle. We show how to achieve constant regret in zero-sum games and O(T^0.25) regret in general-sum games while using only O(log t) best-response queries at a given iteration t, thus improving over the best prior result, which required O(T) queries per iteration. Moreover, our framework yields the first last-iterate convergence guarantees for self-play with best-response oracles in zero-sum games. This convergence occurs at a linear rate, though with a condition-number dependence. We go on to show a O(T^(-0.5)) best-iterate convergence rate without such a dependence. Our results build on linear-rate convergence results for variants of the Frank-Wolfe (FW) algorithm for strongly convex and smooth minimization problems over polyhedral domains. These FW results depend on a condition number of the polytope, known as facial distance. In order to enable application to settings such as EFGs, we show two broad new results: 1) the facial distance for polytopes in standard form is at least γ/k where γ is the minimum value of a nonzero coordinate of a vertex of the polytope and k≤n is the number of tight inequality constraints in the optimal face, and 2) the facial distance for polytopes of the form Ax=b, Cx≤d, x≥0 where x∈R^n, C≥0 is a nonzero integral matrix, and d≥0, is at least 1/(c√n), where c is the infinity norm of C. This yields the first such results for several problems such as sequence-form polytopes, flow polytopes, and matching polytopes.