Forward-backward Algorithm Research Articles

Convergence of a class of algorithms with the level-set subdifferential error bound

In this paper, we study the convergence of algorithms for a class of nonconvex and nonsmooth optimization problems via an level-set subdifferential error bound. Many algorithms (cf. forward-backward splitting algorithm and alternating proximal minimization algorithm) satisfy sufficient descent and relative error conditions. We also find that these algorithms satisfy a quadratic error condition. Under these conditions and the level-set subdifferential error bound, we study the global convergence and the convergence rates of these algorithms without the Kurdyka-Łojasiewicz property which is stronger than the level-set subdifferential error bound. Two examples are given to illustrate the broader applicability of our findings across these algorithms.

Increasing Parallelism in Forward-backward Distributed Algorithm for Finding Strongly Connected Components of Directed Graphs

The paper proposes a modification of the existing distributed forward-backward algorithm for finding strongly connected components in directed graphs. The modification aims at improving the parallelism of the algorithm by increasing the branching factor while dividing the workload. Instead of randomly picking the pivot vertex, a heuristic technique is used which allows more sub-tasks to be generated, on average, for the subsequent step of the algorithm. The work describes suitable algorithm modifications and presents empirical results, proving suitability of the approach in question.

Three-step projected forward–backward algorithms for constrained minimization problem

Three-step projected forward–backward algorithms for constrained minimization problem

Screening osteoporosis in elderly using a new two inertial‎ ‎projective forward-backward splitting algorithm

Screening osteoporosis in elderly using a new two inertial‎ ‎projective forward-backward splitting algorithm

Parallel inertial forward–backward splitting methods for solving variational inequality problems with variational inclusion constraints

The inertial forward–backward splitting algorithm can be considered as a modified form of the forward–backward algorithm for variational inequality problems with monotone and Lipschitz continuous cost mappings. By using parallel and inertial techniques and the forward–backward splitting algorithm, in this paper, we propose a new parallel inertial forward–backward splitting algorithm for solving variational inequality problems, where the constraints are the intersection of common solution sets of a finite family of variational inclusion problems. Then, strong convergence of proposed iteration sequences is showed under standard assumptions imposed on cost mappings in a real Hilbert space. Finally, some numerical experiments demonstrate the reliability and benefits of the proposed algorithm.

Fast power flow calculation for distribution networks based on graph models and hierarchical forward-backward sweep parallel algorithm

IntroductionIn response to the issues of complexity and low efficiency in line loss calculations for actual distribution networks, this paper proposes a fast power flow calculation method for distribution networks based on Neo4j graph models and a hierarchical forward-backward sweep parallel algorithm.MethodsFirstly, Neo4j is used to describe the distribution network structure as a simple graph model composed of nodes and edges. Secondly, a hierarchical forward-backward sweep method is adopted to perform power flow calculations on the graph model network. Finally, during the computation of distribution network subgraphs, the method is combined with the Bulk Synchronous Parallel (BSP) computing model to quickly complete the line loss analysis.Results and DiscussionResults from the IEEE 33-node test system demonstrate that the proposed method can calculate network losses quickly and accurately, with a computation time of only 0.175s, which is lower than the MySQL and Neo4j graph methods that do not consider hierarchical parallel computing.

Novel algorithms based on forward-backward splitting technique: effective methods for regression and classification

In this paper, we introduce two novel forward-backward splitting algorithms (FBSAs) for nonsmooth convex minimization. We provide a thorough convergence analysis, emphasizing the new algorithms and contrasting them with existing ones. Our findings are validated through a numerical example. The practical utility of these algorithms in real-world applications, including machine learning for tasks such as classification, regression, and image deblurring reveal that these algorithms consistently approach optimal solutions with fewer iterations, highlighting their efficiency in real-world scenarios.

A fixed-time stable forward–backward dynamical system for solving generalized monotone inclusions

AbstractWe propose a forward–backward splitting dynamical system for solving inclusion problems of the form $$0\in A(x)+B(x)$$ 0 ∈ A ( x ) + B ( x ) in Hilbert spaces, where A is a maximal operator and B is a single-valued operator. Involved operators are assumed to satisfy a generalized monotonicity condition, which is weaker than the standard monotone assumptions. Under mild conditions on parameters, we establish the fixed-time stability of the proposed dynamical system. In addition, we consider an explicit forward Euler discretization of the dynamical system leading to a new forward backward algorithm for which we present the convergence analysis. Applications to other optimization problems such as constrained optimization problems, mixed variational inequalities, and variational inequalities are presented and some numerical examples are given to illustrate the theoretical results.

Fast and Accurate Estimation of Selection Coefficients and Allele Histories from Ancient and Modern DNA.

We here present CLUES2, a full-likelihood method to infer natural selection from sequence data that is an extension of the method CLUES. We make several substantial improvements to the CLUES method that greatly increases both its applicability and its speed. We add the ability to use ancestral recombination graphs on ancient data as emissions to the underlying hidden Markov model, which enables CLUES2 to use both temporal and linkage information to make estimates of selection coefficients. We also fully implement the ability to estimate distinct selection coefficients in different epochs, which allows for the analysis of changes in selective pressures through time, as well as selection with dominance. In addition, we greatly increase the computational efficiency of CLUES2 over CLUES using several approximations to the forward-backward algorithms and develop a new way to reconstruct historic allele frequencies by integrating over the uncertainty in the estimation of the selection coefficients. We illustrate the accuracy of CLUES2 through extensive simulations and validate the importance sampling framework for integrating over the uncertainty in the inference of gene trees. We also show that CLUES2 is well-calibrated by showing that under the null hypothesis, the distribution of log-likelihood ratios follows a χ2 distribution with the appropriate degrees of freedom. We run CLUES2 on a set of recently published ancient human data from Western Eurasia and test for evidence of changing selection coefficients through time. We find significant evidence of changing selective pressures in several genes correlated with the introduction of agriculture to Europe and the ensuing dietary and demographic shifts of that time. In particular, our analysis supports previous hypotheses of strong selection on lactase persistence during periods of ancient famines and attenuated selection in more modern periods.

Low-carbon oriented planning of shared photovoltaics and energy storage systems in distribution networks via carbon emission flow tracing

With the targets of carbon peaking and carbon neutrality, carbon emission reduction measures have become significant issues in new-type power systems. Distribution networks (DNs), which multiple power sources and flexible loads access to, play an essential role in exploiting the potential of reducing carbon emissions on the demand side. To achieve a global carbon emission reduction considering the carbon quota of each customer, shared photovoltaics (PVs) and energy storage systems (ESSs) are allocated with a centralized calculation and optimization conducted by DNs. To solve two key points in demand-side planning of shared PVs and ESSs in distribution networks, i.e., the accuracy of carbon emission flow (CEF) calculation and carbon quota-oriented optimization planning, this paper proposes a low-carbon oriented planning method for shared PVs and ESSs via CEF tracing. Firstly, a novel CEF calculation method is proposed based on the forward current sweep and backward voltage sweep algorithm, which considers the tracing of reactive power and power losses. Next, carbon emission index and economic index of allocating shared PVs and ESSs are defined. Subsequently, a bi-level optimization model is presented, considering the excess carbon emissions of each load based on carbon accounting and carbon quotas. Finally, the superiority of the proposed method is verified through a modified IEEE 33-node distribution network, and the effects of various investment constraints and carbon quotas are discussed.

Modeling Multimodal Curbside Usage in Dynamic Networks

The proliferation of emerging mobility technology has led to a significant increase in demand for ride-hailing services, on-demand deliveries, and micromobility services, transforming curb spaces into valuable public infrastructure for which multimodal transportation competes. However, the increasing utilization of curbs by different traffic modes has substantial societal impacts, further altering travelers’ choices and polluting the urban environment. Integrating the spatiotemporal characteristics of various behaviors related to curb utilization into general dynamic networks and exploring mobility patterns with multisource data remain a challenge. To address this issue, this study proposes a comprehensive framework of modeling curbside usage by multimodal transportation in a general dynamic network. The framework encapsulates route choices, curb space competition, and interactive effects among different curb users, and it embeds the dynamics of curb usage into a mesoscopic dynamic network model. Furthermore, a curb-aware dynamic origin-destination demand estimation framework is proposed to reveal the network-level spatiotemporal mobility patterns associated with curb usage through a physics-informed data-driven approach. The framework integrates emerging real-world curb use data in conjunction with other mobility data represented on computational graphs, which can be solved efficiently using the forward-backward algorithm on large-scale networks. The framework is examined on a small network as well as a large-scale real-world network. The estimation results on both networks are satisfactory and compelling, demonstrating the capability of the framework to estimate the spatiotemporal curb usage by multimodal transportation. History: This paper has been accepted for the Transportation Science Special Issue on ISTTT25. Funding: This material is based upon work supported by the Office of Energy Efficiency and Renewable Energy, U.S. Department of Energy [Award DE-EE0009659]. Supplemental Material: The online appendix is available at https://doi.org/10.1287/trsc.2024.0522 .

A Bayesian non-stationary heteroskedastic time series model for multivariate critical care data.

We propose a multivariate GARCH model for non-stationary health time series by modifying the observation-level variance of the standard state space model. The proposed model provides an intuitive and novel way of dealing with heteroskedastic data using the conditional nature of state-space models. We follow the Bayesian paradigm to perform the inference procedure. In particular, we use Markov chain Monte Carlo methods to obtain samples from the resultant posterior distribution. We use the forward filtering backward sampling algorithm to efficiently obtain samples from the posterior distribution of the latent state. The proposed model also handles missing data in a fully Bayesian fashion. We validate our model on synthetic data and analyze a data set obtained from an intensive care unit in a Montreal hospital and the MIMIC dataset. We further show that our proposed models offer better performance, in terms of WAIC than standard state space models. The proposed model provides a new way to model multivariate heteroskedastic non-stationary time series data. Model comparison can then be easily performed using the WAIC.

Scaled forward-backward algorithm and the modified superiorized version for solving the split monotone variational inclusion problem

In this paper, we propose an inexact scaled forward-backward algorithm to solve the split monotone variational inclusion problem in a real Hilbert space and prove the strong convergence of it under appropriate conditions. Based on this, we discuss the bounded perturbation resilience of the exact algorithm for introducing the corresponding superiorized version and the superiorization algorithm with restarted perturbations. The numerical experiments illustrate that the proposed algorithms perform well and the superiorization version with restarted perturbations has advantage in decreasing the number of the iterations.

Third Order Dynamical Systems for the Sum of Two Generalized Monotone Operators

In this paper, we propose and analyze a third-order dynamical system for finding zeros of the sum of two generalized operators in a Hilbert space H\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\mathcal {H}$$\\end{document}. We establish the existence and uniqueness of the trajectories generated by the system under appropriate continuity conditions, and prove exponential convergence to the unique zero when the sum of the operators is strongly monotone. Additionally, we derive an explicit discretization of the dynamical system, which results in a forward–backward algorithm with double inertial effects and larger range of stepsize. We establish the linear convergence of the iterates to the unique solution using this algorithm. Furthermore, we provide convergence analysis for the class of strongly pseudo-monotone variational inequalities. We illustrate the effectiveness of our approach by applying it to structured optimization and pseudo-convex optimization problems.

Forward-Backward Algorithm for Functions with Locally Lipschitz Gradient: Applications to Mean Field Games

Forward-Backward Algorithm for Functions with Locally Lipschitz Gradient: Applications to Mean Field Games

RUNGE–KUTTA LIKE METHOD FOR THE SOLUTION OF OPTIMAL CONTROL MODEL OF REAL INVESTMENT AND FISH MANAGEMENT

This study develops the Runge-Kutta Like Method (RKLM), which uses Pontryagin's principle to solve optimal control problems numerically using forward-backward sweep methods. It is based on the Patade and Bhalekar methodology. The RKLM's stability properties and its convergence are examined. The Forward-backward sweep algorithm and the RKLM algorithm are implemented using MATLAB code. Physical optimum control problems are solved with the RKLM. The first problem's conclusion demonstrates that, when investment declines, the capital first grow to boost production before it depreciates. The outcome of the second problem demonstrates that a larger weight parameter causes the harvesting rate to reach zero more quickly and the total fish mass to reach its maximum level more quickly. The findings obtained demonstrate the effectiveness of using RKLM in conjunction with forward-backward sweep methods to solve optimal control problems.

Relaxed Inertial Projective Forward-backward Splitting Algorithms for Regularized Least Square Problems

This study presents flexible conditions of the inertial extrapolation parameter for easy implementation that is added to the algorithm in faster convergence. Modified inertial forward-backward splitting algorithms for solving variational inclusion problems are introduced to apply to solve the LASSO problem for image restoration and signal recovery, and the elastic net model for classification problem. Projection methods are used in the final step for narrowing down the search field, resulting in a better solution.

Convergence of inertial prox-penalization and inertial forward–backward algorithms for solving monotone bilevel equilibrium problems

The main focus of this paper is on bilevel optimization on Hilbert spaces involving two monotone equilibrium bifunctions. We present a new achievement consisting on the introduction of inertial methods for solving these types of problems. Indeed, two several inertial type methods are suggested: a proximal algorithm and a forward–backward one. Under suitable conditions and without any restrictive assumption on the trajectories, the weak and strong convergence of the sequence generated by the both iterative methods are established. Two particular cases illustrating the proposed methods are thereafter discussed with respect to hierarchical minimization problems and equilibrium problems under a saddle point constraint. Furthermore, numerical examples are given to demonstrate the implementability of our algorithms. The algorithms and their convergence results improve and develop previous results in the field.

Kalman filter with impulse noised outliers: a robust sequential algorithm to filter data with a large number of outliers.

Impulse noised outliers are data points that differ significantly from other observations. They are generally removed from the data set through local regression or the Kalman filter algorithm. However, these methods, or their generalizations, are not well suited when the number of outliers is of the same order as the number of low-noise data (often called nominal measurement). In this article, we propose a new model for impulsed noise outliers. It is based on a hierarchical model and a simple linear Gaussian process as with the Kalman Filter. We present a fast forward-backward algorithm to filter and smooth sequential data and which also detects these outliers. We compare the robustness and efficiency of this algorithm with classical methods. Finally, we apply this method on a real data set from a Walk Over Weighing system admitting around 60 % of outliers. For this application, we further develop an (explicit) EM algorithm to calibrate some algorithm parameters.

Splitting algorithms for solving state-dependent maximal monotone inclusion problems

We consider the state-dependent maximal monotone inclusion problem and propose forward-backward splitting algorithms for solving it. Strong convergence of the proposed algorithms is established under suitable conditions. For the special separable case, we present an improved Douglas–Rachford variant that can be easily implemented. Moreover, some accelerated forward-backward splitting algorithms are also presented. Preliminary numerical experiments with comparisons to existing results are presented, illustrating the advantages of our methods.