Single-loop Algorithm Research Articles

Structural Estimation of Markov Decision Processes in High-Dimensional State Space with Finite-Time Guarantees

Researchers have introduced a new algorithm to estimate structural models of dynamic decisions by human agents, addressing the challenge of high computational complexity. Traditionally, this task involves a nested structure: an inner problem identifying an optimal policy and an outer problem maximizing a measure of fit. Previous methods have struggled with large discrete state spaces or high-dimensional continuous state spaces, often sacrificing reward estimation accuracy. The new approach combines policy improvement with a stochastic gradient step for likelihood maximization, ensuring accurate reward estimation without compromising computational efficiency. This single-loop algorithm, designed to handle high-dimensional state spaces, converges to a stationary solution with finite-time guarantees. When the reward is linearly parameterized, it approximates the maximum likelihood estimator sublinearly, offering a robust solution for complex decision modeling tasks.

A Single-Loop Algorithm for Weighted Sum Rate Maximization in Multiuser MIMO Systems With Per-Antenna Power Constraints

This paper investigates the linear precoder design for multiuser multiple-input-multiple-output (MIMO) systems, in which the problem of maximizing weighted sum rate (WSR) subject to per-antenna power constraints (PAPC) is considered. This problem is hard to solve due to its nonconvexity and the existence of multiple quadratic constraints. Conventional methods to tackle this problem, such as the iterative weighted minimum mean squared error (WMMSE) algorithm, typically consist of two nested loops and thus suffer from high computational complexity. By leveraging the inherent separability of the PAPC constraint set, we propose a low-complexity single-loop algorithm for solving the WSR maximization problem under PAPC, in which each updating step is done efficiently with closed form. Theoretically, we prove the convergence of the proposed algorithm to stationary solutions. Then we extend the proposed algorithm to the multi-carrier scenario where the power constraints are shared over the subcarriers. Complexity analysis and numerical results show that the proposed single-loop algorithm maintains the same WSR performance as existing methods but dramatically reduces the computational complexity.

Proximal Stochastic Recursive Momentum Methods for Nonconvex Composite Decentralized Optimization

Consider a network of N decentralized computing agents collaboratively solving a nonconvex stochastic composite problem. In this work, we propose a single-loop algorithm, called DEEPSTORM, that achieves optimal sample complexity for this setting. Unlike double-loop algorithms that require a large batch size to compute the (stochastic) gradient once in a while, DEEPSTORM uses a small batch size, creating advantages in occasions such as streaming data and online learning. This is the first method achieving optimal sample complexity for decentralized nonconvex stochastic composite problems, requiring O(1) batch size. We conduct convergence analysis for DEEPSTORM with both constant and diminishing step sizes. Additionally, under proper initialization and a small enough desired solution error, we show that DEEPSTORM with a constant step size achieves a network-independent sample complexity, with an additional linear speed-up with respect to N over centralized methods. All codes are made available at https://github.com/gmancino/DEEPSTORM.

On the maximum penalized full likelihood approach for Cox model with extreme value for heavily censored survival data

In this paper, to deal with heavily censored survival data, a penalized full likelihood (PFL) with extreme value is proposed to estimate the regression coefficients and baseline hazard function in Cox model simultaneously, where multiple penalties are used for variable selection. We present a single-loop algorithm to fit the tail of the baseline distribution beyond a threshold with an extreme value model. The proposed maximum PFL estimators are proved to possess good asymptotic properties, which are validated by simulations and real data analysis.

A new Lagrangian-based first-order method for nonconvex constrained optimization

In this paper, we introduce a new form of Lagrangian function and propose a simple first-order primal-dual algorithm for solving nonconvex optimization with nonlinear equality constraints. We show that the algorithm generates bounded primal-dual iterates, and establish the convergence to KKT points under standard assumptions. The key features of the proposed method are: (i) it does not require boundedness assumptions on dual iterates generated by the algorithm as well as the set of multipliers; (ii) it is a single-loop algorithm that does not involve any penalty subproblems.

A unified single-loop alternating gradient projection algorithm for nonconvex–concave and convex–nonconcave minimax problems

Much recent research effort has been directed to the development of efficient algorithms for solving minimax problems with theoretical convergence guarantees due to the relevance of these problems to a few emergent applications. In this paper, we propose a unified single-loop alternating gradient projection (AGP) algorithm for solving smooth nonconvex-(strongly) concave and (strongly) convex–nonconcave minimax problems. AGP employs simple gradient projection steps for updating the primal and dual variables alternatively at each iteration. We show that it can find an \(\varepsilon \)-stationary point of the objective function in \({\mathcal {O}}\left( \varepsilon ^{-2} \right) \) (resp. \({\mathcal {O}}\left( \varepsilon ^{-4} \right) \)) iterations under nonconvex-strongly concave (resp. nonconvex–concave) setting. Moreover, its gradient complexity to obtain an \(\varepsilon \)-stationary point of the objective function is bounded by \({\mathcal {O}}\left( \varepsilon ^{-2} \right) \) (resp., \({\mathcal {O}}\left( \varepsilon ^{-4} \right) \)) under the strongly convex–nonconcave (resp., convex–nonconcave) setting. To the best of our knowledge, this is the first time that a simple and unified single-loop algorithm is developed for solving both nonconvex-(strongly) concave and (strongly) convex–nonconcave minimax problems. Moreover, the complexity results for solving the latter (strongly) convex–nonconcave minimax problems have never been obtained before in the literature. Numerical results show the efficiency of the proposed AGP algorithm. Furthermore, we extend the AGP algorithm by presenting a block alternating proximal gradient (BAPG) algorithm for solving more general multi-block nonsmooth nonconvex-(strongly) concave and (strongly) convex–nonconcave minimax problems. We can similarly establish the gradient complexity of the proposed algorithm under these four different settings.

Stochastic first-order methods for convex and nonconvex functional constrained optimization

Functional constrained optimization is becoming more and more important in machine learning and operations research. Such problems have potential applications in risk-averse machine learning, semisupervised learning and robust optimization among others. In this paper, we first present a novel Constraint Extrapolation (ConEx) method for solving convex functional constrained problems, which utilizes linear approximations of the constraint functions to define the extrapolation (or acceleration) step. We show that this method is a unified algorithm that achieves the best-known rate of convergence for solving different functional constrained convex composite problems, including convex or strongly convex, and smooth or nonsmooth problems with stochastic objective and/or stochastic constraints. Many of these rates of convergence were in fact obtained for the first time in the literature. In addition, ConEx is a single-loop algorithm that does not involve any penalty subproblems. Contrary to existing primal-dual methods, it does not require the projection of Lagrangian multipliers into a (possibly unknown) bounded set. Second, for nonconvex functional constrained problems, we introduce a new proximal point method which transforms the initial nonconvex problem into a sequence of convex problems by adding quadratic terms to both the objective and constraints. Under certain MFCQ-type assumption, we establish the convergence and rate of convergence of this method to KKT points when the convex subproblems are solved exactly or inexactly. For large-scale and stochastic problems, we present a more practical proximal point method in which the approximate solutions of the subproblems are computed by the aforementioned ConEx method. Under a strong feasibility assumption, we establish the total iteration complexity of ConEx required by this inexact proximal point method for a variety of problem settings, including nonconvex smooth or nonsmooth problems with stochastic objective and/or stochastic constraints. To the best of our knowledge, most of these convergence and complexity results of the proximal point method for nonconvex problems also seem to be new in the literature.

Nonconvex Regularized Gradient Projection Sparse Reconstruction for Massive MIMO Channel Estimation

Novel sparse reconstruction algorithms are proposed for beamspace channel estimation in massive multiple-input multiple-output systems. The proposed algorithms minimize a least-squares objective having a nonconvex regularizer. This regularizer removes the penalties on a few large-magnitude elements from the conventional <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\ell _{1}$ </tex-math></inline-formula> -norm regularizer, and thus it only forces penalties on the remaining elements that are expected to be zeros. Accurate and fast reconstructions can be achieved by performing gradient projection updates within the framework of difference of convex functions (DC) programming. A double-loop algorithm and a single-loop algorithm are proposed via different DC decompositions, and these two algorithms have distinct computational complexities and convergence rates. An extension algorithm is further proposed by designing new step sizes for the single-loop algorithm. The extension algorithm has a faster convergence rate and can achieve approximately the same level of accuracy as the proposed double-loop algorithm. Numerical results show significant advantages of the proposed algorithms over existing reconstruction algorithms in terms of reconstruction accuracies and runtimes. Compared with the benchmark channel estimation approaches, the proposed algorithms can achieve smaller channel reconstruction error and higher achievable spectral efficiency.

An improved maximum entropy method via fractional moments with Laplace transform for reliability analysis

The fractional moment-based maximum entropy method (FM-MEM) attracts more and more attention in reliability analysis recently, comparing with the common integer moment-based maximum entropy method. In the present paper, a new approach for reliability analysis is proposed from the improvement of the fractional moment-based maximum entropy method via the Laplace transformation and dimension reduction method (DRM). Different with the traditional FM-MEM with a double-loop multivariate optimization formulation, the proposed method introduces a single-loop univariate optimization algorithm through solving a set of linear equations. Firstly, the single-loop algorithm is proposed for bounded positive random variable and then is extended to arbitrary positive random variable with the help of Laplace transform. Then, a univariate strategy is given to simplify the calculation for the unknown parameters in the single-loop algorithm. Through the improvement, not only the proposed method can predict failure probability accurately but also the computational cost is considerably reduced. Due to the univariate optimization strategy, the proposed method is more robust than common reliability analysis methods, which are sensitive to the initial points, such as the traditional FM-MEM and the first-order reliability method. Several numerical examples are studied to illustrate the efficiency, robustness, and accuracy of the proposed method for the prediction of the failure probability in comparison with other methods.

A Parallel Algorithm for Large-Scale Nonconvex Penalized Quantile Regression

ABSTRACTPenalized quantile regression (PQR) provides a useful tool for analyzing high-dimensional data with heterogeneity. However, its computation is challenging due to the nonsmoothness and (sometimes) the nonconvexity of the objective function. An iterative coordinate descent algorithm (QICD) was recently proposed to solve PQR with nonconvex penalty. The QICD significantly improves the computational speed but requires a double-loop. In this article, we propose an alternative algorithm based on the alternating direction method of multiplier (ADMM). By writing the PQR into a special ADMM form, we can solve the iterations exactly without using coordinate descent. This results in a new single-loop algorithm, which we refer to as the QPADM algorithm. The QPADM demonstrates favorable performance in both computational speed and statistical accuracy, particularly when the sample size n and/or the number of features p are large. Supplementary material for this article is available online.

Reliability-Based Robust Design Optimization of Structures Considering Uncertainty in Design Variables

This paper investigates the structural design optimization to cover both the reliability and robustness under uncertainty in design variables. The main objective is to improve the efficiency of the optimization process. To address this problem, a hybrid reliability-based robust design optimization (RRDO) method is proposed. Prior to the design optimization, the Sobol sensitivity analysis is used for selecting key design variables and providing response variance as well, resulting in significantly reduced computational complexity. The single-loop algorithm is employed to guarantee the structural reliability, allowing fast optimization process. In the case of robust design, the weighting factor balances the response performance and variance with respect to the uncertainty in design variables. The main contribution of this paper is that the proposed method applies the RRDO strategy with the usage of global approximation and the Sobol sensitivity analysis, leading to the reduced computational cost. A structural example is given to illustrate the performance of the proposed method.

Single-loop system reliability-based topology optimization considering statistical dependence between limit-states

This paper presents a single-loop algorithm for system reliability-based topology optimization (SRBTO) that can account for statistical dependence between multiple limit-states, and its applications to computationally demanding topology optimization (TO) problems. A single-loop reliability-based design optimization (RBDO) algorithm replaces the inner-loop iterations to evaluate probabilistic constraints by a non-iterative approximation. The proposed single-loop SRBTO algorithm accounts for the statistical dependence between the limit-states by using the matrix-based system reliability (MSR) method to compute the system failure probability and its parameter sensitivities. The SRBTO/MSR approach is applicable to general system events including series, parallel, cut-set and link-set systems and provides the gradients of the system failure probability to facilitate gradient-based optimization. In most RBTO applications, probabilistic constraints are evaluated by use of the first-order reliability method for efficiency. In order to improve the accuracy of the reliability calculations for RBDO or RBTO problems with high nonlinearity, we introduce a new single-loop RBDO scheme utilizing the second-order reliability method and implement it to the proposed SRBTO algorithm. Moreover, in order to overcome challenges in applying the proposed algorithm to computationally demanding topology optimization problems, we utilize the multiresolution topology optimization (MTOP) method, which achieves computational efficiency in topology optimization by assigning different levels of resolutions to three meshes representing finite element analysis, design variables and material density distribution respectively. The paper provides numerical examples of two- and three-dimensional topology optimization problems to demonstrate the proposed SRBTO algorithm and its applications. The optimal topologies from deterministic, component and system RBTOs are compared with one another to investigate the impact of optimization schemes on final topologies. Monte Carlo simulations are also performed to verify the accuracy of the failure probabilities computed by the proposed approach.

2106 不確定性を考慮した熱問題に対するレベルセット法に基づくトポロジー最適化(OS21.計算力学と最適化(2),オーガナイズドセッション)

This research applies the reliability-based topology optimization (RBTO) method to a heat transfer problem under variation on coefficient of heat transfer. The design problem is formulated to minimize the structural volume subject to the reliability constraint concerning thermal problem under variation on coefficient of heat transfer. The topology optimization uses level set boundary expressions based on the concept of the phase field method for obtaining clear boundary shapes. The topology optimization is combined with the reliability-based optimization method that is formulated as a single loop algorithm by using Single Loop Single Variable (SLSV) method to reduce computational time. Through numerical example, difference between the optimum configurations on deterministic method and reliability-based method is demonstrated and the effectiveness of this method is discussed.