Non-convex Optimization Research Articles

Resource management for computational offload in MEC networks with energy harvesting and relay assistance

In mobile edge computing (MEC) networks, the integrated application of energy harvesting (EH) and relay cooperation is a promising technology, which ensures the real-time performance of the system and computing power supply. Relay and EH cooperation also expand communication range and extend the battery life of energy-constrained devices. However, in wireless powered and relay-assisted MEC networks, it is challenging to jointly dispatch energy, and computing resources to coordinate heterogeneous performance requirements. To fill this gap, this paper examines the fundamental compromise between utility energy efficiency (UEE) and stable queue length in wireless powered and relay-assisted MEC network systems, and the amount of data transferred for each unit of energy is called UEE. The data queue and energy queue models are introduced and the constraints of stability of two queues in long term are included in the formulated problem. To tackle the fractional and nonconvex optimization problems, the Dinkelsbach method is utilized. Since optimization problems change from time to time and the long-term averaging queue stability is considered, the Lyapunov optimization theory is introduced to convert the non-convex problems into the solvable one. An online resource offloading allocation algorithm is proposed to determine the solution. In addition, the algorithm implements the control parameter V to achieve the compromise between UEE and stable queue length, while the impact of various parameters are also revealed on system performance.

Low-Resolution Optimization for an Unmanned Aerial Vehicle Communication Network under a Passive Reconfigurable Intelligent Surface and Active Reconfigurable Intelligent Surface

This paper investigates the optimization of an unmanned aerial vehicle (UAV) network serving multiple downlink users equipped with single antennas. The network is enhanced by the deployment of either a passive reconfigurable intelligent surface (RIS) or an active RIS. The objective is to jointly design the UAV’s trajectory and the low-bit, quantized, RIS-programmable coefficients to maximize the minimum user rate in a multi-user scenario. To address this optimization challenge, an alternating optimization framework is employed, leveraging the successive convex approximation (SCA) method. Specifically, for the UAV trajectory design, the original non-convex optimization problem is reformulated into an equivalent convex problem through the introduction of slack variables and appropriate approximations. On the other hand, for the RIS-programmable coefficient design, an efficient algorithm is developed using a penalty-based approximation approach. To solve the problems with the proposed optimization, high-performance optimization tools such as CVX are utilized, despite their associated high time complexity. To mitigate this complexity, a low-complexity algorithm is specifically tailored for the optimization of passive RIS-programmable reflecting elements. This algorithm relies solely on closed-form expressions to generate improved feasible points, thereby reducing the computational burden while maintaining reasonable performance. Extensive simulations are created to validate the performance of the proposed algorithms. The results demonstrate that the active RIS-based approach outperforms the passive RIS-based approach. Additionally, for the passive RIS-based algorithms, the low-complexity variant achieves a reduced time complexity with a moderate loss in performance.

Efficient image restoration via non-convex total variation regularization and ADMM optimization

This article presents a novel approach to image restoration utilizing a unique non-convex l1/2-TV regularization model. This model integrates the l1/2-quasi norm as a regularization function, introducing non-convexity to promote sparsity and unevenly penalize elements, thereby enhancing restoration outcomes. To tackle this model, an efficient algorithm named the Alternating Direction Method of Multipliers, based on the Lagrangian multiplier, is introduced. This effectively prevents the penalty parameter from reaching infinity and ensures excellent convergence behavior. The proposed algorithm decomposes the optimization problem into subproblems, for which closed-form solutions are derived, particularly addressing the challenging l1/2 regularization problem. To validate its effectiveness, a comprehensive set of experiments are conducted to compare its performance with existing methods. The experimental results demonstrate that the proposed model performs well in both qualitative and quantitative evaluations. Consequently, the proposed model is not only efficient and stable but also exhibits excellent convergence behavior.

Online Stochastic DCA With Applications to Principal Component Analysis.

Stochastic algorithms are well-known for their performance in the era of big data. In this article, we study nonsmooth stochastic Difference-of-Convex functions (DC) programs-the major class of nonconvex stochastic optimization, which have a variety of applications in divers domains, in particular, machine learning. We propose new online stochastic algorithms based on the state-of-the-art DC Algorithm (DCA)-a powerful approach in nonconvex programming framework, in the online context of streaming data continuously generated by some (unknown) source distribution. The new schemes use the stochastic approximations (SAs) principle: deterministic quantities of the standard DCA are replaced by their noisy estimators constructed using newly arriving samples. The convergence analysis of the proposed algorithms is studied intensively with the help of tools from modern convex analysis and martingale theory. Finally, we study several aspects of the proposed algorithms on an important problem in machine learning: the expected problem in principal component analysis (PCA).

Distributed Event-Triggered Stochastic Gradient-Tracking for Nonconvex Optimization

Distributed Event-Triggered Stochastic Gradient-Tracking for Nonconvex Optimization

Artificial intelligence warm-start approach: optimizing the generalization capability of QAOA in complex energy landscapes

To address the issue of the quantum approximate optimization algorithm frequently encountering local minima and the cost of parameter optimization within complex non-convex optimization energy landscapes, we consider a warm-start method. This approach leverages the characteristics of transition states in the enhanced optimizer, specifically descending along unique negative curvature directions, to find smaller local minima. Our research results indicate that with the assistance of an enhanced pre-training structure of the AlphaZero AI model, the initialization generalization ability of the new optimizer is significantly enhanced across various test sets. We train on 2-SAT training sets with clause densities between α ≈ 2.6 and α ≈ 2.89, and transfer to more complex test sets. Additionally, the average residual energy density in transfer learning consistently remains below 0.01, even achieving a high transfer success probability of 98% in hard instances with α ≈ 3.7. The search efficiency, pre-trained by ensemble learning, was significantly enhanced, while only requiring simple interpolation of a few transition points to transfer on the global optimal solutions at higher sample clause densities.

A Joint Communication and Computation Design for Probabilistic Semantic Communications.

In this paper, the problem of joint transmission and computation resource allocation for a multi-user probabilistic semantic communication (PSC) network is investigated. In the considered model, users employ semantic information extraction techniques to compress their large-sized data before transmitting them to a multi-antenna base station (BS). Our model represents large-sized data through substantial knowledge graphs, utilizing shared probability graphs between the users and the BS for efficient semantic compression. The resource allocation problem is formulated as an optimization problem with the objective of maximizing the sum of the equivalent rate of all users, considering the total power budget and semantic resource limit constraints. The computation load considered in the PSC network is formulated as a non-smooth piecewise function with respect to the semantic compression ratio. To tackle this non-convex non-smooth optimization challenge, a three-stage algorithm is proposed, where the solutions for the received beamforming matrix of the BS, the transmit power of each user, and the semantic compression ratio of each user are obtained stage by stage. The numerical results validate the effectiveness of our proposed scheme.

On regularization schemes for data-driven optimization based on Cressie–Read divergence and CVaR

On regularization schemes for data-driven optimization based on Cressie–Read divergence and CVaR

Sparse linear array synthesis using artificial neural network and nonconvex optimization

AbstractThe concept of sparse array synthesis has attracted considerable attention due to its potential to reduce hardware costs in array design and enhance the efficiency of array aperture utilization. In contrast to conventional artificial neural networks (ANNs), a more efficient encoder–decoding ANN framework is proposed in this letter for designing sparse linear arrays. The decoder is pretrained with a randomly generated training set, enabling its ability to rapidly predictiction of far‐field patterns. This process considers the actual coupling effect among the antenna elements. The output of the encoder is used as the input of the decoder to synthesize the sparse arrays by providing the desired pattern to the encoder. During the training process, a novel ‐norm minimization nonconvex optimization method is integrated into the proposed framework to reduce the number of activated elements. The effectiveness and efficiency of the method are demonstrated through numerical and full‐wave simulations for synthesizing various linear sparse arrays, including large‐scale arrays. In comparison to existing algorithms, our proposed method exhibits superior computational efficiency and predicted accuracy with fewer elements.

Joint Base Station Selection and Power Allocation Design for Reconfigurable Intelligent Surface-Aided Cell-Free Networks

Cell-free (CF) networks can reduce cell boundaries by densely deploying base stations (BSs) with additional hardware costs and power sources. Integrating a reconfigurable intelligent surface (RIS) into CF networks can cost-effectively increase the capacity and coverage of future wireless systems. This paper considers an RIS-aided CF system where each user is supported by a devoted RIS and can establish connections with multiple BSs for coherent transmission. Specifically, each RIS can enhance signal transmission between users and their selected BSs through passive beam-forming, but also randomly scattered signals from other non-selected BSs to users, causing additional signals and interference in the network. To gain insights into the system performance, we first derive the average signal-to-interference-plus-noise ratio (SINR) received by each user in a closed-form expression. Subsequently, we formulate an optimization problem aimed at maximizing the weighted sum-SINR of all users in the RIS-CF network. This optimization considers both BS transmit power allocation and BS selections as variables to be jointly optimized. To tackle the complexity of this nonconvex optimization problem, we develop an innovative two-layer iterative approach that offers both efficiency and efficacy. This algorithm iteratively updates the transmit power allocation and BS selections to converge to a locally optimal solution. Numerical results demonstrate significant performance improvement for the RIS-CF network using our proposed scheme. These results also highlight the effectiveness of jointly optimizing BS transmit power allocation and BS selections in the RIS-CF network.

A Reentry Trajectory Planning Algorithm via Pseudo-Spectral Convexification and Method of Multipliers

The reentry trajectory planning problem of hypersonic vehicles is generally a continuous and nonconvex optimization problem, and it constitutes a critical challenge within the field of aerospace engineering. In this paper, an improved sequential convexification algorithm is proposed to solve it and achieve online trajectory planning. In the proposed algorithm, the Chebyshev pseudo-spectral method with high-accuracy approximation performance is first employed to discretize the continuous dynamic equations. Subsequently, based on the multipliers and linearization methods, the original nonconvex trajectory planning problem is transformed into a series of relaxed convex subproblems in the form of an augmented Lagrange function. Then, the interior point method is utilized to iteratively solve the relaxed convex subproblem until the expected convergence precision is achieved. The convex-optimization-based and multipliers methods guarantee the promotion of fast convergence precision, making it suitable for online trajectory planning applications. Finally, numerical simulations are conducted to verify the performance of the proposed algorithm. The simulation results show that the algorithm possesses better convergence performance, and the solution time can reach the level of seconds, which is more than 97% less than nonlinear programming algorithms, such as the sequential quadratic programming algorithm.

Energy efficiency enhancement in millimetre-wave MIMO-NOMA using three layer user grouping and adaptive power allocation algorithm

Massive multi-input multi-output (MIMO) is realized as the principal technology in the emerging fifth generation communication network system. Hybrid structure uplink communication is considered for the MIMO Non-orthogonal multiple access (MIMO-NOMA) system’s beam forming and power efficiency improvement through the novel three-layer user grouping. In the three-layer user grouping, the K-means algorithm is adopted in the initial layer for grouping users among different clusters and rectifying clustering errors in the third layer. The second layer used the agglomerative nesting (AGNES) algorithm for merging smaller clusters based on the channel correlation and angles of arrival similarity. The beam selection is carried out to minimize the intrusion of defined beam elements and to overcome beam overlapping problems. The non-convex optimization of the power allocating problem is modified as a convex problem by introducing a Quadratic transform (QT) to minimize each user’s data rate requirement. The algorithm of coati optimization is proposed to iteratively optimize the power allocation problem. The simulation results show that our proposed methodology goes beyond the existing schemes in terms of energy efficiency beyond the maximum power and achievable sum rate can be achieved.

Convergence and complexity guarantees for a wide class of descent algorithms in nonconvex multi-objective optimization

We address conditions for global convergence and worst-case complexity bounds of descent algorithms in nonconvex multi-objective optimization. Specifically, we define the concept of steepest-descent-related directions. We consider iterative algorithms taking steps along such directions, selecting the stepsize according to a standard Armijo-type rule. We prove that methods fitting this framework automatically enjoy global convergence properties. Moreover, we show that a slightly stricter property, satisfied by most known algorithms, guarantees the same complexity bound of O(ϵ−2) as the steepest descent method.

Quantization avoids saddle points in distributed optimization

Distributed nonconvex optimization underpins key functionalities of numerous distributed systems, ranging from power systems, smart buildings, cooperative robots, vehicle networks to sensor networks. Recently, it has also merged as a promising solution to handle the enormous growth in data and model sizes in deep learning. A fundamental problem in distributed nonconvex optimization is avoiding convergence to saddle points, which significantly degrade optimization accuracy. We find that the process of quantization, which is necessary for all digital communications, can be exploited to enable saddle-point avoidance. More specifically, we propose a stochastic quantization scheme and prove that it can effectively escape saddle points and ensure convergence to a second-order stationary point in distributed nonconvex optimization. With an easily adjustable quantization granularity, the approach allows a user to control the number of bits sent per iteration and, hence, to aggressively reduce the communication overhead. Numerical experimental results using distributed optimization and learning problems on benchmark datasets confirm the effectiveness of the approach.

Adaptive Dataset Management Scheme for Lightweight Federated Learning in Mobile Edge Computing.

Federated learning (FL) in mobile edge computing has emerged as a promising machine-learning paradigm in the Internet of Things, enabling distributed training without exposing private data. It allows multiple mobile devices (MDs) to collaboratively create a global model. FL not only addresses the issue of private data exposure but also alleviates the burden on a centralized server, which is common in conventional centralized learning. However, a critical issue in FL is the imposed computing for local training on multiple MDs, which often have limited computing capabilities. This limitation poses a challenge for MDs to actively contribute to the training process. To tackle this problem, this paper proposes an adaptive dataset management (ADM) scheme, aiming to reduce the burden of local training on MDs. Through an empirical study on the influence of dataset size on accuracy improvement over communication rounds, we confirm that the amount of dataset has a reduced impact on accuracy gain. Based on this finding, we introduce a discount factor that represents the reduced impact of the size of the dataset on the accuracy gain over communication rounds. To address the ADM problem, which involves determining how much the dataset should be reduced over classes while considering both the proposed discounting factor and Kullback-Leibler divergence (KLD), a theoretical framework is presented. The ADM problem is a non-convex optimization problem. To solve it, we propose a greedy-based heuristic algorithm that determines a suboptimal solution with low complexity. Simulation results demonstrate that our proposed scheme effectively alleviates the training burden on MDs while maintaining acceptable training accuracy.

Reentry trajectory planning for hypersonic vehicles via an improved sequential convex programming method

The reentry trajectory planning problem of hypersonic vehicles is generally a continuous and nonconvex optimization problem. The solution efficiency and convergence properties of the planning algorithm become an essential point when considering the practical application scenarios. In this paper, an augmented-Lagrange-based improved sequential convex programming algorithm is proposed to solve it and achieve superior solutions with higher accuracy and efficiency. Firstly, the mapped Chebyshev pseudo-spectral method is utilized to convert the continuous optimal control problem into a highly approximately discretized one. In this framework, the discretization technique is employed to discretize the states and control variables at the time interval, which leads to the improvement of ill-conditioned differential matrix and better approximation properties of the variables. Moreover, by introducing slack variables, automatically updated penalty parameters and Lagrange multipliers during the convexification processing, the original nonconvex problem is transformed into an augmented-Lagrange-function-formed relaxed convex one. Then, the arrived convex subproblem can be solved iteratively by the matured interior-point method until the predetermined precision is satisfied. Based on this transformation, it can effectively alleviate the numerical difficulty and improve the convergence property of the proposed improved sequential convex programming algorithm. Finally, Rigorous theoretical analysis is conducted to prove that the solution sequence generated by the proposed algorithm could converge to an approximate global solution of the original problem. Numerical simulations are also presented, and it is demonstrated that the solution time of the proposed algorithm is more than 95% less than that of the traditional nonlinear programming methods without losing the merits of high precision.

Joint beamforming and power splitting design for MISO downlink communication with SWIPT: a comparison between cell-free massive MIMO and small-cell deployments

Simultaneous wireless information and power transfer (SWIPT) has been advocated as a highly promising technology for enhancing the capabilities of 5G and 6G devices. However, the challenge of dealing with large propagation path loss poses a significant hurdle. To address this issue, massive multiple-input multiple-output (MIMO) is employed to enhance the efficiency of SWIPT in cellular-based networks with multiple small cells, and especially increase the energy for cell-edge users. In addition, by leveraging a large set of spatially distributed base stations to collaboratively serve SWIPT-enabled user equipment, the cell-free massive MIMO has the potential to provide even better performance than the conventional small-cell systems. In this work, we extend the investigation to include the application of SWIPT technology with alternating current (AC) logic in the cell-free networks and the small-cell networks and propose joint beamforming and power splitting optimization frameworks to maximize the system sum-rate, subject to the constraints on harvested energy, AC logic energy supply, and total transmit power. The optimization problem is shown to be non-convex, posing a significant challenge. To address this challenge, we resort to a two-stage decomposition approach. Specifically, we first introduce quadratic transform-based fractional programming (FP) algorithms to iteratively solve the non-convex optimization problems in the first stage, achieving near-optimal solutions with low time complexities. To further reduce the complexities, we also incorporate conventional schemes such as zero forcing, maximum ratio transmission, and signal-to-leakage-and-noise ratio for the design of beamforming vectors. Second, to determine the optimal power splitting ratio within the framework, we develop a one-dimensional (1-D) search algorithm to tackle the single variable optimization problem reduced in the second stage. These algorithms are then evaluated in the context of cell-free MIMO and small-cell networks with numerical experiments. The results show that the FP-based algorithms can consistently outperform those utilizing the conventional beamforming schemes, and the solutions of this work can achieve up to fivefold improvement in the system sum-rate than the small-cell counterpart while providing different but comparable performance trends in energy harvesting (EH).

Fluorescence molecular tomography based on an online maximum a posteriori estimation algorithm

Fluorescence molecular tomography (FMT) is a non-invasive, radiation-free, and highly sensitive optical molecular imaging technique for early tumor detection. However, inadequate measurement information along with significant scattering of near-infrared light within the tissue leads to high ill-posedness in the inverse problem of FMT. To improve the quality and efficiency of FMT reconstruction, we build a reconstruction model based on log-sum regularization and introduce an online maximum a posteriori estimation (OPE) algorithm to solve the non-convex optimization problem. The OPE algorithm approximates a stationary point by evaluating the gradient of the objective function at each iteration, and its notable strength lies in the remarkable speed of convergence. The results of simulations and experiments demonstrate that the OPE algorithm ensures good reconstruction quality and exhibits outstanding performance in terms of reconstruction efficiency.

A class of accelerated GADMM-based method for multi-block nonconvex optimization problems

A class of accelerated GADMM-based method for multi-block nonconvex optimization problems

Stochastic nested primal-dual method for nonconvex constrained composition optimization

In this paper we study the nonconvex constrained composition optimization, in which the objective contains a composition of two expected-value functions whose accurate information is normally expensive to calculate. We propose a STochastic nEsted Primal-dual (STEP) method for such problems. In each iteration, with an auxiliary variable introduced to track the inner layer function values we compute stochastic gradients of the nested function using a subsampling strategy. To alleviate difficulties caused by possibly nonconvex constraints, we construct a stochastic approximation to the linearized augmented Lagrangian function to update the primal variable, which further motivates to update the dual variable in a weighted-average way. Moreover, to better understand the asymptotic dynamics of the update schemes we consider a deterministic continuous-time system from the perspective of ordinary differential equation (ODE). We analyze the Karush-Kuhn-Tucker measure at the output by the STEP method with constant parameters and establish its iteration and sample complexities to find an ϵ \epsilon -stationary point, ensuring that expected stationarity, feasibility as well as complementary slackness are below accuracy ϵ \epsilon . To leverage the benefit of the (near) initial feasibility in the STEP method, we propose a two-stage framework incorporating a feasibility-seeking phase, aiming to locate a nearly feasible initial point. Moreover, to enhance the adaptivity of the STEP algorithm, we propose an adaptive variant by adaptively adjusting its parameters, along with a complexity analysis. Numerical results on a risk-averse portfolio optimization problem and an orthogonal nonnegative matrix decomposition reveal the effectiveness of the proposed algorithms.