Alternating Minimization Algorithm Research Articles

Image restoration for ring-array photoacoustic tomography system based on blind spatially rotational deconvolution

Ring-array photoacoustic tomography (PAT) system has been widely used in noninvasive biomedical imaging. However, the reconstructed image usually suffers from spatially rotational blur and streak artifacts due to the non-ideal imaging conditions. To improve the reconstructed image towards higher quality, we propose a concept of spatially rotational convolution to formulate the image blur process, then we build a regularized restoration problem model accordingly and design an alternating minimization algorithm which is called blind spatially rotational deconvolution to achieve the restored image. Besides, we also present an image preprocessing method based on the proposed algorithm to remove the streak artifacts. We take experiments on phantoms and in vivo biological tissues for evaluation, the results show that our approach can significantly enhance the resolution of the image obtained from ring-array PAT system and remove the streak artifacts effectively.

Structural knowledge-driven meta-learning for task offloading in vehicular networks with integrated communications, sensing and computing

Task offloading is a potential solution to satisfy the strict requirements of computation-intensive and latency-sensitive vehicular applications due to the limited onboard computing resources. However, the overwhelming upload traffic may lead to unacceptable uploading time. To tackle this issue, for tasks taking environmental data as input, the data perceived by roadside units (RSU) equipped with several sensors can be directly exploited for computation, resulting in a novel task offloading paradigm with integrated communications, sensing and computing (I-CSC). With this paradigm, vehicles can select to upload their sensed data to RSUs or transmit computing instructions to RSUs during the offloading. By optimizing the computation mode and network resources, in this paper, we investigate an I-CSC-based task offloading problem to reduce the cost caused by resource consumption while guaranteeing the latency of each task. Although this non-convex problem can be handled by the alternating minimization (AM) algorithm that alternatively minimizes the divided four sub-problems, it leads to high computational complexity and local optimal solution. To tackle this challenge, we propose a creative structural knowledge-driven meta-learning (SKDML) method, involving both the model-based AM algorithm and neural networks. Specifically, borrowing the iterative structure of the AM algorithm, also referred to as structural knowledge, the proposed SKDML adopts long short-term memory (LSTM) network-based meta-learning to learn an adaptive optimizer for updating variables in each sub-problem, instead of the handcrafted counterpart in the AM algorithm. Furthermore, to pull out the solution from the local optimum, our proposed SKDML updates parameters in LSTM with the global loss function. Simulation results demonstrate that our method outperforms both the AM algorithm and the meta-learning without structural knowledge in terms of both the online processing time and the network performance.

Efficient and Fast Joint Sparse Constrained Canonical Correlation Analysis for Fault Detection.

The canonical correlation analysis (CCA) has attracted wide attention in fault detection (FD). To improve the detection performance, we propose a new joint sparse constrained CCA (JSCCCA) model that integrates the l2,0 -norm joint sparse constraints into classical CCA. The key idea is that JSCCCA can fully exploit the joint sparse structure to determine the number of extracted variables. We then develop an efficient alternating minimization algorithm using the improved iterative hard thresholding and manifold constrained gradient descent method. More importantly, we establish the convergence guarantee with detailed analysis. Finally, we provide extensive numerical studies on the simulated dataset, the benchmark Tennessee Eastman process, and a practical cylinder-piston process. In some cases, the computing time is reduced by 600 times, and the FD rate is increased by 12.62% compared with classical CCA. The results suggest that the proposed approach is efficient and fast.

An Enhanced Extreme Learning Machine Based on Square-Root Lasso Method

Extreme learning machine (ELM) is one of the most notable machine learning algorithms with many advantages, especially its training speed. However, ELM has some drawbacks such as instability, poor generalizability and overfitting in the case of multicollinearity in the linear model. This paper introduces square-root lasso ELM (SQRTL-ELM) as a novel regularized ELM algorithm to deal with these drawbacks of ELM. A modified version of the alternating minimization algorithm is used to obtain the estimates of the proposed method. Various techniques are presented to determine the tuning parameter of SQRTL-ELM. The method is compared with the basic ELM, RIDGE-ELM, LASSO-ELM and ENET-ELM on six benchmark data sets. Performance evaluation results show that the SQRTL-ELM exhibits satisfactory performance in terms of testing root mean squared error in benchmark data sets for the sake of slightly extra computation time. The superiority level of the method depends on the tuning parameter selection technique. As a result, the proposed method can be considered a powerful alternative to avoid performance loss in regression problems .

Robust low tubal rank tensor recovery via [formula omitted]E criterion

Tensor analysis has received enormous attention as the increasing prevalence of high-dimensional data in science research and engineering application. Tensor recovery is an important and meaningful problem for tensor analysis. It aims to complete a tensor from an observed subset of its entries disturbed by noise. However, classical methods either develop on the second-order statistics or Lasso-type penalty, leading to them not effectively dealing with gross or dense noise effectively. To address such issues, we propose a robust tensor recovery model for simultaneously completing a low tubal rank tensor with complex noise and missing data. Based on tensor–tensor product (t-product), we first develop a tensor factor Frobenius norm to exploit the low tubal rank property which is closely related to tensor nuclear norm and tubal rank. By utilizing the robust L2 criterion, we derive the nonconvex objective function to accommodate the low tubal rank tensor. An implementable alternating minimization algorithm has been developed to estimate the low tubal rank tensor. It is worth noting that our method is able to jointly estimate the precision parameter to capture the hidden complex noise pattern. Furthermore, some convergence properties of the proposed algorithm are presented. A series of numerical experiments are conducted on both synthetic and real-world data to demonstrate the effectiveness and robustness of the proposed approach in comparison with the state-of-the-art methods.

RISIR: Rapid Infrared Spectral Imaging Restoration Model for Industrial Material Detection in Intelligent Video Systems

Material detection by industrial infrared imaging spectrometer (IRIS) is a key technique in multiple industrial applications, including garbage collection, material analysis, and robot vision. However, IRIS often suffers from overlapped bands and random noises, which limit the precision of subsequent processing. In this article, we propose a novel Gabor transform-based mid-wave infrared (MWIR) spectrum restoration model by successfully exploring the intrinsic structure of the clean MWIR spectrum from the degraded one. First, a total variation-regularized Gabor coefficient adjustment descriptor is designed and incorporated into the spectrum restoration model. In addition to adjusting the Gabor coefficient distribution by total variation-norm, the L2-norm of gradient is leveraged to regulate the smoothness of the instrument degradation function. Then, the proposed model is inferred using an efficient optimization approach based on split Bregman iteration method and alternating minimization algorithm. Quantitative and qualitative experimental results demonstrate that the proposed model favorably outperforms the state-of-the-art approaches. The restored high-resolution MWIR spectrum can be used to rapidly detect different materials in intelligent video systems.

Hybrid distributed online and alternating convex optimization‐based initial value accelerated hybrid precoding for millimeter wave multiple‐input‐multiple‐output

SummaryThe hybrid precoding problem is considered a Frobenius norm reduction problem for the narrowband channel in a millimeter wave (mmWave) with multiple inputs and outputs (mmWave MIMO). This work proposes a hybrid distributed online and alternating convex optimization (HDO‐ACO) algorithm to improve hybrid precoding (HP), interpreted as a trace minimization problem. HDO‐ACO alternately determines the digital and analog precoders to reduce the trace by keeping the other constant. Initially, HDO‐ACO uses Lagrange's method to determine the digital precoding subproblem. Then, it uses the integrated distributed online convex optimization and alternating minimization algorithm in the analog precoder design. In the HP method, the digital and analog precoders are iteratively updated until the highest number of iterations or convergence is reached. But this hybrid precoding method requires an initial analog precoding matrix input to begin the iteration. The algorithms converge gradually and fall into a suboptimal solution when the initial analog precoding matrix is set randomly. Hence, an initial value acceleration‐based heuristic approach is used in the HDO‐ACO alternating minimization algorithm that calculates the initial feasible value of an alternating minimization method using channel conditions. The simulation results of the proposed HDO‐ACO algorithm are presented under bit error rate (BER), spectral efficiency (SE), and convergence behavior by comparing it with modified block coordinate descent–HP (MBCD‐HP), manifold optimization–alternating minimization (MO‐AltMin), and semi‐definite relaxation‐based alternating optimization (SDR‐AO). The proposed HDO‐ACO attains maximum SE and less BER than other hyper‐precoding designs.

A phase‐field formulation based on an extended F‐criterion for rock fracture

AbstractIn this paper a phase‐field formulation based on an extended F‐criterion (the normalized strain energy release rate criterion) is proposed to simulate tensile‐compressive‐shear rock fractures. By applying the F‐criterion, the phase‐field crack‐driving energy decomposition is determined by a direction search which maximizes the local fracture dissipation. In compressive‐shear states, the computation is supplemented by an explicitly expressed confinement‐dependent mode‐II fracture energy release rate, and the cracking angle is determined by both the fracture energy and strain states. The hybrid formulation and alternate minimization algorithm are adopted for the numerical examples in this paper. Fractures for rock and rock‐like specimens subjected to compression demonstrate the ability of the present model in capturing tensile‐compressive‐shear rock fracture behaviors.

Massively parallelizable proximal algorithms for large‐scale stochastic optimal control problems

AbstractScenario‐based stochastic optimal control problems suffer from the curse of dimensionality as they can easily grow to six and seven figure sizes. First‐order methods are suitable as they can deal with such large‐scale problems, but may perform poorly and fail to converge within a reasonable number of iterations. To achieve a fast rate of convergence and high solution speeds, in this article, we propose the use of two proximal quasi‐Newtonian limited‐memory algorithms—minfbe applied to the dual problem and the Newton‐type alternating minimization algorithm (nama)—which can be massively parallelized on lockstep hardware such as graphics processing units. In particular, we use minfbe and nama to solve scenario‐based stochastic optimal control problems with affine dynamics, convex quadratic cost functions (with the stage cost functions being strongly convex in the control variable) and joint state‐input convex constraints. We demonstrate the performance of these methods, in terms of convergence speed and parallelizability, on large‐scale problems involving millions of variables.

Low-Rankness Guided Group Sparse Representation for Image Restoration.

As a spotlighted nonlocal image representation model, group sparse representation (GSR) has demonstrated a great potential in diverse image restoration tasks. Most of the existing GSR-based image restoration approaches exploit the nonlocal self-similarity (NSS) prior by clustering similar patches into groups and imposing sparsity to each group coefficient, which can effectively preserve image texture information. However, these methods have imposed only plain sparsity over each individual patch of the group, while neglecting other beneficial image properties, e.g., low-rankness (LR), leads to degraded image restoration results. In this article, we propose a novel low-rankness guided group sparse representation (LGSR) model for highly effective image restoration applications. The proposed LGSR jointly utilizes the sparsity and LR priors of each group of similar patches under a unified framework. The two priors serve as the complementary priors in LGSR for effectively preserving the texture and structure information of natural images. Moreover, we apply an alternating minimization algorithm with an adaptively adjusted parameter scheme to solve the proposed LGSR-based image restoration problem. Extensive experiments are conducted to demonstrate that the proposed LGSR achieves superior results compared with many popular or state-of-the-art algorithms in various image restoration tasks, including denoising, inpainting, and compressive sensing (CS).

On the optimal rank-1 approximation of matrices in the Chebyshev norm

The problem of low rank approximation is ubiquitous in science. Traditionally this problem is solved in unitary invariant norms such as Frobenius or spectral norm due to existence of efficient methods for building approximations. However, recent results reveal the potential of low rank approximations in Chebyshev norm, which naturally arises in many applications. In this paper we tackle the problem of building optimal rank-1 approximations in the Chebyshev norm. We investigate the properties of alternating minimization algorithm for building the low rank approximations and demonstrate how to use it to construct optimal rank-1 approximation. As a result we propose an algorithm that is capable of building optimal rank-1 approximations in Chebyshev norm for moderate matrices.

Robust enhanced indexation optimization with sparse industry layout constraint

In this paper, we investigate an enhanced indexation methodology using robust Conditional Value-at-Risk (CVaR) and group-sparse optimization. A featured difference from the existing literatures is to describe the tail risk using the worst-case CVaR of excess returns, and the process of industry selection using a weighted ℓ∞,1-norm constraint. We develop an accelerated alternating minimization algorithm for solving this problem. At each iteration, this method usually alternately solves a convex cone program, which admits a closed-form solution via convex duality theory, and a projection problem onto a weighted infinity-to-one-ball, where a fixed-point iteration projection method is developed, terminating in finite number of iterations. The global convergence rates in terms of the primal and dual residuals are also provided. Empirical tests on actual data sets are presented to demonstrate the superior out-of-sample performance of our proposed strategy.

Near-optimal bounds for generalized orthogonal Procrustes problem via generalized power method

Given multiple point clouds, how to find the rigid transform (rotation, reflection, and shifting) such that these point clouds are well aligned? This problem, known as the generalized orthogonal Procrustes problem (GOPP), has found numerous applications in statistics, computer vision, and imaging science. While one commonly-used method is finding the least squares estimator, it is generally an NP-hard problem to obtain the least squares estimator exactly due to the notorious nonconvexity. In this work, we apply the semidefinite programming (SDP) relaxation and the generalized power method to solve this generalized orthogonal Procrustes problem. In particular, we assume the data are generated from a signal-plus-noise model: each observed point cloud is a noisy copy of the same unknown point cloud transformed by an unknown orthogonal matrix and also corrupted by additive Gaussian noise. We show that the generalized power method (equivalently alternating minimization algorithm) with spectral initialization converges to the unique global optimum to the SDP relaxation, provided that the signal-to-noise ratio is high. Moreover, this limiting point is exactly the least squares estimator and also the maximum likelihood estimator. Our theoretical bound is near-optimal in terms of the information-theoretic limit (only loose by a factor of the dimension and a log factor). Our results significantly improve the state-of-the-art results on the tightness of the SDP relaxation for the generalized orthogonal Procrustes problem, an open problem posed by Bandeira et al. (2014) [8].

Metalearning-Based Alternating Minimization Algorithm for Nonconvex Optimization.

In this article, we propose a novel solution for nonconvex problems of multiple variables, especially for those typically solved by an alternating minimization (AM) strategy that splits the original optimization problem into a set of subproblems corresponding to each variable and then iteratively optimizes each subproblem using a fixed updating rule. However, due to the intrinsic nonconvexity of the original optimization problem, the optimization can be trapped into a spurious local minimum even when each subproblem can be optimally solved at each iteration. Meanwhile, learning-based approaches, such as deep unfolding algorithms, have gained popularity for nonconvex optimization; however, they are highly limited by the availability of labeled data and insufficient explainability. To tackle these issues, we propose a meta-learning based alternating minimization (MLAM) method that aims to minimize a part of the global losses over iterations instead of carrying minimization on each subproblem, and it tends to learn an adaptive strategy to replace the handcrafted counterpart resulting in advance on superior performance. The proposed MLAM maintains the original algorithmic principle, providing certain interpretability. We evaluate the proposed method on two representative problems, namely, bilinear inverse problem: matrix completion and nonlinear problem: Gaussian mixture models. The experimental results validate the proposed approach outperforms AM-based methods.

Distributed online multi‐task sparse identification for multiple systems with asynchronous updates

AbstractSimultaneous online identification of the structure and parameters for multiple systems can use the multi‐task learning to share parameter sparse patterns across different identification tasks to improve the estimation accuracy. To reduce the communication and computational burden, improve the convergence rate and increase the reliability of the identification task, a distributed asynchronous multi‐task sparse identification algorithm is proposed. The alternating minimization algorithm is applied to design a distributed multi‐task loss function. Then by constructing the auxiliary matrix and transforming the sparse constraint on the parameter matrix to the constructed auxiliary matrix, the asynchronous parameter update and sparse model update can be realized. The algorithm uses a recursive form to complete online identification which is feasible to analyze. After proving the convergence of the identification algorithm, we present an example to support the theoretical analysis and show the advantages of the proposed algorithm.

Sparse kernel k-means for high-dimensional data

The kernel k-means method usually loses its power when clustering high-dimensional data, due to a large number of irrelevant features. We propose a novel sparse kernel k-means clustering (SKKM) to extend the advantages of kernel k-means to the high-dimensional cases. We assign each feature a 0-1 indicator and optimize an equivalent kernel k-means loss function while penalizing the sum of the indicators. An alternating minimization algorithm is proposed to estimate both the class labels and the feature indicators. We prove the consistency of both clustering and feature selection of the proposed method. In addition, we apply the proposed framework to the normalized cut. In the numerical experiments, we demonstrate that the proposed method provides better/comparable performance compared to the existing high-dimensional clustering methods.

Tensor completion with noisy side information

We develop a new model for tensor completion which incorporates noisy side information available on the rows and columns of a 3-dimensional tensor. This method learns a low rank representation of the data along with regression coefficients for the observed noisy features. Given this model, we propose an efficient alternating minimization algorithm to find high-quality solutions that scales to large data sets. Through extensive computational experiments, we demonstrate that this method leads to significant gains in out-of-sample accuracy filling in missing values in both simulated and real-world data. We consider the problem of imputing drug response in three large-scale anti-cancer drug screening data sets: the Genomics of Drug Sensitivity in Cancer (GDSC), the Cancer Cell Line Encyclopedia (CCLE), and the Genentech Cell Line Screening Initiative (GCSI). On imputation tasks with 20% to 80% missing data, we show that the proposed method TensorGenomic matches or outperforms state-of-the-art methods including the original tensor model and a multilevel mixed effects model. With 80% missing data, TensorGenomic improves the R^2 from 0.404 to 0.552 in the GDSC data set, 0.407 to 0.524 in the CCLE data set, and 0.331 to 0.453 in the GCSI data set compared to the tensor model which does not take into account genomic side information.

A non–convex total generalized variation model for image denoising

We propose an effective regularization model based on second–order total generalized variation for image restoration with mixed Poisson–Gaussian noise. An efficient alternating minimization algorithm is employed to solve the considered model. Finally, provided numerical results show that our proposed model can preserve more details and get higher image visual quality than recent state-of-the-art methods.

Using Alternating Minimization and Convexified Carleman Weighted Objective Functional for a Time-Domain Inverse Scattering Problem

This paper considers a 1D time-domain inverse scattering problem for the Helmholtz equation in which penetrable scatterers are to be determined from boundary measurements of the scattering data. It is formulated as a coefficient identification problem for a wave equation. Using the Laplace transform, the inverse problem is converted into an overdetermined nonlinear system of partial differential equations. To solve this system, a Carleman weighted objective functional, which is proved to be strictly convex in an arbitrary set in a Hilbert space, is constructed. An alternating minimization algorithm is used to minimize the Carleman weighted objective functional. Numerical results are presented to illustrate the performance of the proposed algorithm.

Two-Phase Image Segmentation by Nonconvex Nonsmooth Models with Convergent Alternating Minimization Algorithms

Two-Phase Image Segmentation by Nonconvex Nonsmooth Models with Convergent Alternating Minimization Algorithms