Sequential Subspace Optimization Research Articles

An accelerated inexact Newton-type regularizing algorithm for ill-posed operator equations

We propose and analyze a new iterative regularization approach, called IN-SETPG, for efficiently solving nonlinear ill-posed operator equations in the Hilbert-space setting. IN-SETPG consists of an outer iteration and an inner iteration. The outer iteration is terminated by the discrepancy principle and consists of an inexact Newton regularization method, while the inner iteration is performed by a sequential subspace optimization method based on the two-point gradient iteration. The key idea behind IN-SETPG is that, unlike the standard Landweber method, it uses multiple search directions per iteration in combination with an adaptive step size in order to reduce the total number of iterations. The regularization property of IN-SETPG has been established, i.e., the iterate converges to a solution of the nonlinear problem with exact data when the noise level tends to zero. Various numerical experiments are presented to demonstrate that, compared with the original inexact Newton iteration, IN-SETPG can achieve better reconstruction results and remarkable acceleration.

Sequential Subspace Optimization Design of a Dual Three-Phase Permanent Magnet Synchronous Hub Motor Based on NSGA III

The multiobjective optimization design of dual three-phase permanent magnet synchronous hub motors (PMSHMs) is challenging due to the high dimension and huge computation cost of finite-element analysis (FEA). A new multiobjective optimization strategy is proposed for dual three-phase PMSHMs in this article. All design parameters are divided into two subspaces according to the Pearson sensitivity analysis results to improve optimization efficiency. A new training method is adopted to improve the accuracy of the approximate model. By improving a multiobjective intelligent optimization algorithm, nondominated sorting genetic algorithm (NSGA) III, a new algorithm is proposed, which will greatly shorten optimization time. It is found that the proposed optimization method can significantly improve the performance, such as smaller torque ripple and higher maximum torque for the investigated PMSHM, while the computation resources are reduced. A prototype based on the optimization results is manufactured, and experiments are conducted on the platform to verify the accuracy of the optimization results and the FEA. The effectiveness of optimization and the accuracy of the simulation are verified by the experimental results.

Imaging based on Compton scattering: model uncertainty and data-driven reconstruction methods

The recent development of scintillation crystals combined with γ-rays sources opens the way to an imaging concept based on Compton scattering, namely Compton scattering tomography. The associated inverse problem rises many challenges: non-linearity, multiple order-scattering and high level of noise. Already studied in the literature, these challenges lead unavoidably to uncertainty of the forward model. This work proposes to study exact and approximated forward models and develops two data-driven reconstruction algorithms able to tackle the inexactness of the forward model. The first one is based on the projective method called regularized sequential subspace optimization (RESESOP). We consider here a finite dimensional restriction of the semi-discrete forward model and show its well-posedness and regularization properties. The second one considers the unsupervised learning method, deep image prior, inspired by the construction of the model uncertainty in RESESOP. The methods are validated on Monte-Carlo data.

Multiobjective and Multiphysics Design Optimization of a Switched Reluctance Motor for Electric Vehicle Applications

Switched reluctance motors (SRMs) have attracted much attention in industry due to the advantages of low cost, robust structure, high fault tolerance and high torque density. However, several disadvantages like high torque ripples and coil temperatures hinder their industrialization for some applications requiring high dynamic performance, like electric vehicles (EVs). In this paper, a multiobjective and multiphysics design optimization method considering both thermal and electromagnetic performance is presented for a 12/10 SRM. First, the topology of the SRM is introduced and the optimal parameters are defined. Then, the electromagnetic finite element model (FEM) is introduced and the 3D transient lumped-parameter thermal model (TLPTM), considering both axial and radial heat transfer for the SRM, is proposed. Second, the objectives and constraints of the optimization are determined. To improve the optimization efficiency, the sequential subspace optimization strategy is employed to find the optimal solution of this high-dimensional design optimization problem. Finally, to validate the effectiveness of the proposed method, both simulation and experimental results are given and discussed. Compared with the initial design, the optimal solution exhibits lower temperature, higher torque, lower torque ripple and less loss.

An accelerated Homotopy-Perturbation-Kaczmarz method for solving nonlinear inverse problems

In this paper, we propose an accelerated homotopy-perturbation-Kaczmarz iteration based on sequential subspace optimization method for solving nonlinear systems of inverse problems. The method is to iteratively project the initial value onto stripes the width of which are controlled by the search direction, the forward operator and the noise level to expedite the convergence. Under some general assumptions, we provide the convergence and regularization analysis for the proposed method. Finally, two numerical examples on inverse potential problems are presented to illustrate the effectiveness of reconstructing the solution and acceleration effect of the method.

A fast two-point gradient algorithm based on sequential subspace optimization method for nonlinear ill-posed problems

In this paper, we propose a fast two-point gradient algorithm for solving nonlinear ill-posed problems, which is based on the sequential subspace optimization method. The key idea, in contrast to the standard two-point gradient method, is to use multiple search directions in each iteration without extra computation, and to get the step size by metric projection. Moreover, a modified discrete backtracking search algorithm is proposed to select the combination parameters in the accelerated two-point gradient method. Under the basic assumptions for iterative regularization methods, we establish the convergence results of the method in the noise-free case. Furthermore, stability and regularity are presented when the algorithm terminated by the discrepancy principle for the case of noisy data. Finally, some numerical simulations are presented, which exhibit that the proposed method leads to a significant reduction of the iteration numbers and the overall computational time.

Accelerated homotopy perturbation iteration method for a non-smooth nonlinear ill-posed problem

In this paper, a fast iterative algorithm based on J-order homotopy perturbation method is proposed for the nonlinear ill-posed problem whose forward operator is not Gâteaux differentiable. The Bouligand subderivative of the forward operator is utilized to replace the Fréchet derivative in iteration system. The sequential subspace optimization technique is introduced to accelerate the convergence speed, which regards the correction term of homotopy perturbation as multiple search directions to update the new iterate. To this end, the current iteration is sequentially projected to the stripes whose width is determined by search directions, the nonlinearity of the forward operator and noise level. We present the convergence analysis based on the asymptotic stability estimates and a generalized tangential cone condition. Numerical experiments are performed to illustrate the effectiveness of the proposed method.

Research on Optimal Design of Double Stator Low-speed High-torque Synchronous Motor Based on Surrogate Model

Optimizing double stator low-speed high-torque synchronous motor (DSLHSM) with many parameters is difficult and time-consuming, so an optimization design method suitable for multi-parameter problems was proposed. Through the method, the design optimization of DSLHSM can be more effective, and the optimization time is greatly shortened. Firstly, the optimization variables were selected after the sensitivity of the key parameters of average torque and torque ripple was analysed by using design of experiments (DOE) and analysis of variance (ANOVA). Secondly high-precision dynamic surrogate models of average torque and torque ripple were established based on response surface model (RSM), and then the objective function was established according to the surrogate model and the genetic algorithm (GA) as well as the sequential subspace optimization method (SSOM) was used to solve the objective function step by step. Finally, the optimal solution was substituted into the finite element simulation model. The target performances of the motor before and after optimization were compared and analysed. The results proved the effectiveness and correctness of the optimization design method.

Inverse problems with inexact forward operator: iterative regularization and application in dynamic imaging

The classic regularization theory for solving inverse problems is built on the assumption that the forward operator perfectly represents the underlying physical model of the data acquisition. However, in many applications, for instance in microscopy or magnetic particle imaging, this is not the case. Another important example represent dynamic inverse problems, where changes of the searched-for quantity during data collection can be interpreted as model uncertainties. In this article, we propose a regularization strategy for linear inverse problems with inexact forward operator based on sequential subspace optimization methods (SESOP). In order to account for local modelling errors, we suggest to combine SESOP with the Kaczmarz’ method. We study convergence and regularization properties of the proposed method and discuss several practical realizations. Relevance and performance of our approach are evaluated at simulated data from dynamic computerized tomography with various dynamic scenarios.

Accelerating monotone fast iterative shrinkage–thresholding algorithm with sequential subspace optimization for sparse recovery

The present paper focuses on accelerating monotone fast iterative shrinkage–thresholding algorithm (MFISTA) that is popular to solve the basis pursuit denoising problem for sparse recovery. Inspired by a recent work that accelerates MFISTA with line search, we alternatively use a much more effective speed-up option, termed sequential subspace optimization. Furthermore, instead of manually setting the number of previous propagation directions in the subspace beforehand, we propose an adaptive method to set it. Additionally, for approximating the absolute value function, we analyze the superiority of a smooth version used in this paper over the one recommended in a previous work, and give an analytical closed-form expression for the shrinkage operator corresponding to the smooth approximation. The experiments presented here show that the proposed method achieves faster convergence speeds in terms of iteration and run-time.

An accelerated sequential subspace optimization method based on homotopy perturbation iteration for nonlinear ill-posed problems

Homotopy perturbation iteration is an effective and fast method for solving nonlinear ill-posed problems. It only needs approximately half the computation time of Landweber iteration to reach the similar recovery precision. In this paper, a Nesterov-type accelerated sequential subspace optimization method based on homotopy perturbation iteration is proposed for solving nonlinear inverse problems. The convergence analysis is provided under the general assumptions for iterative regularization methods. The numerical experiments on inverse potential problem indicate that the proposed method dramatically reduces the total number of iterations and time consumption to obtain satisfying approximations, especially for the problems with costly solution of forward calculation.

Acceleration of sequential subspace optimization in Banach spaces by orthogonal search directions

Conjugate Gradient (CG) methods are one of the most effective iterative methods to solve linear equations in Hilbert spaces. So far, they have been inherently bound to these spaces since they make use of the inner product structure. In more general Banach spaces one of the most prominent iterative solvers are Landweber-type methods that essentially resemble the Steepest Descent method applied to the normal equation. More advanced are subspace methods that take up the idea of a Krylov-type search space, wherein an optimal solution is sought. However, they do not share the conjugacy property with CG methods. In this article we propose that the Sequential Subspace Optimization (SESOP) method can be considered as an extension of CG methods to Banach spaces. We employ metric projections to orthogonalize the current search direction with respect to the search space from the last iteration. For the l2-space our method then exactly coincides with the Polak-Ribi\`ere type of the CG method when applied to the normal equation. We show that such an orthogonalized search space still leads to weak convergence of the subspace method. Moreover, numerical experiments on a random matrix toy problem and 2D computerized tomography on lp-spaces show superior convergence properties over all p compared to non-orthogonalized search spaces. This especially holds for lp-spaces with small p. We see that the closer we are to an l2-space, the more we recover of the conjugacy property that holds in these spaces, i. e., as expected, the more the convergence behaves independently of the size of the truncated search space.

Sequential subspace optimization for nonlinear inverse problems

Abstract In this work we discuss a method to adapt sequential subspace optimization (SESOP), which has so far been developed for linear inverse problems in Hilbert and Banach spaces, to the case of nonlinear inverse problems. We start by revising the technique for linear problems. In a next step, we introduce a method using multiple search directions that are especially designed to fit the nonlinearity of the forward operator. To this end, we iteratively project the initial value onto stripes whose width is determined by the search direction, the nonlinearity of the operator and the noise level. We additionally propose a fast algorithm that uses two search directions. Finally, we will show convergence and regularization properties for the presented method.

System-Level Design Optimization Methods for Electrical Drive Systems: Deterministic Approach

Electrical drive systems are key components in modern appliances, industry equipment, and systems, e.g., hybrid electric vehicles. To obtain the best performance of these drive systems, the motors and their control systems should be designed and optimized at the system level rather than the component level. This paper presents an effort to develop system-level design and optimization methods for electrical drive systems. Two system-level design optimization methods are presented in this paper: 1) single-level method (only at system level); and 2) multilevel method. Meanwhile, the approximate models, the design of experiments technique, and the sequential subspace optimization method are presented to improve the optimization efficiency. Finally, a drive system consisting of a permanent-magnet transverse flux machine with a soft magnetic composite core is investigated, and detailed results are presented and discussed. This is a high-dimensional optimization problem with 14 parameters mixed with both discrete and continuous variables. The finite-element analysis model and method are verified by the experimental results on the motor prototype. From the discussion, it can be found that the proposed multilevel method can increase the performance of the whole drive system, such as bigger output power and lower material cost, and decrease the computation cost significantly compared with those of single-level design optimization method.

Multilevel Design Optimization of a FSPMM Drive System by Using Sequential Subspace Optimization Method

In our previous research, flux-switching permanent magnet machine (FSPMM) was investigated for the application in hybrid electric vehicles. To obtain the best performance of the whole drive system, a new multilevel design optimization method is presented for this kind of machine and a field oriented control system. The proposed multilevel optimization method is based on sequential subspace optimization method. In the implementation, three levels are employed to obtain the optimal design scheme at the system level. Meanwhile, sequential optimization method is employed to reduce the computation cost of finite element analysis on the motor level. Finally, from the design analysis, it can be found that the proposed method can provide design scheme with better performance, while the needed computation cost is reduced greatly for this FSPMM drive system.

Non-negative constraint research of Tikhonov regularization inversion for dynamic light scattering

In dynamic light scattering (DLS) technology, a non-negative constraint on the solution can improve the inversion accuracy of the particle size distribution (PSD). Different non-negative constraint methods have different effects on the inversion results. Combined with the Tikhonov regularization inversion method, the following non-negativity constraint methods: negative to zero (N-to-Z), multi-negative to zero (Multi-N-to-Z), Lin-projected gradient (LPG), oblique projected Landweber (OPL), projected sequential subspace optimization (PSESOP), interior point Newton (IPN), gradient projection conjugate gradient (GPCG) and trust-region method based on the interior reflective Newton (TR-IRN) method are studied in DLS inversion. In different inversion ranges and noise levels, autocorrelation functions of unimodal and bimodal particle distributions were inverted using different non-negativity constraint methods. From the inversion results, the characteristics of the various methods were obtained, which can be treated as a reference for the implementation of non-negative constraints in Tikhonov regularization inversion of DLS.

Sequential Subspace Optimization Method for Electromagnetic Devices Design With Orthogonal Design Technique

We present two new sequential optimization strategies, a sequential subspace optimization method (SSOM) and an improved sequential optimization method (SOM) with orthogonal experimental design technique, to deal with optimization design problems of electromagnetic devices in this work. To implement the proposed methods, we first divide the whole design factors into three sets, namely highly-significant, significant, and nonsignificant factors. Then the whole design space can be correspondingly divided into three subspaces with these three sets of factors. Thereafter, SSOM is presented to sequentially optimize those subspaces. In the subspace, we present an improved SOM based on orthogonal experimental design technique to get optimal solutions. Finally, by investigating TEAM benchmark problem 22, we can see that the sampling efficiency can be improved significantly and the computational cost of finite element analysis can be saved remarkably by the proposed methods.

L1-L2 Optimization in Signal and Image Processing

Sparse, redundant representations offer a powerful emerging model for signals. This model approximates a data source as a linear combination of few atoms from a prespecified and over-complete dictionary. Often such models are fit to data by solving mixed ¿ <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> -¿ <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sub> convex optimization problems. Iterative-shrinkage algorithms constitute a new family of highly effective numerical methods for handling these problems, surpassing traditional optimization techniques. In this article, we give a broad view of this group of methods, derive some of them, show accelerations based on the sequential subspace optimization (SESOP), fast iterative soft-thresholding algorithm (FISTA) and the conjugate gradient (CG) method, present a comparative performance, and discuss their potential in various applications, such as compressed sensing, computed tomography, and deblurring.

Acceleration of the generalized Landweber method in Banach spaces via sequential subspace optimization

The Landweber method is a well-known iterative procedure to compute regularized solutions of linear operator equations in Hilbert spaces. Unfortunately it is also known to be very slow. Likewise its generalization to Banach spaces has good regularizing properties but slow convergence. This article intends to give a short survey about the use of sequential subspace optimization to accelerate this method while preserving its regularizing properties.

Fast regularizing sequential subspace optimization in Banach spaces

We are concerned with fast computations of regularized solutions of linear operator equations in Banach spaces in case only noisy data are available. To this end we modify recently developed sequential subspace optimization methods in such a way that the therein employed Bregman projections onto hyperplanes are replaced by Bregman projections onto stripes whose width is in the order of the noise level.