Descent Step Research Articles

Regularized Dual Averaging Image Reconstruction for Full-Wave Ultrasound Computed Tomography.

Ultrasound computed tomography (USCT) holds great promise for breast cancer screening. Waveform inversion-based image reconstruction methods account for higher order diffraction effects and can produce high-resolution USCT images, but are computationally demanding. Recently, a source encoding technique has been combined with stochastic gradient descent (SGD) to greatly reduce image reconstruction times. However, this method bundles the stochastic data fidelity term with the deterministic regularization term. This limitation can be overcome by replacing SGD with a structured optimization method, such as the regularized dual averaging method, that exploits knowledge of the composition of the cost function. In this paper, the dual averaging method is combined with source encoding techniques to improve the effectiveness of regularization while maintaining the reduced reconstruction times afforded by source encoding. It is demonstrated that each iteration can be decomposed into a gradient descent step based on the data fidelity term and a proximal update step corresponding to the regularization term. Furthermore, the regularization term is never explicitly differentiated, allowing nonsmooth regularization penalties to be naturally incorporated. The wave equation is solved by the use of a time-domain method. The effectiveness of this approach is demonstrated through computer simulation and experimental studies. The results suggest that the dual averaging method can produce images with less noise and comparable resolution to those obtained by the use of SGD.

A memory gradient method based on the nonmonotone technique

In this paper, we present a new nonmonotone memory gradient algorithm for unconstrained optimization problems. An attractive property of the proposed method is that the search direction always provides sufficient descent step at each iteration. This property is independent of the line search used. Under mild assumptions, the global and local convergence results of the proposed algorithm are established respectively. Numerical results are also reported to show that the proposed method is suitable to solve large-scale optimization problems and is more stable than other similar methods in practical computation.

An algorithm for constrained one-step inversion of spectral CT data

We develop a primal-dual algorithm that allows for one-step inversion of spectral CT transmission photon counts data to a basis map decomposition. The algorithm allows for image constraints to be enforced on the basis maps during the inversion. The derivation of the algorithm makes use of a local upper bounding quadratic approximation to generate descent steps for non-convex spectral CT data discrepancy terms, combined with a new convex-concave optimization algorithm. Convergence of the algorithm is demonstrated on simulated spectral CT data. Simulations with noise and anthropomorphic phantoms show examples of how to employ the constrained one-step algorithm for spectral CT data.

Relative pose estimation of a lander using crater detection and matching

Future space exploration missions require precise information about the lander pose during the descent and landing steps. An effective algorithm that utilizes crater detection and matching is presented to determine the lander pose with respect to the planetary surface. First, the projections of the crater circular rims in the descent image are detected and fitted into ellipses based on the geometric distance and coplanar circles constraint. Second, the detected craters are metrically rectified through a two-dimensional homography and matched with the crater database by similarity transformation. Finally, the lander pose is calculated by a norm-based optimization method. The algorithm is tested by synthetic and real trials. The experimental results show that our presented algorithm can determine the lander pose accurately and robustly.

Bounded perturbation resilience of projected scaled gradient methods

We investigate projected scaled gradient (PSG) methods for convex minimization problems. These methods perform a descent step along a diagonally scaled gradient direction followed by a feasibility regaining step via orthogonal projection onto the constraint set. This constitutes a generalized algorithmic structure that encompasses as special cases the gradient projection method, the projected Newton method, the projected Landweber-type methods and the generalized expectation-maximization (EM)-type methods. We prove the convergence of the PSG methods in the presence of bounded perturbations. This resilience to bounded perturbations is relevant to the ability to apply the recently developed superiorization methodology to PSG methods, in particular to the EM algorithm.

DLM: Decentralized Linearized Alternating Direction Method of Multipliers

This paper develops the Decentralized Linearized Alternating Direction Method of Multipliers (DLM) that minimizes a sum of local cost functions in a multiagent network. The algorithm mimics operation of the decentralized alternating direction method of multipliers (DADMM) except that it linearizes the optimization objective at each iteration. This results in iterations that, instead of successive minimizations, implement steps whose cost is akin to the much lower cost of the gradient descent step used in the distributed gradient method (DGM). The algorithm is proven to converge to the optimal solution when the local cost functions have Lipschitz continuous gradients. Its rate of convergence is shown to be linear if the local cost functions are further assumed to be strongly convex. Numerical experiments in least squares and logistic regression problems show that the number of iterations to achieve equivalent optimality gaps are similar for DLM and ADMM and both much smaller than those of DGM. In that sense, DLM combines the rapid convergence of ADMM with the low computational burden of DGM.

Less is more: Basic variable neighborhood search for minimum differential dispersion problem

In this paper, we propose a basic variable neighborhood search for solving Minimum differential dispersion problem using only the swap neighborhood structure in both descent (intensification) and shaking (diversification) steps. It has become a trend in the metaheuristic literature to use hybrid metaheuristics, i.e., combination of several metaheuristic paradigms, for solving some particular optimization problem. We show that our simple method, which relies on the basic Variable neighborhood search, significantly outperforms the hybrid one that combines GRASP, Variable neighborhood search, and Exterior path relinking metaheuristics. Thus, simplicity is not only the desired user friendly property of a heuristic but can lead to more efficient and effective method than if complex hybrid metaheuristic is used: less is more.

Improvement of Long Binary Sequence Merit Factors using Modified Legendre Algorithms

Low autocorrelation binary sequence (LABS) detection is a classic problem in the literature. We use these sequences in many real-life applications. The detection of these sequences involves many problems. In the literature, various methods have been developed to approach the LABS issue. Based on the length of the sequence, an appropriate method can be selected and implemented. For short length sequences, linear search is possible and as the length increases we can implement various stochastic optimization algorithms. In our case that is for long binary sequences, we can use construction methods. Kristiansen and Parker [1] in their work have shown that Legendre sequences with periodic rotation can achieve a merit factor of 6.34.We have applied these Legendre sequences to steepest descent and prime step algorithms with some modifications. We call these techniques as modified Legendre algorithms. Using these improved methods we were able to achieve a merit factor of 6.4245 for long binary sequences.

Multiple Sparse Measurement Gradient Reconstruction Algorithm for DOA Estimation in Compressed Sensing

A novel direction of arrival (DOA) estimation method in compressed sensing (CS) is proposed, in which the DOA estimation problem is cast as the joint sparse reconstruction from multiple measurement vectors (MMV). The proposed method is derived through transforming quadratically constrained linear programming (QCLP) into unconstrained convex optimization which overcomes the drawback thatl1-norm is nondifferentiable when sparse sources are reconstructed by minimizingl1-norm. The convergence rate and estimation performance of the proposed method can be significantly improved, since the steepest descent step and Barzilai-Borwein step are alternately used as the search step in the unconstrained convex optimization. The proposed method can obtain satisfactory performance especially in these scenarios with low signal to noise ratio (SNR), small number of snapshots, or coherent sources. Simulation results show the superior performance of the proposed method as compared with existing methods.

On-line relational and multiple relational SOM

In some applications and in order to address real-world situations better, data may be more complex than simple numerical vectors. In some examples, data can be known only through their pairwise dissimilarities or through multiple dissimilarities, each of them describing a particular feature of the data set. Several variants of the Self-Organizing Map (SOM) algorithm were introduced to generalize the original algorithm to the framework of dissimilarity data. Whereas median SOM is based on a rough representation of the prototypes, relational SOM allows representing these prototypes by a virtual linear combination of all elements in the data set, referring to a pseudo-Euclidean framework. In the present article, an on-line version of relational SOM is introduced and studied. Similar to the situation in the Euclidean framework, this on-line algorithm provides a better organization and is much less sensible to prototype initialization than standard (batch) relational SOM. In a more general case, this stochastic version allows us to integrate an additional stochastic gradient descent step in the algorithm which can tune the respective weights of several dissimilarities in an optimal way: the resulting multiple relational SOM thus has the ability to integrate several sources of data of different types, or to make a consensus between several dissimilarities describing the same data. The algorithms introduced in this paper are tested on several data sets, including categorical data and graphs. On-line relational SOM is currently available in the R package SOMbrero that can be downloaded at http://sombrero.r-forge.r-project.org/ or directly tested on its Web User Interface at http://shiny.nathalievilla.org/sombrero.

Knee and ankle mechanics differentiate between the transitional steps of stair descent in older women

L. Alcock1,2,∗, T.D. O’Brien3,4, N. Vanicek2,5 1 Institute for Ageing and Health, Newcastle University, UK 2 Department of Sport, Health and Exercise Science, University of Hull, UK 3 School of Sport and Exercise Sciences, Liverpool John Moores University, UK 4 School of Sport, Health and Exercise Sciences, Bangor University, UK 5 Discipline of Exercise and Sport Science, The University of Sydney, Australia

Two-dimensional permittivity and conductivity imaging by full waveform inversion of multioffset GPR data: a frequency-domain quasi-Newton approach

Full waveform inversion of ground-penetrating radar data is an emerging technique for the quantitative, high-resolution imaging of the near subsurface. Here, we present a 2-D frequency-domain full waveform inversion for the simultaneous reconstruction of the dielectric permittivity and of the electrical conductivity. The inverse problem is solved with a quasi-Newton optimization scheme, where the influence of the Hessian is approximated by the L-BFGS-B algorithm. This formulation can be considered to be fully multiparameter since it enables to update permittivity and conductivity values within the same descent step, provided we define scales of measurement through a reference permittivity, a reference conductivity, and an additional scaling factor. Numerical experiments on a benchmark from the literature demonstrate that the inversion is very sensitive to the parameter scaling, despite the consideration of the approximated Hessian that should correct for parameter dimensionalities. A proper scaling should respect the natural sensitivity of the misfit function and give priority to the parameter that has the most impact on the data (the permittivity, in our case). We also investigate the behaviour of the inversion with respect to frequency sampling, considering the selected frequencies either simultaneously or sequentially. As the relative imprint of permittivity and conductivity in the data varies with frequency, the simultaneous reconstruction of both parameters takes a significant benefit from broad frequency bandwidth data, so that simultaneous or cumulative strategies should be favoured. We illustrate our scaling approach with a realistic synthetic example for the imaging of a complex subsurface from on-ground multioffset data. Considering data acquired only from the ground surface increases the ill-posedness of the inverse problem and leads to a strong indetermination of the less-constrained conductivity parameters. A Tikhonov regularization can prevent the creation of high-wavenumber artifacts in the conductivity model that compensate for erroneous low-wavenumber structures, thus enabling to select model solutions. We propose a workflow for multiparameter imaging involving both parameter scaling and regularization. Optimal combinations of scaling factors and regularization weights can be identified by seeking regularization levels that exhibit a clear minimum of final data misfit with respect to parameter scaling. We confirm this workflow by inverting noise-contaminated synthetic data. In a surface-to-surface acquisition configuration, we have been able to reconstruct an accurate permittivity structure and a smooth version of the conductivity distribution, based entirely on the analysis of the data misfit with respect to parameter scaling, for different regularization levels.

A Parallel Bundle Framework for Asynchronous Subspace Optimization of Nonsmooth Convex Functions

An algorithmic framework is presented for optimizing general convex functions by nonsynchronized parallel processes. Each process greedily picks a suitable adaptive subset of coordinates and runs a bundle method on a corresponding restricted problem stopping whenever a descent step is encountered or predicted decrease is reduced sufficiently. No prior knowledge on the dependencies between variables is assumed. Instead, dependency information is collected automatically by analyzing aggregate subgradient properties required for ensuring convergence. Within this framework three strategies are discussed for supporting varying scenarios of structural independence: a single convex function, the sum of partially separable convex functions, and a scenario tuned to problem decomposition by Lagrangian relaxation of packing-type constraints. The theoretical framework presented here generalizes a practical method proposed by the authors for Lagrangian relaxation of large scale packing problems and simplifies the analysis.

Faster Gradient Descent and the Efficient Recovery of Images

Much recent attention has been devoted to gradient descent algorithms where the steepest descent step size is replaced by a similar one from a previous iteration or gets updated only once every second step, thus forming a faster gradient descent method. For unconstrained convex quadratic optimization these methods can converge much faster than steepest descent. But the context of interest here is application to certain ill-posed inverse problems, where the steepest descent method is known to have a smoothing, regularizing effect, and where a strict optimization solution is not necessary. Specifically, in this paper we examine the effect of replacing steepest descent by a faster gradient descent algorithm in the practical context of image deblurring and denoising tasks. We also propose several highly efficient schemes for carrying out these tasks independently of the step size selection, as well as a scheme for the case where both blur and significant noise are present. In the above context there are situations where many steepest descent steps are required, thus building slowness into the solution procedure. Our general conclusion regarding gradient descent methods is that in such cases the faster gradient descent methods offer substantial advantages. In other situations where no such slowness buildup arises the steepest descent method can still be very effective.

A full‐spectral Bayesian reconstruction approach based on the material decomposition model applied in dual‐energy computed tomography

Dual-energy computed tomography (DECT) makes it possible to get two fractions of basis materials without segmentation. One is the soft-tissue equivalent water fraction and the other is the hard-matter equivalent bone fraction. Practical DECT measurements are usually obtained with polychromatic x-ray beams. Existing reconstruction approaches based on linear forward models without counting the beam polychromaticity fail to estimate the correct decomposition fractions and result in beam-hardening artifacts (BHA). The existing BHA correction approaches either need to refer to calibration measurements or suffer from the noise amplification caused by the negative-log preprocessing and the ill-conditioned water and bone separation problem. To overcome these problems, statistical DECT reconstruction approaches based on nonlinear forward models counting the beam polychromaticity show great potential for giving accurate fraction images. This work proposes a full-spectral Bayesian reconstruction approach which allows the reconstruction of high quality fraction images from ordinary polychromatic measurements. This approach is based on a Gaussian noise model with unknown variance assigned directly to the projections without taking negative-log. Referring to Bayesian inferences, the decomposition fractions and observation variance are estimated by using the joint maximum a posteriori (MAP) estimation method. Subject to an adaptive prior model assigned to the variance, the joint estimation problem is then simplified into a single estimation problem. It transforms the joint MAP estimation problem into a minimization problem with a nonquadratic cost function. To solve it, the use of a monotone conjugate gradient algorithm with suboptimal descent steps is proposed. The performance of the proposed approach is analyzed with both simulated and experimental data. The results show that the proposed Bayesian approach is robust to noise and materials. It is also necessary to have the accurate spectrum information about the source-detector system. When dealing with experimental data, the spectrum can be predicted by a Monte Carlo simulator. For the materials between water and bone, less than 5% separation errors are observed on the estimated decomposition fractions. The proposed approach is a statistical reconstruction approach based on a nonlinear forward model counting the full beam polychromaticity and applied directly to the projections without taking negative-log. Compared to the approaches based on linear forward models and the BHA correction approaches, it has advantages in noise robustness and reconstruction accuracy.

A geometric approach to joint inversion with applications to contaminant source zone characterization

This paper presents a new joint inversion approach to shape-based inverse problems. Given two sets of data from distinct physical models, the main objective is to obtain a unified characterization of inclusions within the spatial domain of the physical properties to be reconstructed. Although our proposed method generally applies to many types of inverse problems, the main motivation here is to characterize subsurface contaminant source zones by processing down-gradient hydrological data and cross-gradient electrical resistance tomography observations. Inspired by Newton's method for multi-objective optimization, we present an iterative inversion scheme in which descent steps are chosen to simultaneously reduce both data-model misfit terms. Such an approach, however, requires solving a non-smooth convex problem at every iteration, which is computationally expensive for a pixel-based inversion over the whole domain. Instead, we employ a parametric level set technique that substantially reduces the number of underlying parameters, making the inversion computationally tractable. The performance of the technique is examined and discussed through the reconstruction of source zone architectures that are representative of dense non-aqueous phase liquid (DNAPL) contaminant release in a statistically homogenous sandy aquifer. In these examples, the geometric configuration of the DNAPL mass is considered along with additional information about its spatial variability within the contaminated zone, such as the identification of low and high saturation regions. Comparison of the reconstructions with the true DNAPL architectures highlights the superior performance of the model-based technique and joint inversion scheme.

Disruption of TAB1/p38α Interaction Using a Cell-permeable Peptide Limits Myocardial Ischemia/Reperfusion Injury

Targeting the adaptor protein (transforming growth factor-β (TGF-β)-activated protein kinase 1 (TAK1)-binding protein 1) (TAB1)-mediated non-canonical activation of p38α to limit ischemia/reperfusion (I/R) injury after an acute myocardial infarction seems to be attractive since TAB1/p38α interaction occurs specifically in very limited circumstances and possesses unique structural basis. However, so far no TAB1/p38α interaction inhibitor has been reported due to the limited knowledge about the interfaces. In this study, we sought to identify key amino acids essential for the unique mode of interaction with computer-guided molecular simulations and molecular docking. After validation of the predicted three-dimensional (3-D) structure of TAB1/p38α complex, we designed several peptides and evaluated whether they could block TAB1/p38α interaction with selectivity. We found that a cell-permeable peptide worked as a selective TAB1/p38α interaction inhibitor and decreased myocardial I/R injury. To our knowledge, this is the first TAB1/p38α interaction inhibitor.

An efficient approach to large-scale genotype–phenotype association analyses

Modern molecular biotechnology generates a great deal of intermediate information, such as transcriptional and metabolic products in bridging DNA and complex traits. In genome-wide linkage analysis and genome-wide association study, regression analysis for large-scale correlated phenotypes is applied to map genes for those by-products that are regarded as quantitative traits. For a single trait, least absolute shrinkage and selection operator with coordinate descent step can be employed to efficiently shrink sparse non-zero genetic effects of quantitative trait loci (QTLs). However, regression analyses in a trait-by-trait basis do not take account of the correlations among the analyzed traits. In this study, conditional phenotype of each trait is defined, given other traits. Large-scale genotype-phenotype association analyses are therefore transformed to separate genotype-conditional phenotype ones. Meanwhile, the correlation architecture between each trait and other traits can also be provided by shrinkage estimation for each conditional phenotype. Simulation demonstrates that the proposed conditional mapping method is generally identical to joint mapping method based on multivariate analysis in terms of statistical detection power and parameter estimation. Application of the method is provided to locate eQTL in yeast.

Direct Optimization of the Dictionary Learning Problem

A novel way of solving the dictionary learning problem is proposed in this paper. It is based on a so-called direct optimization as it avoids the usual technique which consists in alternatively optimizing the coefficients of a sparse decomposition and in optimizing dictionary atoms. The algorithm we advocate simply performs a joint proximal gradient descent step over the dictionary atoms and the coefficient matrix. After having derived the algorithm, we also provided in-depth discussions on how the stepsizes of the proximal gradient descent have been chosen. In addition, we uncover the connection between our direct approach and the alternating optimization method for dictionary learning. We have shown that it can be applied to a broader class of non-convex optimization problems than the dictionary learning one. As such, we have denoted the algorithm as a one-step block-coordinate proximal gradient descent. The main advantage of our novel algorithm is that, as suggested by our simulation study, it is more efficient than alternating optimization algorithms.

Forward–backward-based descent methods for composite variational inequalities

We consider the monotone composite variational inequality (CVI) where the underlying mapping is formed as the sum of two monotone mappings. We combine the forward–backward and descent direction ideas together, and thus present the unified algorithmic framework of forward–backward-based descent methods for solving the CVI. A new iterate of such a method is generated by a prediction–correction fashion, where the predictor is yielded by the forward–backward method and then the predictor is corrected by a descent step. We derive some implementable forward–backward-based descent algorithms for some concrete cases of the CVI, and verify their numerical efficiency via preliminary numerical experiments.