Step Gradient Research Articles

A fast proximal iteratively reweighted nuclear norm algorithm for nonconvex low-rank matrix minimization problems

In this paper, we propose a fast proximal iteratively reweighted nuclear norm algorithm with extrapolation for solving a class of nonconvex low-rank matrix minimization problems. The proposed method incorporates two different extrapolation steps with respect to the previous iterations into the backward proximal step and the forward gradient step of the classic proximal iteratively reweighted method. We prove that the proposed method generates a convergent subsequence under general parameter constraints, and that any limit point is a stationary point of the problem. Furthermore, we prove that if the objective function satisfies the Kurdyka-Łojasiewicz property, the algorithm is globally convergent to a stationary point of the considered problem. Finally, we perform numerical experiments on a practical matrix completion problem with both synthetic and real data, the results of which demonstrate the efficiency and superior performance of the proposed algorithm.

The development of a generic analysis method for natural and synthetic dyes by ultra-high-pressure liquid chromatography with photo-diode-array detection and triethylamine as an ion-pairing agent

In cultural heritage the characterization of organic colorants is a challenging task. Currently, different chromatographic techniques are used to analyze natural and synthetic dyes separately, since the classes differ significantly in chemical properties and, therefore, chromatographic behavior. To save time, costs and sample material, we developed a method suitable for a wide variety of organic colorants using ultra-high-performance liquid chromatography coupled to a photo-diode-array detector. Gradient elution was performed with a mobile phase consisting of water and methanol with 5 mM triethylamine added as an ion-pairing agent at a pH of 3. Both linear and step gradients were optimized using the ‘Program for Interpretive Optimization of Two-dimensional Resolution’ (PIOTR) Pirok et al. [22]. Two optimized linear gradients and two step gradients were evaluated experimentally. The method was applied on a complex dye mixture containing nearly 130 natural- and synthetic-dye reference compounds. More than 100 of these compounds could be identified in a single experiment. The feasibility of the method was demonstrated by analyzing samples of several precious objects that were found in the Texel shipwreck Vos et al. [2] and of two embroideries of Emile Bernard, the results of which are described in this paper.

Effects of UV/H2O2 degradation and step gradient ethanol precipitation on Sargassum fusiforme polysaccharides: Physicochemical characterization and protective effects against intestinal epithelial injury

In this study, the degraded purified fraction from Sargassum fusiforme polysaccharides (SFP), named DSFP, was produced by the treatment of ultraviolet/hydrogen peroxide (UV/H2O2) degradation and step gradient ethanol precipitation. Results showed that the treatment significantly reduced the molecular weight of polysaccharides, from 282.83 kDa to 18.54 kDa, and influenced their surface morphology and roughness. SFP and DSFP were typical sulfated polysaccharides, mainly composed of fucose, galacturonic acid, glucuronic acid, galactose, and mannose. Both SFP and DSFP increased cell migration during intestinal epithelial wound healing and stimulated the cell cycle progression by promoting the transition from G0/G1 to S phase in the rat intestine epithelium cells (IEC-6). But DSFP had a stronger positive effect on wound healing and cell migration than SFP. It reinforced the intestinal barrier function and attenuated lipopolysaccharides-induced intestinal inflammation. DSFP significantly downregulated the expression of Toll-like receptor 4, tumor necrosis factor-α, interleukin-6, interleukin-1β, and inducible nitric oxide synthase by 53.14%, 92.41%, 66.01%, 68.24%, and 78.09%, respectively, and upregulated that of interleukin-10 by 2.48 folds when compared to the model. Therefore, the treatment (UV/H2O2 degradation and step gradient ethanol precipitation) could effectively improve the protective effects against intestinal epithelial injury.

An MXene-Based Metal Anode with Stepped Sodiophilic Gradient Structure Enables a Large Current Density for Rechargeable Na-O2 Batteries.

The metal anode is the pivotal component for advanced sodium-metal batteries such as Na-O2 batteries. Designing a 3D confinement scaffold is a promising strategy for constructing dendrite-free sodium-metal anodes; however, cycling stability at a large current density (>10 mA cm-2 ) is still difficult to realize. Herein, the design of new lightweight and fibrous hydroxylated Ti3 C2 (h-Ti3 C2 ) MXene based scaffolds with stepped sodiophilic gradient structure (h-M-SSG) is reported, and its thickness can be controlled (80-250µm). The sodiophilic gradient structure (adjusted by h-Ti3 C2 ) can effectively induce sodium ions to preferentially deposit at the bottom of the scaffold, thus inhibiting dendrite growth. h-M-SSG/Na-based symmetrical batteries exhibit a low polarization voltage and long cycling life at a high current density (40 mA cm-2 ) and a high cut-off capacity (40 mAh cm-2 ). Moreover, a Na-O2 battery with an h-M-SSG/Na anode exhibits a low potential gap of 0.137V after 45 cycles at 1000 mA g-1 and 1000 mAh g-1 . This deposition-regulation strategy would inspire the design of 3D scaffolds for high-performance sodium-metal-anode-based batteries.

Rapid Screening of Lipase Inhibitors from Ophiopogonis Radix Using High-Performance Thin Layer Chromatography by Two Step Gradient Elution Combined with Bioautographic Method.

In this study, a high-performance thin layer chromatography (HPTLC) method by two step gradient elution with two mobile phases was developed for the simultaneous analysis of seven constituents in Ophiopogonis Radix. The chromatography was performed on silica gel 60 F254 plate with dichloromethane-methanol-ethyl acetate-water (70:25:12:3, v/v/v/v) and dichloromethane-methanol (300:1, v/v) as the mobile phase for two step gradient elution. Then, the HPTLC profiles were observed after derivatization with 10% sulfuric acid in ethanol solution. The obtained HPTLC images were further analyzed by chemometric approaches and the samples could be clustered based on regions and/or growth years, which were two important factors affecting the constituents in Ophiopogonis Radix. Furthermore, five compounds including ophiopogonin D, ophiopojaponin C, ophiopogonin D’, ophiopogonin C’ and methylophiopogonanone B were screened as potential lipase inhibitors from Ophiopogonis Radix by the HPTLC-bioautographic method. The binding modes and interactions between the five compounds and lipase were further explored by molecular docking analysis. The developed HPTLC method could be used for quality control of Ophiopogonis Radix and screening of the potential lipase inhibitors.

Adaptive stochastic gradient descent for optimal control of parabolic equations with random parameters

AbstractIn this paper we extend the adaptive gradient descent (AdaGrad) algorithm to the optimal distributed control of parabolic partial differential equations with uncertain parameters. This stochastic optimization method achieves an improved convergence rate through adaptive scaling of the gradient step size. We prove the convergence of the algorithm for this infinite dimensional problem under suitable regularity, convexity, and finite variance conditions, and relate these to verifiable properties of the underlying system parameters. Finally, we apply our algorithm to the optimal thermal regulation of lithium battery systems under uncertain loads.

Analysis and experimental demonstration of temperature step gradients in preparative liquid chromatography

The amount of substance adsorbed on solid surface depends on temperature. Therefore, the migration velocities of the solutes in a chromatographic column can be altered by introducing temperature gradients. Such gradients designed to change retention behaviours can be exploited to improve the separation performances in preparative chromatography. To describe key process features, we used analytical solutions of the equilibrium model with instant stepwise shift of temperature. To achieve a more realistic description, the equilibrium dispersion model was additionally applied to treat finite column efficiencies. The effect of temperature gradients was illustrated experimentally using two identical columns sequentially connected. Temperature of the second column was modulated by thermostats. Wide pulse injections of a single component led to instructive elution profiles in a preliminary investigation. The observations were found to be in qualitative agreement with predictions of the equilibrium dispersion model. Subsequently, the separation of a ternary model mixture was investigated considering a simple two-step temperature gradient. To support the quantitative analysis and to identify suitable switching and cycle times, the temperature dependencies of the Henry constants were determined by short pulse injections. A meaningful variation of the parameters of the temperature gradient is required for adjusting the cycle times, which is the time difference between two consecutive injections that needs to shorten. Decreasing this time is connected with a desirable increase in process productivity. The results achieved revealed that relatively simple to implement stepwise temperature gradients offer an option to improve and fine-tune the performance of repetitive batch chromatography.

On the acceleration of the Barzilai–Borwein method

The Barzilai–Borwein (BB) gradient method is efficient for solving large-scale unconstrained problems to modest accuracy due to its ingenious stepsize which generally yields nonmonotone behavior. In this paper, we propose a new stepsize to accelerate the BB method by requiring finite termination for minimizing the two-dimensional strongly convex quadratic function. Based on this new stepsize, we develop an efficient gradient method for quadratic optimization which adaptively takes the nonmonotone BB stepsizes and certain monotone stepsizes. Two variants using retard stepsizes associated with the new stepsize are also presented. Numerical experiments show that our strategies of properly inserting monotone gradient steps into the nonmonotone BB method could significantly improve its performance and our new methods are competitive with the most successful gradient descent methods developed in the recent literature.

Sinkhorn Adversarial Attack and Defense.

Adversarial attacks have been extensively investigated in the recent past. Quite interestingly, a majority of these attacks primarily work in the lp space. In this work, we propose a novel approach for generating adversarial samples using Wasserstein distance. Unlike previous approaches, we use an unbalanced optimal transport formulation which is naturally suited for images. We first compute an adversarial sample using a gradient step and then project the resultant image into Wasserstein ball with respect to original sample. The attack introduces perturbation in the form of pixel mass distribution which is guided by a cost metric. Elaborate experiments on MNIST, Fashion-MNIST, CIFAR-10 and Tiny ImageNet demonstrate a sharp decrease in the performance of state-of-art classifiers. We also perform experiments with adversarially trained classifiers and show that our system achieves superior performance in terms of adversarial defense against several state-of-art attacks. Our code and pre-trained models are available at https://bit.ly/2SQBR4E.

A Meta-Learning Approach to the Optimal Power Flow Problem Under Topology Reconfigurations

Recently there has been a surge of interest in adopting deep neural networks (DNNs) for solving the optimal power flow (OPF) problem in power systems. Computing optimal generation dispatch decisions using a trained DNN takes significantly less time when compared to conventional optimization solvers. However, a major drawback of existing work is that the machine learning models are trained for a specific system topology. Hence, the DNN predictions are only useful as long as the system topology remains unchanged. Changes to the system topology (initiated by the system operator) would require retraining the DNN, which incurs significant training overhead and requires an extensive amount of training data (corresponding to the new system topology). To overcome this drawback, we propose a DNN-based OPF predictor that is trained using a meta-learning (MTL) approach. The key idea behind this approach is to find a common initialization vector that enables fast training for any system topology. The developed OPF-predictor is validated through simulations using benchmark IEEE bus systems. The results show that the MTL approach achieves significant training speed-ups and requires only a few gradient steps with a few data samples to achieve high OPF prediction accuracy and outperforms other pretraining techniques.

Unmatched Preconditioning of the Proximal Gradient Algorithm

This work addresses the resolution of penalized least-squares problems using the proximal gradient algorithm (PGA). PGA can be accelerated by preconditioning strategies. However, typical effective choices of preconditioners may correspond to intricate matrices that are not easily inverted, leading to increased complexity in the computation of the proximity step. To relax these requirements, we propose an unmatched preconditioning approach where two metrics are used in the gradient step and the proximity step. We provide convergence conditions for this new iterative scheme and characterize its limit point. Simulations for tomographic image reconstruction from undersampled measurements show the benefits of our approach for various simple choices of metrics.

DMML-Net: Deep Metametric Learning for Few-Shot Geographic Object Segmentation in Remote Sensing Imagery

Geographic object segmentation is a fundamental yet challenging problem for remote sensing image interpretation. The prevalent paradigm to solve this problem is to train deep neural networks on massive labeled samples. Although remarkable achievements have been attained, these methods suffer from the severe dependence on the large-scale dataset and require a long training process with high computation burden. To address these issues, a deep metametric learning framework, named DMML-Net, consisting of the metametric learner and the base-metric learner, is proposed for few-shot geographic object segmentation. First, DMML-Net formulates the segmentation as the metric-based pixel classification and develops a deep feature pyramid comparison network as the architecture of the metric learner for multiscale metric learning. Benefiting from this design, the segmentation can be efficiently solved, as well as being robust to deal with the scale variations of geographic objects. Second, an affinity-based fusion mechanism is introduced to adaptively reweight and fuse the semantic information across samples, effectively calibrating the deviation of prototypes induced by the intraclass variations. Third, considering the impact of the large interclass distribution divergences, DMML-Net presents a metametric training paradigm to provide the metric model with flexible scalability for fast adaptation to novel tasks. After metatraining, DMML-Net can be applied for the few-shot segmentation tasks of novel geographic objects with only a few gradient steps on the small training set. Experimental results on two benchmark remote sensing datasets demonstrate the validity and the superiority of our method in low-shot conditions where there are only one to ten labeled samples.

Fast User Adaptation for Human Motion Prediction in Physical Human–Robot Interaction

Accurate prediction of human movements is required to enhance the efficiency of physical human-robot interaction. Behavioral differences across various users are crucial factors that limit the prediction of human motion. Although recent neural network-based modeling methods have improved their prediction accuracy, most did not consider an effective adaptations to different users, thereby employing the same model parameters for all users. To deal with this insufficiently addressed challenge, we introduce a meta-learning framework to facilitate the rapid adaptation of the model to unseen users. In this study, we propose a model structure and a meta-learning algorithm specialized to enable fast user adaptation in predicting human movements in cooperative situations with robots. The proposed prediction model comprises shared and adaptive parameters, each addressing the user's general and individual movements. Using only a small amount of data from an individual user, the adaptive parameters are adjusted to enable user-specific prediction through a two-step process: initialization via a separate network and adaptation via a few gradient steps. Regarding the motion dataset that has 20 users collaborating with a robotic device, the proposed method outperforms existing meta-learning and non-meta-learning baselines in predicting the movements of unseen users.

Revisiting the Primal-Dual Method of Multipliers for Optimisation Over Centralised Networks

The primal-dual method of multipliers (PDMM) was originally designed for solving a decomposable optimisation problem over a general network. In this paper, we revisit PDMM for optimisation over a centralised network. We first note that the recently proposed method <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">FedSplit</i> [1] implements PDMM for a centralised network. In [1], Inexact FedSplit (i.e., gradient based FedSplit) was also studied both empirically and theoretically. We identify the cause for the poor reported performance of Inexact FedSplit, which is due to the suboptimal initialisation in the gradient operations at the client side. To fix the issue of Inexact FedSplit, we propose two versions of Inexact PDMM, which are referred to as gradient-based PDMM (GPDMM) and accelerated GPDMM (AGPDMM), respectively. AGPDMM accelerates GPDMM at the cost of transmitting two times the number of parameters from the server to each client per iteration compared to GPDMM. We provide a new convergence bound for GPDMM for a class of convex optimisation problems. Our new bounds are tighter than those derived for Inexact FedSplit. We also investigate the update expressions of AGPDMM and SCAFFOLD to find their similarities. It is found that when the number <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">K</i> of gradient steps at the client side per iteration is <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">K=1</i> , both AGPDMM and SCAFFOLD reduce to vanilla gradient descent with proper parameter setup. Experimental results indicate that AGPDMM converges faster than SCAFFOLD when <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">K</i> >1 while GPDMM converges slightly slower than SCAFFOLD.

Novel anion exchange membrane chromatography method for the separation of empty and full adeno-associated virus.

A challenge in the production of recombinant Adeno-Associated Virus (AAV) for gene therapies is the presence of capsids that lack the required gene of interest. The impact of these empty vectors in therapies is not fully understood, however the ability to control the ratio of empty to full particles, which contain the genetic payload, is a necessary step in the purification of these viruses. In this study, a novel anion exchange chromatography elution method for enrichment of full AAV particles is demonstrated. A step gradient with small conductivity increases of around 1 mS cm-1 provides more efficient separation of empty and full AAV serotype 5 across membrane media as compared to conventional linear gradient method. The use of this approach in optimizing a simpler method for manufacturing processes and scalability to a larger chromatographic volume is explored. With this approach, the authors achieved greater than 4-fold enrichment of full capsids, to give a total of ≈50%-60% full capsids, using a 25mM Bis-Tris Propane pH 9.0 buffer system with NaCl as the eluting salt. Results suggest that this elution method can be implemented into a scalable process and can provide insight into development of elution methods for other AAV serotypes.

Large steps in inverse rendering of geometry

Inverse reconstruction from images is a central problem in many scientific and engineering disciplines. Recent progress on differentiable rendering has led to methods that can efficiently differentiate the full process of image formation with respect to millions of parameters to solve such problems via gradient-based optimization. At the same time, the availability of cheap derivatives does not necessarily make an inverse problem easy to solve. Mesh-based representations remain a particular source of irritation: an adverse gradient step involving vertex positions could turn parts of the mesh inside-out, introduce numerous local self-intersections, or lead to inadequate usage of the vertex budget due to distortion. These types of issues are often irrecoverable in the sense that subsequent optimization steps will further exacerbate them. In other words, the optimization lacks robustness due to an objective function with substantial non-convexity. Such robustness issues are commonly mitigated by imposing additional regularization, typically in the form of Laplacian energies that quantify and improve the smoothness of the current iterate. However, regularization introduces its own set of problems: solutions must now compromise between solving the problem and being smooth. Furthermore, gradient steps involving a Laplacian energy resemble Jacobi's iterative method for solving linear equations that is known for its exceptionally slow convergence. We propose a simple and practical alternative that casts differentiable rendering into the framework of preconditioned gradient descent. Our pre-conditioner biases gradient steps towards smooth solutions without requiring the final solution to be smooth. In contrast to Jacobi-style iteration, each gradient step propagates information among all variables, enabling convergence using fewer and larger steps. Our method is not restricted to meshes and can also accelerate the reconstruction of other representations, where smooth solutions are generally expected. We demonstrate its superior performance in the context of geometric optimization and texture reconstruction.

The dual-polarized staggered stacked patches antenna

A new dual-polarized staggered and stacked patches antenna with wide impedance bandwidth and high isolation is proposed. The antenna consists of two groups of radiation patches, in 7 layers, and uses the orthogonal adjacent coupling structure on staggered layer to excite a pair of linear polarization modes. Thanks to the staggered feeder mode, it has increased the isolation performance between ports and compressed the transverse size of the antenna. As a result of the combination of staggered stack-up between the patches and the stepped gradient shape of the main radiating patches, it has effectively expanded the impedance bandwidth of the antenna. The proposed antenna is simulated, fabricated and measured. The staggered feeding structure effectively reduces the cross-sectional area of the antenna, and greatly improves the isolation between feeding ports. The measurement results show that the impedance bandwidths for vertical and horizontal polarization modes are 40.2% (638–960 MHz) and 40.0% (645968 MHz) respectively when the return loss is lower than −10 dB, and the isolation between feeding ports is better than −30 dB. Meanwhile, the antenna has a stable and symmetrical radiation pattern across the working band, therefore making it suitable to be used as antenna and antenna array element of mobile wireless communication base stations.

Understanding the effects of system differences for parameter estimation and scale-up of high throughput chromatographic data

In this paper, we evaluate how employing fraction collection and multistep gradients with RoboColumns® (Repligen, formally Atoll) affects both comparison to benchtop experimental data and column simulation parameter estimation. These operational differences arise from the RoboColumn® system (operated on an automated liquid handling device) requiring offline analysis for determination of elution profiles rather than the continuous in-line UV curves obtained with larger scale systems. In addition, multistep gradients are used to model the smooth linear gradients of larger scale systems because sequential injections are used to provide liquid flow. Comparisons of two sets of column simulations was first carried out to demonstrate that fraction collection reduced the first moments of the elution peaks by 1/2 of the fraction volumes. Additional column simulations determined that the effect of a multistep gradient approximation on retention volume was dependent upon the gradient step length. An empirical transformation was then developed to correct the first moments obtained from gradient experimental data using the RoboColumn® system. These corrected values provided a more direct comparison of the experimental data at different scales and resulted in a significant improvement in agreement with results obtained using a 20 mL benchtop column. Linear steric mass-action (SMA) parameters were then estimated using the corrected values and employed to successfully predict the performance of the benchtop system data. Finally, these parameters were demonstrated to be well suited for modeling the RoboColumn® gradient data when properly accounting for multistep gradients and fraction collection. This work continues previous investigations into understanding system differences associated with robotic liquid handling devices and proposes a methodology for properly accounting for operational differences to predict operation at larger scales using conventional chromatography systems.

Feature Extraction of Music Signal Based on Adaptive Wave Equation Inversion

The digitization, analysis, and processing technology of music signals are the core of digital music technology. There is generally a preprocessing process before the music signal processing. The preprocessing process usually includes antialiasing filtering, digitization, preemphasis, windowing, and framing. Songs in the popular wav format and MP3 format on the Internet are all songs that have been processed by digital technology and do not need to be digitalized. Preprocessing can affect the effectiveness and reliability of the feature parameter extraction of music signals. Since the music signal is a kind of voice signal, the processing of the voice is also applicable to the music signal. In the study of adaptive wave equation inversion, the traditional full-wave equation inversion uses the minimum mean square error between real data and simulated data as the objective function. The gradient direction is determined by the cross-correlation of the back propagation residual wave field and the forward simulation wave field with respect to the second derivative of time. When there is a big gap between the initial model and the formal model, the phenomenon of cycle jumping will inevitably appear. In this paper, adaptive wave equation inversion is used. This method adopts the idea of penalty function and introduces the Wiener filter to establish a dual objective function for the phase difference that appears in the inversion. This article discusses the calculation formulas of the accompanying source, gradient, and iteration step length and uses the conjugate gradient method to iteratively reduce the phase difference. In the test function group and the recorded music signal library, a large number of simulation experiments and comparative analysis of the music signal recognition experiment were performed on the extracted features, which verified the time-frequency analysis performance of the wave equation inversion and the improvement of the decomposition algorithm. The features extracted by the wave equation inversion have a higher recognition rate than the features extracted based on the standard decomposition algorithm, which verifies that the wave equation inversion has a better decomposition ability.

Fast selection of nonlinear mixed effect models using penalized likelihood

Nonlinear Mixed effects models are hidden variables models that are widely used in many fields such as pharmacometrics. In such models, the distribution characteristics of hidden variables can be specified by including several parameters such as covariates or correlations which must be selected. Recent development of pharmacogenomics has brought averaged/high dimensional problems to the field of nonlinear mixed effects modeling for which standard covariates selection techniques like stepwise methods are not well suited. The selection of covariates and correlation parameters using a penalized likelihood approach is proposed. The penalized likelihood problem is solved using a stochastic proximal gradient algorithm to avoid inner-outer iterations. Speed of convergence of the proximal gradient algorithm is improved using component-wise adaptive gradient step sizes. The practical implementation and tuning of the proximal gradient algorithm are explored using simulations. Calibration of regularization parameters is performed by minimizing the Bayesian Information Criterion using particle swarm optimization, a zero-order optimization procedure. The use of warm restart and parallelization allowed computing time to be reduced significantly. The performance of the proposed method compared to the traditional grid search strategy is explored using simulated data. Finally, an application to real data from two pharmacokinetics studies is provided, one studying an antifibrinolytic and the other studying an antibiotic.