Guaranteed Convergence Properties Research Articles

Online Parameter Estimation of the Polarization Curve of a Fuel Cell With Guaranteed Convergence Properties: Theoretical and Experimental Results

Online Parameter Estimation of the Polarization Curve of a Fuel Cell With Guaranteed Convergence Properties: Theoretical and Experimental Results

Memory-efficient model-based deep learning with convergence and robustness guarantees.

Computational imaging has been revolutionized by compressed sensing algorithms, which offer guaranteed uniqueness, convergence, and stability properties. Model-based deep learning methods that combine imaging physics with learned regularization priors have emerged as more powerful alternatives for image recovery. The main focus of this paper is to introduce a memory efficient model-based algorithm with similar theoretical guarantees as CS methods. The proposed iterative algorithm alternates between a gradient descent involving the score function and a conjugate gradient algorithm to encourage data consistency. The score function is modeled as a monotone convolutional neural network. Our analysis shows that the monotone constraint is necessary and sufficient to enforce the uniqueness of the fixed point in arbitrary inverse problems. In addition, it also guarantees the convergence to a fixed point, which is robust to input perturbations. We introduce two implementations of the proposed MOL framework, which differ in the way the monotone property is imposed. The first approach enforces a strict monotone constraint, while the second one relies on an approximation. The guarantees are not valid for the second approach in the strict sense. However, our empirical studies show that the convergence and robustness of both approaches are comparable, while the less constrained approximate implementation offers better performance. The proposed deep equilibrium formulation is significantly more memory efficient than unrolled methods, which allows us to apply it to 3D or 2D+time problems that current unrolled algorithms cannot handle.

Hierarchical continuous-time inhomogeneous hidden Markov model for cancer screening with extensive followup data.

Continuous-time hidden Markov models are an attractive approach for disease modeling because they are explainable and capable of handling both irregularly sampled, skewed and sparse data arising from real-world medical practice, in particular to screening data with extensive followup. Most applications in this context consider time-homogeneous models due to their relative computational simplicity. However, the time homogeneous assumption is too strong to accurately model the natural history of many diseases including cancer. Moreover, cancer risk across the population is not homogeneous either, since exposure to disease risk factors can vary considerably between individuals. This is important when analyzing longitudinal datasets and different birth cohorts. We model the heterogeneity of disease progression and regression using piece-wise constant intensity functions and model the heterogeneity of risks in the population using a latent mixture structure. Different submodels under the mixture structure employ the same types of Markov states reflecting disease progression and allowing both clinical interpretation and model parsimony. We also consider flexible observational models dealing with model over-dispersion in real data. An efficient, scalable Expectation-Maximization algorithm for inference is proposed with the theoretical guaranteed convergence property. We demonstrate our method's superior performance compared to other state-of-the-art methods using synthetic data and a real-world cervical cancer screening dataset from the Cancer Registry of Norway. Moreover, we present two model-based risk stratification methods that identify the risk levels of individuals.

On generation of virtual outputs via signal injection: Application to observer design for electromechanical systems

Probing signal injection is a well-established technique to extract additional information from a weakly (or non) observable dynamical system. Using averaging theory, a framework to analyse such schemes for general nonlinear systems has been recently proposed in Combes et. al., [9], where it is shown that the signal injection may be used to generate a new high frequency component of the systems output that can be used for state observation or controller design. A key step for the success of this technique is the implementation of a filter to reconstruct this virtual output from the measurement of the overall systems output. The main contribution of this paper is to propose a new filter with guaranteed convergence properties that outperforms the classical designs. The method is applied to a general class of electromechanical systems, and its performance is assessed via simulations and experiments on the benchmark example of a 1-dof magnetic levitation system.

Trajectory tracking for a quadrotor under wind perturbations: sliding mode control with state-dependent gains

The problem of position tracking of a mini drone subject to wind perturbations is investigated. The solution is based on a detailed unmanned aerial vehicle (UAV) model, with aerodynamic coefficients and external disturbance components, which is introduced in order to better represent the impact of the wind field. Then, upper bounds of wind-induced disturbances are characterized, which allow a sliding mode control (SMC) technique to be applied with guaranteed convergence properties. The peculiarity of the considered case is that the disturbance upper bounds depend on the control amplitude itself (i.e. the system is nonlinear in control), which leads to a new procedure for the control tuning presented in the paper. The last part of the paper is dedicated to the analysis and reduction of chattering effects, as well as investigation of rotor dynamics issues. Conventional SMC with constant gains, proposed first order SMC, and proposed quasi-continuous SMC are compared. Nonlinear UAV simulator, validated through in-door experiments, is used to demonstrate the effectiveness of the proposed controls.

Removal of incoherent noise from an averaged cross-spectral matrix.

Measured cross-spectral matrices (CSMs) from a microphone array will in some cases be contaminated by severe incoherent noise signals in the individual channels. A typical example is flow noise generated in the individual microphones when measuring in a wind tunnel. Assuming stationary signals and performing long-time averaging, the contamination will be concentrated on the CSM diagonal. When the CSM is used for traditional frequency-domain beamforming, diagonal removal (DR) will avoid use of the diagonal. DR is effective at suppressing the contamination effects, but it also has some side effects. With other beamforming algorithms and in connection with acoustic holography, however, the diagonal of the CSM is needed. The present paper describes a method for removal of incoherent noise contamination from the CSM diagonal. The method formulates the problem as a semidefinite program, which is a convex optimization problem that can be solved very efficiently and with guaranteed convergence properties. A first numerical study investigates the question, whether the semidefinite program formulation will provide in all cases the desired output. A second numerical study investigates the limitations introduced by off-diagonal noise contributions due to finite averaging time. The results of that study are backed up by results from a practical measurement.

Optimal control of viscous Burgers equation via an adaptive nonmonotone Barzilai–Borwein gradient method

ABSTRACTAn adaptive nonmonotone spectral gradient method for the solution of distributed optimal control problem (OCP) for the viscous Burgers equation is presented in a black-box framework. Regarding the implicit function theorem, the OCP is transformed into an unconstrained nonlinear optimization problem (UNOP). For solving UNOP, an adaptive nonmonotone Barzilai–Borwein gradient method is proposed in which to make a globalization strategy, first an adaptive nonmonotone strategy which properly controls the degree of nonmonotonicity is presented and then is incorporated into an inexact line search approach to construct a more relaxed line search procedure. Also an adjoint technique is used to effectively evaluate the gradient. The low memory requirement and the guaranteed convergence property make the proposed method quite useful for large-scale OCPs. The efficiency of the presented method is supported by numerical experiments.

Toward autonomous avian-inspired grasping for micro aerial vehicles*

Micro aerial vehicles, particularly quadrotors, have been used in a wide range of applications. However, the literature on aerial manipulation and grasping is limited and the work is based on quasi-static models. In this paper, we draw inspiration from agile, fast-moving birds such as raptors, that are able to capture moving prey on the ground or in water, and develop similar capabilities for quadrotors. We address dynamic grasping, an approach to prehensile grasping in which the dynamics of the robot and its gripper are significant and must be explicitly modeled and controlled for successful execution. Dynamic grasping is relevant for fast pick-and-place operations, transportation and delivery of objects, and placing or retrieving sensors. We show how this capability can be realized (a) using a motion capture system and (b) without external sensors relying only on onboard sensors. In both cases we describe the dynamic model, and trajectory planning and control algorithms. In particular, we present a methodology for flying and grasping a cylindrical object using feedback from a monocular camera and an inertial measurement unit onboard the aerial robot. This is accomplished by mapping the dynamics of the quadrotor to a level virtual image plane, which in turn enables dynamically-feasible trajectory planning for image features in the image space, and a vision-based controller with guaranteed convergence properties. We also present experimental results obtained with a quadrotor equipped with an articulated gripper to illustrate both approaches.

A nonmonotone spectral projected gradient method for large-scale topology optimization problems

An efficient gradient-based method to solve the volume constrained topology optimization problems is presented. Each iterate of this algorithm is obtained by the projection of a Barzilai-Borwein step onto the feasible set consisting of box and one linear constraints (volume constraint). To ensure the global convergence, an adaptive nonmonotone line search is performed along the direction that is given by the current and projection point. The adaptive cyclic reuse of the Barzilai-Borwein step is applied as the initial stepsize. The minimum memory requirement, the guaranteed convergence property, and almost only one function and gradient evaluations per iteration make this new method very attractive within common alternative methods to solve large-scale optimal design problems. Efficiency and feasibility of the presented method are supported by numerical experiments.

Design and prototyping of a low-cost vehicle localization system with guaranteed convergence properties

This work presents a low-cost vehicle localization system, using measurements from one gyroscope, two wheel speed sensors and a GPS, to estimate the heading, velocity and position of a vehicle. Taking advantage of the nonholonomic constraints, the design of the observer (or “filter”) takes into account imperfections of the embedded sensor measurements, such as a slowly time-varying gyroscope bias or some uncertainty in the wheel diameter value. Thanks to a well-chosen nonlinear structure, the estimator is easy to tune, easy to implement, and well-behaved even at very low speed. Moreover, the proposed filter has guaranteed convergence properties when GPS is available and keeps providing good estimations during GPS losses. Simulation and experiment results in urban area illustrate the good performance of the algorithm.

The Burbea-Rao and Bhattacharyya Centroids

We study the centroid with respect to the class of information-theoretic Burbea-Rao divergences that generalize the celebrated Jensen-Shannon divergence by measuring the non-negative Jensen difference induced by a strictly convex and differentiable function. Although those Burbea-Rao divergences are symmetric by construction, they are not metric since they fail to satisfy the triangle inequality. We first explain how a particular symmetrization of Bregman divergences called Jensen-Bregman distances yields exactly those Burbea-Rao divergences. We then proceed by defining skew Burbea-Rao divergences, and show that skew Burbea-Rao divergences amount in limit cases to compute Bregman divergences. We then prove that Burbea-Rao centroids can be arbitrarily finely approximated by a generic iterative concave-convex optimization algorithm with guaranteed convergence property. In the second part of the paper, we consider the Bhattacharyya distance that is commonly used to measure overlapping degree of probability distributions. We show that Bhattacharyya distances on members of the same statistical exponential family amount to calculate a Burbea-Rao divergence in disguise. Thus we get an efficient algorithm for computing the Bhattacharyya centroid of a set of parametric distributions belonging to the same exponential families, improving over former specialized methods found in the literature that were limited to univariate or “diagonal” multivariate Gaussians. To illustrate the performance of our Bhattacharyya/Burbea-Rao centroid algorithm, we present experimental performance results for <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">k</i> -means and hierarchical clustering methods of Gaussian mixture models.

Estimation of a covariance matrix with zeros

We consider estimation of the covariance matrix of a multivariate random vector under the constraint that certain covariances are zero. We first present an algorithm, which we call Iterative Conditional Fitting, for computing the maximum likelihood estimator of the constrained covariance matrix, under the assumption of multivariate normality. In contrast to previous approaches, this algorithm has guaranteed convergence properties. Dropping the assumption of multivariate normality, we show how to estimate the covariance matrix in an empirical likelihood approach. These approaches are then compared via simulation and on an example of gene expression.

Global Frequency Estimators Through Marino-Tomei Observers

We show how adaptive observers of the Marino-Tomei type can be designed for the classical problem of online estimation of the frequency of a sinusoidal signal. For the later problem, a global solution is provided recently by (Hsu et al., 1999) to system control theorists through a variation of an adaptive notch filter (ANF) which is originated from signal processing. The note is to provide another global solution through Marino-Tomei observers. This new solution does not only enjoy simpler, more a control system theoretical structure, more elegant global stability proof, but also guaranteed convergence properties.

Learning with permutably homogeneous multiple-valued multiple-threshold perceptrons

The (k,s)-perceptrons partition the input space {0,..., k-1}/sup n/ into s+1 regions using s parallel hyperplanes. Their learning abilities are examined in this paper. The previously studied homogeneous (k, k-1)-perceptron learning algorithm is generalized to the permutably homogeneous (k,s)-perceptron learning algorithm with guaranteed convergence property. We also introduce a powerful learning method that learns any permutably homogeneously separable k-valued logic function given as input.

Performance comparison of SAM and SQP methods for structural shape optimization

This paper presents a numerical performance comparison of a modern version of the well-established sequential quadratic programming (SQP) method and the more recent spherical approximation method (SAM). The comparison is based on the application of these algorithms to examples with nonlinear objective and constraint functions, among others: weight minimization problems in structural shape optimization. The comparison shows that both the SQP and SAM-algorithms are able to converge to accurate minimum weight values. However, because of the lack of a guaranteed convergence property of the SAM method, it exhibits an inability to consistently converge to a fine tolerance. This deficiency is manifested by the appearance of small oscillations in the neighbourhood of the solution.

Learning with discrete multivalued neurons

Analog neural networks of limited precision are essentially k -ary neural networks. That is, their processors classify the input space into k regions using k — 1 parallel hyperplanes by computing k -ary weighted multilinear threshold functions. The ability of k -ary neural networks to learn k -ary weighted multilinear threshold functions is examined. The well-known perceptron learning algorithm is generalized to a k -ary perceptron algorithm with guaranteed convergence property. Littlestone's winnow algorithm is superior to the perceptron learning algorithm when the ratio of the sum of the weight to the threshold value of the function being learned is small. A k -ary winnow algorithm with a mistake bound which depends on this value and the ratio between the largest and smallest thresholds is presented.

Hierarchical dynamic optimization for induction motors regulation

A hierarchical dynamic optimization technique is developed to tentatively regulate the performance of non-linear induction motor systems. The decomposition process of the non-linear model is achieved using a new method called the parameters-transformation method. Manipulation of the interaction patterns is performed via a predictive-type algorithmic procedure. The developed computation structure has the features of relatively simple calculations at the lower level, minimal storage requirements at the higher level, moderate information processing between hierarchical levels and guaranteed convergence properties.