Non-smooth Components Research Articles

Gradient sliding for composite optimization

We consider in this paper a class of composite optimization problems whose objective function is given by the summation of a general smooth and nonsmooth component, together with a relatively simple nonsmooth term. We present a new class of first-order methods, namely the gradient sliding algorithms, which can skip the computation of the gradient for the smooth component from time to time. As a consequence, these algorithms require only $$\mathcal{O}(1/\sqrt{\epsilon })$$O(1/∈) gradient evaluations for the smooth component in order to find an $$\epsilon $$∈-solution for the composite problem, while still maintaining the optimal $$\mathcal{O}(1/\epsilon ^2)$$O(1/∈2) bound on the total number of subgradient evaluations for the nonsmooth component. We then present a stochastic counterpart for these algorithms and establish similar complexity bounds for solving an important class of stochastic composite optimization problems. Moreover, if the smooth component in the composite function is strongly convex, the developed gradient sliding algorithms can significantly reduce the number of graduate and subgradient evaluations for the smooth and nonsmooth component to $$\mathcal{O} (\log (1/\epsilon ))$$O(log(1/∈)) and $$\mathcal{O}(1/\epsilon )$$O(1/∈), respectively. Finally, we generalize these algorithms to the case when the smooth component is replaced by a nonsmooth one possessing a certain bi-linear saddle point structure.

Classification of edges using compactly supported shearlets

We analyze the detection and classification of singularities of functions f = χ S , where S ⊂ R d and d = 2 , 3 . It will be shown how the set ∂ S can be extracted by a continuous shearlet transform associated with compactly supported shearlets. Furthermore, if ∂ S is a d − 1 dimensional piecewise smooth manifold with d = 2 or 3, we will classify smooth and non-smooth components of ∂ S . This improves previous results given for shearlet systems with a certain band-limited generator, since the estimates we derive are uniform. Moreover, we will show that our bounds are optimal. Along the way, we also obtain novel results on the characterization of wavefront sets in 3 dimensions by compactly supported shearlets. Finally, geometric properties of ∂ S such as curvature are described in terms of the continuous shearlet transform of f .

Penalized least squares estimation in the additive model with different smoothness for the components

We consider the standard additive regression model consisting of two components f0 and g0. The first component f0 is assumed to be in some sense “smoother” than the second g0. It is known that in that case one can construct estimators that estimate the smoother component f0 with fast minimax rate as if the non-smooth component g0 were known. Our contribution shows that this phenomenon also occurs when one uses the penalized least squares estimator (fˆ,gˆ) of (f0,g0). We describe smoothness in terms of a semi-norm on the class of regression functions. This covers the case of Sobolev smoothness where the penalized estimator is a standard spline estimator. The theory is illustrated by a simulation study. Our proofs rely on recent results from empirical process theory.

Mirror Prox algorithm for multi-term composite minimization and semi-separable problems

In the paper, we develop a version of Mirror Prox algorithm for solving convex---concave saddle point problems and monotone variational inequalities of special structure, allowing to cover saddle point/variational analogies of what is usually called composite minimization (minimizing a sum of an easy-to-handle nonsmooth and a general-type smooth convex functions as if there were no nonsmooth component at all). We demonstrate that the Mirror Prox inherits the favourable (and unimprovable already in the large-scale bilinear saddle point case) [InlineEquation not available: see fulltext.] efficiency estimate of its prototype. We demonstrate that the proposed approach can be successfully applied to Lasso-type problems with several penalizing terms (e.g. acting together $$\ell _1$$l1 and nuclear norm regularization) and to problems of semi-separable structures considered in the alternating directions methods, implying in both cases methods with the [InlineEquation not available: see fulltext.] complexity bounds.

IMPACT OSCILLATORS: FUNDAMENTALS AND APPLICATIONS

The paper presents results of the analysis of nonlinear oscillators with nonsmooth elements and nonlinear systems with nonsmooth forcing components. Main attention has been focused on highlighting specific properties of nonsmooth systems compared to their smooth counterparts. Nonsmooth transformation of the time variable and the replacement of initial issues by boundary problems have been taken as the base for the analytical method. Results of numerical simulations and computing in the form of graphs of displacements and velocity waveforms and attractors are presented. To fully identify the system's behaviour and meet high performance specifications recourse to model all dynamics together with their interactions has been taken into account. Strong interactions among the parts of the system are considered and the phenomenon of the impact is exhibited. It has been found that non-smooth dynamical systems reveal significant wealth of nonlinear phenomena, including a chaotic, that are unique to this potentially important class of nonlinear systems. In non-smooth systems at small change of parameters, a sudden transition from a stable periodic oscillation to the full range of chaotic oscillations may often occur. The dynamics of nonsmooth oscillations with shock external forcing is analysed by using a relatively new mathematical tool, which appears to be hyperbolic algebra. The key idea of this tool is steeped in of non-smooth time transformations (NSTT) for strongly nonlinear, but still smooth models.

A smoothing stochastic gradient method for composite optimization

We consider the unconstrained optimization problem whose objective function is composed of a smooth and a non-smooth components where the smooth component is the expectation of a random function. This type of problem arises in some interesting applications in machine learning. We propose a stochastic gradient descent algorithm for this class of optimization problems. When the non-smooth component has a particular structure, we propose a stochastic gradient descent algorithm by incorporating a smoothing method into our first algorithm. The proofs of the convergence rates of these two algorithms are given and we show the numerical performance of our algorithm by applying them to regularized linear regression and logistic regression problems with different sets of synthetic data.

Solving Variational Inequalities with Stochastic Mirror-Prox Algorithm

In this paper we consider iterative methods for stochastic variational inequalities (s.v.i.) with monotone operators. Our basic assumption is that the operator possesses both smooth and nonsmooth components. Further, only noisy observations of the problem data are available. We develop a novel Stochastic Mirror-Prox (SMP) algorithm for solving s.v.i. and show that with the convenient stepsize strategy it attains the optimal rates of convergence with respect to the problem parameters. We apply the SMP algorithm to Stochastic composite minimization and describe particular applications to Stochastic Semidefinite Feasibility problem and deterministic Eigenvalue minimization.

The Stripping Principle to the System Synthesis for a Kind of Complex Networked Systems

In this paper, based on the philosophical thinking of `On Contradiction', we propose a method to the system synthesis for a kind of uncertain networked systems which are composed of some heteromorphic subsystems. We call it the stripping principle. Based on the principle, we design, for each node, the integral feedback with variable gain so that it can remove or eliminate the interconnected parts, uncertainties and disturbances. Then we translate the problem of networked control into that of decentralized control. So we need only to consider the synthesis for each of the subsystems respectively without concerning the influence of the non-principal factors. Therefore, with little restrictions to the networked system and using simple control structure, we can get satisfactory control results even for systems which are with non-smooth components. The theoretical analysis and simulation results are shown to indicate the effectiveness of the proposed method.

Properties and changes in empirical orthogonal components of temperature fields in Northern Eurasia in the 20th century

Using data from 55 meteorological stations in the Northern Hemisphere for the 20th century as a whole and separately for 1901–1950 and 1951–2000, the norms and empirical orthogonal components (EOCs) of the fields of the annual mean surface air temperature were calculated. These fields were found to have two components: smooth and nonsmooth. The field of differences between the temperature norms of the first and second halves of the 20th century was found to be projected almost completely on the very first component of fixed sign; i.e., the current climate trend of the Northern Hemisphere is spatially homogeneous. No substantial changes were found in the transition from the first half to the second half of the 20th century, either in the form of components or in the spectrum of eigenvalues, with the only exception being a small growth (in magnitude) in part of the eigenvalues corresponding to the nonsmooth component. This can be explained by the progressing urbanization of northern Eurasia. All coefficients of the temperature-field expansion by natural components are distributed normally, and their temporal correlation functions for smooth components of fields have a form characteristic of processes with long-term memory. The latter manifests itself particularly in the motions of the two main waves of the smooth component and reflects two stages of the current climate warming.

Solving variational inequalities with stochastic mirror-prox algorithm

In this paper we consider iterative methods for stochastic variational inequalities (s.v.i.) with monotone operators. Our basic assumption is that the operator possesses both smooth and nonsmooth components. Further, only noisy observations of the problem data are available. We develop a novel Stochastic Mirror-Prox (SMP) algorithm for solving s.v.i. and show that with the convenient stepsize strategy it attains the optimal rates of convergence with respect to the problem parameters. We apply the SMP algorithm to Stochastic composite minimization and describe particular applications to Stochastic Semidefinite Feasibility problem and deterministic Eigenvalue minimization.

On a Certain Class of Expanding Systems

On a Certain Class of Expanding Systems The interesting properties of a class of expanding systems are discussed. The operation of the considered systems can be described as follows: the input signal is processed by a linear dynamic converter in subsequent time intervals, each of them is equal to Ti. Processing starts at the moments n · Ti, always after zeroing of converter initial conditions. For smooth input signals and a given transfer function of the converter one can suitably choose Ti and the gain coefficient in order to realize the postulated linear operations on input signals, which is quite different comparing it to the operation realized by the converter. The errors of postulated operations are mainly caused by non-smooth components of the input signal. The principles for choice of system parameters and rules for system optimization are presented in the paper. The referring examples are attached too.

ACCPM with a nonlinear constraint and an active set strategy to solve nonlinear multicommodity flow problems

This paper proposes an implementation of a constrained analytic center cutting plane method to solve nonlinear multicommodity flow problems. The new approach exploits the property that the objective of the Lagrangian dual problem has a smooth component with second order derivatives readily available in closed form. The cutting planes issued from the nonsmooth component and the epigraph set of the smooth component form a localization set that is endowed with a self-concordant augmented barrier. Our implementation uses an approximate analytic center associated with that barrier to query the oracle of the nonsmooth component. The paper also proposes an approximation scheme for the original objective. An active set strategy can be applied to the transformed problem: it reduces the dimension of the dual space and accelerates computations. The new approach solves huge instances with high accuracy. The method is compared to alternative approaches proposed in the literature.

On a class of non-smooth dynamical systems: a sufficient condition for smooth versus non-smooth solutions

In this paper we present a possible classification of the elements of a class of dynamical systems, whose underlying mathematical models contain non-smooth components. For this purpose a sufficient condition is introduced. To illustrate and motivate this classification, three nontrivial and realistic examples are considered.

Generalized Hyperinterpolation on the Sphere and the Newman—Shapiro Operators

Hyperinterpolation on the sphere, as introduced by Sloan in 1995, is a constructive approximation method which is favorable in comparison with interpolation, but still lacking in pointwise convergence in the uniform norm. For this reason we combine the idea of hyperinterpolation and of summation in a concept of generalized hyperinterpolation. This is no longer projectory, but convergent if a matrix method A is used which satisfies some assumptions. Especially we study A partial sums which are defined by some singular integral used by Newman and Shapiro in 1964 to derive a Jackson-type inequality on the sphere. We could prove in 1999 that this inequality is realized even by the corresponding discrete operators, which are generalized hyperinterpolation operators. In view of this result the Newman—Shapiro operators themselves gain new attention. Actually, in their case, A furnishes second-order approximation, which is best possible for positive operators. As an application we discuss a method for tomography, which reconstructs smooth and nonsmooth components at their adequate rate of convergence. However, it is an open question how second-order results can be obtained in the discrete case, this means in generalized hyperinterpolation itself, if results of this kind are possible at all.

The Analysis of Designed Experiments and Longitudinal Data by Using Smoothing Splines

SUMMARY In designed experiments and in particular longitudinal studies, the aim may be to assess the effect of a quantitative variable such as time on treatment effects. Modelling treatment effects can be complex in the presence of other sources of variation. Three examples are presented to illustrate an approach to analysis in such cases. The first example is a longitudinal experiment on the growth of cows under a factorial treatment structure where serial correlation and variance heterogeneity complicate the analysis. The second example involves the calibration of optical density and the concentration of a protein DNase in the presence of sampling variation and variance heterogeneity. The final example is a multienvironment agricultural field experiment in which a yield–seeding rate relationship is required for several varieties of lupins. Spatial variation within environments, heterogeneity between environments and variation between varieties all need to be incorporated in the analysis. In this paper, the cubic smoothing spline is used in conjunction with fixed and random effects, random coefficients and variance modelling to provide simultaneous modelling of trends and covariance structure. The key result that allows coherent and flexible empirical model building in complex situations is the linear mixed model representation of the cubic smoothing spline. An extension is proposed in which trend is partitioned into smooth and non-smooth components. Estimation and inference, the analysis of the three examples and a discussion of extensions and unresolved issues are also presented.

Collective coordinates for ion dynamics

Utilising a link between microfield smoothness and ion dynamics, a method for calculating the autocorrelation function of spectral line shapes in a plasma, including ion dynamics, is presented. The method is based on a time-independent simulation, in that the dynamics are handled semi-analytically via a separable kernel approximation in Dyson's equation. For each perturber configuration this reduces to the solution of a linear algebraic system. In addition this method naturally leads to a set of (approximate) relevant collective coordinates. It is shown that the microfield must be smooth and this implies the existence of new collective coordinates, namely the microfield values at a few time points. Nonsmooth components are separated out and treated analytically by the ion impact theory. This approach is valid for all parameter ranges and may be utilized either as an alternative to solving the Schrödinger equation in a simulation-like manner, or by using the collective coordinate approach. In the later case, knowledge of the many-times joint probability density is desirable.

Deterministic representation of chaos with application to turbulence

Chaos and unpredictability in some classical dynamic systems are eliminated by referring the governing equation to a specially selected rapidly oscillating (non-inertial) frame of reference in which the stabilization effect is caused by inertia forces. The resulting motion is found as a sum of smooth and non-smooth (rapidly oscillating) parts. The solution is stable and reproducible in the sense that small changes in initial conditions lead to small changes in both smooth and non-smooth components. In this interpretation, conceptually the closure problem in turbulence is reduced to the problem of finding such a frame of reference where the high Reynolds number instability is eliminated. The usefulness of the approach is illustrated by examples.