Smooth Objects Research Articles

An active-set with barrier method and trust-region mechanism to solve a nonlinear Bilevel programming problem

<abstract><p>Nonlinear Bilevel programming (NBLP) problem is a hard problem and very difficult to be resolved by using the classical method. In this paper, Karush-Kuhn-Tucker (KKT) condition is used with Fischer-Burmeister function to convert NBLP problem to an equivalent smooth single objective nonlinear programming (SONP) problem. An active-set strategy is used with Barrier method and trust-region technique to solve the smooth SONP problem effectively and guarantee a convergence to optimal solution from any starting point. A global convergence theory for the active-set barrier trust-region (ACBTR) algorithm is studied under five standard assumptions. An applications to mathematical programs are introduced to clarify the effectiveness of ACBTR algorithm. The results show that ACBTR algorithm is stable and capable of generating approximal optimal solution to the NBLP problem.</p></abstract>

Distributed Anytime-Feasible Resource Allocation Subject to Heterogeneous Time-Varying Delays

This paper considers distributed allocation strategies, formulated as a distributed sum-preserving (fixed-sum) allocation of resources over a multi-agent network in the presence of heterogeneous arbitrary time-varying delays. We propose a double time-scale scenario for unknown delays and a faster single time-scale scenario for known delays. Further, the links among the nodes are considered subject to certain nonlinearities (e.g, quantization and saturation/clipping). We discuss different models for nonlinearities and how they may affect the convergence, sum-preserving feasibility constraint, and solution optimality over general weight-balanced uniformly strongly connected networks and, further, time-delayed undirected networks. Our proposed scheme works in a variety of applications with general non-quadratic strongly-convex smooth objective functions. The non-quadratic part, for example, can be due to additive convex penalty or barrier functions to address the local box constraints. The network can change over time, is not necessarily connected at all times, but is only assumed to be uniformly-connected. The novelty of this work is to address all-time feasible Laplacian gradient solutions in presence of nonlinearities, switching digraph topology (not necessarily all-time connected), and heterogeneous time-varying delays.

Improving the Transient Times for Distributed Stochastic Gradient Methods

We consider the distributed optimization problem where <inline-formula><tex-math notation="LaTeX">$n$</tex-math></inline-formula> agents, each possessing a local cost function, collaboratively minimize the average of the <inline-formula><tex-math notation="LaTeX">$n$</tex-math></inline-formula> cost functions over a connected network. Assuming stochastic gradient information is available, we study a distributed stochastic gradient algorithm, called exact diffusion with adaptive stepsizes (EDAS) adapted from the Exact Diffusion method [1] and NIDS [2] and perform a non-asymptotic convergence analysis. We not only show that EDAS asymptotically achieves the same network independent convergence rate as centralized stochastic gradient descent (SGD) for minimizing strongly convex and smooth objective functions, but also characterize the transient time needed for the algorithm to approach the asymptotic convergence rate, which behaves as <inline-formula><tex-math notation="LaTeX">$K_{T}=\mathcal {O}(\frac{n}{1-\lambda _{2}})$</tex-math></inline-formula>, where <inline-formula><tex-math notation="LaTeX">$1-\lambda _{2}$</tex-math></inline-formula> stands for the spectral gap of the mixing matrix. To the best of our knowledge, EDAS achieves the shortest transient time when the average of the <inline-formula><tex-math notation="LaTeX">$n$</tex-math></inline-formula> cost functions is strongly convex and each cost function is smooth. Numerical simulations further corroborate and strengthen the obtained theoretical results.

Global attractor and limit points for nonsmooth ADMM

The alternating direction method of multipliers (ADMM) is frequently used to solve optimization problems derived from machine learning and statistics. Hence it is important to investigate the asymptotic behavior of ADMM. Most works in this direction deal with smooth objective functions excluding various important applications in the nonsmooth case. In this paper, we study the asymptotic behavior of the solutions of differential inclusions arising from ADMM applied to nonsmooth objective functions. More precisely, we associate a multiflow to this differential inclusion and analyze the structure of both ω-limit points and global attractors for this multiflow.

Compressed Gradient Tracking for Decentralized Optimization Over General Directed Networks

In this paper, we propose two communication-efficient decentralized optimization algorithms over a general directed multi-agent network. The first algorithm, termed Compressed Push-Pull (CPP), combines the gradient tracking Push-Pull method with communication compression. We show that CPP is applicable to a general class of unbiased compression operators and achieves linear convergence rate for strongly convex and smooth objective functions. The second algorithm is a broadcast-like version of CPP (B-CPP), and it also achieves linear convergence rate under the same conditions on the objective functions. B-CPP can be applied in an asynchronous broadcast setting and further reduce communication costs compared to CPP. Numerical experiments complement the theoretical analysis and confirm the effectiveness of the proposed methods.

A Semisupervised CRF Model for CNN-Based Semantic Segmentation With Sparse Ground Truth

Convolutional neural networks (CNNs) represent the new reference approach for semantic segmentation of very-high-resolution (VHR) images, due to their ability to automatically capture semantic information while learning relevant features. However, as for most supervised methods, the map accuracy depends on the quantity and quality of ground truth (GT) used to train them. The use of densely annotated data (i.e., a detailed, exhaustive, pixel-level GT) allows to obtain effective CNN models but normally implies high efforts in annotation. Such ground truth is often available in benchmark datasets on which new methods are tested, but not on real data for land-cover applications, where only sparse annotations might be sufficiently cost effective. A CNN model trained with such incomplete GT maps has the tendency to smooth object boundaries because they are never precisely delineated in the GT. To cope with those shortcomings, we propose to exploit the intermediate activation maps of the CNN and to deploy a semisupervised fully connected conditional random field (CRF). In comparison with competitors using the same sparse annotations, the proposed method is able to better fill part of the performance gap compared to a CNN trained on the densely annotated, but generally unavailable, GTs.

The notion of fractals in tumour angiogenic sprout initiation model based on cellular automata

Tumour growth is considered chaotic, poorly controlled growth as its cells and vasculatures are irregular in shape. The vascular system of tumour is a complex network and its architectural complexity and geometrical characteristics cannot be defined with Euclidian geometry as it is only capable of expressing regular or smooth objects whereas, fractal geometry describes an object with irregularities and it is therefore well suited to quantify those morphological characteristics. In this paper, we develop a cellular automata (CA) based discrete model that tries to mimic all the features and make useful predictions of tumour angiogenesis with computer-coded rules. From the interactions of several species in 2-D tissue space, we study the effects of these various biological factors on the formation of the capillary sprouts. The model also captures anastomoses and branching phenomena in capillary blood vessels. It is observed that chemotaxis and randomness are the essential components for the capillary sprout motility whereas, haptotaxis works as an add-on feature. The model is also validated with partial sensitivity analysis. The fractal properties of the growing capillary sprouts in the model are investigated. The characterization of these fractal objects (capillary sprouts) is provided by the box-counting method. The model shows, the morphological change in the early stages of the vascularization process where the capillary sprouts are formed from the surrounding blood vessels. The model successfully simulates the morphological aspects of capillary vessel growth during tumour angiogenesis. The estimated fractal dimension is very close to the previously measured range. This is established our hypothesis about the fractal nature of vascular network pattern and our CA-based discrete model for capillary sprout initiation during tumour angiogenesis.

A novel algorithm of the digital nervous tissue phantom creation based on 3D Voronoi diagram application

The essential part of mathematical modelling of nutrients convectional reaction-diffusion is creation of a digital phantom of considered biological object. This process becomes an especial problem which needs to be solved before numerical calculations of the concentration gradients will be done. There are two principal ways to get the solution in this case. The first approach is the reconstruction of a digital phantom on the base of the experimental data directly. The second one is the creation of a virtual object according to the experimental evidence and the known principals de novo. The main advantage of the created phantom is a high adaptability to modelling demands and a physical problem formulation. In the present study a new algorithm of a digital phantom creation has been established. The principles of the claimed procedures are demonstrated by the example of a nervous tissue. Initially, one needs to create N 3D objects according to Voronoi diagrams. Each object has 144 edges and 69 boundaries on average. Having chosen M rear objects, a long 3D structure mimicking neurons axons is created according to a loft procedure from the start boundaries to the end ones. Then, the set of Boolean operations has been applied to form continuous smooth objects. The remain (N-(M+s)) objects are combined into several whole bodies using the loft procedures between the closet neighbours. The final structure has a good conformity with a nervous tissue architecture. Furthermore, the obtained phantom is correct to the mesh application and further numerical calculations.

Determination of the refractive index profile and surface topography of optically smooth objects using interference of optical vortices

In this paper, we present the results of the propagational dynamics of vortex beams in the scope of their possible applications for interferometric non-contact robust and precision optical surface profilometry with nanoscale longitudinal resolution. The result of coaxial superposition of the reference plane wave with singly charged vortex beams represents a dynamically changing intensity distribution. The nature of this changes, namely, rotational effects of intensity zeros, allows to determine directly the optical path difference which is introduced by the surfaces and internal structure of test object. We have proposed the experimental setup for examination of reflecting and transmitting objects.

On Seven Fundamental Optimization Challenges in Machine Learning

Many recent successes of machine learning went hand in hand with advances in optimization. The exchange of ideas between these fields has worked both ways, with machine learning building on standard optimization procedures such as gradient descent, as well as with new directions in the optimization theory stemming from machine learning applications. In this thesis, we discuss new developments in optimization inspired by the needs and practice of machine learning, federated learning, and data science. In particular, we consider seven key challenges of mathematical optimization and develop a solution to each. Our first contribution is the resolution of a key open problem in Federated Learning: we establish the first theoretical guarantees for the famous Local SGD algorithm in the heterogeneous data regime. As the second challenge, we close the gap between the upper and lower bounds for the theory of two algorithms known as Random Reshuffling (RR) and Shuffle-Once that are widely used in practice, and set as the default data selection strategies for SGD in modern machine learning software. Our third contribution can be seen as a combination of our new theory for proximal RR and Local SGD yielding a new algorithm, which we call FedRR. Unlike Local SGD, FedRR can provably beat gradient descent in communication complexity in the heterogeneous data regime. The fourth challenge is related to the class of adaptive methods. In particular, we present the first parameter-free stepsize rule for gradient descent that provably works for any locally smooth convex objective. The fifth challenge is the development of an algorithm for distributed optimization with quantized updates that preserves linear convergence of gradient descent. Finally, in our sixth and seventh challenges, we develop new VR mechanisms applicable to the non-smooth setting based on proximal operators.

A new hybrid method between equivalent source method and boundary element method for modeling diffraction

The Equivalent Source Method (ESM) allows to model the acoustical radiation of an object with a set of monopole sources inside the object. This method efficiently describes acoustical radiation of an object with smooth surface, but is not performant for addressing diffraction induced by sharp edge. In this study, we use the simple source formulation to establish ESM as an integral formulation. In this form, ESM belong to the Boundary Element Method (BEM) family, with an implicit mesh. Therefore, we propose an alternative method similar to ESM with sources on the surface object, as the singularities can now be handled with the integral formulation. This new method is more performant than ESM for modeling diffraction, and is easier to implement than standard BEM based on Gauss Quadrature while it still takes advantage of the ESM implicit mesh. Comparisons of this hybrid method with ESM and standard BEM approaches show that the developed method is a good compromise between the two other approaches: the new method is more accurate than BEM, but less than ESM for a smooth object; while it is equivalent to BEM and more accurate than ESM for an edgy object. Work Funded by AMIDEX foundation.

Load Sharing Design of a Multi-legged Adaptable Gripper With Gecko-Inspired Controllable Adhesion

Gecko-inspired controllable dry adhesion has promising applications in various fields such as transfer printing, advanced robotics, and space technology. In this study, we proposed a multi-legged self-adaptive gripper based on gecko-inspired controllable adhesion to manipulate both flat and curved objects. It consists of four symmetric adhesive units, each modulated by two adaptive-locking mechanisms for compression and rotation, respectively, and one peeling mechanism. The two adaptive-locking mechanisms adapt to surfaces with height and curvature differences to ensure intimate contact and then lock the adaption configuration to enable equal load sharing for strong attachment. The peeling mechanism rapidly peels the adhesive surfaces from the substrate for easy detachment. Therefore, this gripper can achieve controllable load sharing to reliably grasp and easily release smooth objects with various surface shapes. Furthermore, the effects of object posture and structural parameters on its grasping performance were analyzed, which can guide its optimal design by improving equal load sharing among adhesive units. This work provides an equal load-sharing strategy for multi-unit adhesive system design and demonstrates its potential for emerging industrial applications.

Про пришвидшення оптимізаційних методів для задачі синтезу багатошарових оптичних покриттів

Розглянуто п’ять способів для пришвидшення багатовимірного пошуку розв’язку задачі синтезу багатошарових оптичних покриттів за допомогою методів нульового та першого порядків. Перший спосіб — це використання аналітичної похідної для цільової функції якості багатошарового покриття. Він дозволяє точно (у межах комп’ютерної арифметики) обчислити значення градієнта гладкої цільової функції та узагальненого градієнта негладкої цільової функції. Перший спосіб потребує таку ж кількість арифметичних операцій, як і скінченно-різницеві способи обчислення градієнта та узагальненого градієнта. Другий спосіб — це використання пришвидшеного знаходження градієнта цільової функції за допомогою використання префікс- та суфікс-масивів у аналітичному способі обчислення градієнта. Цей прийом дозволяє зменшити кількість арифметичних операцій втричі для задач великої розмірності. Третій спосіб — це використання табуляції значень тригонометричних функцій для обчислення характеристичних матриць. Цей прийом зменшує час виконання операцій множення характеристичних матриць у десятки разів в залежності від характеристик комп’ютера. Для деяких архітектур комп’ютера ця перевага становить більше ніж 140 разів. Четвертий спосіб — це використання методу золотого перерізу для одновимірної оптимізації в задачах синтезу оптичних покриттів. Зокрема, при розв’язанні однієї часткової задачі показано, що метод тернарного пошуку потребує приблизно на 40 % більше часових затрат, ніж метод золотого перерізу. П’ятий спосіб — це використання ефективної реалізації множення двох матриць. Вона полягає у зміні порядку другого і третього циклів для загальновідомого методу множення двох матриць та фіксації у звичайній змінній значення елемента першої матриці. Це дозволяє суттєво прискорити виконання операції множення двох матриць. Для матриць розмірності 1000×1000 придшвидшення складає від 2 до 15 разів — залежно від характеристик комп’ютера.

SQuARM-SGD: Communication-Efficient Momentum SGD for Decentralized Optimization

In this paper, we propose and analyze SQuARM-SGD, a communication-efficient algorithm for decentralized training of large-scale machine learning models over a network. In SQuARM-SGD, each node performs a fixed number of local SGD steps using Nesterov's momentum and then sends sparsified and quantized updates to its neighbors regulated by a locally computable triggering criterion. We provide convergence guarantees of our algorithm for general (non-convex) and convex smooth objectives, which, to the best of our knowledge, is the first theoretical analysis for compressed decentralized SGD with momentum updates. We show that the convergence rate of SQuARM-SGD matches that of vanilla SGD. We empirically show that including momentum updates in SQuARM-SGD can lead to better test performance than the current state-of-the-art which does not consider momentum updates.

Preconditioned Accelerated Gradient Descent Methods for Locally Lipschitz Smooth Objectives with Applications to the Solution of Nonlinear PDEs

We develop a theoretical foundation for the application of Nesterov’s accelerated gradient descent method (AGD) to the approximation of solutions of a wide class of partial differential equations (PDEs). This is achieved by proving the existence of an invariant set and exponential convergence rates when its preconditioned version (PAGD) is applied to minimize locally Lipschitz smooth, strongly convex objective functionals. We introduce a second-order ordinary differential equation (ODE) with a preconditioner built-in and show that PAGD is an explicit time-discretization of this ODE, which requires a natural time step restriction for energy stability. At the continuous time level, we show an exponential convergence of the ODE solution to its steady state using a simple energy argument. At the discrete level, assuming the aforementioned step size restriction, the existence of an invariant set is proved and a matching exponential rate of convergence of the PAGD scheme is derived by mimicking the energy argument and the convergence at the continuous level. Applications of the PAGD method to numerical PDEs are demonstrated with certain nonlinear elliptic PDEs using pseudo-spectral methods for spatial discretization, and several numerical experiments are conducted. The results confirm the global geometric and mesh size-independent convergence of the PAGD method, with an accelerated rate that is improved over the preconditioned gradient descent (PGD) method.

A fluid-supported 3D hydrogel bioprinting method

Hydrogels are used in many biomedical applications, including regenerative medicine and surgical training phantoms. However, the ability to shape these materials into complex anatomical structures using additive manufacturing is limited in part by their low mechanical stiffness. We developed a hydrogel 3D printer, that projects patterns directly onto a thin layer of fluid-supported hydrogel precursor, which serves as a floating, liquid projection screen. This approach avoids inadvertent adhesion that affects typical resin-based 3D printers, and enables fast, continuous printing. As a consequence, we can print smooth objects free of layering artifacts, at rates of 200 mm/h along the Z-axis. We demonstrate the versatility of our approach by printing various complex structures, including free-standing channel networks with 500 μm-thick walls, using hydrogels with a wide range of stiffness from 7 kPa to more than 4 MPa. Lastly, because the printer features a free surface, we combined it with an extruder to perform multi-material printing. We use this strategy to create centimeter-scale, cell-laden hydrogels containing channels, that help address the key nutrient supply problem in bioprinting.

Asymptotics for M-type smoothing splines with non-smooth objective functions

M-type smoothing splines are a broad class of spline estimators that include the popular least-squares smoothing spline but also spline estimators that are less susceptible to outlying observations and model misspecification. However, available asymptotic theory only covers smoothing spline estimators based on smooth objective functions and consequently leaves out frequently used resistant estimators such as quantile and Huber-type smoothing splines. We provide a general treatment in this paper and, assuming only the convexity of the objective function, show that the least-squares (super-)convergence rates can be extended to M-type estimators whose asymptotic properties have not been hitherto described. We further show that auxiliary scale estimates may be handled under significantly weaker assumptions than those found in the literature and we establish optimal rates of convergence for the derivatives, which have not been obtained outside the least-squares framework. A simulation study and a real-data example illustrate the competitive performance of non-smooth M-type splines in relation to the least-squares spline on regular data and their superior performance on data that contain anomalies.

Asynchronous Delayed Optimization With Time-Varying Minibatches

Large-scale learning and optimization problems are often solved in parallel. In a master-worker distributed setup, worker nodes are most often assigned fixed-sized minibatches of data points to process. However, workers may take different amounts of time to complete their per-batch calculations. To deal with such variability in processing times, an alternative approach has recently been proposed wherein each worker is assigned a fixed duration to complete the calculations associated with each batch. This fixed-time approach results in time-varying minibatch sizes and has been shown to outperform the fixed minibatch approach in synchronous optimization. In this paper we make a number of contributions in the analysis and experimental verification of such systems. First, we formally present a system model of an asynchronous optimization scheme with variable-sized minibatches and derive the expected minibatch size. Second, we show that for our fixed-time asynchronous approach, the expected gradient staleness does not depend on the number of workers contrary to existing schemes. Third, we prove that for convex smooth objective functions the asynchronous variable minibatch method achieves the optimal regret and optimality gap bounds. Finally, we run experiments comparing the performances of the asynchronous fixed-time and fixed-minibatch methods. We present results for CIFAR-10 and ImageNet.

Game-Based Multiobjective Optimization of Suspension System for In-Wheel Motor Drive Electric Vehicle

Since the driving motor is embedded in the wheel, the unsprung mass and the wheel rotational inertia of the in-wheel motor drive electric vehicle both increase, which not only affect the vehicle smoothness but also worsen the motor’s working condition due to its own vertical vibration. The evaluation index of in-wheel motor’s vertical vibration is introduced on the basis of vehicle smoothness analysis. The parameters’ optimization of vibration absorber and suspension are carried out, respectively, and the optimization results show the contradictory relationship between smoothness objective and the motor’s vertical vibration acceleration objective. Regarding the contradictory indices of the smoothness and the motor’s vertical acceleration as the objective function, a multiobjective optimization scheme is designed. Then, the orthogonal experimentation and fuzzy clustering method are applied in the multiobjective optimized design based on the game decision analysis, and the Nash equilibrium and cooperative competition game theory are used to optimize the parameters of suspension and vibration absorber. The optimized results verify the game relation between the optimization variables, the game optimization obtains better optimized results than traditional linear weight sum method, and the in-wheel motor functional stability and the vehicle smoothness can be both achieved. Compared with the traditional complex iterative process and the manmade preassurance weight allocation, the game optimization has the advantages of less iterations, faster convergence, and less influence by human experience.

Smoothing inertial neurodynamic approach for sparse signal reconstruction via [formula omitted]-norm minimization

In this paper, we propose a smoothing inertial neurodynamic approach (SINA) which is used to deal with Lp-norm minimization problem to reconstruct sparse signals. Note that the considered optimization problem is nonsmooth, nonconvex and non-Lipschitz. First, the problem is transformed into a smooth optimization problem based on smoothing approximation method, and the Lipschitz property of gradient of the smooth objective function is discussed. Then, SINA based on Karush–Kuhn–Tucker (KKT) condition, smoothing approximation and inertial dynamical approach, is designed to handle smooth optimization problem. The existence, uniqueness, global convergence and optimality of the solution of the SINA are discussed by the Cauchy–Lipschitz–Picard theorem, energy function and KKT condition. In addition, for p=1, the SINA has a mean sublinear convergence rate O1∕t under some mild conditions. Finally, some numerical examples on sparse signal reconstruction and image restoration are given to illustrate the theoretical results and the efficiency of SINA.