Decreasing Step Size Research Articles

Linear convergence of decentralized estimation for statistical estimation using gradient method

There has been a growing interest in solving consensus optimization problems in a multi-agent system. It is known that there is an exactness-speed dilemma for decentralized optimization for the naive gradient method, where either we have fast linear convergence to a neighborhood of size O(η) using a fixed constant step size η, or slow convergence to the exact optimizer with a decreasing step size. We reinvestigate a special case of this problem from a statistical point of view, where each agent’s local function is a sample estimate of a common population function. It is shown that when considering the convergence to the population target instead of to the exact consensual solution, the convergence is linear up to the statistical accuracy of the problem, using a step size only depending on the network characteristics instead of the sample size N. The objective function is not required to be strongly convex (but their statistical limit is), which allows for example the absolute deviation function which is used in median regression. Such results are in stark contrast to the optimization literature, where strong convexity of the objective function is necessary for linear convergence.

Orbit-attitude-structure coupled modelling method in local translational coordinate frame for multibody systems

The absolute coordinate-based (ACB) method, including absolute nodal coordinate formulation and natural coordinate formulation, is a widely-used dynamic modelling method for multibody systems. However, the problem of accuracy loss would arise when multi-scale motions are described in the same global coordinate system, such as large-scale orbital motions, small-scale attitude motions and structural vibrations, and micro-scale contact/collision. To address this problem, an ACB method in local translational coordinate system is proposed such that the large-scale orbital motions can be separated from other smaller-scale motions. The dynamic equations are derived by the Hamilton's equations and the Kronecker product. The validity of the proposed method is studied using an orbit-attitude coupled model of an orbiting rigid plate. It is found that the attitude errors of the original ACB method increase as time step size decreases. The reason for this counterintuitive phenomenon is that more serious accuracy loss is encountered for smaller time step size. In addition, more iterations are required for the original ACB method to obtain the prescribed tolerable error in each time step due to accuracy loss. In comparison, the proposed method is more accurate and efficient in numerical simulations. Finally, the proposed method is applied to the orbit-attitude-structure coupled simulation of a space robotic system.

Sequential Regularization Method for the Identification of Mold Heat Flux during Continuous Casting Using Inverse Problem Solutions Techniques

A two-dimensional transient inverse heat-conduction problem (2DIHCP) was established to determine the mold heat flux using observed temperatures. The sequential regularization method (SRM) was used with zeroth-, first-, and second-order spatial regularization to solve the 2DIHCP. The accuracy of the 2DIHCP was investigated under two strict test conditions (Case 1: heat flux with time-spatial periodically varying, and Case 2: that with sharp variations). The effects of the number of future time steps, regularization parameters, order of regularization, discrete grids, and time step size on the accuracy of the 2DIHCP were analyzed. The results showed that the minimum relative error (epred) of the predicted Case 1 heat flux is 5.05%, 5.39%, and 5.88% for zeroth-, first-, and second-order spatial regularization, respectively. The corresponding values for the predicted Case 2 heat flux are 6.31%, 6.30%, and 6.36%. Notably, zeroth- and first-order spatial regularization had higher accuracy than second-order spatial regularization, while zeroth-order spatial regularization was comparable to first-order. Additionally, first-order spatial regularization was more accurate in reconstructing heat flux containing sharp spatial variations. The CPU time of the predicted Case 2 heat flux is 1.71, 1.71, and 1.70 s for zeroth-, first-, and second-order spatial regularization, respectively. The corresponding values for the predicted Case 1 heat flux are 6.18, 6.15, and 6.17 s. It is noteworthy that the choice of spatial regularization order does not significantly impact the required computing time. Lastly, the minimum epred of Case 2 heat flux with zeroth-order spatial regularization is 7.96%, 6.42%, and 7.87% for time step sizes of 1/fs, 1/2fs, and 1/5fs, respectively. The accuracy of the inverse analysis displays an initial improvement followed by degradation as the time step size decreases. A recommended time step size is 1/2fs, where fs denotes the temperature-sampling rate.

Formability and Dimensional Accuracy Enhancement with Controlled Tool Rotational Speed in Stretch Forming Process

We present a new stretch forming method for axisymmetric sheet metal parts production. A test bench is designed, fabricated, and mounted on a drilling machine. The blank is fixed rigidly around its periphery in the apparatus, and the forming tool is mounted on the machine spindle. Thus, the control of tool rotational speed, feed rate, and vertical motion from the upper sheet surface becomes possible. Experimental tests are then conducted to form a reference shape. Rotational speed, feed rate, lubrication effect on the final thickness distribution, and geometric profiles are studied. A new multi-pass forming method is also developed and verified. The feed rate decrease, within the range of 0.11–0.18[Formula: see text]mm/rev, reduces the sheet thinning, while its increase improves the dimensional accuracy. The rotational speed increase, within the range of 112–710[Formula: see text]rpm, reduces the thickness, while its decrease enhances the geometrical accuracy. Mineral oil seems to be more effective than greases, so the thickness drop is reduced and the dimensional accuracy is improved. The use of the multi-pass strategy avoids the sheet cracking at great forming depth. The decrease in step size, within the range of 0.3–1[Formula: see text]mm, reduces the thickness minima, while good dimensional accuracy can be obtained in the range of 0.5–1[Formula: see text]mm.

New subgradient extragradient algorithm for solving variational inequalities in Hadamard manifold

This paper introduces a schematic approximation method for a solution to a variational inequality problem in the framework of Hadamard manifold. The method is a combination of the subgradient extragradient technique and Popov extragradient method. Using this method, some convergence algorithms were proved when the cost operators are pseudomonotone and strongly pseudomonotone, respectively. In the construction of this method, the dependence on Lipschitz constants of the operators is dispensed with by the use of a monotone decreasing step size. We give an application of our main result to the constrained convex minimization problem. Finally, we report some numerical examples to illustrate the efficiency and applicability of the method.

Superconvergent postprocessing of the discontinuous Galerkin time stepping method for nonlinear Volterra integro-differential equations

We propose a very simple but efficient postprocessing technique for improving the global accuracy of the discontinuous Galerkin (DG) time stepping method for solving nonlinear Volterra integro-differential equations. The key idea of the postprocessing technique is to add a higher order generalized Jacobi polynomial of degree k+1 with parameters (−1,0) to the DG approximation of degree k. We prove that the postprocessed DG approximations converge one order faster than the unprocessed DG approximations in the L2-, H1- and L∞-norms. Based on these postprocessed superconvergence results, we construct several a posteriori error estimators and prove that they are asymptotically exact as the step-size decreases. Numerical experiments are carried out to verify the theoretical results.

Artificial Intelligence-Enhanced UUV Actuator Control

This manuscript compares deterministic artificial intelligence to a model-following control applied to DC motor control, including an evaluation of the threshold computation rate to let unmanned underwater vehicles correctly follow the challenging discontinuous square wave command signal. The approaches presented in the main text are validated by simulations in MATLAB®, where the motor process is discretized at multiple step sizes, which is inversely proportional to the computation rate. Performance is compared to canonical benchmarks that are evaluated by the error mean and standard deviation. With a large step size, discrete deterministic artificial intelligence shows a larger error mean than the model-following self-turning regulator approach (the selected benchmark). However, the performance improves with a decreasing step size. The error mean is close to the continuous deterministic artificial intelligence when the step size is reduced to 0.2 s, which means that the computation rate and the sampling period restrict discrete deterministic artificial intelligence. In that case, continuous deterministic artificial intelligence is the most feasible and reliable selection for future applications on unmanned underwater vehicles, since it is superior to all the approaches investigated at multiple computation rates.

Monte Carlo analysis of focused Gaussian beam in scattering media: Curvature correction and Mie scattering

We introduce curvature correction into the Monte Carlo (MC) technique to determine the suitable photon-step size and initial photon distribution in the focusing lens aperture, that can account for the effects of the radius of curvature on the axial intensity profile of a focused Gaussian beam that is propagated through a scattering medium. The scattering anisotropy of the medium is determined via the Mie scattering theory for given values of the scattering particle radius, density, and phase distribution values while optical ray tracing is used to determine the occurrence of scattering events. We evaluate the performance of the modified MC technique for random and periodic media by examining the axial intensity profiles of the propagating focused beam. Relative to the predictions of the scalar diffraction theory, curvature correction improves the accuracy of the MC results with decreasing step size s and increasing number of steps Ns. In the absence of scattering, the axial beam distribution approaches that of the scalar diffraction theory for a given numerical aperture (NA≤0.5), s, and Ns based on the Linfoot’s image quality criteria of fidelity, correlation quality and structural content.

Federated learning for minimizing nonsmooth convex loss functions

Federated learning possesses a distributed learning framework for protecting privacy where local clients need to collaboratively train a shared model via a central server. Many existing methods in the literature of federated learning are analyzed under some smoothness conditions of loss functions and need access to gradient information to perform local optimization algorithms such as stochastic gradient descent or dual average. Some methods even need a strong convexity of the loss function. However, in many situations of real world applications such as readability of texts, first-order gradient information is difficult to obtain, and the strong smoothness and strong convexity of loss functions are not satisfied. We consider methods to overcome such situations. This paper aims at providing an understanding of federated learning in the situation that the loss functions are nonsmooth and gradient information is unavailable, also a strong convexity condition is not needed. Based on Nesterov's zeroth-order (gradient-free) techniques, we propose a zeroth-order stochastic federated learning method. Constant and decreasing step size strategies are considered. Moreover, a new type of approximating sequence is proposed in federated learning for strictly decreasing step sizes. Expected error bounds for the proposed approximating sequence and learning rates of the proposed method are derived under some selection rules of the step sizes and smoothing parameters.

Evaluation of scanning resolution, detector choice and detector orientation to be used for accurate and time-efficient commissioning of a 6MV clinical linear accelerator

The present study is aimed at exploring different scanning parameters, detectors and their orientations for time-efficient and accurate commissioning of a 6 MV clinical linear accelerator (LINAC). Beam profiles and percentage depth dose (PDD) curves were measured with a PTW dosimetry diode, a PTW Semiflex and a PinPoint ion chamber in different orientations. To acquire beam data, equidistant (step size of 0.5mm, 1mm, 2mm and 3mm) and fanline (step size of 2-0.5mm, 2-1mm, 3-0.5mm and 3-1mm) scanning modes were employed and data measurement time was recorded. Scan time per measurement point was also varied (0.2s, 0.5s and 1.0s) to investigate its effect on the accuracy and acquisition time of beam data. Accuracy of the measured data was analyzed on the basis of the variation between measured data and data modeled by a treatment planning system. Beam profiles (particularly in penumbra region) were found to be sensitive to variation in scanning resolution and showed an improved accuracy with decrease in step size, while PDD curves were affected negligibly. The accuracy of beam data obtained with the PTW dosimetry diode and the PinPoint ion chamber was higher than those obtained with the PTW Semiflex ion chamber for small fields (2 × 2cm2 and 3 × 3cm2). However, the response of the PTW diode and the PinPoint ion chamber was significantly indifferent in these fields. Furthermore, axial orientation of the PTW Semiflex ion chamber improved accuracy of profiles and PDDs as compared to radial orientation, while such a difference was not significant for the PinPoint ion chamber. It is concluded that a scan time of 0.2s/point with a fanline scanning resolution of 2-1mm for beam profiles and 3mm for PDDs are most favorable in terms of accuracy and time efficiency. For small fields (2 × 2cm2 and 3 × 3cm2), a PinPoint ion chamber in radial orientation or a dosimetry diode in axial orientation are recommended for both beam profiles and PDDs. If a PinPoint ion chamber and a PTW dosimetry diode are not available, a Semiflex ion chamber in axial orientation may be used for small fields.

Effective computational discretization scheme for nonlinear dynamical systems

Numerical methods are essential to investigate and apply nonlinear continuous-time dynamical systems in many fields of science and engineering and discretization schemes are required to obtain the solutions of such dynamical systems. Although computing power has been speedily growing in recent decades, embedded and large-scale problems have motivated significant research to improve the computational efficiency. Nevertheless, few studies have focused on finite precision limitation on discretization schemes due to round-off effects in floating-point number representation. In this paper, a computational effective discretization scheme for nonlinear dynamical systems is introduced. By means of a theorem, it is shown that high-order terms in the Runge-Kutta method can be neglected with no accuracy loss. The proposed approach is illustrated using three well-known systems, namely the Rössler systems, the Lorenz equations and the Sprott B system. The number of mathematical operations and simulation time have reduced up to 81.1% and 90.7%, respectively. Furthermore, as the step-size decreases, the number of neglected terms increases due to the precision of the computer. Yet, accuracy, observability of dynamical systems and the largest Lyapunov are preserved. The adapted scheme is effective, reliable and suitable for embedded and large-scale applications.

Robust echo state network with sparse online learning

Echo state network (ESN) is an effective tool for nonlinear systems modeling. To handle irregular noises or outliers in practical systems and alleviate the overfitting issue, the robust echo state network with sparse online learning (RESN-SOL) is proposed. Firstly, the ε-insensitive loss function is introduced to replace the commonly used quadratic loss function, which is theoretically optimal for Gaussian noise distribution. Secondly, the online gradient descent algorithm is used to calculate the network readout. Notably, the better learning performance can be achieved by the constant learning rate rather than the decreasing step size. Based on this observation, the sparse online learning algorithm (SOL) is proposed, in which the constant step size is used. Particularly, the SOL is able to truncate the small weights in network readout to zero for achieving sparsity. Furthermore, the convergence of RESN-SOL is theoretically analyzed, which implies the tradeoff between learning performance and readout sparsity can be controlled by a predefined sparsity parameter. Finally, the proposed method is verified in two simulated benchmarks and an actual dynamical wastewater treatment system. Experimental results demonstrate that the RESN-SOL exhibits better robustness against outliers, network compactness and modeling accuracy than other existing algorithms.

Optimizing Neural Network Model Performance for Wind Energy Forecasting

High variability and intermittency of wind create difficulties in managing and optimizing wind farms. Short-term forecasts are essential for a power plant’s safe operation. The aim of this work was to develop an efficient model for forecasting wind energy in the short term using machine learning and metaheuristics methods. The study improved a fruit Fly Optimization Algorithm (FOA) with decreasing step size to enhance the forecasting accuracy of the backpropagation neural network and radial basis function neural network. The efficiencies of the proposed methods were evaluated by comparing the values of the mean absolute percentage error, the root-mean-square error, and the standard deviation error. It was found that the optimized models demonstrate the high efficiency of forecasting in comparison with actual meteorological data. The error estimation analysis showed that the error values for the optimized models are 4-5 times lower than those for the same models without optimization. It has been shown that FOA with decreasing step size for neural network improves accuracy and computational speed for short-term wind energy forecasts. This approach can be applied in programs for real wind farms and studied for other network parameters, such as weights and offsets.

On stochastic mirror descent with interacting particles: Convergence properties and variance reduction

An open problem in optimization with noisy information is the computation of an exact minimizer that is independent of the amount of noise. A standard practice in stochastic approximation algorithms is to use a decreasing step-size. This however leads to a slower convergence. A second alternative is to use a fixed step-size and run independent replicas of the algorithm and average these. A third option is to run replicas of the algorithm and allow them to interact. It is unclear which of these options works best. To address this question, we reduce the problem of the computation of an exact minimizer with noisy gradient information to the study of stochastic mirror descent with interacting particles. We study the convergence of stochastic mirror descent and make explicit the tradeoffs between communication and variance reduction. We provide theoretical and numerical evidence to suggest that interaction helps to improve convergence and reduce the variance of the estimate.

Research on Maritime Emergency Material Dispatch Based on Beetle Antennae Search Algorithm

The research on dispatching of emergency supplies for maritime emergencies is the key to improve the use efficiency of marine emergency resources and the success rate of search and rescue. In view of the characteristics of emergency accidents at sea, in this paper, a material scheduling model with the shortest time is built for multi-accident points, multi-ship dispatching and multi-material supply points, and the optimal material scheduling scheme is obtained. The step-size updating coefficient eta is introduced into the variable step-size optimization search algorithm ( BAS ), so that the step-size decreases with the depth of optimization, which improves the searching precision and accuracy of the search algorithm. The simulation experiment verifies that this algorithm can solve the optimization scheme of material scheduling quickly and efficiently.

A Golden Ratio Algorithm for Solving Nonmonotone Variational Inequalities

A gradient projection method was considered for solving variational inequalities, and a golden ratio gradient algorithm for solving nonmonotonic mapping was given. The characteristics of the algorithm combine those of the inertial acceleration method, without the knowledge of the mapping’s Lipschitz constant but with a nonmonotonically decreasing step size. Under suitable assumptions, the convergence of the algorithm was proved. Finally, numerical experiments were given.

A Broyden–Fletcher–Goldfarb–Shanno algorithm for reliability-based design optimization

Classic mean value methods of performance measure approach in reliability-based design optimization may become computationally challenging due to their instability and inefficiency as the degree of nonlinearity grows in problems. In this paper, sequential quadratic programming method associated with the Broyden–Fletcher–Goldfarb–Shanno algorithm is introduced to take information about the curvature of the performance function into account and determine the search direction of the minimum performance target point within the performance measure approach. A new mean value method of performance measure approach is developed in accordance with the search direction defined by a technique akin to the non-monotone line search of globalization strategy in the unconstrained optimization problem. The minimum performance target point, on the one hand, is rapidly located in the local domain along the search direction by the steepest descend method, and on the other hand, globally explored along a direction from the origin with exponentially decreasing step sizes. The behavior of the proposed method is illustrated and compared with some other existing algorithms through two reliability examples with highly-nonlinear performance functions and two representative reliability-based design optimization problems in engineering practice. The results show that the proposed method is robust, accurate and efficient with relatively less iterations in contrast with the other representative algorithms of performance measure approach in the literature.

Effect of Ultrasonic Surface Rolling Process Parameters on Surface Properties of S30408 Austenitic Stainless Steel

S30408 austenitic stainless steel was treated by using hawking processing which is a type of ultrasonic surface rolling process (USRP). The effects of USRP parameters on surface roughness, surface hardness and corrosion resistance were investigated. Compared with the untreated specimen, the surface roughness and surface hardness is decreased by 87%, the surface hardness is increased by 51% and the corrosion rate reduces after USRP. The surface roughness decreases obviously as the step-size decreases, while the step-size has little effect on the surface hardness and corrosion resistance.

Transient wave propagation in inhomogeneous media with enriched overlapping triangular elements

We study and use overlapping triangular finite elements enriched by trigonometric functions and implicit time integration to solve transient wave propagations in inhomogeneous media. We show explicitly that the total dispersion error of the calculated solutions can be split into two parts, the spatial error and temporal error. The study of the spatial dispersion error shows the effectiveness of the enriched overlapping finite elements compared to the traditional finite elements and the overlapping finite elements without enrichment. The study of the temporal error of the Bathe time integration scheme shows monotonic convergence to zero with decreasing time step size. The result is that we see monotonic convergence to exact solutions as the mesh of enriched overlapping finite elements is refined and the time step is decreased. We demonstrate the effectiveness of using the proposed scheme in the solution of waves traveling in inhomogeneous media at different speeds, where reflected and transmitted waves are predicted accurately by “simply” using a fine enough mesh and small enough time step.

Separate Time Integration Based on the Hilber, Hughes, Taylor Scheme for Flexible Bodies With a Large Number of Modes

Abstract In the current development of flexible multibody dynamics, the efficient and accurate consideration of distributed and nonlinear forces is an active area of research. Examples are, forces due to body-body contact or due to elastohydrodynamics (EHD). This leads to many additional modes for representing the local deformations in the areas on which those forces act. Recent publications show that these can be several hundred to several thousand additional modes. A conventional, monolithic numerical time integration scheme would lead to unacceptable computing times. This paper presents a method for an efficient time integration of such systems. The core idea is to treat the equations associated with modes representing local deformations separately. Using the Newmark formulas, a fixed point iteration is proposed for these separated equations, which can always be stabilized with decreasing step size. The concluding examples underline this property, as well as the fact that the proposed method massively outperforms the conventional, monolithic time integration with increasing number of modes.