Dynamic Step Size Research Articles

Convergence rates of subgradient methods for quasi-convex optimization problems

Quasi-convex optimization acts a pivotal part in many fields including economics and finance; the subgradient method is an effective iterative algorithm for solving large-scale quasi-convex optimization problems. In this paper, we investigate the quantitative convergence theory, including the iteration complexity and convergence rates, of various subgradient methods for solving quasi-convex optimization problems in a unified framework. In particular, we consider a sequence satisfying a general (inexact) basic inequality, and investigate the global convergence theorem and the iteration complexity when using the constant, diminishing or dynamic stepsize rules. More importantly, we establish the linear (or sublinear) convergence rates of the sequence under an additional assumption of weak sharp minima of Holderian order and upper bounded noise. These convergence theorems are applied to establish the iteration complexity and convergence rates of several subgradient methods, including the standard/inexact/conditional subgradient methods, for solving quasi-convex optimization problems under the assumptions of the Holder condition and/or the weak sharp minima of Holderian order.

Compressed sensing channel estimation for STBC‐SM based hybrid MIMO‐OFDM system for visible light communication

SummaryThis paper presents the idea of sparse channel estimation using compressed sensing (CS) method for space–time block coding (STBC), and spatially multiplexing (SM) derived hybrid multiple‐input multiple‐output (MIMO) Asymmetrically clipped optical‐orthogonal frequency division multiplexing (ACO‐OFDM) optical wireless communication system. This hybrid system accounts multiplexing gain of SM and diversity gain of STBC technique. We present a new variant of sparsity adaptive matching pursuit (SaMP) algorithm called dynamic step‐size SaMP (DSS‐SaMP) algorithm. It makes use of the inherent and implicit structure of SaMP, along with dynamic adaptivity of step‐size feature which is compatible with the energy of the input signal, thus the name dynamic step size. Existing CS‐based recovery algorithms like orthogonal matching pursuit, SaMP, adaptive step‐size SaMP, and proposed DSS‐SaMP were compared for hybrid MIMO‐ACO‐OFDM visible light communication system. The performance analysis is demonstrated through simulation results with respect to bit error rate, symbol error rate, mean square error, computational complexity, and peak‐to‐average power ratio. Simulation results show that the proposed technique gives improved performance and lesser computational complexity in comparison with conventional estimation algorithms.

Solving the Split Feasibility Problem and the Fixed Point Problem of Left Bregman Firmly Nonexpansive Mappings via the Dynamical Step Sizes in Banach Spaces

In this work, we suggest a new self-adaptive method using the dynamical step size technique for solving the split feasibility problem and the fixed point problem of left Bregman firmly nonexpansive mappings in a certain Banach space. We establish the strong convergence without the computation of the operator norms. Finally, we give some examples including its numerical experiments to show the efficiency and the implementation of our proposed method.

Using Cuckoo Search Algorithm with Q-Learning and Genetic Operation to Solve the Problem of Logistics Distribution Center Location

Cuckoo search (CS) algorithm is a novel swarm intelligence optimization algorithm, which is successfully applied to solve some optimization problems. However, it has some disadvantages, as it is easily trapped in local optimal solutions. Therefore, in this work, a new CS extension with Q-Learning step size and genetic operator, namely dynamic step size cuckoo search algorithm (DMQL-CS), is proposed. Step size control strategy is considered as action in DMQL-CS algorithm, which is used to examine the individual multi-step evolution effect and learn the individual optimal step size by calculating the Q function value. Furthermore, genetic operators are added to DMQL-CS algorithm. Crossover and mutation operations expand search area of the population and improve the diversity of the population. Comparing with various CS algorithms and variants of differential evolution (DE), the results demonstrate that the DMQL-CS algorithm is a competitive swarm algorithm. In addition, the DMQL-CS algorithm was applied to solve the problem of logistics distribution center location. The effectiveness of the proposed method was verified by comparing with cuckoo search (CS), improved cuckoo search algorithm (ICS), modified chaos-enhanced cuckoo search algorithm (CCS), and immune genetic algorithm (IGA) for both 6 and 10 distribution centers.

A new projection method with a dynamic stepsize for the split equality problem

A new projection method with a dynamic stepsize for the split equality problem

Modeling and optimization of urban rail transit scheduling with adaptive fruit fly optimization algorithm

Abstract Despite the rapid development of urban rail transit in China, there are still some problems in train operation, such as low efficiency and poor punctuality. To realize a proper allocation of passenger flows and increase train frequency, this paper has proposed an improved urban rail transit scheduling model and solved the model with an adaptive fruit fly optimization algorithm (AFOA). For the benefits of both passengers and operators, the shortest average waiting time of passengers and the least train frequency are chosen as the optimization objective, and train headway is taken as the decision variable in the proposed model. To obtain higher computational efficiency and accuracy, an adaptive dynamic step size is built in the conventional FOA. Moreover, the data of urban rail transit in Zhengzhou was simulated for case study. The comparison results reveal that the proposed AFOA exhibits faster convergence speed and preferable accuracy than the conventional FOA, particle swarm optimization, and bacterial foraging optimization algorithms. Due to these superiorities, the proposed AFOA is feasible and effective for optimizing the scheduling of urban rail transit.

Bacterial foraging optimization algorithm with mutation to solve constrained problems

A simple version of a Swarm Intelligence algorithm called bacterial foraging optimization algorithm with mutation and dynamic stepsize (BFOAM-DS) is proposed. The bacterial foraging algorithm has the ability to explore and exploit the search space through its chemotactic operator. However, premature convergence is a disadvantage. This proposal uses a mutation operator in a swim, similar to evolutionary algorithms, combined with a dynamic stepsize operator to improve its performance and allows a better balance between the exploration and exploitation of the search space. BFOAM-DS was tested in three well-known engineering design optimization problems. Results were analyzed with basic statistics and common measures for nature-inspired constrained optimization problems to evaluate the behavior of the swim with a mutation operator and the dynamic stepsize operator. Results were compared against a previous version of the proposed algorithm to conclude that BFOAM-DS is competitive and better than a previous version of the algorithm.

Life prediction of highly reliable products based on wiener process based on random shock response

Aiming at the life prediction problem of high reliability products with random shock response, this paper analyzes the two-dimensional degradation of highly reliable products, the nonlinear Wiener process model was used to analyze the degradation process of the product performance index, and the multivariate nonlinear Wiener process model was established. Based on the sliding window mechanism and the variable step length method, a dynamic variable step size nonlinear Wiener process degradation model was established by dynamically evaluating the parameters of the degradation model. The approximate expression of the probability density function of the residual life was further given. Finally, the validity and practicability of the proposed model and method were verified by an example.

Path-based incremental target level algorithm on Riemannian manifolds

ABSTRACTIn this paper, we propose and analyse a path-based incremental target level algorithm for minimizing a constrained convex optimization on complete Riemannian manifolds with lower bounded sectional curvature, where the object function consists of the sum of a large number of component functions. This algorithm extends, to the context of Riemannian manifolds, an incremental subgradient method emplying a version of dynamic stepsize rule. Some convergence results and iteration-complexity bounds of the algorithm are established.

RePlAce: Advancing Solution Quality and Routability Validation in Global Placement

The Nesterov’s method approach to analytic placement has recently demonstrated strong solution quality and scalability. We dissect the previous implementation strategy and show that solution quality can be significantly improved using two levers: 1) constraint-oriented local smoothing and 2) dynamic step size adaptation. We propose a new density function that comprehends local overflow of area resources; this enables a constraint-oriented local smoothing at per-bin granularity. Our improved dynamic step size adaptation automatically determines step size and effectively allocates optimization effort to significantly improve solution quality without undue runtime impact. Our resulting global placement tool, RePlAce, achieves an average of 2.00% half-perimeter wirelength (HPWL) reduction over all best known ISPD-2005 and ISPD-2006 benchmark results, and an average of 2.73% over all best known modern mixed-size (MMS) benchmark results, without any benchmark-specific code or tuning. We further extend our global placer to address routability, and achieve on average 8.50%–9.59% scaled HPWL reduction over previous leading academic placers for the DAC-2012 and ICCAD-2012 benchmark suites. To our knowledge, RePlAce is the first work to achieve superior solution quality across all the ISPD-2005, ISPD-2006, MMS, DAC-2012, and ICCAD-2012 benchmark suites with a single global placement engine.

Incremental quasi-subgradient methods for minimizing the sum of quasi-convex functions

The sum of ratios problem has a variety of important applications in economics and management science, but it is difficult to globally solve this problem. In this paper, we consider the minimization problem of a sum of a number of nondifferentiable quasi-convex component functions over a closed and convex set, which includes the sum of ratios problem as a special case. The sum of quasi-convex component functions is not necessarily to be quasi-convex, and so, this study goes beyond quasi-convex optimization. Exploiting the structure of the sum-minimization problem, we propose a new incremental subgradient method for this problem and investigate its convergence properties to a global optimal solution when using the constant, diminishing or dynamic stepsize rules and under a homogeneous assumption and the H\"{o}lder condition of order $p$. To economize on the computation cost of subgradients of a large number of component functions, we further propose a randomized incremental subgradient method, in which only one component function is randomly selected to construct the subgradient direction at each iteration. The convergence properties are obtained in terms of function values and distances of iterates from the optimal solution set with probability 1. The proposed incremental subgradient methods are applied to solve the sum of ratios problem, as well as the multiple Cobb-Douglas productions efficiency problem, and the numerical results show that the proposed methods are efficient for solving the large sum of ratios problem.

Linear Convergence of an Iterative Algorithm for Solving the Multiple-Sets Split Feasibility Problem

In this paper, we propose the simultaneous sub-gradient projection algorithm with the dynamic step size (SSPA for short) for solving the multiple-sets split feasibility problem (MSSFP for short) and investigate its linear convergence. We involve a notion of bounded linear regularity for the MSSFP and construct several sufficient conditions to prove the linear convergence for the SSPA. In particular, the SSPA is an easily calculated algorithm that uses orthogonal projection onto half-spaces. Furthermore, some numerical results are provided to verify the effectiveness of our proposed algorithm.

Face Recognition Based on SVM Optimized by the Improved Bacterial Foraging Optimization Algorithm

Support vector machine (SVM) is always used for face recognition. However, kernel function selection (kernel selection and its parameters selection) is a key problem for SVMs, and it is difficult. This paper tries to make some contributions to this problem with focus on optimizing the parameters in the selected kernel function. Bacterial foraging optimization algorithm, inspired by the social foraging behavior of Escherichia coli, has been widely accepted as a global optimization algorithm of current interest for distributed optimization and control. Therefore, we proposed to optimize the parameters in SVM by an improved bacterial foraging optimization algorithm (IBFOA). In the improved version of bacterial foraging optimization algorithm, a dynamical elimination-dispersal probability in the elimination-dispersal step and a dynamical step size in the chemotactic step are used to improve the performance of bacterial foraging optimization algorithm. Then the optimized SVM is used for face recognition. Simultaneously, an improved local binary pattern is proposed to extract features of face images in this paper to improve the accuracy rate of face recognition. Numerical results show the advantage of our algorithm over a range of existing algorithms.

Noise-Free Maximum Correntropy Criterion Algorithm in Non-Gaussian Environment

In this brief, a noise-free maximum correntropy criterion (NFMCC) algorithm is proposed for system identification in non-Gaussian environments. The proposed algorithm utilizes correntropy theory to construct a cost function which is realized based on a normalized Gaussian kernel. In addition, a new dynamic step size scheme is proposed to enhance the performance of the proposed algorithm, which is implemented by minimizing the noise-free a posteriori error signal, and the mean square deviation (MSD) is greatly decreased. The proposed NFMCC algorithm shows significant property in reducing the detrimental effects of outliers and impulsive noise with different input signals. Moreover, a Students’ T distributed noise is employed to evaluate the effectiveness of the proposed algorithm in terms of the MSD and convergence for heavy tailed noising environment. The parameter effects on the NFMCC algorithm are also presented, and its performance is investigated on a real-life channel that is measured in underwater. Simulation results prove the effectiveness of the proposed algorithm which provides a considerable computational complexity and an acceptable running time.

A Path-Planning Strategy for Unmanned Surface Vehicles Based on an Adaptive Hybrid Dynamic Stepsize and Target Attractive Force-RRT Algorithm

It is well known that path planning has always been an important study area for intelligent ships, especially for unmanned surface vehicles (USVs). Therefore, it is necessary to study the path-planning algorithm for USVs. As one of the basic algorithms for USV path planning, the rapidly-exploring random tree (RRT) is popular due to its simple structure, high speed and ease of modification. However, it also has some obvious drawbacks and problems. Designed to perfect defects of the basic RRT and improve the performance of USVs, an enhanced algorithm of path planning is proposed in this study, called the adaptive hybrid dynamic stepsize and target attractive force-RRT(AHDSTAF-RRT). The ability to pass through a narrow area and the forward speed in open areas of USVs are improved by adopting the AHDSTAF-RRT in comparison to the basic RRT algorithm. The improved algorithm is also applied to an actual gulf map for simulation experiments, and the experimental data is collected and organized. Simulation experiments show that the proposed AHDSTAF-RRT in this paper outperforms several existing RRT algorithms, both in terms of path length and calculating speed.

On gradient projection methods for strongly pseudomonotone variational inequalities without Lipschitz continuity

In this paper, we propose two new self-adaptive algorithms for solving strongly pseudomonotone variational inequalities. Our algorithms use dynamic step-sizes, chosen based on information of the previous step and their strong convergence is proved without the Lipschitz continuity of the underlying mappings. In contrast to the existing self-adaptive algorithms in Bello Cruz et al. (Comput Optim Appl 46:247–263, 2010), Santos et al. (Comput Appl Math 30:91–107, 2011), which use fast diminishing step-sizes, the new algorithms use arbitrarily slowly diminishing step sizes. This feature helps to speed up our algorithms. Moreover, we provide the worst case complexity bound of the proposed algorithms, which have not been done in the previous works. Some preliminary numerical experiences and comparisons are also reported.

A Modified Adaptive Thresholding Method using Cuckoo Search Algorithm for Detecting Surface Defects

There are various mathematical optimization problems that can be effectively solved by meta-heuristic algorithms. The improvement of these algorithms is that they carry out iterative search processes which resourcefully act upon exploration and exploitation in spatial domain containing global and local optima. An innovative robust Cuckoo Optimization Algorithm (COA) with adaptive thresholding is proposed to solve the problem of detection and estimation of surface defects on metal coating surfaces. The proposed method is developed through implementing changes to COA and improved the performance. For improving capability of local search as well to keep the global search effect, the enhanced methods such as level set is associated with the proposed method. Also, the method adapts dynamic step size, adaptively changing with the search process for improving the rate of convergence and the ability of local search. The algorithm performance is scrutinized from the experimental analysis and results. Also, the segmentation effectiveness is further enhanced by adapting suitable methods for preprocessing and post processing. The comparison and analysis of the results accomplished with the proposed method and results of earlier methods shows superior performance of the proposed method.

Efficient primal–dual fixed point algorithms with dynamic stepsize for composite convex optimization problems

In this paper, we propose a new primal–dual fixed point algorithm with dynamic stepsize ( $$\hbox {PDFP}^{2}O_{DS_{n}}$$ ) for solving convex minimization problems involving the sum of a smooth function with Lipschitzian gradient and the composition of a nonsmooth convex function with a continuous linear operator. Based on modified Mann iteration and the firmly nonexpansive properties of the proximity operator, we achieve the convergence of the proposed algorithm. Moreover, we give the connection of the proposed algorithm with other existing $$\hbox {PDFP}^{2}O$$ (Chen et al. in Inverse Probl 29:025011–025033, 2013). Finally, we illustrate the efficiency of $$\hbox {PDFP}^{2}O_{DS_{n}}$$ through some numerical examples on the CT image reconstruction problem. Numerical results show that our iterative algorithm ( $$\hbox {PDFP}^{2}O_{DS}$$ ) performs better than the original one ( $$\hbox {PDFP}^{2}O$$ ).

A novel nonlinear modeling for the prediction of blast-induced airblast using a modified conjugate FR method

Prediction of the blast-induced is an important issue in Shur river dam, Iran. The nonlinear mathematical models can provide an appropriate flexibility to achieve the accurate predictions of the blast-induced airblast. In this paper, a set of nonlinear mathematical models with eight empirical relations, which are added based on the logarithmic and power basic functions, are selected to calibrate of the mine blasting airblast using two input variables, i.e. maximum charge per delay (MC) and distance from the blast-point (DI). A general regression analysis is proposed to calibrate the nonlinear models using a modified conjugate Fletcher and Reeves (FR) method using a limited scalar factor and dynamic step size to achieve the stabilization in nonlinear modeling. Finally, three simple empirical models are chosen to implement the prediction of the blast-induced airblast. The proposed empirical models were compared with the United States Bureau of Mines (USBM) model using several error statistics. The results indicate that the proposed modified FR model provides an appropriate calibration for the nonlinear regression analysis. Also, it was found that the empirical model proposed in this study, with the root mean square error (RMSE) of 3.79, is more accurate than USBM, with the RMSE of 4.22, and can be applied to other sites for predicting the airblast.

Traffic‐aware resource allocation scheme for mMTC in dynamic TDD systems

The asymmetry traffic between downlink (DL) and uplink (UL) in massive machine-type communication (mMTC) systems is so prominent that it makes the traditionally fixed frame protocols insufficient to handle. Meanwhile, the dynamic time-division duplexing (D-TDD) is a promising and attractive technology since its number of time slots for the DL and UL can be asymmetric and adjusted dynamically. In the cellular and mMTC co-existing network, to balance the discrepancy of the UL/DL ratio and alleviate the interference as well, in this study, the authors design a D-TDD-based transmission frame structure to first fulfil the basic transmission requirements of the human-type communication (HTC) users with a low power almost blank subframe. Herein, the stochastic geometry methods are adopted to calculate spectral efficiencies of the HTC user equipment. Then, focusing on the worst queue state in mMTC, they devise the slot allocation problem with the min–max objective of the UL/DL queues and utilise the sub-gradient descent (SGD) method for a solution. Simulation results show that the proposed traffic aware sub-frame configuration is more appropriate for the dynamical asymmetry environment. Meanwhile, the adopted dynamic step size SGD algorithm can achieve a trade-off between the worst-case queue and the network throughput.