Pure Adaptive Search Research Articles

Particle Swarm Optimization and Pure Adaptive Search in Finish Goods’ Inventory Management

Inventory management deals with a tradeoff between the benefits of keeping stocks of goods that allows fulfillment of the customer’s demand, and the cost of carrying inventory. Inventory control techniques are very important components and the most organizations can substantially reduce their costs associated with the flow of materials. This paper presents new inventory management model based on particle swarm optimization and pure adaptive search global optimization algorithm in production-inventory system. The proposed model is focusing on planned level of demand for finished goods, production and raw materials cost, production capacity as the norm, change of the production cost and inventory capital cost, all of which are typical factors in automobile manufacture industry. The model determines different factors such as the minimizing inventory quantity, minimizing inventory value, and minimizing production cost based on demand for production items. The model is tested with original real-world dataset obtained from the automotive company Lear from US and its factory in Novi Sad, Serbia.

Solving fractional problems with dynamic multistart improving hit-and-run

Fractional programming has numerous applications in economy and engineering. While some fractional problems are easy in the sense that they are equivalent to an ordinary linear program, other problems like maximizing a sum or product of several ratios are known to be hard, as these functions are highly nonconvex and multimodal. In contrast to the standard Branch-and-Bound type algorithms proposed for specific types of fractional problems, we treat general fractional problems with stochastic algorithms developed for multimodal global optimization. Specifically, we propose Improving Hit-and-Run with restarts, based on a theoretical analysis of Multistart Pure Adaptive Search (cf. the dissertation of Khompatraporn (2004)) which prescribes a way to utilize problem specific information to sample until a certain level α of confidence is achieved. For this purpose, we analyze the Lipschitz properties of fractional functions, and then utilize a unified method to solve general fractional problems. The paper ends with a report on numerical experiments.

Backtracking Adaptive Search: Distribution of Number of Iterations to Convergence

Backtracking adaptive search is a simplified stochastic optimiza-tion procedure which permits the acceptance of worsening objective function values. It generalizes the hesitant adaptive search, which in turn is a gener-alization of the pure adaptive search. In this paper, we use ideas from the theory of stochastic processes to determine the full distribution of the number of iterations to convergence for the backtracking adaptive search.

Expected hitting times for Backtracking Adaptive Search

This article analyses a counting process associated with a stochastic process arising in global optimisation. Backtracking adaptive search (BAS) is a theoretical stochastic global optimisation algorithm modelling the temporary acceptance of solutions of lower quality. BAS generalises the pure adaptive search and hesitant adaptive search algorithms, whose full search duration distributions are known. This article gives the exact expected search duration for backtracking adaptive search.

Expected search duration for finite backtracking adaptive search

Backtracking adaptive search is an optimisation algorithm which generalises pure adaptive search and hesitant adaptive search. This paper considers the number of iterations for which the algorithm runs, on a problem with finitely many range levels, in order to reach a sufficiently extreme objective function level. A difference equation for the expectation of this quantity is derived and solved. Several examples of backtracking adaptive search on finite problems are presented, including special cases that have received attention in earlier papers.

Implementing Pure Adaptive Search with Grover's Quantum Algorithm

Pure adaptive search (PAS) is an idealized stochastic algorithm for unconstrained global optimization. The number of PAS iterations required to solve a problem increases only linearly in the domain dimension. However, each iteration requires the generation of a random domain point uniformly distributed in the current improving region. If no regularity conditions are known to hold for the objective function, then this task requires a number of classical function evaluations varying inversely with the proportion of the domain constituted by the improving region, entirely counteracting the PAS apparent speedup. The Grover quantum computational search algorithm provides a way to generate the PAS iterates. We show that the resulting implementation, which we call the Grover adaptive search (GAS), realizes PAS for functions satisfying certain conditions, and we believe that, when quantum computers will be available, GAS will be a practical algorithm.

Convergence and first hitting time of simulated annealing algorithms for continuous global optimization

In this paper simulated annealing algorithms for continuous global optimization are considered.Under the simplifying assumption of known optimal value, the convergence of the algorithms and an upper bound for the expected first hitting time, i.e. the expected number of iterations before reaching the global optimum value within accuracy e, are established. The obtained results are compared with those for the ideal algorithm PAS (Pure Adaptive Search) and for the simple PRS (Pure Random Search) algorithm.

Hesitant adaptive search: the distribution of the number of iterations to convergence

Hesitant adaptive search is a stochastic optimisation procedure which accommodates hesitation, or pausing, at objective function values. It lies between pure adaptive search (which strictly improves at each iteration) and simulated annealing with constant temperature (which allows backtracking, or the acceptance of worse function values). In this paper we build on an earlier work and make two contributions; first, we link hesitant adaptive search to standard counting process theory, and second, we use this to derive the exact distribution of the number of iterations of hesitant adaptive search to termination.

Implementing pure adaptive search for global optimization using Markov chain sampling

The Pure Adaptive Search (PAS) algorithm for global optimization yields a sequence of points, each of which is uniformly distributed in the level set corresponding to its predecessor. This algorithm has the highly desirable property of solving a large class of global optimization problems using a number of iterations that increases at most linearly in the dimension of the problem. Unfortunately, PAS has remained of mostly theoretical interest due to the difficulty of generating, in each iteration, a point uniformly distributed in the improving feasible region. In this article, we derive a coupling equivalence between generating an approximately uniformly distributed point using Markov chain sampling, and generating an exactly uniformly distributed point with a certain probability. This result is used to characterize the complexity of a PAS-implementation as a function of (a) the number of iterations required by PAS to achieve a certain solution quality guarantee, and (b) the complexity of the sampling algorithm used. As an application, we use this equivalence to show that PAS, using the so-called Random ball walk Markov chain sampling method for generating nearly uniform points in a convex region, can be used to solve most convex programming problems in polynomial time.

On success rates for controlled random search

Controlled Random Search (CRS) is a simple population based algorithm which despite its attractiveness for practical use, has never been very popular among researchers on Global Optimization due to the difficulties in analysing the algorithm. In this paper, a framework to study the behaviour of algorithms in general is presented and embedded into the context of our view on questions in Global Optimization. By using as a reference a theoretical ideal algorithm called N-points Pure Adaptive Search (NPAS) some new analytical results provide bounds on speed of convergence and the Success Rate of CRS in the limit once it has settled down into simple behaviour. To relate the performance of the algorithm to characteristics of functions to be optimized, constructed simple test functions, called extreme cases, are used.

On Uniform Covering, Adaptive Random Search and Raspberries

The problem of generating a random sample over a level set, called Uniform Covering, is considered. A variant is discussed of an algorithm known as Pure Adaptive Search which is a global optimisation ideal with a desirable complexity. The algorithm of Uniform Covering by Probabilistic Rejection is discussed as an approach to the practical realisation of PAS. Consequences for the complexity and practical performance in comparison to other algorithms are illustrated.

A new theoretical framework for analyzing stochastic global optimization algorithms

In this paper, we develop a new theoretical framework by means of the absorbin Markov process theory for analyzing some stochastic global optimization algorithms. Applying the framework to the pure random search, we prove that the pure random search converges to the global minimum in probability and its time has geometry distribution. We also analyze the pure adaptive search by this framework and turn out that the pure adaptive search converges to the global minimum in probability and its time has Poisson distribution.

Hesitant adaptive search for global optimisation

Pure adaptive search is a stochastic algorithm which has been analysed in distinct ways for finite and continuous global optimisation. In this paper, motivated by the behaviour of practical algorithms such as simulated annealing, we extend these ideas. We present a unified theory which yields both the finite and continuous results for pure adaptive search. At the same time, we allow our extended algorithm to “hesitate” before improvement continues. Results are obtained for the expected number of iterations to convergence for such an algorithm. © 1998 The Mathematical Programming Society, Inc. Published by Elsevier Science B.V.

Pure adaptive search for finite global optimization

Pure Adaptive Search is a stochastic algorithm which has been analyzed for continuous global optimization. When a uniform distribution is used in PAS, it has been shown to have complexity which is linear in dimension. We define strong and weak variations of PAS in the setting of finite global optimization and prove analogous results. In particular, for then-dimensional lattice {1,⋯,k}n, the expected number of iterations to find the global optimum is linear inn. Many discrete combinatorial optimization problems, although having intractably large domains, have quite small ranges. The strong version of PAS for all problems, and the weak version of PAS for a limited class of problems, has complexity the order of the size of the range.

Towards Pure Adaptive Search

The algorithm known as Pure Adaptive Search is a global optimisation ideal with desirable complexity. In this paper we temper it to a framework we term Somewhat Adaptive Search. This retains the desirable complexity, but allows scope for a practical realisation. We introduce a new algorithm termed Pure Localisation Search which attempts to reach the practical ideal. For a certain class of one variable functions the gap is bridged.

A comparison of algorithms for global characterization of confidence regions for nonlinear models

Environmental models are often highly nonlinear, and parameters have to be estimated from noisy data. The standard approach of locally linearizing the model, which leads to ellipsoid confidence regions, is inappropriate in this situation. A straightforward technique to characterize arbitrary-shaped confidence regions is to calculate model output on a grid of parameter values. Each parameter value P results in a goodness of fit G(P), which allows delineation of the set of parameters corresponding to G(P) < Gc, with Gc some threshold level (e.g., 5% probability). This approach is impractical and time-consuming for complex models, however. This article aims at finding an efficient alternative. It is first shown that the most general approach is to generate parameter values uniformly covering the set G(P) < Gc rather than finding the boundary G(P) = Gc. It is argued that the most efficient method of generating a uniform cover is by a (theoretical) algorithm known as pure adaptive search (PAS); the presently proposed method (uniform covering by probabilistic rejection; UCPR) is shown to be a good approximation to PAS. The UCPR is compared with alternative methods for a number of test problems. It is illustrated that for complex models (where model run time dominates total computer time) UCPR is considerably faster and its cover of Gc more uniform than existing alternatives. An intrinsic problem common to all methods is that the amount of work increases at least quadratically with the number of parameters considered, making them of limited use for high-dimensional problems.

A COMPARISON OF ALGORITHMS FOR GLOBAL CHARACTERIZATION OF CONFIDENCE REGIONS FOR NONLINEAR MODELS

Environmental models are often highly nonlinear, and parameters have to be estimated from noisy data. The standard approach of locally linearizing the model, which leads to ellipsoid confidence regions, is inappropriate in this situation. A straightforward technique to characterize arbitrary-shaped confidence regions is to calculate model output on a grid of parameter values. Each parameter value P results in a goodness of fit G(P), which allows delineation of the set of parameters corresponding to G(P) < Gc, with Gc some threshold level (e.g., 5% probability). This approach is impractical and time-consuming for complex models, however. This article aims at finding an efficient alternative. It is first shown that the most general approach is to generate parameter values uniformly covering the set G(P) < Gc rather than finding the boundary G(P) = Gc. It is argued that the most efficient method of generating a uniform cover is by a (theoretical) algorithm known as pure adaptive search (PAS); the presently proposed method (uniform covering by probabilistic rejection; UCPR) is shown to be a good approximation to PAS. The UCPR is compared with alternative methods for a number of test problems. It is illustrated that for complex models (where model run time dominates total computer time) UCPR is considerably faster and its cover of Gc more uniform than existing alternatives. An intrinsic problem common to all methods is that the amount of work increases at least quadratically with the number of parameters considered, making them of limited use for high-dimensional problems.

Pure adaptive search in global optimization

Pure adaptive seach iteratively constructs a sequence of interior points uniformly distributed within the corresponding sequence of nested improving regions of the feasible space. That is, at any iteration, the next point in the sequence is uniformly distributed over the region of feasible space containing all points that are strictly superior in value to the previous points in the sequence. The complexity of this algorithm is measured by the expected number of iterations required to achieve a given accuracy of solution. We show that for global mathematical programs satisfying the Lipschitz condition, its complexity increases at mostlinearly in the dimension of the problem.

Pure adaptive search in monte carlo optimization

Pure adaptive search constructs a sequence of points uniformly distributed within a corresponding sequence of nested regions of the feasible space. At any stage, the next point in the sequence is chosen uniformly distributed over the region of feasible space containing all points that are equal or superior in value to the previous points in the sequence. We show that for convex programs the number of iterations required to achieve a given accuracy of solution increases at most linearly in the dimension of the problem. This compares to exponential growth in iterations required for pure random search.