Restart Techniques Research Articles

A stochastic gradient approach for the reliability maximization of passively controlled structures

This paper addresses the reduction of the computational burden of structural reliability maximization of passively controlled buildings subject to transient loading. Such a reduction is accomplished by means of Stochastic Gradient Descent (SGD) algorithms, which replace expensive multi-dimensional Monte Carlo integration by a singleton multi-iteration integration. In order to be able to apply SGD methods, the time-dependent structural reliability evaluation was constructed based on the out-crossing rate approach. Design of single and multi Friction Tuned Mass Dampers (FTMDs) are considered as design examples. The proposed SGD algorithm, the accelerated stochastic gradient descent (ASGD) algorithm, efficiently combines Nesterov acceleration, Polyak–Ruppert averaging and restart techniques. The main result drew from the numerical analysis is that SGD algorithms were able to maximize the structural reliability by providing more accurate results and requiring lower computational cost than well-known optimization methods. Finally, the limitation of the proposed optimization procedure is linked to the validity of the out-crossing rate to approximate the time-dependent structural reliability.

On restarting the tensor infinite Arnoldi method

An efficient and robust restart strategy is important for any Krylov-based method for eigenvalue problems. The tensor infinite Arnoldi method (TIAR) is a Krylov-based method for solving nonlinear eigenvalue problems (NEPs). This method can be interpreted as an Arnoldi method applied to a linear and infinite dimensional eigenvalue problem where the Krylov basis consists of polynomials. We propose new restart techniques for TIAR and analyze efficiency and robustness. More precisely, we consider an extension of TIAR which corresponds to generating the Krylov space using not only polynomials, but also structured functions, which are sums of exponentials and polynomials, while maintaining a memory efficient tensor representation. We propose two restarting strategies, both derived from the specific structure of the infinite dimensional Arnoldi factorization. One restarting strategy, which we call semi-explicit TIAR restart, provides the possibility to carry out locking in a compact way. The other strategy, which we call implicit TIAR restart, is based on the Krylov–Schur restart method for the linear eigenvalue problem and preserves its robustness. Both restarting strategies involve approximations of the tensor structured factorization in order to reduce the complexity and the required memory resources. We bound the error introduced by some of the approximations in the infinite dimensional Arnoldi factorization showing that those approximations do not substantially influence the robustness of the restart approach. We illustrate the effectiveness of the approaches by applying them to solve large scale NEPs that arise from a delay differential equation and a wave propagation problem. The advantages in comparison to other restart methods are also illustrated.

The importance of implementation details and parameter settings in black-box optimization: a case study on Gaussian estimation-of-distribution algorithms and circles-in-a-square packing problems

We consider a scalable problem that has strong ties with real-world problems, can be compactly formulated and efficiently evaluated, yet is not trivial to solve and has interesting characteristics that differ from most commonly used benchmark problems: packing n circles in a square (CiaS). Recently, a first study that used basic Gaussian EDAs indicated that typically suggested algorithmic parameter settings do not necessarily transfer well to the CiaS problem. In this article, we consider also AMaLGaM, an enhanced Gaussian EDA, as well as arguably the most powerful real-valued black-box optimization algorithm to date, CMA-ES, which can also be seen as a further enhanced Gaussian EDA. We study whether the well-known performance on typical benchmark problems extends to the CiaS problem. We find that although the enhancements over a basic Gaussian EDA result in superior performance, the further efficiency enhancements in CMA-ES are not highly impactful. Instead, the most impactful features are how constraint handling is performed, how large the population size is, whether a full covariance matrix is used and whether restart techniques are used. We further show that a previously published version of AMaLGaM that does not require the user to set the the population size parameter is capable of solving the problem and we derive the scalability of the required number of function evaluations to solve the problem up to 99.99 % of the known optimal value for up to 30 circles.

Numerical recovery strategies for parallel resilient Krylov linear solvers

SummaryAs the computational power of high‐performance computing systems continues to increase by using a huge number of cores or specialized processing units, high‐performance computing applications are increasingly prone to faults. In this paper, we present a new class of numerical fault tolerance algorithms to cope with node crashes in parallel distributed environments. This new resilient scheme is designed at application level and does not require extra resources, that is, computational unit or computing time, when no fault occurs. In the framework of iterative methods for the solution of sparse linear systems, we present numerical algorithms to extract relevant information from available data after a fault, assuming a separate mechanism ensures the fault detection. After data extraction, a well‐chosen part of missing data is regenerated through interpolation strategies to constitute meaningful inputs to restart the iterative scheme. We have developed these methods, referred to as interpolation–restart techniques, for Krylov subspace linear solvers. After a fault, lost entries of the current iterate computed by the solver are interpolated to define a new initial guess to restart the Krylov method. A well‐suited initial guess is computed by using the entries of the faulty iterate available on surviving nodes. We present two interpolation policies that preserve key numerical properties of well‐known linear solvers, namely, the monotonic decrease of the A‐norm of the error of the conjugate gradient or the residual norm decrease of generalized minimal residual algorithm for solving. The qualitative numerical behavior of the resulting scheme has been validated with sequential simulations, when the number of faults and the amount of data losses are varied. Finally, the computational costs associated with the recovery mechanism have been evaluated through parallel experiments. Copyright © 2016 John Wiley & Sons, Ltd.

Combining restarts, nogoods and bag-connected decompositions for solving CSPs

From a theoretical viewpoint, the (tree-)decomposition methods offer a good approach for solving Constraint Satisfaction Problems (CSPs) when their (tree)-width is small. In this case, they have often shown their practical interest. So, the literature (coming from Mathematics, OR or AI) has concentrated its efforts on the minimization of a single parameter, namely the tree-width. Nevertheless, experimental studies have shown that this parameter is not always the most relevant to consider when solving CSPs. So, in this paper, we highlight two fundamental problems related to the use of tree-decomposition and for which we offer two particularly appropriate solutions. First, we experimentally show that the decomposition algorithms of the state of the art produce clusters (a tree-decomposition is a rooted tree of clusters) having several connected components. We highlight the fact that such clusters create a real disadvantage which affects significantly the efficiency of solving methods. To avoid this problem, we consider here a new graph decomposition called Bag-Connected Tree-Decomposition, which considers only tree-decompositions such that each cluster is connected. We analyze such decompositions from an algorithmic point of view, especially in order to propose a first polynomial time algorithm to compute them. Moreover, even if we consider a very well suited decomposition, it is well known that sometimes, a bad choice for the root cluster may significantly degrade the performance of the solving. We highlight an explanation of this degradation and we propose a solution based on restart techniques. Then, we present a new version of the BTD algorithm (for Backtracking with Tree-Decomposition Jegou and Terrioux, Artificial Intelligence, 146 43---75 28) integrating restart techniques. From a theoretical viewpoint, we prove that reduced nld-nogoods can be safely recorded during the search and that their size is smaller than ones recorded by MAC+RST+NG (Lecoutre et al., JSAT, 1(3---4) 147---167 34). We also show how structural (no)goods may be exploited when the search restarts from a new root cluster. Finally, from a practical viewpoint, we show experimentally the benefits of using independently bag-connected tree-decompositions and restart techniques for solving CSPs by decomposition methods. Above all, we experimentally highlight the advantages brought by exploiting jointly these improvements in order to respond to two major problems generally encountered when solving CSPs by decomposition methods.

Stochastic Constriction Cockroach Swarm Optimization for Multidimensional Space Function Problems

The effect of stochastic constriction on cockroach swarm optimization (CSO) algorithm performance was examined in this paper. A stochastic constriction cockroach swarm optimization (SCCSO) algorithm is proposed. A stochastic constriction factor is introduced into CSO algorithm for swarm stability enhancement; control cockroach movement from one position to another while searching for solution to avoid explosion; enhanced local and global searching capabilities. SCCSO performance was tested through simulation studies and its performance on multidimensional functions is compared with that of original CSO, modified cockroach swarm optimization (MCSO), and one of the well-known global optimization techniques in the literature known as line search restart techniques (LSRS). Standard benchmarks that have been widely used for global optimization problems are considered for evaluating the proposed algorithm. The selected benchmarks were solved up to 3000 dimensions by the proposed algorithm.

Compiler-Assisted Checkpointing of Parallel Codes: The Cetus and LLVM Experience

With the evolution of high-performance computing, parallel applications have developed an increasing necessity for fault tolerance, most commonly provided by checkpoint and restart techniques. Checkpointing tools are typically implemented at one of two different abstraction levels: at the system level or at the application level. The latter has become an interesting alternative due to its flexibility and the possibility of operating in different environments. However, application-level checkpointing tools often require the user to manually insert checkpoints in order to ensure that certain requirements are met (e.g. forcing checkpoints to be taken at the user code and not inside kernel routines). This paper examines the transformations required to enable automatic checkpointing of parallel applications in the CPPC application-level checkpointing framework. These transformations have been implemented on two very different compiler infrastructures: Cetus and LLVM. Cetus is a Java-based compiler infrastructure aiming to provide an easy to use and clean IR and API for program transformation. LLVM is a low-level, SSA-based toolchain. The fundamental differences of both approaches are analyzed from the structural, behavioral and performance perspectives.

Correlated set coordination in fault tolerant message logging protocols for many‐core clusters

SUMMARYWith our current expectation for the exascale systems, composed of hundred of thousands of many‐core nodes, the mean time between failures will become small, even under the most optimistic assumptions. One of the most scalable checkpoint restart techniques, the message logging approach, is the most challenged when the number of cores per node increases because of the high overhead of saving the message payload. Fortunately, for two processes on the same node, the failure probability is correlated, meaning that coordinated recovery is free. In this paper, we propose an intermediate approach that uses coordination between correlated processes but retains the scalability advantage of message logging between independent ones. The algorithm still belongs to the family of event logging protocols but eliminates the need for costly payload logging between coordinated processes. Copyright © 2012 John Wiley & Sons, Ltd.

A restarted estimation of distribution algorithm for solving sudoku puzzles

In this paper, we describe a stochastic algorithm to solve sudoku puzzles. Our method consists in computing probabilities for each symbol of each cell updated at each step of the algorithm using estimation of distributions algorithms (EDA). This update is done using the empirical estimators of these probabilities for a fraction of the best puzzles according to a cost function. We develop also some partial restart techniques in the RESEDA algorithm to obtain a convergence for the most difficult puzzles. Our algorithm is tested numerically on puzzles with various levels of difficulty starting from very easy ones to very hard ones including the famous puzzle AI Escargot. The CPU times vary from few hundreds of a second for the easy ones to about one minute for the most difficult one.

Integrated approach towards adaptive state-tracking job migration for maximising performance benefit

Presented is a new state-tracking migration policy which is adaptively integrated with immediate restart and selective preemption techniques for risk-resilient job allocation in high-throughput computing environments. The proposed approach guarantees the maximum reduction of inevitable delays with relatively low migration costs. Simulation results show that the integrated migration policy averagely outperforms the conventional mechanisms by 26% in makespan and 33% in throughput, respectively, under various job's burstiness.

Iterative restart technique for solving timetabling problems

Restart techniques for randomizing complete search algorithms were proposed recently by Gomes et al., for solving hard combinatorial problems. A new iterative algorithm that uses restart techniques is proposed, and its behavior analyzed on random timetabling problems. Employee timetabling problems (ETPs) can naturally represented by constraints networks for real world instances which are large and difficult. Complete search algorithms, even with good heuristics are unable to solve large enough instances of ETPs. In fact, several local search techniques have been proposed in the past decade for solving timetabling problems. In particular, it has been shown that local search can efficiently solve large ETPs. Random instances of ETPs that was generated based on real world ETPs are used to test the proposed iterative restart algorithm––GIRA. The two main parameters of GIRA are pointed out and investigated: the initial cutoff value for restart and the number of idle iterations. For random sets of a large range of hardness of ETPs, the successful values of these two parameters tend to be surprisingly low. We conclude by recommending an optimal range of values for the initial cutoff and demonstrate experimentally that the number of idle iterations is of little consequence.

RESTARTING TECHNIQUES FOR THE LANCZOS ALGORITHM AND THEIR IMPLEMENTATION IN PARALLEL COMPUTING ENVIRONMENTS: ARCHITECTURAL INFLUENCES

The Lanczos algorithm is one of the principal algorithms for the computation of a few of the extreme eigenvalues and their corresponding eigenvectors of very large, sparse, symmetric matrices and is currently the subject of intense research. In this paper two explicit restart techniques for the algorithm are proposed and their suitability for implementation in two different parallel environments is explored. Both techniques adopt a deflation approach in which exactly one eigenpair at a time is computed. Thus, p eigenpairs are computed using exactly p applications of the chosen Lanczos method. Newly generated Lanczos vectors are orthogona-lized with respect to all previously converged eigenvectors. Each of the techniques has been embedded in a version of the Lanczos algorithm which incorporates full reorthogonalization of the Lanczos vectors as well as a new convergence monitoring routine. Both of the modified Lanczos algorithms have been implemented in a distributed memory SIMD environment and in a shared memory MIMD environment. The execution time efficiency of the two algorithms in each environment for the solution of a variety of matrices selected from the Harwell-Boeing collection of sparse matrices is presented and their performances are compared with those of Sorensen's implicit state-of-the-art routine taken from the ARPACK library. The experiments show that, in an environment in which matrix-vector products may be executed very efficiently, the explicit methods often significantly outperform the implicit method. This result points to the desirability of selecting implementations of algorithms built (as far as possible) from operations which may be efficiently performed in the given parallel environment.

A comparison of the GEVT and RESTART techniques for the simulation of rare events in ATM networks

This paper contains a performance comparison of two simulation techniques, GEVT and RESTART, in the context of the estimation of rare events in ATM networks. An example of rare event is the occurrence of a cell loss due to buffer overflow in the network switches. The comparison is conducted by estimating the probability of buffer occupancy in four queuing models (M/M/1; M/D/1; Geo/D/1; MMPP/D/1) of ATM switches/multiplexers and comparing the results obtained with those predicted by exact analysis. Both methods allow a reduction of 4 to 5 orders of magnitude in the simulation time with respect to the Monte Carlo method. RESTART has however a broader domain of application and requires a less critical choice of the simulation parameters.

Availability in parallel systems: Automatic process restart

Parallel and clustered architectures are increasingly being used as a foundation for high-capacity servers. At the same time, the availability expectations are also rising rapidly, since the effects of down time become more apparent and have higher economic consequences for larger systems. The use of parallel structures generally implies more hardware and software components. The presence of more and larger components increases the chances that an individual component will fail, and that failure has the potential to hurt the overall availability of the system. This paper discusses the use of “restart techniques” as an important strategy in providing increased availability in a parallel structure. The paper covers a set of functions that have been developed for the S/390® Parallel Sysplex™.

Evaluation of timestamp-based concurrency control mechanisms incorporating livelock avoidance

A simulation model of a distributed database system is defined; it is then used to analyze the behavior of various concurrency control mechanisms based on timestamp ordering. The model can reflect different network configurations, different data distribution/replication schemes, and different users' activity characteristics. Various concurrency control algorithms are accepted, and their relative performance can be compared. This model is used to show that the standard transaction restart policy used by time-stamp-based algorithms leads to infinite cyclic restarts of transactions. To remedy this problem, three alternative restart techniques are defined and are formally shown to guarantee the absence of cyclic restarts. Simulation is used to determine the relative advantages of the resulting robust concurrency control mechanisms.