Simplex Implementation Research Articles

RESAC: A redundancy strategy involving approximate computing for error-tolerant applications

Given the continuing miniaturization of underlying transistors, electronic functional units (circuits/systems) become increasingly susceptible to high-energy radiation, encountered in applications like space. Hence, redundancy is employed as a radiation hardening by design strategy to cope with faults of functional units used in such applications and to maintain their correct operation. Triple modular redundancy (TMR), which is a subset of N-modular redundancy (NMR), that can mask any single fault or a faulty functional unit has been widely used. However, compared to a simplex implementation a TMR implementation requires two additional functional units and a majority voting logic therefore a TMR implementation's area and power overheads are greater by over 200 %. This is burdensome for resource-constrained applications like space where low power and energy efficiency are important considerations. This paper presents a new redundancy strategy involving approximate computing called RESAC for error-tolerant applications such as digital image/video/audio processing, which is used in space systems. We evaluated the feasibility of RESAC for an image processing case study and the results confirm the usefulness. For implementation using a 28-nm CMOS technology, RESAC achieves reductions in area, delay, and power by 22.3 %, 15.3 %, and 24.9 % compared to TMR. Nonetheless, RESAC can address any NMR.

Analysis of Redundancy Techniques for Electronics Design—Case Study of Digital Image Processing

Electronic circuits/systems operating in harsh environments such as space are likely to experience faults or failures due to the impact of high-energy radiation. Given this, to overcome any faults or failures, redundancy is usually employed as a hardening-by-design approach. Moreover, low power and a small silicon footprint are also important considerations for space electronics since these translate into better energy efficiency, less system weight, and less cost. Therefore, the fault-tolerant design of electronic circuits and systems should go hand in hand with the optimization of design metrics, especially for resource-constrained electronics such as those used in space systems. A single circuit or system (also called a simplex implementation) is not fault-tolerant as it may become a single point of failure and is not used for a space application. As an alternative, a triple modular redundancy (TMR) implementation, which uses three identical copies of a circuit or system and a voter to perform majority voting of the circuits and systems outputs, may be used. However, in comparison with a simplex implementation, a TMR implementation consumes about 200% more area and dissipates 200% more power when circuits or systems are triplicated. To mitigate the area and power overheads of a TMR implementation compared to a simplex implementation, researchers have suggested alternative redundancy approaches such as selective TMR (STMR) insertion, partially approximate TMR (PATMR), fully approximate TMR (FATMR), and majority voting-based reduced precision redundancy (VRPR). Among these, VRPR appears to be promising, especially for inherently error-tolerant applications such as digital image/video/audio processing, which is relevant to space systems. However, the alternative redundancy approaches mentioned are unlikely to be suitable for the implementation of control logic. In this work, we analyze various redundancy approaches and evaluate the performance of TMR and VRPR for a digital image processing application. We provide MATLAB-based image processing results corresponding to TMR and VRPR and physical implementation results of functional units based on TMR and VRPR using a 28-nm CMOS technology.

A Hybrid Multi-GPU Implementation of Simplex Algorithm with CPU Collaboration

The simplex algorithm has been successfully used for many years in solving linear programming (LP) problems. Due to the intensive computations required (especially for the solution of large LP problems), parallel approaches have also extensively been studied. The computational power provided by the modern GPUs as well as the rapid development of multicore CPU systems have led OpenMP and CUDA programming models to the top preferences during the last years. However, the desired efficient collaboration between CPU and GPU through the combined use of the above programming models is still considered a hard research problem. In the above context, we demonstrate here an excessively efficient implementation of standard simplex, targeting to the best possible exploitation of the concurrent use of all the computing resources, on a multicore platform with multiple CUDA-enabled GPUs. More concretely, we present a novel hybrid collaboration scheme which is based on the concurrent execution of suitably spread CPU-assigned (via multithreading) and GPU-offloaded computations. The experimental results extracted through the cooperative use of OpenMP and CUDA over a notably powerful modern hybrid platform (consisting of 32 cores and two high-spec GPUs – Titan Rtx and Rtx 2080Ti) highlight that the performance of the presented here hybrid GPU/CPU collaboration scheme is clearly superior to the GPU-only implementation under almost all conditions. The corresponding measurements validate the value of using all resources concurrently, even in the case of a multi-GPU configuration platform. Furthermore, the given implementations are completely comparable (and slightly superior in most cases) to other related attempts in the bibliography, and clearly superior to the native CPU implementation with 32 cores. Keywords—Parallel Computing; Linear Programming; Simplex Method; Multicore Platform; Hardware Acceleration, GPGPU; OpenMP; CUDA

A triangulation and fill-reducing initialization procedure for the simplex algorithm

The computation of an initial basis is of great importance for simplex algorithms since it determines to a large extent the number of iterations and the computational effort needed to solve linear programs. We propose three algorithms that aim to construct an initial basis that is sparse and will reduce the fill-in and computational effort during LU factorization and updates that are utilized in modern simplex implementations. The algorithms rely on triangulation and fill-reducing ordering techniques that are invoked prior to LU factorization. We compare the performance of the CPLEX 12.6.1 primal and dual simplex algorithms using the proposed starting bases against CPLEX using its default crash procedure over a set of 95 large benchmarks (NETLIB, Kennington, Meszaros, Mittelmann). The best proposed algorithm utilizes METIS (Karypis and Kumar in SIAM J Sci Comput 20:359–392, 1998), produces remarkably sparse starting bases, and results in 5% reduction of the geometric mean of the execution time of CPLEX’s primal simplex algorithm. Although the proposed algorithm improves CPLEX’s primal simplex algorithm across all problem types studied in this paper, it performs better on hard problems, i.e., the instances for which the CPLEX default requires over 1000 s. For these problems, the proposed algorithm results in 37% reduction of the geometric mean of the execution time of CPLEX’s primal simplex algorithm. The proposed algorithm also reduces the execution time of CPLEX’s dual simplex on hard instances by 10%. For the instances that are most difficult for CPLEX, and for which CPLEX experiences numerical difficulties as it approaches the optimal solution, the best proposed algorithm speeds up CPLEX by more than 10 times. Finally, the proposed algorithms lead to a natural way to parallelize CPLEX with speedups over CPLEX’s dual simplex of 1.2 and 1.3 on two and four cores, respectively.

Fast implementation of linear and nonlinear simplex growing algorithm for hyperspectral endmember extraction

Like any other techniques for hyperspectral image analysis, high performance computing is also the urgent requirement for endmember extraction. In this paper, we investigate the fast implementation of two endmember extraction algorithms named as new simplex growing algorithm (NSGA) and kernel new simplex growing algorithm (KNSGA), which are recently developed as linear and nonlinear alternative to the SGA algorithm, respectively, and shown to be two promising endmember extraction techniques. Due to the fact that when implementing NSGA and KNSGA only the new endmember is processed at a time with the former endmembers staying the same and remaining unchanged, the time cost of simplex volume computation is reduced by simplifying matrix determinant based on the partitioned matrix determinant formula. Experimental results of simulated and real data show that the proposed fast algorithms can greatly reduce computational complexity while their performance remains invariant.

Parallel distributed-memory simplex for large-scale stochastic LP problems

We present a parallelization of the revised simplex method for large extensive forms of two-stage stochastic linear programming (LP) problems. These problems have been considered too large to solve with the simplex method; instead, decomposition approaches based on Benders decomposition or, more recently, interior-point methods are generally used. However, these approaches do not provide optimal basic solutions, which allow for efficient hot-starts (e.g., in a branch-and-bound context) and can provide important sensitivity information. Our approach exploits the dual block-angular structure of these problems inside the linear algebra of the revised simplex method in a manner suitable for high-performance distributed-memory clusters or supercomputers. While this paper focuses on stochastic LPs, the work is applicable to all problems with a dual block-angular structure. Our implementation is competitive in serial with highly efficient sparsity-exploiting simplex codes and achieves significant relative speed-ups when run in parallel. Additionally, very large problems with hundreds of millions of variables have been successfully solved to optimality. This is the largest-scale parallel sparsity-exploiting revised simplex implementation that has been developed to date and the first truly distributed solver. It is built on novel analysis of the linear algebra for dual block-angular LP problems when solved by using the revised simplex method and a novel parallel scheme for applying product-form updates.

A distributed, scaleable simplex method

We present a simple, scaleable, distributed simplex implementation for large linear programs. It is designed for coarse-grained computation, particularly, readily available networks of workstations. Scalability is achieved by using the standard form of the simplex rather than the revised method. Virtually all serious implementations are based on the revised method because it is much faster for sparse LPs, which are most common. However, there are advantages to the standard method as well. First, the standard method is effective for dense problems. Although dense problems are uncommon in general, they occur frequently in some important applications such as wavelet decomposition, digital filter design, text categorization, and image processing. Second, the standard method can be easily and effectively extended to a coarse grained, distributed algorithm. Such an implementation is presented here. The effectiveness of the approach is supported by experiment and analysis.

A conjugate Rosen’s gradient projection method with global line search for piecewise linear concave optimization

The Kelley cutting plane method is one of the methods commonly used to optimize the dual function in the Lagrangian relaxation scheme. Usually the Kelley cutting plane method uses the simplex method as the optimization engine. It is well known that the simplex method leaves the current vertex, follows an ascending edge and stops at the nearest vertex. What would happen if one would continue the line search up to the best point instead? As a possible answer, we propose the face simplex method, which freely explores the polyhedral surface by following the Rosen’s gradient projection combined with a global line search on the whole surface. Furthermore, to avoid the zig-zagging of the gradient projection, we propose a conjugate gradient version of the face simplex method. For our preliminary numerical tests we have implemented this method in Matlab. This implementation clearly outperforms basic Matlab implementations of the simplex method. In the case of state-of-the-art simplex implementations in C or similar, our Matlab implementation is only competitive for the case of many cutting planes.

Progress in the dual simplex method for large scale LP problems: practical dual phase 1 algorithms

The dual simplex algorithm has become a strong contender in solving large scale LP problems. One key problem of any dual simplex algorithm is to obtain a dual feasible basis as a starting point. We give an overview of methods which have been proposed in the literature and present new stable and efficient ways to combine them within a state-of-the-art optimization system for solving real world linear and mixed integer programs. Furthermore, we address implementation aspects and the connection between dual feasibility and LP-preprocessing. Computational results are given for a large set of large scale LP problems, which show our dual simplex implementation to be superior to the best existing research and open-source codes and competitive to the leading commercial code on many of our most difficult problem instances.

An efficient approach to updating simplex multipliers in the simplex algorithm

The simplex algorithm computes the simplex multipliers by solving a system (or two triangular systems) at each iteration. This note offers an efficient approach to updating the simplex multipliers in conjunction with the Bartels–Golub and Forrest–Tomlin updates for LU factors of the basis. It only solves one triangular system. The approach was implemented within and tested against MINOS 5.51 on 129 problems from Netlib, Kennington and BPMPD. Computational results show that the new approach improves simplex implementations.

Advanced preprocessing techniques for linear and quadratic programming

The paper presents an overview on the preprocessing techniques of linear programming. A new reduction technique is also introduced and the presolve is extended to mixed integer and quadratic programming problems. Numerical results are presented to demonstrate the impact of presolving in interior point and simplex implementations. The demonstrative results are given on large-scale linear, mixed integer and quadratic programming test problems.

Parallelizing the Dual Simplex Method

We study the parallelization of the steepest-edge version of the dual simplex algorithm. Three different parallel implementations are examined, each of which is derived from the CPLEX dual simplex implementation. One alternative uses PVM, one general-purpose System V shared-memory constructs, and one the PowerC extension of C on a Silicon Graphics multi-processor. These versions were tested on different parallel platforms, including heterogeneous workstation clusters, Silicon Graphics multi-processors, and an IBM SP2. We report on our computational experience.

A Subtree-Partitioning Algorithm for Inducing Parallelism in Network Simplex Dual Updates

This paper reports on the development of a very efficient method for partitioning the network simplex basis subtree in which dual values must be updated during a pivot. The partitioning procedure may be concurrently executed by multiple processes. The resulting rapid decomposition of the subtree allows an arbitrary number of processes to be utilized in the actual dual update. This approach alleviates a primary limitation of the most efficient parallel network simplex implementation published to date. The new code performs at least as well as the previous implementation on medium-scale problems and reduces average solution time by over 34% on large-scale problems.

The Water Resource Management System (WREMS): Linking data management and operational optimization

There have been many management science models proposed for optimizing various operational aspects of municipal water systems. However, our practical experience indicates that water managers view such modeling attempts with skepticism. Too much effort has been spent on developing elegant, complex models, instead of focusing on providing water managers with systems that support their total information needs. This paper presents the design and development of the Water Resource Management System (WREMS). The main distinction of the WREMS system is that the optimization model used to offer operational guidance is imbedded within an inventory data management information system, the "hook" that interests the city water managers in trusting and, ultimately, using the system. The optimization model incorporated within WREMS is a customized network simplex imple-mentation of a minimum cost network flow problem, designed to assist in making daily decisions such as which wells to produce and how to manage water levels in reservoirs. Details of the linkages between the data management and optimization components of the system are highlighted, illustrating how IS and MS/OR can be synergistically coupled.

Centralized and decentralized stochastic routing models in telecommunication networks

With the advent of non-hierarchical routing in circuit-switched telecommunication networks, on-line routing policies have been developed with the objective of optimizing some measure of gain or performance. These policies are decentralized. However, traditional planning models are centralized models. We present a decentralized routing model to be used in network planning. We compare it theoretically and empirically with a centralized multicommodity flow model previously presented. The two models are solved by the same type of algorithm, a convex simplex implementation, adapted differently according to the model. Comparative results between planning models reproducing the two policies are discussed.

A Dual Simplex Implementation of a Constraint Selection Algorithm for Linear Programming

The constraint selection approach to linear programming begins by solving a relaxed version of the problem using only a few of the original constraints. If the solution obtained to this relaxation satisfies the remaining constraints it is optimal for the original LP. Otherwise, additional constraints must be incorporated in a larger relaxation. The procedure successively generates larger subproblems until an optimal solution is obtained which satisfies all of the original constraints. Computational results for a dual simplex implementation of this technique indicate that solving several small subproblems in this manner is more computationally efficient than solving the original LP using the revised simplex method.

Solving Piecewise-Linear Programs: Experiments with a Simplex Approach

We report tests of CPLP, a piecewise-linear simplex implementation based on the XMP subroutine library. Using test problems of various shapes and origins, we judge CPLP against a comparable linear bounded-variable simplex code that is applied to equivalent linear programs. Our evidence indicates that CPLP is typically 2 to 3 times faster in solving the test problems that are most significantly piecewise-linear; previously predicted advantages of the piecewise-linear simplex approach are observed to occur, though with different combinations of advantages being relevant in different cases. The savings achieved by CPLP appear likely to extend, moreover, to the costs involved in formulating, generating and analyzing piecewise-linear programs. INFORMS Journal on Computing, ISSN 1091-9856, was published as ORSA Journal on Computing from 1989 to 1995 under ISSN 0899-1499.

A Dual Simplex Implementation of a Constraint Selection Algorithm for Linear Programming

A Dual Simplex Implementation of a Constraint Selection Algorithm for Linear Programming

A Dual Simplex Implementation of a Constraint Selection Algorithm for Linear Programming

A Dual Simplex Implementation of a Constraint Selection Algorithm for Linear Programming

Implementing interior point linear programming methods in the Optimization Subroutine Library

This paper discusses the implementation of interior point (barrier) methods for linear programming within the framework of the IBM Optimization Subroutine Library. This class of methods uses quite different computational kernels than the traditional simplex method. In particular, the matrices we must deal with are symmetric and, although still sparse, are considerably denser than those assumed in simplex implementations. Severe rank deficiency must also be accommodated, making it difficult to use off-the-shelf library routines. These features have particular implications for the exploitation of the newer IBM machine architectural features. In particular, interior methods can benefit greatly from use of vector architectures on the IBM 3090™ series computers and “super-scalar“ processing on the RISC System/6000™ series.