Approximation of Transcendental Functions With Guaranteed Algorithmic QoS by Multilayer Pareto Optimization

Abstract

Design space exploration of approximate computing is deemed tough. While restricting the optimization to reduce approximate errors at the individual-function level might simplify the problem to be tackled, the increased hardware cost should be well justified by the improvement of the quality of service (QoS) at the algorithm level. In light of the loose correlation between atomic errors and algorithmic QoS, it is imperative but computationally expensive to explore approximate design space incorporating a variety of alternatives for evaluation. Despite being addressed extensively in the literature, we consider the transcendental sigmoid ( $\sigma$ ) and hyperbolic tangent (tanh) functions as typical examples manifesting the dilemma of approximate computing. In this work, we leverage Pareto-front optimization at three hierarchies, from the parameter layer up to the structure and algorithm layers, to effectively tailor the approximate design space of the $\sigma $ and tanh functions for hardware-efficient as well as algorithm-feasible implementations. Our investigations are performed based on a comprehensive design library consisting of representative approximate schemes in radically different hardware structures featuring both linear and nonlinear approximations. As tested on MNIST with 99% accuracy and Wisconsin Breast Cancer data set with 96.6% accuracy, we identify a novel and compact shift-based approximation that directly applies to the two’s complement numbers achieving a $1.5\times $ area reduction and a $3.4\times $ energy reduction compared with the prior art. Provided the flexibility in approximate functions is of concern, we also present a uniform yet concise structure for implementing the Chebyshev-polynomials-based approximation adaptive in silicon to arbitrary nonlinear functions and error constraints.