Abstract

A model of magnitude relative error in floating point multiplication is developed and is analyzed stochastically for various choices of computer design parameters. These parameters include the base, the type of rounding rule, the number of guard digits, and whether the post-arithmetic normalization shift (if needed) is done before or after rounding. Under the assumption of logarithmic distribution for the fraction (mantissa), the major stochastic conclusions are: (1) The average magnitude relative error in multiplication increases as the base increases. (2) The average error is minimized by selecting the machine base to be binary with a hidden bit, and is larger for base 16. Other models of relative error are developed and analyzed.

Full Text
Published version (Free)

Talk to us

Join us for a 30 min session where you can share your feedback and ask us any queries you have

Schedule a call