Probabilistic Error Models for Machine Learning Kernels Implemented on Stochastic Nanoscale Fabrics

Abstract

Presented in this paper are probabilistic error models for machine learning kernels implemented on low-SNR circuit fabrics where errors arise due to voltage overscaling (VOS), process variations, or defects. Four different variants of the additive error model are proposed that describe the error probability mass function (PMF): additive over Reals Error Model with independent Bernoulli RVs (REM-i), additive over Reals Error Model with joint Bernoulli random variables (RVs) (REM-j), additive over Galois field Error Model with independent Bernoulli RVs (GEM-i), and additive over Galois field Error Model with joint Bernoulli RVs (GEM-j). Analytical expressions for the error PMF is derived. Kernel level model validation is accomplished by comparing the Jensen-Shannon divergence DJS between the modeled PMF and the PMFs obtained via HDL simulations in a commercial 45nm CMOS process of MAC units used in a 2nd order polynomial support vector machine (SVM) to classify data from the UCI machine learning repository. Results indicate that at the MAC unit level, DJS for the GEM-j models are 1-to-2-orders-of-magnitude lower (better) than the REM models for VOS and process variation errors. However, when considering errors due to defects, DJS for REM-j is between 1-to-2-orders-of-magnitude lower than the others. Performance prediction of the SVM using these models indicate that when compared with Monte Carlo with HDL generated error statistics, probability of detection pdet estimated using GEM-j is within 3% for VOS error when the error rate pη ≤ 80%, and within 5% for process variation error when supply voltage Vdd is between 0.3V and 0.7V. In addition, pdet using REM-j is within 2% for defect errors when the defect rate (the percentage of circuit nets subject to stuck-at-faults) psaf is between 10−3 and 0.2.