Minimum Precision Requirements of General Margin Hyperplane Classifiers

Abstract

Margin hyperplane classifiers such as support vector machines have achieved considerable success in various classification tasks. Their simplicity makes them suitable candidates for the design of embedded intelligent systems. Precision is an effective parameter to trade-off accuracy and resource utilization. We present analytical bounds on the precision requirements of general margin hyperplane classifiers. In addition, we propose a principled precision reduction scheme based on the trade-off between input and weight precisions. We present simulation results that support our analysis and illustrate the gains of our approach in terms of reducing resource utilization. For instance, we show that a linear margin classifier with precision assignment dictated by our approach and applied to the “two versus four” task of the MNIST dataset is $\sim 2\times $ more accurate than a standard 8 bits low precision implementation in spite of using $\sim 2\times 10^{4}$ fewer 1 bit full adders and $\sim 2\times 10^{3}$ fewer bits for data and weight representation.