SATB-Nets: Training Deep Neural Networks with Segmented Asymmetric Ternary and Binary Weights

Abstract

Deep convolutional neural networks (CNNs) are both computationally intensive and memory intensive, making them difficult to deploy on embedded systems with limited hardware resources efficiently. To address this limitation, we introduce SATB-Nets, a method which trains CNNs with segmented asymmetric ternary weights for convolutional layers and binary weights for the fully-connected layers. We compare SATB-Nets with previous proposed ternary weight networks (TWNs), binary weight networks (BWNs) and full precision networks (FPWNs) on CIFAR-10 and ImageNet datasets. The result shows that our SATB-Nets model outperforms full precision model VGG16 by 0.65% on CIFAR-10 and achieves up to \(29\times \) model compression rate. On ImageNet, there is \(31\times \) model compression rate and only 0.15% accuracy degradation over the full-precision AlexNet model of Top-1 accuracy.