Abstract
Most capsule network designs rely on traditional matrix multiplication between capsule layers and computationally expensive routing mechanisms to deal with the capsule dimensional entanglement that the matrix multiplication introduces. By using Homogeneous Vector Capsules (HVCs), which use element-wise multiplication rather than matrix multiplication, the dimensions of the capsules remain unentangled. In this work, we study HVCs as applied to the highly structured MNIST dataset in order to produce a direct comparison to the capsule research direction of Geoffrey Hinton, et al. In our study, we show that a simple convolutional neural network using HVCs performs as well as the prior best performing capsule network on MNIST using 5.5× fewer parameters, 4× fewer training epochs, no reconstruction sub-network, and requiring no routing mechanism. The addition of multiple classification branches to the network establishes a new state of the art for the MNIST dataset with an accuracy of 99.87% for an ensemble of these models, as well as establishing a new state of the art for a single model (99.83% accurate).
Highlights
Capsules have become a more active area of research since [1], which demonstrated near state of the art performance on MNIST [2] classification by using capsules and a routing algorithm to determine which capsules in a previous layer feed capsules in the subsequent layer
In [6], we proposed a capsule design that used element-wise multiplication between capsules in subsequent layers and relied on backpropagation to do the work that prior capsule designs were relying on routing mechanisms for
We proposed using a simple convolutional neural network and established design principles as a basis for a network architecture
Summary
Capsules (vector-valued neurons) have become a more active area of research since [1], which demonstrated near state of the art performance on MNIST [2] classification (at 99.75%) by using capsules and a routing algorithm to determine which capsules in a previous layer feed capsules in the subsequent layer. In [6], we proposed a capsule design that used element-wise multiplication between capsules in subsequent layers and relied on backpropagation to do the work that prior capsule designs were relying on routing mechanisms for. We referred to this capsule design as homogeneous vector capsules (HVCs). We directly extend the work of [7,1] on capsules applied to MNIST by applying HVCs to MNIST.By using this capsule design, we avoid
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
