Survey on Capsule Network’s Depth Scalability and Learned Feature Spatial Relationships Retention

Abstract

AbstractIn 2017, the first working architecture of capsule networks called Capsule Network with Dynamic Routing (CapsNet) was introduced and shown to perform better than Convolutional Neural Networks (CNNs) in the MNIST dataset. It was also speculated that capsule networks’ ability to retain spatial relationships among learned features are better than CNNs. Ever since then, much research has been devoted to improving capsule network architectures. However, most proposed architectures are relatively shallow and not scalable depth-wise. Also, some works have shown that the said retention ability is not as good as speculated. In this survey, we reviewed research works on capsule networks that focus on depth scaling and the said retention ability. We found that the routing algorithm used in a capsule network plays a crucial role in both the depth scalability and the said retention ability. We believe that future research on capsule networks should be focused on improving the routing algorithm to ensure the success of capsule networks.KeywordsCapsNetConvolutional neural networkSpatial relationshipsDepth scalability