HAMMER: Hardware-Friendly Approximate Computing for Self-Attention With Mean-Redistribution And Linearization

Abstract

The recent advancement of the natural language processing (NLP) models is the result of the ever-increasing model size and datasets. Most of these modern NLP models adopt the Transformer based model architecture, whose main bottleneck is exhibited in the self-attention mechanism. As the computation required for self-attention increases rapidly as the model size gets larger, self-attentions have been the main challenge for deploying NLP models. Consequently, there are several prior works which sought to address this bottleneck, but most of them suffer from significant design overheads and additional training requirements. In this work, we propose HAMMER, hardware-friendly approximate computing solution for self-attentions employing mean-redistribution and linearization, which effectively increases the performance of self-attention mechanism with low overheads. Compared to previous state-of-the-art self-attention accelerators, HAMMER improves performance by <inline-formula><tex-math notation="LaTeX">$1.2-1.6\times$</tex-math></inline-formula> and energy efficiency by <inline-formula><tex-math notation="LaTeX">$1.2-1.5\times$</tex-math></inline-formula> .