Regularizing Data for Improving Execution Time of NLP Model

Abstract

Natural language processing (NLP) is a very important part of machine learning that can be applied to different real applications. Several NLP models with huge training datasets are proposed. The primary purpose of these large-scale NLP models is the downstream tasks. However, because of the diversity and rapidly increasing the size of these datasets, they consume a lot of resources and time. In this study, we propose a state-of-the-art method to reduce the training time of NLP models on downstream tasks while maintaining accuracy. Our method focuses on removing unimportant data from the input data set and optimizing the padding of tokens to reduce the processing time for the NLP model. Experiments are conducted on many different GLUE benchmark datasets demonstrated that our method can reduce the most up to 57% in training time compared to other methods.