A Unified Search Framework for Data Augmentation and Neural Architecture on Small-Scale Image Data Sets

Abstract

Data augmentation is an effective technique to enrich the training data’s diversity and reduce the risk of overfitting. However, different datasets have distinct preferences on various augmentation techniques. Recently, automated data augmentation (auto-augmentation), which could engineer augmentation policy automatically, drew a growing interest. Previous auto-augmentation methods usually utilize a Density Matching strategy, which highly depends on a large data scale to ensure a precise policy evaluation. When facing small-scale datasets, it usually achieves an inferior performance. To address the problem, an improved method named Augmented Density Matching is proposed by augmenting the train data with policies uniformly sampled from a prior distribution, making the policy evaluation more precise. Moreover, we propose a unified search framework for data augmentation and neural architecture (USAA) by formulating the search processes with one formulation. As a result, both optimal augmentation policy and neural architecture could be obtained within one round of search process. Extensive experiments have been conducted on a bag of medical datasets with general small scales, and the results show that the proposed Augmented Density Matching and USAA can outperform the state-of-the-art auto-augmentation and AutoML approaches, respectively.