A Survey of Deep Active Learning for Foundation Models

Abstract

Active learning (AL) is an effective sample selection approach that annotates only a subset of the training data to address the challenge of data annotation, and deep learning (DL) is data-intensive and reliant on abundant training data. Deep active learning (DeepAL) benefits from the integration of AL and DL, offering an efficient solution that balances model performance and annotation costs. The importance of DeepAL has been increasingly recognized with the emergence of large foundation models that depend heavily on substantial computational resources and extensive training data. This survey endeavors to provide a comprehensive overview of DeepAL. Specifically, we first analyze and summarize various sample query strategies, data querying considerations, model training paradigms, and real-world applications of DeepAL. In addition, we discuss the challenges that arise in the era of foundation models and propose potential directions for future AL research. The survey aims to bridge a gap in the existing literature by organizing and summarizing current approaches, offering insights into DeepAL and highlighting the necessity of developing specialized DeepAL techniques tailored to foundation models. By critically examining the current state of DeepAL, this survey contributes to a more profound understanding of the field and serves as a guide for researchers and practitioners interested in DeepAL techniques.