Automated Machine Learning on High Dimensional Big Data for Prediction Tasks

Abstract

Machine learning techniques have become deeply embedded in our daily lives. Artificial intelligence (AI) is a term used to describe the development and creation of “thinking machines” capable of emulating, learning, and replacing human intelligence. Machine Learning is one of the subsets of AI that has recently made major strides in enhancing real-time application processes and efficiency. The rapid growth of ML applications, as well as the field’s rapid expansion, has created a huge demand for ML methods that can be employed quickly and do not require any expert knowledge. However, many data scientists in charge of handling high-dimensional big data spend 60% of their time cleaning data and processing it with machine learning algorithms. AutoML is a field of study that tries to make machine learning more automated over time by using optimization and machine learning ideas. Automated machine learning (AutoML) is emerging as a promising approach for developing an AI system without the need for human intervention, and an increasing number of academics are focusing on AutoML. This paper at first presents an overview of AutoML. Secondly, the custom hybrid algorithm of the AutoML pipeline and its steps are discussed. Then, a comparison and experimental results of various AutoML tools for high-dimensional data have been illustrated. Finally, the open challenges and prospective research initiatives have been explained.