LLM is All You Need : How Do LLMs Perform On Prediction and Classification Using Historical Data

Abstract

This study investigates the utility of large language models (LLMs) in performing traditional machine learning tasks such as prediction, and explores the potential of refinement architectures to enhance their effectiveness in these roles. Utilizing the Titanic survival dataset, we conducted a com- parative analysis using both conventional machine learning tools and LLM-based approaches. Our findings indicate that while LLMs differ fundamentally from traditional ML models in prediction tasks, there exist specific architectural modifications, termed Thought Refinement Architectures, which can significantly improve their performance. These results highlight the potential for inte- grating LLMs into traditional ML workflows, thereby expanding their applicability and enhancing predictive accuracy.