Cloud big data decision support system for machine learning on AWS: Analytics of analytics

Abstract

Machine Learning algorithms on large datasets can be executed in the Cloud. Amazon Web Services (AWS) provides over 60 different On-Demand EC2 instances [1]. The instance prices range from $0.0059 (t2.nano) to $14.4 (p2.16xlarge) per hour. We decided to build an automatic recommendation system to choose the best instance for a dataset and a machine learning algorithm to optimize time and money spent. After running multiple algorithms for different Big Data sets on assorted AWS instances and collecting the results in the NoSQL DynamoDB database, we have trained machine learning models to predict time and cost using assorted regression ML methods.