Enabling Awareness of Quality of Training and Costs in Federated Machine Learning Marketplaces

Abstract

The proliferation of data and machine learning (ML) as a service, coupled with advanced federated and distributed training techniques, fosters the development of federated ML marketplaces. One important, but under-researched, aspect is to enable the stakeholder interactions centered around the quality of training and costs in the marketplace and the service models in federated ML training. This paper conceptualizes a federated ML marketplace and proposes a framework to enable the awareness of the quality of training and a variety of costs where both data providers and ML model consumers can easily value the contribution of each data source to ML model performance in nearly real-time. This improves the transparency and explainability iv.r.t. the quality and costs for participation in the marketplace. Based on that, we design and implement the quality of training and cost awareness framework for an edge federated ML marketplace. Experimental realistic scenarios show the usefulness of cost and quality details that provides insightful information for various purposes, including potential budget management and training optimization.