Abstract
Now-a-days, in the world of enterprise, machine learning workloads have become mainstream. However, there is an abundance of choices that can be made around multi-cloud infrastructure and machine learning toolkits, making it complex to balance their costs and performance. Microservices architecture has been the preferred architecture style for a few years now and there’s been rapid growth in its adoption, never failing to provide exceptionally testable & maintainable services. To have a lot more simplified services management, deployment and to orchestrate tools, Kubernetes is recommended. Kubeflow, a known and widely adopted open source container management platform that manages machine learning stack on Kubernetes. This paper discusses the development and validation of Kubeflow components such as PyTorch, TensorFlow, & Notebook Servers. It includes PodDefault functionalities for notebooks and container builder API to build docker images using Kaniko. Using Helm, Kubeflow upgrade operation is performed to enhance the configured resources whenever required for the distributed training jobs & workloads. Hence, providing data scientists a scalable platform to run machine learning workloads without having to worry about resources, costs, time, and portability.
Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callMore From: International Journal for Research in Applied Science and Engineering Technology
Disclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
