A Machine Learning Process-based Training Task Execution Method in the Field of Power Grid Regulation

Abstract

With the rapid development of artificial intelligence technology, the power industry has entered the era of big data, business data is rapidly accumulated, and the traditional Spring-Boot-based microservice architecture raises more and more requirements for hardware resources, which can no longer meet our requirements for service invocation performance, data consistency, elastic scaling and flexible deployment requirements. In response to the above problems, the distributed container technology which is based on Kubernetes and Docker is introduced, and a unified JSON-based machine learning process description language structure is proposed, some useful configuration templates are provided for machine learning training processes, including algorithm selection, hyper-parameter setting, loss function, optimization function and execution plan. In response to the needs of enterprise business development, a machine learning model training task scheduling system adapted to business scenarios in the field of power grid regulation is designed and constructed, which solves the problems of inability to reuse sample data and waste of resources and realizes resource isolation and elastic scaling. By building a visualized machine learning task process, implementing model training and evaluation, supporting real-time display of the execution status of each algorithm node, the platform implements a multi-tenant resource isolation and elastic scaling containerized machine learning model training environment. Keywords