Интеграция унаследованных приложений в экосистему научных исследований "Эни"

Abstract

The article describes the development of a digital platform for the Eni research ecosystem, based on a microservice approach that automatically scales the computing resources of the system when working with big data. A brief description of the structure of the digital platform, its main components and the functionality of the basic subsystems is provided. Two types of digital platform consumers are presented - legacy and platform applications. Their capabilities and limitations are shown. The subsystem of distributed computing is described, which provides continuous management and monitoring of the microservice architecture of the platform, in particular, it is responsible for: load balancing, service discovery, system recovery after failures, end-to-end authentication, "canary rollouts", access control. The high-performance computing subsystem is presented. It includes models and methods for organizing parallel calculations on various hardware devices, such as a multiprocessor system, a cluster of computing devices connected by a local network, and a graphics processor. As software solutions for organizing parallel computing, multithreaded data processing technologies, MPI messaging interface, and CUDA technology are used. Also described is a data mining subsystem designed to deploy different types of neural networks with different architectures, including direct distribution, convolutional, recurrent and generative neural networks. The integration of the legacy application MD-SLAG-MELT v13.0 into the ecosystem of scientific research using platform integration adapters is presented. The architectures and the main components of the original software package and the integrated software package MD-SLAG-MELT v14.0 are presented. The results of load testing with the analysis of performance metrics and response time of an inherited application, when processing big data, are presented.