Cluster Management System Research Articles

On the Optimization of Kubernetes toward the Enhancement of Cloud Computing

With the vigorous development of big data and cloud computing, containers are becoming the main platform for running applications due to their flexible and lightweight features. Using a container cluster management system can more effectively manage multiocean containers on multiple machine nodes, and Kubernetes has become a leader in container cluster management systems, with its powerful container orchestration capabilities. However, the current default Kubernetes components and settings have appeared to have a performance bottleneck and are not adaptable to complex usage environments. In particular, the issues are data distribution latency, inefficient cluster backup and restore leading to poor disaster recovery, poor rolling update leading to downtime, inefficiency in load balancing and handling requests, poor autoscaling and scheduling strategy leading to quality of service (QoS) violations and insufficient resource usage, and many others. Aiming at the insufficient performance of the default Kubernetes platform, this paper focuses on reducing the data distribution latency, improving the cluster backup and restore strategies toward better disaster recovery, optimizing zero-downtime rolling updates, incorporating better strategies for load balancing and handling requests, optimizing autoscaling, introducing better scheduling strategy, and so on. At the same time, the relevant experimental analysis is carried out. The experiment results show that compared with the default settings, the optimized Kubernetes platform can handle more than 2000 concurrent requests, reduce the CPU overhead by more than 1.5%, reduce the memory by more than 0.6%, reduce the average request time by an average of 7.6%, and reduce the number of request failures by at least 32.4%, achieving the expected effect.

Modern food protection

The article provides data on grown vegetables such as tomatoes, cucumbers, radishes, peppers, various greens such as dill, celery, ginger, basil and cilantro, as well as fruits such as apples, pears, grapes, pomegranates that need protection. So, a lot of work is being done to protect food products, including by farms and dekhkan farms to protect them. Everywhere they are switching to a cluster management system. According to the information service of the Ministry of Agriculture of the Republic of Uzbekistan, today there are 342.5 thousand hectares of orchards in our country. Of these, 250.7 thousand hectares belong to farms and agricultural enterprises, the rest belongs to households. The main task is to separate and sell ready-made food products or semi-finished products on the markets, without throwing away processed products of various types and varieties.

Eigen: End-to-End Resource Optimization for Large-Scale Databases on the Cloud

Increasingly, cloud database vendors host large-scale geographically distributed clusters to provide cloud database services. When managing the clusters, we observe that it is challenging to simultaneously maximizing the resource allocation ratio and resource availability. This problem becomes more severe in modern cloud database clusters, where resource allocations occur more frequently and on a greater scale. To improve the resource allocation ratio without hurting resource availability, we introduce Eigen, a large-scale cloud-native cluster management system for large-scale databases on the cloud. Based on a resource flow model, we propose a hierarchical resource management system and three resource optimization algorithms that enable end-to-end resource optimization. Furthermore, we demonstrate the system optimization that promotes user experience by reducing scheduling latencies and improving scheduling throughput. Eigen has been launched in a large-scale public-cloud production environment for 30+ months and served more than 30+ regions (100+ available zones) globally. Based on the evaluation of real-world clusters and simulated experiments, Eigen can improve the allocation ratio by over 27% (from 60% to 87.0%) on average, while the ratio of delayed resource provisions is under 0.1%.

Modeling of Distributed Control System for Network of Mineral Water Wells

The article is devoted to solving the problem of designing a distributed control system for a network of production wells on the example of mineral water deposits in the Caucasus Mineral Waters region, Russia. The purpose was to determine the set of parameters of the control system to ensure technologically effective and safe operating modes of mineral water deposits. A mathematical model of the deposit was developed taking into account the given configuration and production rate of the network of the wells. The detailed algorithm is presented for designing the control system under consideration based on the frequency concept of analysis and synthesis for distributed control systems. The experimental tests and model validation were performed at the production wells facility of “Narzan”, Kislovodsk, Russia. The results of modeling and field experiments confirmed the adequacy of the mathematical model and the effectiveness of the proposed algorithm. The authors came to the conclusion that the adapted mathematical model can be used to create a regional automated field cluster management system for monitoring, operational management and forecasting the nature of real hydrogeological processes and ensuring their stability.

AutoInfer: Self-Driving Management for Resource-Efficient, SLO-Aware Machine=Learning Inference in GPU Clusters

As Internet-of-Things (IoT) keeps growing, IoT-side intelligence services, such as intelligent personal assistant, healthcare surveillance, and smart home service, offload more and more complex machine learning (ML) inference workloads to cloud clusters. GPUs have been widely adopted to accelerate the execution of these ML inference workloads. However, current cluster management systems guarantee low tail latency for ML inferences using resource over-provisioning and small batch sizes, resulting in a serious waste of GPU resources and increasing the service costs greatly. To mitigate poor GPU utilization, we present AutoInfer, a self-driving cluster management system for ML inference serving in GPU clusters, where users express only the latency and accuracy requirements for their workloads without needing to specify the model variant, GPU provisioning strategy, and batching mechanism. AutoInfer extends the matrix factorization model to automatically recommend model variants for each new incoming ML inference workload with respect to latency and accuracy requirements, by identifying similarities to previously scheduled workloads. During runtime, AutoInfer leverages online telemetry data and deep reinforcement learning to adaptively adjust the GPU allocation and batch size to account for load variations while minimizing the effects on tail latency Service Level Objectives (SLOs). Testbed experiments show that AutoInfer is able to improve the average GPU utilization by up to 77% and keep the tail latency SLO violations under 5.5%.

Application of Social Network Analysis to Visualization and Description of Industrial Clusters: A Case of the Textile Industry

This paper discusses the issues of industrial cluster analysis. Initially, the authors explore theoretical approaches to understanding the clusters phenomenon and their identification and analysis. Looking at industrial clusters as network structures connected by various forms of interaction between members, such as ownership linkages, transactions, the presence of common counterparts, and participation in arbitration processes, the authors propose visualizing clusters using social network analysis metrics. This approach helps to address one of the main difficulties when contacting the members of industrial clusters for a subsequent survey or in-depth interviewing. The analysis concludes with a discussion of the proposed method as a way to identify cluster members and determine the most significant ones that are the primary nodes of the network. These key members usually possess enough relevant information about the structure, coordination mechanisms, general strategy, and cluster management system. Therefore, it is possible to limit the list of interviewed respondents without a substantial loss in empirical data quality. The case of the textile industry cluster presented in this paper confirms the applicability of social network analysis to the visualization and description of industrial clusters.

Amber industry cluster management organization

The review of the short-term experience of industrial clustering in Russia provides a summary on the effective management structuring for the recent Amber Industry Cluster in the Kaliningrad Region. Unlike the international practice allowing flexible rather than rigid management of clusters, the domestic approach assumes a strict vertical in the cluster management system to control considerable government sponsorship of activities of the cluster members. The structure of a cluster management includes the federal and regional executive authorities, specific departments of state corporations, as well as the actually governing bodies of operational and commercial activities (management companies, specialized organizations). However, such strict requirement appears to be unsupported with specific recommendations concerning the composition and structure of a cluster administration, as well as distribution of management functions within the administration. As a result, it is impossible to utilize in full the positive potential of the cluster-wise development of territories due to the actual inability of a management company or a specific organization to arrange and direct problem solution within the integrated innovation-and-investment projects set for a cluster. To this end, this article recommends a methodical approach to creation of an organization structure of a cluster management, as well as a composition and distribution of authorities between its elements. The distinctive feature of this approach is the orientation both at traditional and realistic objectives and functions of management according to its present-day concept.

A multiparty session typing discipline for fault-tolerant event-driven distributed programming

This paper presents a formulation of multiparty session types (MPSTs) for practical fault-tolerant distributed programming. We tackle the challenges faced by session types in the context of distributed systems involving asynchronous and concurrent partial failures – such as supporting dynamic replacement of failed parties and retrying failed protocol segments in an ongoing multiparty session – in the presence of unreliable failure detection. Key to our approach is that we develop a novel model of event-driven concurrency for multiparty sessions. Inspired by real-world practices, it enables us to unify the session-typed handling of regular I/O events with failure handling and the combination of features needed to express practical fault-tolerant protocols. Moreover, the characteristics of our model allow us to prove a global progress property for well-typed processes engaged in multiple concurrent sessions, which does not hold in traditional MPST systems. To demonstrate its practicality, we implement our framework as a toolchain and runtime for Scala, and use it to specify and implement a session-typed version of the cluster management system of the industrial-strength Apache Spark data analytics framework. Our session-typed cluster manager composes with other vanilla Spark components to give a functioning Spark runtime; e.g., it can execute existing third-party Spark applications without code modification. A performance evaluation using the TPC-H benchmark shows our prototype implementation incurs an average overhead below 10%.

Aerodynamics of the ejection-injection method of separation of entomophages

This article reveals the peculiarities of the transition to a cluster management system in the agro-industrial complex (AIC) of the Republic of Uzbekistan based on the use of innovative techniques and technology. It is shown that, despite the widespread use of agrotechnical, chemical methods of combating pests, diseases and sulfur plants, the issues of mechanization of biological control agents still remain unresolved, and improvised means of controlling them do not give the expected results. As the top priority tasks of the study, the author poses the issues of mechanization of the technological process of dispersal of eggs or pupae of entomophages using an unmanned drone equipped with an ejector-injection dispenser.The work also contains questions of the theory of dispersal of eggs of entomophages through an ejector-injection dispenser under the action of the lifting force of a local air jet or a suction pneumodynamic (injection) force created in the area of the dispenser windows, which allowed not only to significantly improve the design of the dispenser, but also to expand the technological capabilities of the proposed unmanned unit. apparatus.

Актуалізація використання кластерної моделі в інноваційному розвитку економіки України

Unfortunately, one of the main problems of the Ukrainian economy is the State’s inability to concentrate available resources on the main, promising directions and involve Ukrainian businesses, both large and medium-sized, into this process. Therefore, the creation and development of clusters as an element of an innovative model is necessary, and the study of the issue of their functioning is quite relevant. The article is aimed at further studying the processes of clustering in Ukraine. An attempt was made, based on the theoretical basis for the development of innovative processes, as well as coming from the worldwide and domestic experience of cluster formation and development, to give an own vision of the essence and classification of clusters, to determine their priority types for the Ukrainian economy, to offer certain cluster strategies and criteria for evaluating the efficiency of cluster formations. The results of the study are presented as follows. First, the authors’ own definition of the cluster as a voluntary association of geographically concentrated, independent economic entities and public authorities without the formation of a separate legal entity for joint activities in the field of entrepreneurship for the more effective use of resources and stimulation of innovative development is proposed. Secondly, on the basis of this definition, an appropriate classification of clusters has been developed. Thirdly, the order of formation of an innovative cluster is suggested, which should consist of several stages: emergence of the idea of the cluster and substantiation of the need to create it; defining the territory for the creation of the cluster based on the assessment of its competitive potential; formation of cluster management system; evaluation of the efficiency of its performance. At this, specific actions are recommended that must be taken at each stage. Fourth, assessments of the effectiveness of the cluster functioning under the conditions of using certain indicators are provided.

ASSOCIATION OF MUNICIPAL NON-PROFIT ENTERPRISES AS AN IMPORTANT COMPONENT OF THE CLUSTER APPROACH IN THE ORGANIZATION OF THE HEALTHCARE SYSTEM OF TERRITORIAL COMMUNITIES

Purpose. The aim of the article is to substantiate the theoretical and methodological basis for the formation of cluster associations in the healthcare system of territorial communities, against the background of various forms of organization and set of relations that arise within the cluster association in the context of decentralization of powers. Methodology of research. The methodological basis of the study is the systemic and institutional approaches, according to which all processes of management of the health care system of the local community comply with current legislation. The following methods were used: structural and functional research – to analyse the main trends of the social and economic situation of development; statistical comparisons – to assess the impact of the medical industry on regional development. The generalization of methodological approaches of developed countries to the formation of a cluster management system has shown that the regional healthcare system is able to function effectively considering management decisions and the results obtained from their implementation, which is reflected in assessing its condition at the appropriate stages of development. Findings. It is substantiated that the effectiveness of healthcare regulation in modern conditions largely depends on ensuring access of all participants in social and economic relations in the field of healthcare to the necessary resources. Regulation needs to focus on realizing the potential of the health sector through the most efficient use of its resources. It is found that one of the most optimal tools for integrating the interests of government, business and the population in this area is the so-called “cluster approach” given its universal mechanisms, high efficiency, and simplicity of organization. Territorial clusters are an active force in economic development and a powerful tool for networking between participants in regional innovation systems in different countries of the world. It is proved that the cluster provides important synergetic effects, without which modern life is impossible, it is a way to mobilize stakeholders (state, business, scientific community) and a tool for science, technology, and innovation policy to achieve the goal of innovative development and technological modernization of the economy in medicine, pharmaceuticals and related industries. It is revealed that the corporation of communal non-profit enterprises of the health care system acts as an active communicator and integrator for the participants of the future medical cluster and health care organizations. In the future, such an association will become an international center of communication and exchange of competencies, and the launch of a portal on medical technology and innovation in healthcare – will broadcast this positive experience to the masses, talk about advances in medicine. The features of business associations (corporations) are systematized. It is established that this is a complex organization, which is created based on a combination of material interests of the participating enterprises, operates on the basis of the memorandum or articles of association as a legal entity. The advantages of merging enterprises in the field of healthcare are formed, namely: merging of communal non-profit enterprises. The main ones are the effective use of human resources, material and technical base, improvement of medical services for patients. Originality. It is proposed to create a corporation of public non-profit enterprises in the healthcare system of the local community, which, unlike the existing one, will allow optimal and efficient use of available resources, generate innovative solutions, and create preconditions for interaction of all medical sector participants. For the first time, after the implementation of the project to create a corporation that will unite non-profit medical enterprises into a horizontally integrated structure, it is proposed to create a full-fledged medical cluster of the territorial community on its platform. The corporation is being transformed into a management company of the regional medical cluster, the main functions of which will be reduced to the general coordination of the cluster’s work and project management. Practical value. Specific recommendations have been developed for the establishment of a corporation – the association of communal non-profit enterprises, with a clear strategy, comprehensive methodology and a formed road map, as part of a full-fledged medical cluster of the territorial community. Key words: healthcare; cluster; territorial community; corporation; municipal enterprise; non-profitenterprise.

Implementation of the cluster economy concept in the accounting system

The paper is devoted to the fundamental aspects of clustering the economy of Ukraine. This topic is important because modern economy of the developed countries, which was previously based on traditional forms, is now impossible without innovation clusters development. Therefore, the investigation of the transformation of innovation systems based on clusters under the influence of crises in the world economy and the consequences of the COVID-19 pandemic is very important scientific and practical task. The theoretical experience and essence of the organizational mechanism of cluster structures are considered in this paper. It is determined that the concept of cluster economy offers new types of interaction between business entities, government and other institutions that are interested in increasing competitiveness. Therefore, the study of the cluster approach should be based on theories investigating the relationship between competition and cluster models. It is established that economy clustering strengthens the relationships between economic entities and contributes to the regions competitiveness increase. It is the strengthening of ties between cluster members that promotes the implementation of innovations. Conceptual approaches to cluster structure management are considered. It is determined that the network form of management is based on the use as the subject of cluster management of an independent business unit, which replaces the functional management structures of independent association members. This form of management is based on the high level of trust between the cluster members, as well as on the development of relations between economic entities. The cluster formation requires appropriate changes in the field of management, and this, in turn, affects its main information carrier – the accounting system. The author emphasizes the need to create accounting and analytical support for the cluster management system. The accounting information system should reflect the facts of financial and economic activities of the integrated structure and provide management with the ability to make tactical and strategic decisions. That is why the development of accounting will contribute to obtain more complete and reliable information about the results of the cluster as a group of legally independent, but interconnected in economic and financial terms of organizations.

Cluster system management

Cluster system management (CSM) was co-designed with the Department of Energy Labs to provide the support necessary to effectively manage the Summit and Sierra supercomputers. The CSM system administration tools provide a unified view of a large-scale cluster and the ability to examine and understand data from multiple sources. CSM consists of five components: 1) application programming interfaces (APIs) and infrastructure; 2) Big Data Store; 3) support for reliability, availability, and serviceability (RAS); 4) Diagnostic and Health Check; and 5) support for job management. APIs and infrastructure provide lightweight daemons for compute nodes, hardware and software inventory collection, job accounting, and RAS. Logs, environmental data, and performance data are collected in the Big Data Store for analysis. RAS events can trigger corrective actions by CSM. Diagnostic and Health Check are provided through a diagnostic framework and test results collection. To support job management, CSM coordinates with the Job Step Manager to provide an overlay network of JSM daemons. CSM is an open source and available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/IBM/CAST</uri> . Documentation can be found at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://cast.readthedocs.io</uri> .

Scalable, fault-tolerant job step management for high-performance systems

Scientific applications on the CORAL systems demanded a fault-tolerant, scalable job launch infrastructure for complex workflows with multiple job steps within an allocation. The distinct design of IBM's Job Step Manager (JSM) infrastructure, working in concert with Load Sharing Facility (LSF) and Cluster System Management (CSM), achieves these goals. JSM demonstrated launching over three-quarters of a million processes in under a minute while providing efficient process management interface for exascale-based services to communication libraries, such as parallel active messaging interface and message passing interface, and tools over the management network. JSM relies on the parallel task support library to provide a fault-tolerant, scalable communication medium between the JSM daemons. Application workflows using job steps harness the unique resource set abstraction concept in JSM to manage CPUs, GPUs, and memory between groups of processes, possibly in discrete job steps, sharing a node. The resource set concept gives JSM the opportunity to better organize process placement to optimize, for example, CPU-to-GPU communication. Applications that need complete control over the shaping of the resource sets and the placement, binding, and ordering of processes within them can leverage JSM's co-designed Explicit Resource File mechanism. This article explores the design decisions, implementation considerations, and performance optimizations of IBM's JSM infrastructure to support scientific discovery on the CORAL systems.

Management system of grain production cluster of the region

The purpose of the study is to develop an organizational model of the grain production cluster in the Nizhny Novgorod region, which will contribute to the formation of sustainable production and, as a result, increase the efficiency of the industry under study and strengthen food security in the region. The design of organizational structures was carried out by the method of analogies, which include the development of typical management structures of industrial and economic organizations and the definition of boundaries and conditions for their application. A cluster management system was developed, which consists of four blocks: a control block; production unit; processing and storage unit; block of education and science. Scientific novelty lies in the author’s approach to the development of a control system for the production of grain clusters. The proposed form will significantly increase the level of production sustainability among potential cluster members and strengthen food security in the Nizhny Novgorod region.

ФОРМИРОВАНИЕ ИННОВАЦИОННОГО МЕДИЦИНСКОГО КЛАСТЕРА В СТРУКТУРЕ РЕГИОНА

The article is devoted to the consideration of the formation of an innovative medical cluster in the structure of the region. The importance of cluster policy as a tool for increasing regional competitiveness is emphasized. A classification of the concept of cluster policy is given from the position of the state’s role in stimulating innovation, as well as the formation by cluster participants of an interaction mechanism based on a breakthrough idea. Tendencies in the development of global cluster initiatives in medicine are systematized. The main types of medical clusters existing in the world practice are highlighted and their key characteristics are given. A comparison of the level of development of European and Russian clusters in the biopharmaceutical and medical industries is given. The main features of the innovative medical cluster in the Sverdlovsk region are formulated, its model is presented. The vision of the medical cluster management system is shown.

Features of the cluster as an organization and system institute and their impact on the accounting development

The institutional nature of the cluster is investigated. It is established that the cluster is defined as an association, which has a voluntary character and which priority is to increase the competitiveness of products and the economic development of the region. The properties of the cluster functioning and its dissimilarity with other types of enterprise associations are determined. It is established that since the current legislation of Ukraine does not define the cluster as an object of economic and legal policy, there are some differences in the economic literature regarding establishing its status as an independent association in the field of entrepreneurship. Formation and effective use of accounting information in the cluster management system can be based on an institutional approach. The consideration of a cluster from the standpoint of institutional theory makes it possible to comprehensively examine it from economic, psychological, sociological, legal and other points of view. The defining features of the cluster as a socio-economic phenomenon are the presence of institutes (institute of cooperation, institute of competition and partnership, institute of communication, institute of coordination), establishment of partnerships between the participating companies, institutional forms of agreements, spatial and technological distribution of the organization, informal distribution of organization, regulation and culture. The authors distinguish the specific institutional features of the cluster (organizational and legal form, presence of participants of different spheres and relationships between them, geographical concentration, management) that influence the construction of the accounting system, in particular the organization and accounting of joint activities in the cluster, the formation of financial statements, access to information and the creation of transaction costs, the sharing of common assets, capital and liabilities.

Effect of security and trustworthiness for a fuzzy cluster management system in VANETs

Recently, smart cities and Internet of Things (IoT) applications, such as Vehicular Ad-hoc Networks (VANETs) and Opportunistic networks have been deeply investigated. However, these kinds of wireless networks have security problems. Also, the vehicles can be not trustworthy, which brings different communication problems. In this work, we present a Fuzzy Cluster Management System (FCMS) for VANETs. We present and compare two fuzzy-based system models: FCMS1 and FCMS2 for clustering of vehicles in VANETs. For FCMS1, we use three input parameters: Vehicle Relative Speed with Vehicle Cluster (VRSVC), Vehicle Degree of Centrality (VDC) and Vehicle Security (VS). The output parameter is Vehicle Remain or Leave Cluster (VRLC). For FCMS2, we consider four input parameters by adding Vehicle Trustworthiness (VT) as a new parameter. We evaluate both systems by simulations. The simulation results show that vehicles with the same VRSVC and with high VDC, VS and VT values have higher possibility to remain in the cluster. By comparing FCMS1 and FCMS2, we found that the FCMS2 can manage better the vehicles in the cluster than FCMS1.

A fuzzy-based approach for cluster management in VANETs: Performance evaluation for two fuzzy-based systems

In recent years, due to the growing interest in smart cities and Internet of Things (IoT) applications, several innovation types of networks, such as Vehicular Ad-hoc Networks (VANETs) and Opportunistic networks have been deeply investigated. Moreover, inter-vehicle communication has attracted attention because it can be applicable not only to alternative networks but also to various communication systems. In this paper, we present a Fuzzy-Based Cluster-Management System (FBCMS) for VANETs. We present and compare two fuzzy-based system models: FBCMS1 and FBCMS2 for clustering of vehicles in VANETs. We evaluate both systems by simulations. The simulation results show that by selecting vehicles with high GS, RA, SC and DC values, the vehicles have high connectivity with other vehicles and are more secure, so they will be selected as the cluster head of the cluster. By comparing FBCMS1 and FBCMS2, we found that the FBCMS2 can manage better the vehicles in the cluster than FBCMS1.

Corp to Cloud: Google’s Virtual Desktops

Over one-fourth of Googlers use internal, data-center-hosted virtual desktops. This on-premises offering sits in the corporate network and allows users to develop code, access internal resources, and use GUI tools remotely from anywhere in the world. Among its most notable features, a virtual desktop instance can be sized according to the task at hand, has persistent user storage, and can be moved between corporate data centers to follow traveling Googlers. Until recently, our virtual desktops were hosted on commercially available hardware on Google’s corporate network using a homegrown open-source virtual cluster-management system called Ganeti. Today, this substantial and Google-critical workload runs on GCP (Google Compute Platform). This article discusses the reasons for the move to GCP, and how the migration was accomplished.