Large Data Arrays Research Articles

Big Data Intelligent Analysis Technology for the Study of Spatial and Time Trends in the Development of Large Cities

The purpose of this research is to study modern problems and prospects for solving the processing of big data containing information about real estate, as well as the possibility of practical implementation of the methodology for processing such data arrays by designing and filling a special graphic abstraction «metahouse» on a practical example.Materials and methods. The study includes a review of bibliographic sources on the problems of big data analysis and their application in the modern field of construction of large cities. During the study, a technique for presenting data in a graphical form – abstraction was used. The mathematical basis of the technique is the use of multidimensional spaces, where measurements are the characteristics of individual objects. Computer simulation of a practical problem was applied using the C# programming language. Big data storage is based on the MongoDB server. To visualize data, a Web interface based on HTML and CSS is used.Results. In the course of the work, the main characteristics of big data were identified, and the specifics of data arrays consisting of information about real estate objects in a large city were described. When processing data consisting of information about real estate objects of a large city, certain difficulties arise. Thereby, methods for effectively solving the set practical task of processing and searching for patterns in a large data array were proposed: «metahouse» abstraction, data aggregator.Tabular data were obtained for a large city by analyzing three million records containing more than 10 data groups, with a basic set of parameters: floor, number of floors, price, area, living area, kitchen area, type, operation. A MongoDB cluster was created on several computers, each of which was working with its own data set without intermediate results.The results of the computational experiment showed that when using the graphical form (vector) of big data representation, the costs and time for interpreting mining data were reduced.Combining big data processing methods and their presentation through graphical abstraction allows getting new results from existing data sets.Conclusion. During the study, it was found that the presentation of groups of the received data in a graphic image has a number of advantages over a tabular presentation of data (a vector image is easy to scale, the ability to compare without plotting).The proposed way for visualizing big data by constructing abstract vector images is an alternative to traditional tables, allowing you to take a different look at data arrays and the results of their processing. The results obtained can be used both for the primary study of big data processing technologies and as a basis for the development of real applications in the following areas: analysis of changes in the area of houses over time, analysis of changes in the number of floors of urban development, dynamics and distribution of supply and demand, etc.

Machine Learning Based Result Analysis Prediction of Student’s Academic Performance

Abstract: Failure at any phase of education happens frequently. The rise in drop - out rates is a result of numerous reasons. Poor grades are one of the biggest causes of school abandonment. This has an influence on performance because so many students find it difficult to adjust to the institution's learning environment once they get there. Other factors include student participation in extracurricular tasks and politics. Learners' performances frequently tend to be unsatisfactory for these different predictable and unpredictable reasons, which have an impact on development. As a result, it's important to examine undergraduate results to identify the real reasons for students' varied level of performance. The primary goal of our research work is to identify the numerous variables that affect achievement at the under-graduation level. Therefore, the main motivation behind this effort is to help students identify the factors that lead to their performance so that they can take action to change their results. The learners, course teachers, and others will have the opportunity to improve the environment once the major elements have been recognized and assessed. This paper highlights the importance of using student data to drive improvement in education planning. It then presents techniques of how to obtain knowledge from databases such as large arrays of student data from academic Institution databases. To early predict the student’s academic performance, we have proposed deep learning model of Recurrent Neural (RNN) classifier. This proposed methodology is compared with various traditional machines learning classification models and deep learning classifier.

The Pareto Principle in the Functions Approximating Processing the Results of Psychological and Sociological Research

The possibilities of approximating functions in the processing of large data arrays of the results of psychological and sociological research are analyzed. An approach to the processing of areas limited by approximating functions is substantiated in accordance with the Pareto principle. Threshold values are obtained that ensure the division of areas in the ratio 1/4 for the most frequently used for approximating the results of studies of approximating functions. The conformity of the law of distribution of histograms of scientific achievements of scientists, approximating them with curves, according to Pearson's goodness-of-fit criterion is checked.

Recurrent Neural Network-Based Autoencoder for Problems of Automatic Time Series Analysis at Power Facilities

Digitalization of the energy sector leads to an increase in the volume and rate of data collection. A primary barrier to the proper management of the technological data is the lack of data labeling corresponding to emergency modes, power equipment technical state, etc. Thus, despite the large amount of data, there is a shortage of labeled data suitable for training, validating and testing the machine learning models. Labeling by an expert takes too much time, so there is an actual task to automatically identify data fragments that are potentially of interest. The aim of the work is to develop an algorithm for prioritizing the fragments of the time series using the compact recurrent autoencoder. To achieve the goal, a neural network architecture was developed based on recurrent encoding and decoding cells, capable of unsupervised learning. The model was tested on two data sets: a synthetic sinusoidal signal with missing values and electric current measurements with thermal limit deviations. The substantial results of the work are the compact architecture of the autocoding model and the high interpretability of the output. The most significant achievements of the study are both the autocoding neural network model, which does not require initial assumption about the type of deviations, and the proposed algorithm for prioritizing the data fragments. The significance of the results is prooved by the reduction of the time for analyzing and labeling large data arrays with technological parameters of the electrical networks, which allows using these data for training, validating and testing.

Management of the development of a metallurgical company on the basis of the algorithm of the pre-investment analysis

The article is devoted to the development of methodological approaches to assessing the effectiveness of investments in innovations and repairs of Russian metallurgical companies based on an analysis of the state of innovation and technological potential in order to maintain competitiveness in the face of tougher sanctions. The analysis of sources and directions of investments and comparison of the current internal rate of return in the leading companies in the metallurgical sector were carried out: Severstal, MMK, NLMK, EVRAZ; their investment activity was studied and the predominant selfinvestment in the development of companies and interest in financial parameters and global trends were confirmed to the detriment of the material component of production. The problem of disunity of owners, managers of the main production, repair facilities and top management in making investment decisions is detailed and methods for its solution are proposed through the use of a company's preinvestment analysis algorithm based on the methodology of predictive analysis of large data arrays. The methodological basis is the theory of strategic management and investment analysis. Methods: classification, comparison, grouping, economic and mathematical modeling, predictive analysis and algorithms. The methodology for evaluating investment projects using the modified internal rate of return (MIRR) has been refined. An algorithm for analyzing the investment environment of metallurgical companies is proposed based on taking into account the service life of equipment and ranking it depending on the need for repair or renewal. The updated analytical tools will be of interest to industry enterprises, corporations and government agencies in substantiating the strategy of innovative development. The developed approaches are of scientific interest in the study of contradictions between production, top management, owners of metallurgical companies and the state.

Statistical study of the level of world digitalization

The modern study of digitalization is based on the differentiation of indicators and types of activities into separate subgroups, one way or another characterizing the level of the digital economy, within a limited (subjective) expert opinion. This study was based on the Network Readiness Index (NRI), as one of the indicators created to assess the level of development of information and communication technologies (ICT) and the network economy (based on Internet technologies) in different countries of the world. The article presents the results of a statistical study of the society’s network readiness index using factor analysis methods to qualitatively change the dimension of a large data array. The classification of the countries of the world according to significant integral indicators of the network readiness index by cluster analysis methods is carried out, conclusions are drawn on the formation of the components of the network readiness index in various typological groups. On the basis of the developed classification, mathematical models were built for each independent group of countries to establish the relationship of the effective attribute from the explanatory integral components – f1 – sufficiency of network space conditions and f2 – technological activity. The resulting factors were GDP, million US dollars (GDP) and the human development index, HDI (HDI) as indicators illustrating the dynamics of changes in the economy and the conditions for the development of society, which should be directly affected by digitalization processes.

Possibilities of using HADOOP and R to analyze large arrays of geospatial data

The main problems associated with the processing of Big Data, in particular arrays containing geospatial data, are analyzed. The Hadoop platform is considered one of the basic approaches to the analysis of large data arrays and the possibility of its integration with the R environment. The potential possibilities of using the Hadoop platform for solving practical problems in the process of analyzing geospatial and spatiotemporal data are analyzed

Measurement of the thermal diffusivity of optical materials and products by a new thermographic express method that does not require cutting samples from bulk

Thermal diffusivity a and thermal conductivity λ are important for many building, structural and functional material applications. They determine the intensity of heat transfer, the quality of thermal insulation, the rate of heating / cooling, reaching a stationary mode, and the efficiency of power equipment. In laser technology, the radiation strength of the optical components of the system depends upon them, and in laser technologies with material removal they determine the speed and quality of processing. Most methods for measuring a and λ in solid materials require cutting out samples of a certain geometry, which makes them unsuitable for testing finished products. The paper proposes and describes an express method for determining a and λ in translucent materials, which does not require cutting a sample from a controlled object. It consists in the analysis of a non-stationary temperature field on the surface of the test object using a high-speed thermal imaging camera. The unsteady heating spot was created by a focused laser beam. It was switched on abruptly and operated in the mode of continuous irradiation with a constant intensity during the entire time of measurements. Heat propagated from this spot to the periphery, creating a non-stationary temperature field containing information about a and λ. The a value was extracted from the primary data using original algorithms and software. A thermal imager, as a recorder of a dynamic temperature field, provides a number of advantages – non-contact, high speed and a large amount of information (each of the many hundreds of thousands of pixels of a professional thermal imager matrix is a temperature sensor in a small surface area). Measurements of a and λ in semitransparent materials of laser optics have their own specifics. The low radiation absorption coefficient and the possible curvature of the surface (for example, in lenses) require special measures, which are described in the article. Due to the large amount of information contained in the dynamic patterns of the thermal field and the possibility of averaging over a large data array, the RMS of the thermal diffusivity measurement does not exceed 2 %.

МОДЕЛЮВАННЯ ЕКОНОМІЧНОГО РОСТУ МЕТОДАМИ ІНТЕЛЕКТУАЛЬНОГО АНАЛІЗУ ДАНИХ

The article substantiates the expediency of using intelligent data analysis in the study and forecasting of economic growth. Intelligent data analysis is a powerful tool that can help researchers and politicians understanding economic growth more deeply and making reasonable decisions by forming state policy. In addition, intelligent data analysis can be used to determine the potential areas of economic growth and investment opportunities. Attention is focused on using CART (predictive modeling tool, which has proved its high efficiency in understanding the non-obvious relationships between several variable statistical importance) and associative rules (intelligent data analysis, which can work with big data arrays and generate clear and effective conclusions, supported by statistical significance indicators). It is concluded that the methods of intelligent data analysis, such as CART and associative rules, are crucial for understanding the complex patterns that exist in large arrays of economic data. The advantages of these methods make them appropriate and effective tools for modeling and understanding of levers and tools of economic growth. Based on the analysis of data for 119 countries of the world for 2021 (the period of post-pandemic recovery), it was established that in order to promote economic growth, countries with low levels of export and import need to focus on the development of national industry. In addition, developing an economic policy aimed at promoting the recovery of the economy, it is necessary to increase the democratic status of the country. In general, countries that are less dependent on exports and imports tend to have more stable and self-sufficient economies. This can be explained by a number of factors, such as a strong domestic market, the development of national industry and less vulnerability to global economic shocks. In addition, a low level of imports may indicate that a country is able to produce and consume a significant portion of its own goods, which may contribute to further economic growth and development. It is important to note that these relationships are based on highlevel statistical correlations.

Извлечение именованных сущностей из рецензий к исходному коду

This paper addresses the problem of named entities recognition from source code reviews. The paper provides a comparative analysis of existing approaches and proposes its own methods to improve the quality of problem solving. Proposed and implemented improvements include: methods to deal with data imbalances, improved tokenization of input data, the use of large arrays of unlabeled data, and the use of additional binary classifiers. To assess quality, a new set of 3,000 user code reviews was collected and manually labeled. It is shown that the proposed improvements can significantly increase the performance measured by quality metrics, calculated both at the token level (+22%) and at the entire entity level (+13%).

Stationary reversibles processes MA and ARMA

The term mathematical diagnostics was introduced by V. F. Demyanov in the early 2000s. The simplest problem of mathematical diagnostics is to determine the relative position of a certain point p and the convex hull C of a finite number of given points in n-dimensional Euclidean space. Of interest is the answer to the following questions: does the point p belong to the set C or not? If p does not belong to C, then what is the distance from p to C? In general problem of mathematical diagnostics two convex hulls are considered. The question is whether they have common points. If there are no common points, then it is required to find the distance between these hulls. From an algorithmic point of view, the problems of mathematical diagnostics are reduced to special problems of linear or quadratic programming, for the solution of which there are finite methods. However, when implementing this approach in the case of large data arrays, serious computational difficulties arise. Infinite but easily implemented methods come to the rescue, which allow obtaining an approximate solution with the required accuracy in a finite number of iterations. These methods include the MDM method. It was developed by Mitchell, Demyanov and Malozemov in 1971 for other purposes, but later found application in machine learning. From a modern point of view, the original version of the MDM method can be used to solve the simplest problems of mathematical diagnostics. This article gives a natural generalization of the MDM-method, oriented towards solving general problems of mathematical diagnostics. The equivalence of the general problem of mathematical diagnostics and the problem of linear separation of two finite sets with the largest width of the margin is established.

Very-large-scale integration device for parallel vertical group computing the sum of squared differences

Is a paper that proposes a new method for computing sum-of-squares differences in a parallel vertical environment. The method is based on a group approach, which allows you to divide the task into several subtasks and calculate them in parallel. The article considers the problem of calculating the sum of squared differences between elements of large data arrays. Applying traditional methods of calculating such sums in parallel environments can be inefficient due to the exchange of large amounts of data between nodes. The proposed method allows to reduce the amount of transmitted data and increase the efficiency of calculations. The article proposes a new method for calculating the sum of squared differences, which allows to increase the efficiency of calculations in a parallel vertical environment. Testing of the method on different data sets shows its high efficiency compared to traditional methods of calculating sums of squared differences in parallel environments. The proposed method can be applied in various areas that require the processing of large volumes of data, and allows to increase the efficiency of calculations and reduce their execution time. The methods, algorithms and structures of devices for computing the sum of squared differences have been analyzed and their defects have been defined in the article. It has been defined that the device for computing the sum of squared differences should support the next: high device utilization; the use of capabilities and benefits of VLSI; short-term development and moderate price. The development of the device has been suggested by computing the sum of squared differences using modularity principles, coordination between data flow and computing capability of the device, pipelining and space parallelism, localization and simplification of links with elements. The proposed method can be useful for researchers in the fields of parallel computing and data processing, and can find applications in various fields such as data science, machine learning, image processing, and bioinformatics.

MDM method for solving the general quadratic problem of mathematical diagnostics

The term mathematical diagnostics was introduced by V. F. Demyanov in the early 2000s. The simplest problem of mathematical diagnostics is to determine the relative position of a certain point p and the convex hull C of a finite number of given points in n-dimensional Euclidean space. Of interest is the answer to the following questions: does the point p belong to the set C or not? If p does not belong to C, then what is the distance from p to C? In general problem of mathematical diagnostics two convex hulls are considered. The question is whether they have common points. If there are no common points, then it is required to find the distance between these hulls. From an algorithmic point of view, the problems of mathematical diagnostics are reduced to special problems of linear or quadratic programming, for the solution of which there are finite methods. However, when implementing this approach in the case of large data arrays, serious computational difficulties arise. Infinite but easily implemented methods come to the rescue, which allow obtaining an approximate solution with the required accuracy in a finite number of iterations. These methods include the MDM method. It was developed by Mitchell, Demyanov and Malozemov in 1971 for other purposes, but later found application in machine learning. From a modern point of view, the original version of the MDM method can be used to solve the simplest problems of mathematical diagnostics. This article gives a natural generalization of the MDM-method, oriented towards solving general problems of mathematical diagnostics. The equivalence of the general problem of mathematical diagnostics and the problem of linear separation of two finite sets with the largest width of the margin is established.

ТЕНДЕНЦИИ РАЗВИТИЯ И ЦИФРОВОЙ ТРАНСФОРМАЦИИ СЕЛЬСКОХОЗЯЙСТВЕННОГО ПРОИЗВОДСТВА

The article examines the readiness of agricultural production for digital transformation. The importance of using digital technologies for the innovative development of agriculture and improving the collection of analytical information is shown. This implies, first of all, the stimulation of investment activity in the industry that creates competitive and progressive technologies for the agricultural sectors. It is concluded that the trends in the development of domestic agriculture are directly related to its digitalization, the introduction of robotics, IT technologies, bio- and nanotechnologies into production, the analytics of large data arrays, new approaches to methods for collecting statistical information based on space technologies and the use of unmanned aerial vehicles. aircraft, as well as the formation of a new technological order.

MODERN FINANCIAL TECHNOLOGIES OF INTRODUCING DIGITALIZATION INTO THE FINANCIAL SYSTEM

Introduction. Many researchers studying the causes of the economic crisis and its consequences, come to the conclusion that this process is bifurcational, in other words, it gives rise to the transition of the economic system to a new stage. The use of a "synergistic approach", in which the object or phenomenon under study is considered as a kind of "evolutionary process", when studying the patterns of functioning of the financial system in the Russian Federation, seems to be the most promising. This approach allows us to describe the operation of the financial and banking systems at the level of the entire state, which certainly allows us to develop and apply modern high-tech products and services created as a result of the digitalization of the financial system at the level of all subjects of the Russian Federation. Methodology. Research methods, such as statistical, analytical, observation and grouping methods, are aimed at identifying objects of digital technology transformation, the use of products and services in the financial system. When solving the problem of predicting the properties of the "digital economy" object, it is important to track the dynamics of the studied indicators. The analysis of large arrays of data gives a large number of analytical studies, the description of which allows us to build a predictive model of strategic development and demand behavior. Results. It has been determined that digitalization in the financial environment is necessary in terms of the transfer of products, services that should be available to the public, organizations, enterprises, be comfortable, and most importantly, aimed at obtaining financial services through a provider, through remote access. Conclusion. The key trends in the development of the financial market in our country, identified by the Central Bank of Russia, create conditions for increasing the efficiency, manufacturability and customer focus of market players.

Parallel methods and algorithms for solving problems of digital filtering of data arrays

A quasisystolic computation method for solving digital filtering problems of various dimensions on specialized computing means – quasisystolic structures is proposed. This method makes it possible to build parallel-pipeline computation algorithms that are optimal in speed and memory usage. Based on the ideas of the pyramid method for parallelization of cycles, parallel algorithms with autonomous branches were built for solving filtering problems on clusters, hybrid architectures and computers with a multi-core processor. Theoretical speed-up estimates were obtained, which confirm the high efficiency of the constructed parallel filtering algorithms. For individual algorithms with autonomous branches, real speed-up estimates were obtained, which are in good agreement with the theoretical ones. The obtained results can be used for preliminary processing of large arrays of input data in various subject areas

Results of clinical developments of early triple negative breast cancer drug treatment: ASCO - 2022

Introduction. This article continues the series of reviews of innovative clinical developments related to drug therapy for leading cancers following the ASCO-2022 Annual Congress. This time we are seeing the results of therapy for early triple negative breast cancer (hereinafter referred to as TNR) is a consequence of the fact that TNR accounts for 15–20% of cases of all breast cancers, while it is the subtype of breast cancer (hereinafter referred to as BC), for which the choice of treatment methods
 is significantly limited. In Russia, state tasks have been set within the framework of the Federal project “Fight against cancer”,
 which provide for a significant reduction in mortality from neoplasms, including by improving the quality
 and effectiveness of drug treatment.
 Materials and methods. This scientific review is based on the results of a search in the databases of the digital educational platform MEDtalks.nl, PubMed/Medline and Google. Dictionary forms were used in the search strings: "kanker", "vroeg stadium", "triple negatief", "mammacarcinoom", "clinical trials", "ASCO-2022" and others in Dutch and English.
 Results and conclusion. The results presented in this article once again substantiate the prospects for further determination of algorithms for biomarker-personalized immunotherapy (in particular, pembrolizumab) as part of the combined antitumor treatment of RMZHER–/PR–/HER2– at early stages. Radiogenomics/radioproteomics technologies are a prime example of low-cost diagnostic tools that allow promptly suggesting a personalized tumor portrait in the period before biopsy. Further improvement of clinical oncoinformatics technologies (rather large arrays of omics data have already been accumulated) will contribute to a better understanding of programs for individualized choice of differentiated therapy for cancer patients, including patients with RMZHER–/PR–/HER2–.
 For citation: Andreev DA, Zavyalov AA. Results of clinical developments of early triple negative breast cancer drug treatment: ASCO-2022. City Healthсare. 2022;3(4): 122–131 doi: 10.47619/2713-2617.zm.2022.v.3i4;122–131

Application of data mining technologies to find correspondences of development patterns in large arrays of web data based on Big Data analysis tools

A number of modern scientific studies interpret corporate strategy as a certain way to implement a specific action plan for an enterprise through the development of management decisions in a highly competitive environment with a limited set of resources. Management is a chain of various solutions, as well as tools that enable the practical implementation of these solutions. The relevance of the topic studied in the article is associated with the development of a global trend for digital technologies and solutions that have affected almost all aspects of society. The digitalisation of management activities directly affects the formation of an enterprise management strategy: in this way, automation of administration allows to systematically transfer relations between various groups of business entities, businesses and consumers to a new unique environment created in digital form, which ultimately allows to qualitatively increase the level of development and decision-making methods effectiveness in the field of management. The article defines the practical areas of using data mining technology in the context of the development of the digital economy. The key trends in the development of intelligent data volume technology are highlighted. The authors consider a variant of the architecture that allows optimizing web data processing.

Data processing on digital platforms: topical issues of improving legislation and practice

The article is devoted to the analysis of current legal problems arising in the process of collecting, using and distributing users’ personal data of modern digital platforms. The paper emphasizes that the issue of protecting user data has a significant legal component and allows to take a fresh look at the usual concept of privacy and the content of the right to privacy. On the example of the property of the network effect of digital platforms, the interest of users in building up large arrays of user data by platforms is substantiated and a conclusion is made about the key role of the authority to control the volume and fate of the information collected about a person in the content of the right to privacy. Based on the experience of the European Union, the author substantiates the need to regulate in domestic legislation the system of requirements for the basic and subsequent settings for the confidentiality of digital services, the acts for designing the protection of personal data by operators who process such information. In addition to this, the author highlights advantages of legal regulation of the procedures for transferring data between digital platforms and exercising the right to be forgotten. As conclusions, recommendations are formulated aimed at developing transparency in the procedures for collecting personal data by platforms, countering parsing and leakage of personal data, as well as deformalizing the dissemination of personal information based on the concept of open data.

TRAINING A NEURAL NETWORK TO CALCULATE THE CLOSEST POINT OF APPROACH

The process of training a neural network on calculation of closest point of approach (CPA) between two ships, and testing its performance and accuracy is described in the paper. The architecture of the neural network, the type of input and output data, and creation of training data set are also described in the paper. Feed Forward Neural Networks with Backpropagation algorithm are used; training method is Supervised with Levenberg-Marquardt algorithm. The input data are positions, courses and speeds of vessels in a certain area, the output data are Closest Points of Approach (CPA) between them. The process of writing a script in MATLAB software environment is described. The script allows a user to generate training data with any number of vessels in an area. Comparison of the time spent on CPA calculation using formulas and using neural networks is carried out. It has been proven that when processing large data arrays, the CPA calculation with neural networks is much faster than by means of formulas. After neural networks training process and the calculations results comparison, one neural network with mean squared error of 0.21 is chosen. It can be used for CPA calculations in MATLAB-based simulations. In the future this network might become a base for a collision-avoidance neural network system, which will allow vessels to manoeuvre safely in order to avoid collisions in a certain area.