Big Data Processing Technology Research Articles

Вопросы внедрения аналитических систем больших данных и других достижений цифровизации для повышения эффективности бизнеса горнодобывающих компаний

Currently, digitalization and widespread adoption of digital technologies are significantly changing people's activities in many areas. Digital technologies provide automation of business processes, data management, analytics, they support strategic decision-making and dictate the need to introduce new approaches to doing business in order to increase its efficiency and profitability, as well as to ensure sustainability of companies' development in modern conditions. It needs to be especially mentioned how the big data processing and analysis technologies and other Industry 4.0 achievements are introduced in the mining industry. The use of big data analytical systems in modern production, including the mining industry, provides an integrated approach to processing and analyzing a large amount of information. It also provides organizations with significant advantages reflected at various levels of management and strategic decision-making. The prospects for implementation and development of these digital solutions currently look very encouraging. Effective management of these processes provides companies with significant opportunities and advantages, allowing them to increase competitiveness, optimize the use of resources and increase the efficiency of their business as a whole.

LEVERAGING BIG DATA TECHNOLOGIES FOR ENHANCED PUBLIC PARTICIPATION IN PUBLIC FINANCIAL MANAGEMENT

The article is devoted to the topical issues regarding the implementation of Big Data technologies in public finance management. The application of Big Data has the potential to enhance transparency and accountability in the use of budgetary resources, increase trust in government, improve the efficiency of budget resource utilization, better understand citizens' needs, and engage the public in public finance management. The purpose of the study is to explore theoretical, methodological, and practical aspects, as well as to develop recommendations for the implementation of Big Data processing and analysis technologies to enhance public participation in public financial management. The article examines traditional methods of civil engagement in the budgetary process, identifies their disadvantages, and explores Big Data technology potential based on Computational Linguistics and Machine Learning to strengthen public participation. Developments in sentiment analysis and opinion mining have been adapted to the field of public finance. A generative model for analyzing public sentiment on social networks regarding public finance management has been constructed and tested. The approaches developed for using Big Data technologies can be implemented in the field of public finance to enhance public participation in their management as advisory tools for the realization of representative democracy and require further theoretical elaboration and practical application to improve the analysis of alternative sentiments, prevent manipulation of public opinion, and abuse within the network.

Personalized Learning Resource Push Evaluation Considering Image Big Data Processing Technology

Personalized Learning Resource Push Evaluation Considering Image Big Data Processing Technology

ANALYSIS OF DATA ORGANIZATION SYSTEM ARCHITECTURES OF THE SMART EUROPEAN REGION "CENTER OF EUROPE" BASED ON THE DATA LAKE TECHNOLOGICAL SOLUTION

In the contemporary landscape of digital transformation, the concept of a "smart region" emerges as a key strategy for regional development, utilizing data integration technologies to enhance the quality of life, optimize service delivery, and promote sustainable growth. The designation of the "Centre of Europe" as a smart region is justified by its strategic geographical location, encompassing Transcarpathia and adjacent areas of neighboring countries, coupled with a relatively low population density. This unique context creates compelling premises for rethinking traditional urban frameworks of smart cities on a broader scale, transforming them into a comprehensive smart region model tailored to the specific needs and opportunities of Central Europe. The rationale for implementing the "Centre of Europe" as a smart region stems from its advantageous position at the heart of the continent, which generates unprecedented opportunities for cross-border collaboration, innovation, and connectivity. The low population density of the region further highlights the potential for the adoption of advanced technological solutions capable of addressing a wide array of challenges and needs. By achieving the status of a smart region, the "Centre of Europe" aspires to become a leading example of how digital and technological innovations can be utilized to create an inclusive, efficient, and progressive regional ecosystem. Objective: The primary objective of this research is to conduct a comparative analysis of advanced data organization system architectures and their role in facilitating the transformation of the "Centre of Europe" into an intelligent European region. It also aims to explore the capabilities of utilizing data lakes for the effective management of large data sets and regional development, taking into account the unique geographical and demographic characteristics of the region. The subject of the study encompasses the architectures of big data organization systems in the smart region of the "Centre of Europe". The object of the study involves the processes of storing and processing large data within the smart region. Main characteristics: This article illuminates the main aspects of implementing big data processing technologies in the smart region of the "Centre of Europe", necessitating scalable and flexible solutions for large data storage, the use of efficient data management practices, and the integration of advanced technologies such as cloud computing, the Internet of Things (IoT), and artificial intelligence (AI) to enhance data processing and analytical capabilities. Furthermore, recommendations for regional authorities on adopting these technologies have been formulated, highlighting the importance of collaboration, support for investments in the region's digital infrastructure as key components for achieving the smart region vision. The construction of modern, high-tech data storage facilities using the Data Lake architecture enables the simultaneous use and processing of large volumes of data from various fields and sources. The authors have proposed conceptual models for forming data complexes from different functional areas of the smart region.

Fabrication of Sb2S3/Sb2Se3 heterostructure for potential resistive switching applications

The exponential growth of large data and the widespread adoption of the Internet of Things (IoT) have created significant challenges for traditional Von Neumann computers. These challenges include complex hardware, high energy consumption, and slow memory access time. Researchers are investigating novel materials and device architectures to address these issues by reducing energy consumption, improving performance, and enabling compact designs. A new study has successfully engineered a heterostructure that integrates Sb2Se3 and Sb2S3, resulting in improved electrical properties. This has generated significant interest in its potential applications in resistive switching. In this study, we have demonstrated the fabrication of a device based on Sb2S3/Sb2Se3 heterostructure that exhibits resistive switching behavior. The device has different resistance states that can be switched between high and low resistance levels when exposed to an external bias (−1 V to 0 V to 1 V). It also has good non-volatile memory characteristics, including low power consumption, high resistance ratio (∼102), and reliable endurance (∼103). The device enables faster data processing, reduces energy consumption, and streamlines hardware designs, contributing to computing advancements amidst modern challenges. This approach can revolutionize resistive switching devices, leading to more efficient computing solutions for big data processing and IoT technologies.

Integration of building information modeling and big data processing technologies

Integration of building information modeling and big data processing technologies

A novel collaborative bearing fault diagnosis method based on multi-scale dynamic fusion network under speed fluctuating condition

Improving bearing fault diagnosis accuracy under speed fluctuation is a challenge in engineering applications. With the development of big data processing technology, a new solution, multi-sensor complementary information, has emerged. However, single-scale dimension compression, which is adopted in most multi-sensor data fusion methods, captures only a small amount of valuable information. To deal with this deficiency, a multi-scale dynamic fusion network (MSDFN) is proposed. First, considering the existence of non-stationary features in the fluctuating speed signal, the FReLU function is adopted to activate the features after considering contextual information. Then, multi-sensor features are fused by multiple scales to obtain richer feature information, and fusion features at different scales are weighted by using the attention mechanism. Finally, batch normalization is employed to standardize the variable speed feature distribution. The validity of the MSDFN is proved by conducting fault diagnosis experiments on two bearings under speed fluctuating conditions. Experimental results indicate that the MSDFN is not only effective in identifying various types of fault samples, but also shows higher stability in multiple trials when compared with other methods.

APPLICATION OF BIG DATA TECHNOLOGIES TO ASSESS THE NATURAL MOISTURE OF THE TERRITORY

Due to the sharp changes in climatic conditions in Western Siberia, the most pressing problem is food security associated with forecasting crop yields. There is a need to estimate the natural wetness of the area on the basis of agroclimatic indicators, among which the sum of active air temperatures and precipitation is currently the most widely used.
 Background. However, for a comprehensive assessment of the wetness of the territory, it is necessary to take into account the climate energy resources associated with evaporation of different time resolutions (day, decades, months, vegetation period, year). The initial meteorological parameters are described in the form of poorly structured information of a large volume. Therefore, various technologies and algorithms of machine processing are used in the work.
 Purpose. The application of big data technologies to assess the natural moisture of the territory
 Materials and methods. With the help of the high-level Python programming language and engineering libraries, a comprehensive assessment of the territory was carried out using the example of Mirny of the Kochenevsky District of the Novosibirsk Region in the context of long-term average data and two years 2019 and 2020. That the use of machine processing technologies related to NoSQL data requests, creation of data set and big data slices allows to store and process meteorological parameters using cloud services of different time resolution. This makes it possible to significantly reduce the time for a comprehensive assessment of the territory according to agroclimatic parameters. As a result of the work, precipitation distribution, temperature, relative air humidity, evaporability, humidification coefficients (Ivanov-Vysotsky and Selyaninov hydrothermal coefficient) were obtained.
 Results. The paper proposes to use technologies of big data processing using Python, including pre-processing of poorly structured hydrometeorological data, execution of NoSQL queries, compilation of summary reports on agroclimatic parameters. Pre-processing consists of processing hourly meteorological data, filling gaps in the data, making slices in big data to process multi-temporal queries (day, month, growing season, year). By the example of Mirny farm in Kochenevsky district of Novosibirsk region (Russian Federation), big data processing was performed to calculate agroclimatic parameters to assess the natural moisture content of the area and yield forecasting.
 Conclusion. The practical significance of the work is as follows:
 - the application of big data processing technologies has significantly reduced the time for the labor-intensive process of assessing agrometeorological parameters;
 - obtained aggregated meteorological parameters of different temporal resolution (hours, days, decades, months) allowed us to identify a strong variability of agroclimatic conditions for the territory of Mirny farm Kochenevsky district, located in the forest-steppe zone of Western Siberia;
 - perform an integral assessment of agroclimatic conditions by calculating the integral indices of moisture, climate continentality, and agroclimatic potential.

Optimization of ecological sports innovation teaching path based on multiple intelligence evaluation model

Abstract Rural sports tourism is an emerging industry that integrates rural tourism and sports tourism and plays an important role in broadening the field of sports and cultural consumption and driving rural economic development. Under the background of big data, how to better integrate rural sports culture and sports tourism becomes an important issue to improve the quality of rural tourism. This paper analyzes the big data of rural sports tourism on the Internet review platform based on big data processing technology, combined with the investigation and research on the current situation of rural sports tourism resources in a town. The big data analysis shows that rural sports tourists are only 7% more female than male, and the overall gender ratio is relatively balanced. In terms of age composition, the largest number of tourists were between 36 and 45 years old, accounting for 42.6%. In addition, rural sports tourists aged 26 to 60 accounted for 82.6% of all, indicating that rural sports tourism is dominated by middle-aged and elderly people. In terms of travel time, the largest number of rural sports tourists chose 2 to 3 days, accounting for 33.8%, and only 2.3% traveled for more than 7 days. In the choice of travel mode, 37.6% of tourists are organized by travel agencies, accounting for the largest number of travelers. From the big data emotional analysis of the evaluation words of sports tourists, the most frequent occurrence is characteristic, reaching 435 times, and the least frequent is danger, but also 89 times. Meanwhile, convenience and health also appear more frequently. Comprehensive analysis of the above, this paper believes that rural sports tourism should promote more local characteristics of sports culture and highlight differentiation while strengthening personal safety protection measures for tourists.

Evaluation Method of Japanese Teaching Effect Based on Feature Offset Compensation

This study applies big data processing technology and parallel computing methods to assess the teaching effect of Japanese in the Flipped Classroom (FC) and task-based teaching mode. We propose a model for evaluating the Japanese teaching effect in this mode using feature offset compensation. We employ distributed mining of association rules to detect the teaching effect and extract ontology information and association rules related to the distribution of Japanese teaching effect in the FC and task-based teaching mode. Furthermore, we construct a fuzzy decision-making model for evaluating the teaching effect. The joint information entropy characteristic value of the teaching effect deviation is calculated, and the extracted characteristic quantity is classified and identified using feature deviation compensation and the C-means clustering method. Based on the classification and identification results, we achieve accurate evaluation of the Japanese teaching effect in the FC and task-based teaching mode. Our simulation results demonstrate high precision rates and confidence levels in the evaluation of Japanese teaching effect in this mode.

Big Data Intelligent Analysis Technology for the Study of Spatial and Time Trends in the Development of Large Cities

The purpose of this research is to study modern problems and prospects for solving the processing of big data containing information about real estate, as well as the possibility of practical implementation of the methodology for processing such data arrays by designing and filling a special graphic abstraction «metahouse» on a practical example.Materials and methods. The study includes a review of bibliographic sources on the problems of big data analysis and their application in the modern field of construction of large cities. During the study, a technique for presenting data in a graphical form – abstraction was used. The mathematical basis of the technique is the use of multidimensional spaces, where measurements are the characteristics of individual objects. Computer simulation of a practical problem was applied using the C# programming language. Big data storage is based on the MongoDB server. To visualize data, a Web interface based on HTML and CSS is used.Results. In the course of the work, the main characteristics of big data were identified, and the specifics of data arrays consisting of information about real estate objects in a large city were described. When processing data consisting of information about real estate objects of a large city, certain difficulties arise. Thereby, methods for effectively solving the set practical task of processing and searching for patterns in a large data array were proposed: «metahouse» abstraction, data aggregator.Tabular data were obtained for a large city by analyzing three million records containing more than 10 data groups, with a basic set of parameters: floor, number of floors, price, area, living area, kitchen area, type, operation. A MongoDB cluster was created on several computers, each of which was working with its own data set without intermediate results.The results of the computational experiment showed that when using the graphical form (vector) of big data representation, the costs and time for interpreting mining data were reduced.Combining big data processing methods and their presentation through graphical abstraction allows getting new results from existing data sets.Conclusion. During the study, it was found that the presentation of groups of the received data in a graphic image has a number of advantages over a tabular presentation of data (a vector image is easy to scale, the ability to compare without plotting).The proposed way for visualizing big data by constructing abstract vector images is an alternative to traditional tables, allowing you to take a different look at data arrays and the results of their processing. The results obtained can be used both for the primary study of big data processing technologies and as a basis for the development of real applications in the following areas: analysis of changes in the area of houses over time, analysis of changes in the number of floors of urban development, dynamics and distribution of supply and demand, etc.

The study of labor pendulum migration using big data processing technologies and the capabilities of the VKontakte social network

Official statistical data do not allow estimating the magnitude of labor pendulum migration in the context of regions and municipalities on a regular basis. The purpose of this research was to study the possibility of using open data of the VKontakte social network to identify and analyze labor pendulum migration on the example of Ekaterinburg and 12 satellite cities. Depersonalized information of users of the VKontakte social network and Rosstat data acted as empirical data. The research established a connection between the distance from the city of residence to Ekaterinburg and the proportion of citizens-users of the VKontakte network included in the process of pendulum labor migration. The analysis of incoming and outgoing flows of labor pendulum migration showed that despite significant differentiation by cities, in all cities the outgoing flow of labor migrants to Ekaterinburg exceeded the incoming one. The results of the research can be used in the analysis of the features, intensity and directions of pendulum migration for their consideration in the elaboration of strategic documents in the field of spatial development of territories. Further research by the authors will be aimed at improving the methodology for studying migration processes that occur between cities of millions and satellite cities, in order to identify agglomeration effects using big data technologies.

Construction of E-commerce for tourism and agricultural products based on Internet of Things technology under the background of big data

With the rise of big data processing technology and agricultural product Electronic business platform (E-COMMERCE), the varieties and channels of agricultural product sales are becoming more and more diversified. It is not popular among the E-COMMERCE, resulting in users not being able to accurately find the products that suit them in the first time. Therefore, the agricultural product E-COMMERCE has become an important means to efficiently meet individual needs. Combined with the technical characteristics of the Internet of things (IOT) and the development trend of online retailers, it provides professional online logistics services and other value-added services throughout all links of logistics for logistics enterprises, suppliers, manufacturers, retailers, partners, customers, governments and banks. Based on the analysis of the current development status of China's agricultural tourism product online retailers, combined with the characteristics of tourism consumption in the era of experience economy, this paper puts forward the innovative development strategy of China's agricultural tourism product E-commerce.

ТЕХНОЛОГИЯ ОБРАБОТКИ БОЛЬШИХ ДАННЫХ (BIG DATA)

The description of big data, big data processing technologies (big data), Hadoop big data storage systems are considered

Using the Potential of Social Media in Profiling and Attributing Precarious Employment of Socio-Demographic Groups in Russia

In the past three decades have seen the rapid development of Internet and its wide distribution across countries and socio-demographic groups. The use of social media for researches may be really useful, as users of media platforms often post data or enter queries related with their current work or job search. The research presents the results of an analysis of data found in social media and open information resources related to precarious employment in Russia. There was used thesaurus technology in the processing of information which consists of scientific articles and public discussions. Automatic monitoring of news, posts and discussions of precarious employment issue was carried out. Identification of precarious employment actors were carried out using the Big Data processing technology. The gender, age and professional connection of precarious employment actors, as well as their geographical location in Russia, has been revealed. The author evaluates the scale of precarious employment in the context of Russian federal districts. There are factors, attributes and consequences of labor precarisation in Russia are identified in result of social media data analysis. Sociodemographic, professional and regional profile of a subject with high risks of precarious employment in Russia has been compiled. Based on the monitoring of references to precarious employment in social networks, blogs and public information resources, its manifestations were systematized in the context of a negative impact on the quality of working life, employment incomes and households’ standard of living.

Strategy to coordinate actions through a plant parameter prediction model during startup operation of a nuclear power plant

The development of automation technology to reduce human error by minimizing human intervention is accelerating with artificial intelligence and big data processing technology, even in the nuclear field. Among nuclear power plant operation modes, the startup and shutdown operations are still performed manually and thus have the potential for human error. As part of the development of an autonomous operation system for startup operation, this paper proposes an action coordinating strategy to obtain the optimal actions. The lower level of the system consists of operating blocks that are created by analyzing the operation tasks to achieve local goals through soft actor-critic algorithms. However, when multiple agents try to perform conflicting actions, a method is needed to coordinate them, and for this, an action coordination strategy was developed in this work as the upper level of the system. Three quantification methods were compared and evaluated based on the future plant state predicted by plant parameter prediction models using long short-term memory networks. Results confirmed that the optimal action to satisfy the limiting conditions for operation can be selected by coordinating the action sets. It is expected that this methodology can be generalized through future research.

Intelligent analysis and processing technology of big data based on clustering algorithm

In order to study the big data intelligent analysis and processing technology based on clustering algorithm, an attribute category clustering method based on hierarchical clustering is proposed, which combines the attribute categories with similar fault type distribution, reduces the data dimension, and binarizes it. Aiming at the problem of more missing values of continuous data, a data completion method based on attribute distribution function is adopted. Then, from the perspective of the selection and estimation of project unit price in construction enterprises, this paper combs and summarizes the data mining process facing the characteristics of project cost data, and puts forward the method of analyzing and processing project cost data based on clustering algorithm. Finally, the processed data sets are subjected to bottom-up hierarchical clustering analysis, and finally the ideal analysis results can be obtained. The experimental results show that the preprocessing method based on attribute clustering proposed in this paper can effectively merge attributes, reduce the dimension after binary transformation and effectively reduce the amount of data under the condition of ensuring data information.

The Construction of Psychological Intervention Mechanism of Deep Learning in the Prevention of Legal Anomie.

The convenience of big data processing technology has played a great advantage in many scenarios, and its deep learning can effectively mine different types of data in data sets. Applying this method to mining psychological prediction data set of legal anomie behavior can effectively prevent the occurrence of illegal behavior. The effective analysis of its psychological characteristics and the changes of psychological emotions will have hidden dangers, so it is necessary to extract this kind of data in such cases. Therefore, the forest algorithm under deep learning uses weight factors, simple sampling, and other ideas to build the intervention mechanism, which will improve psychological quality to prevent anomie. The experimental results of this paper are as follows: (1) In the case of testing psychological fluctuation by three algorithms, the forest algorithm calculates the psychological change area steadily, with the increase of the number of samples. (2) In the data sample extraction of different dimensions, the test value increases with the increase of dimensions, and it is stable at 12 dimensions. (3) Preventive intervention is extremely important to construct the mechanism before the anomie behavior occurs, and the longest running time of the roll neural network is as much as 950 s. (4) The data mining technology for constructing the intervention mechanism accounts for 15.4% of counseling, which also shows that the auxiliary measures for preventing behavior are particularly important.

Challenges and Countermeasures of Arab Immigrants and International Trade in the Era of Big Data

In recent years, the development of intelligent iteration technology and the use of big data processing technology have set off an upsurge, and the analysis and application of artificial intelligence algorithms have been paid more and more attention. In order to face the challenges of Arab migration and international trade, this paper constructs the basic structure of the Arab migration action imitation model. In this paper, a simulated servo clustering algorithm based on big data and intelligent iteration is used. Then, through the analysis of pseudo servo clustering algorithm, an optimization model is established, and a big data analysis system is formed. This paper focuses on the wide application of big data statistics to solve the construction of Arab immigration and entrepreneurship data system. This paper studies and applies the big data statistics and intelligent iterative algorithm of Arab immigration behavior, focuses on the annual ladder degree of Arab immigrants, and constructs a pseudo servo cluster system based on Intelligent iterative algorithm. Finally, the simulation experiment verifies whether the clustering model can accurately retrieve the behavior of Arab immigrants in China. The era of big data provides good development opportunities for Arab immigrants and international trade, but it also faces severe challenges. The study provides a reference for strengthening the analysis of international trade, and puts forward perfect countermeasures in combination with the actual situation, so as to improve the efficiency of international trade management and promote the better implementation of international trade.

Improvement of Business Analysis Method of E-Commerce System from the Perspective of Intelligent Recommendation System

In recent years, with the continuous development of the country’s Internet platforms, China has gradually entered the e-commerce era of national online shopping, and more and more e-commerce platforms and stores have adopted intelligent recommendation systems to increase transaction rates. However, it is not easy for consumers to filter out the products they want from a large amount of information. The emergence of intelligent recommendation systems provides great convenience for people to screen out personalized products that meet their own characteristics. However, the algorithms used in traditional recommendation technology focus on the single-computer environment and do not consider the performance of the recommendation method when distributed parallel processing is required in the big data environment, so it cannot meet the personalized needs of users in the big data environment. Aiming at the new requirements for the development of e-commerce intelligent recommendation technology in the big data environment, this paper uses the big data processing technology based on cloud computing and focuses on the realization technology of the e-commerce intelligent recommendation algorithm and the comprehensive evaluation method of the recommendation system in the big data environment. A prototype system of personalized intelligent recommendation based on cloud computing has been developed, which is of great importance to meet the needs of e-commerce personalized intelligent recommendation in the big data environment, improve the effectiveness, scale, and real-time performance of the personalized intelligent recommendation system, and improve the level of personalized precision marketing., which is of theoretical significance and economic value.