Big Data Processing Research Articles

Application of Deep Learning in Public Network Security Management

With the rapid advancement of information technology, network security issues have become increasingly prominent, posing significant challenges in the field of public administration. Deep learning, as a cutting-edge technology in artificial intelligence, is emerging as a critical tool for enhancing network security management due to its exceptional performance in big data processing and pattern recognition. This paper reviews the basic concepts of deep learning and its current applications in network security management, discusses the design and implementation of deep learning-based network security management systems, and analyzes their effectiveness and evaluation through specific application cases. The results show that deep learning excels in areas such as intrusion detection, malware detection, network traffic analysis, and anomaly detection, significantly enhancing network security defenses. However, the application of deep learning in network security still faces challenges such as data privacy, model robustness, and computational resources. The paper concludes by proposing future development directions, providing theoretical support and practical references for further improving public network security management.

Evolutionary Bioinformatics with Veiled Biological Database for Health Care Operations

The tremendous growth of biological data processing systems in the realm of health care applications has made real-time information accessible to everyone with no processing lags. Bioinformatics is even integrated into most wireless technology applications to account for all physical characteristics. The planned model focuses on evolutionary bioinformatics for medical sensor applications in health care. The optimization scenario is executed by combining genetic and ant colony optimization methods (GACO). In the proposed technique, the design concerns are implemented with appropriate transmitting and receiving modules, and individual bits are framed for extra bioinformatics data processing components. a design that completely minimizes all errors in the big data processing stage. Such a design completely lowers the overall error in the huge data processing state since all channels can be accessed in accordance with the framed bits. Furthermore, the quality of service is maximized because all channels carrying bioinformatics data are kept at high quality bits, increasing utility rates. The experiments were conducted using five scenarios to evaluate the effectiveness of the proposed design. The findings indicate that the proposed technique can handle bioinformatics data for healthcare in real time with a service quality of 95%.

Enhancing Business Competitiveness through Accounting Digitalization

Introduction: Amid rapid changes in the external and internal environment of modern business, more and more companies are turning to the introduction of the latest digital technologies to improve their accounting systems. This helps to increase the efficiency and accuracy of processes, as well as reduce the cost of routine work. However, the use of modern digital tools in accounting also involves certain risks and limitations, as well as raises questions about the ability to use the data for further management decision-making. Given the proven relevance of the topic, the purpose of the study can be defined as identifying key digital tools for the development and improvement of accounting to ensure the competitiveness of modern business in the long term. Methods: To obtain the results, the author used general scientific methods of analysis, synthesis, systematisation, processing of statistical material, generalisation and specification, as well as a specific method of hierarchy analysis, which allowed to identify the most optimal areas of accounting optimisation for a modern enterprise. Results: The study identified the main digital tools that can be used in the field of accounting and their impact on the system of accounting procedures in the company. The possibilities of applying various digital tools were identified and it was found that they are necessary to increase competitiveness in the long term. Conclusions: By applying the hierarchy analysis method, it is determined that the most promising for the development of the accounting system may be the tools for processing big data and using digital twins.

Research Progress in the Field of MapReduce Framework Based on Serverless Computing

In recent years, big data technology has presented a rapid development trend with each passing day, and its application in all walks of life is becoming more and more extensive and in-depth. At the same time, traditional computing frameworks are gradually showing serious challenges in performance bottlenecks and scalability in the face of massive data processing requirements, making it difficult to meet the growing needs of data processing and highly concurrent access scenarios. Firstly, this paper introduces MapReduce framework and serverless platforms, and then introduces the MapReduce framework model based on serverless platforms from several parts of implementation, application scenarios and technical principles. This paper argues that the serverless-based MapReduce framework will provide a more efficient and flexible solution for big data processing and analyzing, which is of great significance in promoting the development of data-driven decision-making and intelligent applications. The main content of this paper will be introduced from the basic overview of MapReduce computing framework and serverless platform, the implementation of serverless-based MapReduce computing framework, technical principles and application scenarios.

Exploring Drug Repurposing for Interstitial Cystitis/Bladder Pain Syndrome: Defining Novel Therapeutic Targets.

Interstitial cystitis/bladder pain syndrome (IC/BPS) is a debilitating pain condition of unknown etiology. Effective therapies for this condition could not have been developed in the last century. Drug repurposing is a practical strategy for enhancing patient access to successful therapies. It is an approach for discovering novel applications for licensed or investigational pharmaceuticals that extend beyond the initial medical indication. This work aims to identify repurposable medications through bioinformatics to discover potential drugs or compounds that can reverse the IC/BPS disease signature. The analysis involved examining the differentially expressed genes in IC/BPS patients with two distinct disease phenotypes (Hunner's lesion disease, non-Hunner's lesion disease) and controls using the datasets GSE11783, GSE28242, and GSE57560. The goal was to assess the reversal of the disease signature on the L1000CDS2 and cMAP platforms. Twenty-one compounds were repurposed, consisting of 11 small molecules, 10 chemical compounds, 3 natural products, and 6 FDA-approved drugs, currently used for clinical indications such as cancer, myelofibrosis, and diabetes. Bioinformatics can be useful for identifying therapeutic agents for IC/BPS by accessing and processing big data on molecular and cellular levels. Prospective in vivo experiments must validate repurposed drugs. The expansion of large-scale genome sequencing, gene expression studies, and clinical data for IC/BPS will improve successful drug selection.

Cloud cost monitoring and forecasting: issues and challenges

Cloud platforms are revolutionizing big data processing, but they also present significant challenges, particularly in cost management. As organizations move from on-premises solutions to cloud-based infrastructures, they often face unexpected costs due to the increasing volumes of structured, semi-structured, and unstructured data they collect and consume. Overcoming this challenge requires a thorough understanding of cost drivers, effective financial management practices, and strategic planning. This article examines the factors that influence the cost of cloud-based data processing and explores financial management best practices that maximize return on investment. It emphasizes the importance of adopting cost optimization strategies tailored to the unique needs of organizations operating in different industries. In addition to financial considerations, the article examines how cloud platforms enable marketing transformation through campaign automation, enhanced analytics, and personalization of the customer experience. These capabilities not only improve operational efficiency, but also drive greater customer engagement and loyalty. The research evaluates the challenges and opportunities associated with cost optimization in cloud systems. It highlights types of forecasts, optimization processes, and monitoring methods as critical components of achieving financial efficiency. A comprehensive three-step approach to effective financial management is proposed, focusing on understanding, controlling, and optimizing costs. In addition, the article evaluates the tools and solutions offered by cloud service providers, providing actionable insights for organizations to gain better control over their expenses. By integrating perspectives from both the financial and marketing dimensions, the study demonstrates how cloud-based systems can serve as a cornerstone for business innovation and growth. It concludes with practical recommendations for leveraging cloud platforms to improve operational and marketing efficiency while maintaining cost transparency and predictability.

Enhancing Big Data Processing Efficiency in AI-Based Healthcare Systems: A Comparative Analysis of Random Forest and Deep

This research focuses on optimizing the speed of Big Data processing using Artificial Intelligence (AI) in healthcare applications. The study integrates Random Forest (RF) and Deep Learning (DL) algorithms with cloud-based computing systems to improve data processing efficiency. The dataset includes both structured data, such as Electronic Health Records (EHR), and unstructured data, like medical images. The results show that RF performs better with structured data, achieving a lower Mean Squared Error (MSE) and higher R-squared (R²) than traditional methods. Meanwhile, DL achieves superior accuracy and Area Under the Curve (AUC) in processing unstructured data. By utilizing the distributed computing power of Spark on a cloud platform, the processing speed was significantly enhanced, as demonstrated by a statistically significant reduction in processing time (p < 0.05) observed through a t-test analysis comparing Spark-based computing with traditional methods. Despite these improvements, challenges such as data privacy and infrastructure costs remain. Despite these improvements, challenges such as data privacy and infrastructure costs remain. This research provides a robust framework for real-time healthcare data analysis, highlighting its potential to improve decision-making processes and patient outcomes in medical services.

Enhancing Big Data Processing Efficiency in AI-Based Healthcare Systems: A Comparative Analysis of Random Forest and Deep

This research focuses on optimizing the speed of Big Data processing using Artificial Intelligence (AI) in healthcare applications. The study integrates Random Forest (RF) and Deep Learning (DL) algorithms with cloud-based computing systems to improve data processing efficiency. The dataset includes both structured data, such as Electronic Health Records (EHR), and unstructured data, like medical images. The results show that RF performs better with structured data, achieving a lower Mean Squared Error (MSE) and higher R-squared (R²) compared to traditional methods. Meanwhile, DL achieves superior accuracy and Area Under the Curve (AUC) in processing unstructured data. By utilizing the distributed computing power of Spark on a cloud platform, the processing speed was significantly enhanced, as demonstrated by a statistically significant reduction in processing time (p < 0.05) observed through a t-test analysis comparing Spark-based computing with traditional methods. Despite these improvements, challenges such as data privacy and infrastructure costs remain. Despite these improvements, challenges such as data privacy and infrastructure costs remain. This research provides a robust framework for real-time healthcare data analysis, highlighting its potential to improve decision-making processes and patient outcomes in medical services.

SCALE FOR DETERMINING THE VIRTUAL MANAGEMENT LEVEL OF AN ENTERPRISE

The soaring rise of the information age, the postmodern management approach, the instantaneous state of the era, the speed, and the processing of big data with different methods, and the compulsory of being socially distanced under the conditions of the new normal have granted vital importance to virtualization. The study aims to propose a scale that will determine the virtual management levels of enterprises by transforming traditional management functions into virtual management functions with the contribution of four features: adaptability-flexibility, teamwork, communications-leadership, and strategic vision. In the study, which was conducted with the participation of 265 people working in 45 different sectors and approved by the ethics committee, 4 main features were confirmed by factor analysis. Confirmatory factor analysis was used to test the validity of the four-factor structure obtained by exploratory factor analysis.

Hybrid CNN-LSTM Model for Accurate Long-Term and Short-Term Temperature Prediction: A Case Study for Bingöl and Tunceli

Extreme and sudden weather events experienced with global warming and climate change reveal the importance of accurate air temperature prediction. For this reason, it can be used to optimize decision-making processes for a wide range of applications from health and agricultural planning to energy consumption strategies. Artificial intelligence methods are successfully applied in many application areas due to their flexibility and efficiency. Traditional weather forecasting models are inefficient in detecting sudden fluctuations and complex, irregular patterns in data. Artificial in-telligence methods overcome these limitations thanks to their ability to process big data and capture long-term temporal dependencies. In this study, the aim is to predict future temperature changes more accurately by capturing patterns in past data with the developed CNN-LSTM hybrid model. The developed hybrid model is compared in detail with RF, SVM, CNN, and LSTM. The compared models were tested using daily average temperature data between 1961-2024 and hourly temperature data between 2020-2024. Experiments have shown that CNN-LSTM outperforms the compared models with R2 value above 0.97 in all scenarios.

Bayesian Inference for Post-Processing of Remote-Sensing Image Classification

A key component of remote-sensing image analysis is image classification, which aims to categorize images into different classes using machine-learning methods. In many applications, machine-learning classifiers assign class probabilities to each pixel. These class probabilities serve as input for post-processing techniques that aim to improve the results of machine-learning algorithms. This paper proposes a new post-processing algorithm based on an empirical Bayes approach. We employ non-isotropic neighborhood definitions to capture the impact of borders between land classes in the statistical model. By incorporating expert knowledge, the algorithm improves the consistency of the classified map. This technique has proven its efficacy for large-scale data processing using image time-series analysis. The proposed method is a key component of a time-first, space-based approach for big Earth-observation data processing. It is available as open source as part of the R package sits.

Illegal ship identification system based on trajectory similarity measurement

The Automatic Identification System (AIS) relies on the identification code (MMSI) of a vessel to identify it and records its trajectory data, which is an important technical means for modern maritime management. However, there are some vessels that violate regulations by opening multiple AIS transponders, resulting in multiple location trajectory records for the same vessel. This behavior is difficult to distinguish through traditional means and has a negative impact on the accuracy and effectiveness of maritime management, thereby threatening the safety of navigation. This system uses big data processing tool Spark to efficiently process and analyze large amounts of AIS data and detects the similarity of vessel trajectories to automatically identify the violating vessels that open multiple AIS transponders. In this way, the system not only improves the detection efficiency of violations, but also provides more reliable evidence for maritime management departments to ensure the safety and order of navigation routes.

High-throughput mesoscopic optical imaging data processing and parsing using differential-guided filtered neural networks

High-throughput mesoscopic optical imaging technology has tremendously boosted the efficiency of procuring massive mesoscopic datasets from mouse brains. Constrained by the imaging field of view, the image strips obtained by such technologies typically require further processing, such as cross-sectional stitching, artifact removal, and signal area cropping, to meet the requirements of subsequent analyse. However, obtaining a batch of raw array mouse brain data at a resolution of 0.65×0.65×3μm3 can reach 220TB, and the cropping of the outer contour areas in the disjointed processing still relies on manual visual observation, which consumes substantial computational resources and labor costs. In this paper, we design an efficient deep differential guided filtering module (DDGF) by fusing multi-scale iterative differential guided filtering with deep learning, which effectively refines image details while mitigating background noise. Subsequently, by amalgamating DDGF with deep learning network, we propose a lightweight deep differential guided filtering segmentation network (DDGF-SegNet), which demonstrates robust performance on our dataset, achieving Dice of 0.92, Precision of 0.98, Recall of 0.91, and Jaccard index of 0.86. Building on the segmentation, we utilize connectivity analysis for ascertaining three-dimensional spatial orientation of each brain within the array. Furthermore, we streamline the entire processing workflow by developing an automated pipeline optimized for cluster-based message passing interface(MPI) parallel computation, which reduces the processing time for a mouse brain dataset to a mere 1.1 h, enhancing manual efficiency by 25 times and overall data processing efficiency by 2.4 times, paving the way for enhancing the efficiency of big data processing and parsing for high-throughput mesoscopic optical imaging techniques.

Development of intelligent systems for monitoring and management of agricultural enterprises

Modern agriculture faces a number of challenges, among which are the growing demand for food, the impact of climate change, limited natural resources and the need to ensure the environmental sustainability of production. In this context, the introduction of new technologies, in particular intelligent systems, becomes an important tool for improving the efficiency and competitiveness of agro-industrial enterprises. Intelligent Systems Based on Use artificial intelligence, machine learning, the Internet of Things (IoT) and big data are opening up new opportunities for monitoring and managing agricultural processes. The article presents the results of a study aimed at the development of intelligent systems that provide comprehensive monitoring of the state of fields and management of production processes at agricultural enterprises. The proposed system integrates data from various sources, such as sensors, drones, satellite images and other IoT devices, which allows you to create a detailed and up-to-date view of the state of crops, soils and climatic conditions. This, in turn, allows you to make prompt and sound management decisions that increase the productivity and efficiency of agricultural production. Particular attention is paid to the development of algorithms for processing big data and machine learning methods, which are used to analyze the data obtained and generate recommendations for optimizing the use of resources. This includes managing water resources, fertilizers, crop protection products, as well as predicting yields and identifying potential threats such as plant diseases or pests. The implementation of such systems allows not only to reduce resource costs, but also to minimize the environmental impact of agriculture by reducing the use of chemicals and water. Thus, the article makes a significant contribution to the development of intelligent technologies in agriculture, offering new approaches to the management and monitoring of agricultural enterprises. These approaches can form the basis for the development of future innovations in this area, contributing to improving the resilience and efficiency of agro-industrial production in response to modern challenges.

Simulation-Based Evaluation of Big Data Caching Mechanisms

This paper provides a simulation-based evaluation that addresses memory management problems throughout Big Data processing. A significant problem occurs with in-memory computing when there is not enough available memory for processing the whole chunk of data, and hence some data must be selected for deletion to make room for new ones. The selected research strategy is to use different cache selection and replacement algorithms, such as Adaptive Replacement Cache (ARC) and Low Inter-Reference Recency Set (LIRS) algorithms, besides the default one, which is Least Recently Used (LRU). A simulator was built by the authors to assess the use of different caching approaches on Big Data platforms. The evaluation showed that the LIRS and the ARC algorithms gave a better hit ratio for different workloads than the LRU algorithm.

NETWORK INTRUSION DETECTION MODEL BASED ON CONVOLUTIONAL NEURAL NETWORKS AND TABULAR DATA CONVERTED INTO IMAGES

The object of the study is the process of identifying the state of a computer systems and network. The subject of the study are the methods of identifying the state of computer systems and networks. The purpose of this paper is to improve the quality of detecting intrusions into computer networks. The UNSW-NB 15 set, which contains information about the normal functioning of the network and during synthetic intrusions, was used as input. Deep neural networks (DL), their advantages and problems in big data processing are considered. It was found that deep neural networks when processing tabular data require their transformation. Modern methods of tabular data transformation were studied. The results obtained. A method of converting tabular data into an image is proposed. The method converts each object of a separate class from a set of tabular data into an image by mapping the attribute values onto a two-dimensional plane. The method was implemented programmatically using the GOOGLE COLAB cloud service based on Jupyter Notebook. Conclusions. It was found that the use of the proposed conversion method of tabular data into an image made it possible to use a classification model based on the CNN neural network and increase the quality of detection of intrusions into computer networks up to 4%.

Standardization of studies on physical development of children in the Russian Federation

BACKGROUND:The importance of studying the growth and development processes of the child population is due to the significance of the pre-clinical approach aimed at primary disease prevention. AIM:To provide the constituent entities of the Russian Federation with standards for the physical development of children and adolescents aged 7–17years. MATERIAL AND METHODS:An analytical study was conducted, big data from the database “Physical Development of Children, Adolescents, and Youth of the Russian Federation in 2000–2021” were analyzed, containing 395,956observations of physical development (length and body weight indicators) of children, adolescents, and young people living in 49constituent entities of the Russian Federation. Priority was given to research in the constituent entities for which regional standards had never been developed or for which they were developed 15–20years or more ago. As a result, at least 2constituent entities were analyzed in each federal district. Big data processing using machine learning was carried out on the basis of the PostgreSQL database. A web application was created that implemented the functions of tabular display of data, construction of histograms with the ability to select parameters and subsequent data publication on the map of the Russian Federation. RESULTS:In Russia at the beginning of the 21st century, the characteristics of physical development of students living in diffe­rent constituent entities of the Russian Federation were established, which allowed us to talk about the presence of “regional scenarios” that must be taken into account when developing standards. At the age of 15, the range of average values of growth indicators for boys in the constituent entities ranged from 161.70±0.20cm in the Kursk Region to 174.55±0.40cm in the Krasnodar Territory (the difference was 12.85cm, p=0.001), for girls— from 154.60±0.50cm in the Republic of Tyva to 165.69±0.55cm in St.Petersburg (the difference was 11.09cm, p=0.001). The range of body weight of 15-year-old boys in the regions was from 50.17±0.70kg in the Kursk region to 66.00±0.80kg in the Khabarovsk region (difference 15.83kg, p=0.001), girls— from 46.90±0.60kg in the Chelyabinsk region to 58.40±0.70kg in the Khabarovsk region (difference 8.50kg, p=0.001). CONCLUSION:40regional standards for the physical development of the child population have been developed in a standar­dized manner, which allows taking into account regional characteristics and makes it possible to use them in practical healthcare in the constituent entities of the Russian Federation.

Analysis of Big Data Literacy Skills of Prospective Mathematics Educators Through Case Method-Based Learning

The aim of this research is to analyze the big data literacy skills that a prospective mathematics educator needs to master. Specifically, the ability to read, analyze, make conclusions, and think based on data and information, especially large amounts of data, so that mathematics education students have a broad mindset, and can carry out new innovations. Kemmis and McTaggart's Classroom Action research model was used as the research method. Its stages have four, namely planning, action, observation, and reflection. The planning and action stages took the form of preparing and validating case method-based learning tools in the Operations Research course and were implemented in 3 classes. Lectures are divided into 3 stages: problem-based learning, case method learning from the business world, and case-based learning (Case Method). During the learning process, observation and reflection are carried out. Learning involving the business world, namely Astra Financial and an expert in data sciences, provides opportunities for students to process company big data. The data literacy ability indicators used are the ability to group data, check data, clean data, do qualitative coding, eliminate data, and develop models. The learning results show that students have carried out all big data processing steps in the very good category (> 80%), except for the developing model phase (65.38%). Mini research field studies at partner institutions for the application of operations research in the community. Learning completeness reached 83.11%, and learning activeness reached 82.14% in the very active category. Learning results show an increase in the Big Data Literacy skills of prospective educators and lecturers, even though mastery of big data processing software still needs to be improved.

Big data processing using hybrid Gaussian mixture model with salp swarm algorithm

The traditional methods used in big data, like cluster creation and query-based data extraction, fail to yield accurate results on massive networks. To address such issues, the proposed approach involves using the Hadoop Distributed File System (HDFS) for data processing, the map-reduce programming paradigm for data processing, and query optimization techniques to quickly and effectively extract accurate outcomes from a variety of options with a high processing capacity. The methodology proposed in this work makes use of Gaussian Mixture Model (GMM) for data clustering and the Salp Swarm Algorithm (SSA) for optimization. The security of preprocessed data stored on networked clusters with interconnections has been ensured by SHA algorithms. Finally, incorporating into consideration the important parameters, evaluation findings for the experimental performance of the model in the indicated methodology are produced. For this work, the estimated range of input file sizes is 60–100 MB. The processing of 100 MB of input files yielded an accuracy of 96% and results for specificity and sensitivity of 90% and 93%, respectively. The outcomes have been compared with well-known methods like fuzzy C-means and K-means approaches, and the results show that the proposed method effectively distributes accurate data processing to cluster nodes with low latency. Moreover, it uses the least amount of memory resources possible when operating on functional CPUs. As a result, the proposed approach outperforms existing techniques.

Оптимізація обробки великих даних за допомогою «лінивих обчислень»: систематичний огляд технологій та застосування

The article examines the concept of lazy operations and its application for efficient processing of large volumes of data. The main principles of lazy computations, their implementation in various programming languages, and strategies for effective use in Big Data processing are analyzed. The advantages and limitations of the lazy approach are investigated, particularly regarding memory savings, performance improvement, and the ability to work with infinite data streams. A concept for selecting computation strategies based on data size and computational complexity is proposed.