Big Data Model Research Articles

Response-Aided Score-Matching Representative Approaches for Big Data Analysis and Model Selection under Generalized Linear Models

In this paper, we propose an efficient method called the response-aided score-matching representative (RASMR) approach to facilitate massive data model selection and data analysis with generalized linear models (GLMs) and a predetermined data partition due to data localization. Similar to the original score-matching representative (SMR) approach, RASMR constructs an artificial data point, called the representative, for each data block. It then fits a GLM on the representative dataset, which provides not only an efficient approach for massive data analysis but also an ideal solution in response to privacy concerns by avoiding the transfer of sensitive data. By further splitting the data blocks according to the values of the response variables, RASMR can obtain more accurate parameter estimates than SMR. Furthermore, by theoretical justifications and simulation studies, we show that RASMR can be more efficiently utilized for model selection and variable selection for a massive dataset by approximating the Akaike information criterion (AIC) and the aggregated prediction errors for cross-validation, which are commonly used for choosing the most appropriate statistical model and drawing reliable conclusions. We also apply the proposed RASMR approach to the airline on-time performance data, which consists of 371 data files labeled by month, and show that RASMR can be successfully used for selecting the most appropriate model for real massive data analysis.

Optimization of Offshore Saline Aquifer CO2 Storage in Smeaheia Using Surrogate Reservoir Models

Machine learning-based Surrogate Reservoir Models (SRMs) can replace/augment multi-physics numerical simulations by replicating the reservoir simulation results with reduced computational effort while maintaining accuracy compared with numerical simulations. This research will demonstrate SRMs’ potential in long-term simulations and optimization of geological carbon storage in a real-world geological setting and address challenges in big data curation and model training. The present study focuses on CO2 storage in the Smeaheia saline aquifer. Two SRMs were created using Deep Neural Networks (DNNs) to predict CO2 saturation and pressure over all grid blocks for 50 years. 18 million samples and 31 features, including reservoir static and dynamic properties, build the input data. Models comprise 3–5 hidden layers with 128–512 units apiece. SRMs showed a runtime improvement of 300 times and an accuracy of 99% compared to the 3D numerical simulator. The genetic algorithm was then employed to determine the optimal rate and duration of CO2 injection, which maximizes the volume of injected CO2 while ensuring storage operations’ safety through constraints. The optimization continued for the reproduction of 100 generations, each containing 100 individuals, without any hyperparameter tuning. Finally, the optimization results confirm the significant potential of Smeaheia for storing 170 Mt CO2.

Ship emission variations during the COVID-19 from global and continental perspectives

The COVID-19 pandemic and the International Maritime Organization's (IMO) 2020 fuel-switching policy have profoundly impacted global maritime activities, leading to unprecedented changes in shipping emissions. This study aimed to examine the effects from different scales and investigate the underlying drivers. The big data model Ship Emission Inventory Model (SEIM) was updated and applied to analyze the spatiotemporal pattern of global ship emissions as well as the main contributors in 2019 and 2020. Overall, ships emitted NOx, CO, HC, CO2, and N2O declined by 7.4 %–13.8 %, while SO2, PM2.5, and BC declined by 40.9 %–81.9 % in 2020 compared with 2019. The decline in CO2 emissions indicated a comparable reduction across vessel tonnages. Ship emissions occurring at cruising status accounted for over 90 % of the ship's CO2 emission reduction. Container ships, chemical tankers, and Ro-Ro vessels were the primary contributors to the emission reductions, with container ships alone responsible for 39.4 % of the CO2 decrease. The ship's CO2 emissions variations revealed the decline-rebound patterns in response to the pandemic. Asian-related routes saw emissions drop in February 2020, followed by a rebound in May, while European and American routes experienced declines starting in May, with a recovery in August. Further analysis of CO2 emission in Exclusive Economic Zones (EEZs) showed high temporal consistency between vessel CO2 emissions, sailing speeds, and international trade volumes across continents, and exhibited heterogeneity in main contributing ship type of emission reduction on continental scale. Our study reveals the short-term fluctuation characteristics of global ship emissions during the pandemic, particularly focusing on their spatiotemporal evolution and the inherent disparities. The results highlight the correlation between global ship emissions and trade, as well as the operational status of ships, and their rigidity.

A Study on the Integration of Big Data With Information Retrieval Technology in the Construction of Translation Talent Pools

This paper conducts a corresponding study on the construction of a translation talent pool internally through the context of information retrieval technology. It first discusses the current state of development of information retrieval technology in various countries in the present era and the current social development of information retrieval technology. A model of the system is constructed by basing it on information retrieval technology. The system model of information retrieval technology combined with the optimisation problems associated with translation talent pools is also investigated, incorporating a factorisation algorithm. The results of the study show that the system model of big data combined with information retrieval technology has good application in concrete practical applications after modelling and performing internal upgrading.

Innovative research aspects of big data modelling, management and analytics

Innovative research aspects of big data modelling, management and analytics

Secure Big Data Model Based on Blockchain Technology

Secure Big Data Model Based on Blockchain Technology

Proposing a maturity model for assessing Artificial Intelligence and Big data in the process industry

Among digital technologies, Artificial Intelligence (AI) and Big Data (BD) have proven capability to support different processes, mainly in discrete manufacturing. Despite a number of AI and BD solutions and applications, no comprehensive assessment of their implementation is available for the Process Industry (i.e. cement, chemical and steel) and it is getting urgent to take into consideration specific operations. Grounding on literature and focus group interaction, this paper contributes to answering this gap by proposing a maturity model (MM) for AI and BD and assessing the current status of the application of these solutions in the process industry. Based on MMs available in the literature, a set of dimensions for the process industry has been identified and contextualised for assessing the level of maturity for AI and BD solutions. Results from applying the MM to a sample of European companies reveal that operations are supported by a relatively high level of maturity of AI and BD implementation with differences in the specific dimensions and operations where it is still necessary to invest. The MM can be used by companies both to self-assess and to benchmark with companies from the same or other sectors.

Generative artificial intelligence of things systems, multisensory immersive extended reality technologies, and algorithmic big data simulation and modelling tools in digital twin industrial metaverse

Research background: Multi-modal synthetic data fusion and analysis, simulation and modelling technologies, and virtual environmental and location sensors shape the industrial metaverse. Visual digital twins, smart manufacturing and sensory data mining techniques, 3D digital twin simulation modelling and predictive maintenance tools, big data and mobile location analytics, and cloud-connected and spatial computing devices further immersive virtual spaces, decentralized 3D digital worlds, synthetic reality spaces, and the industrial metaverse. Purpose of the article: We aim to show that big data computing and extended cognitive systems, 3D computer vision-based production and cognitive neuro-engineering technologies, and synthetic data interoperability improve artificial intelligence-based digital twin industrial metaverse and hyper-immersive simulated environments. Geolocation data mining and tracking tools, image processing computational and robot motion algorithms, and digital twin and virtual immersive technologies shape the economic and business management of extended reality environments and the industrial metaverse. Methods: Quality tools: AMSTAR, BIBOT, CASP, Catchii, R package and Shiny app citationchaser, DistillerSR, JBI SUMARI, Litstream, Nested Knowledge, Rayyan, and Systematic Review Accelerator. Search period: April 2024. Search terms: “digital twin industrial metaverse” + “artificial Intelligence of Things systems”, “multisensory immersive extended reality technologies”, and “algorithmic big data simulation and modelling tools”. Selected sources: 114 out of 336. Published research inspected: 2022–2024. PRISMA was the reporting quality assessment tool. Dimensions and VOSviewer were deployed as data visualization tools. Findings & value added: Simulated augmented reality and multi-sensory tracking technologies, explainable artificial intelligence-based decision support and cloud-based robotic cooperation systems, and ambient intelligence and deep learning-based predictive analytics modelling tools are instrumental in augmented reality environments and in the industrial metaverse. The economic and business management of the industrial metaverse necessitates connected enterprise production and big data computing systems, simulation and modelling technologies, and virtual reality-embedded digital twins.

Exploring influential factors in Chinese engineering professors' industrial experience: A big data and CV analysis approach

There is a long-standing criticism in academic circles about the inadequacy of engineering industrial experience for professors in Chinese research universities, but there is a lack of analysis of the manifestations and root causes of the problem. The research aims to contribute meaningful insights into the factors influencing the industrial experience of engineering professors in Chinese research universities, facilitating informed policymaking and fostering collaborative efforts between academia and industry. This paper examines Chinese engineering professors' industrial experience using big data mining, computing, modeling, and image rendering. The data was collected from the publicly available CVs from the websites of various Chinese universities which contains data on colleges and universities across China and analyzed engineering professors' professional trajectories and industrial experience using the method of resume analysis. A nonlinear logistic regression model is fitted to determine the impact of multiple independent variables on the probability of engineering industrial experience among professors, with logit transformation and maximum likelihood estimation. The results show that a low percentage of engineering university professors in Chinese research universities have had industry employment after obtaining their Ph.D. degrees. The regression model indicates that gender, level of inbreeding, overseas study experience, university level, and birth age significantly affect the engineering experience of professors. This study proposes several policy recommendations based on comparative analysis and its research. The study's findings are critical for policymakers to create policies that promote industrial partnerships as a fundamental aspect of the professional development of engineering faculty and for further research in the field.

Prediction of Electric Load Neural Network Prediction Model for Big Data

In this study, the two-dimensional hexagonal Photonic crystal energy band structure was simulated using COMSOL Multiphysics field simulation software. The 2D hexagonal Photonic crystal structure was constructed by setting parameters such as periodic boundary conditions and air hole radius. Using the frequency domain solver of COMSOL software, the transmission and reflection spectra of the structure were calculated, and the energy band structure diagram was obtained. The effects of different parameters on the energy band structure were analyzed by adjusting the structure parameters. The results show that the fine tuning of the energy band structure of 2D hexagonal Photonic crystals can be realized by adjusting the radius of air holes and the periodic boundary conditions. This study provides a useful reference for further research on the properties and applications of 2D photonic crystals.

An Intelligent Model and Methodology for Predicting Length of Stay and Survival in a Critical Care Hospital Unit

This paper describes the design and methodology for the development and validation of an intelligent model in the healthcare domain. The generated model relies on artificial intelligence techniques, aiming to predict the length of stay and survival rate of patients admitted to a critical care hospitalization unit with better results than predictive systems using scoring. The proposed methodology is based on the following stages: preliminary data analysis, analysis of the architecture and systems integration model, the big data model approach, information structure and process development, and the application of machine learning techniques. This investigation substantiates that automated machine learning models significantly surpass traditional prediction techniques for patient outcomes within critical care settings. Specifically, the machine learning-based model attained an F1 score of 0.351 for mortality forecast and 0.615 for length of stay, in contrast to the traditional scoring model’s F1 scores of 0.112 for mortality and 0.412 for length of stay. These results strongly support the advantages of integrating advanced computational techniques in critical healthcare environments. It is also shown that the use of integration architectures allows for improving the quality of the information by providing a data repository large enough to generate intelligent models. From a clinical point of view, obtaining more accurate results in the estimation of the ICU stay and survival offers the possibility of expanding the uses of the model to the identification and prioritization of patients who are candidates for admission to the ICU, as well as the management of patients with specific conditions.

Big Data Model for Third-party Stability Assessment of Major Decisions: Generation Logic and Path Construction

The role of digital and intelligent technology in social risk governance and assessment is increasingly prominent. However, existing research often focuses on the technical aspects of development, neglecting the synergy and integration between technology and administrative practices. To address this, the paper constructs a four-dimensional theoretical analysis framework based on collaborative governance theory, which includes “subject integration, object unification, relationship coupling, and function complementation.” This framework is applied in a case study to analyze the generation logic and pathway construction of the third-party stability assessment the “big data” model adopted by City A. The study concludes that the dominant role of administrative efficiency is core, the supportive role of technological empowerment is key, and the participatory role of social forces is essential in building the pathway. These findings provide empirical support for the digital construction of major decision-making third-party stability assessment nationwide. They offer new perspectives to enhance governance efficiency and contribute to the ongoing improvement and high-quality development of major decision-making processes in social stability risk governance and assessment.

Manufacturing Industry Resource Endowment Index and Its Big Data Model

This study aims to demonstrate the application of big data models in the evaluation of manufacturing competitiveness across different countries and economic entities. It involves analyzing algorithms and data structures, designing algorithms for evaluating manufacturing competitiveness suitable for different countries and economic entities, and providing algorithmic and theoretical support for building computer assessment models for manufacturing competitiveness in economic entities. The research adopts qualitative and quantitative methods, comparing and analyzing empirical data on manufacturing resource endowment and competitiveness in various countries, combining theoretical analysis and empirical data. By conducting a literature review and empirical research, relevant information and data are collected to establish the relationship between manufacturing resource endowment index and competitiveness. Through case studies and comparative research, this study explores the main factors affecting manufacturing competitiveness in different countries and their interrelationships. It confirms that the competitiveness of manufacturing industries in different countries is influenced by factors such as resource endowment. Further analysis quantifies the relationship between national manufacturing resource endowment and competitiveness, leading to the design of computer algorithms, data structures, and processing logic. The results demonstrate the significant role of resource endowment in manufacturing competitiveness, serving as a crucial basis for formulating comprehensive national strategies. The innovations of this study include: 1. Introducing the manufacturing resource endowment index as a key variable for evaluating competitiveness, enriching the content of competitiveness big data models. 2. Designing comprehensive data structures, algorithms, and processing logic for assessing manufacturing competitiveness from multiple perspectives. 3. In-depth analysis of the relationship and analytical methods between manufacturing strategies, resource endowment, and competitiveness, providing theoretical and model references for the design of computer big data operation platforms.

Analysis of Factors Affecting Voluntourism Network Attention in China: An empirical Test Based on Big Data Analysis

Based on the Baidu index platform, this study takes panel data of 31 provinces, autonomous regions and municipality directly under the central government in China from 2019 to 2023 as a research sample, and and uses big data to empirically analyze the influencing factors of the Voluntourism Network Attention in China. This study chose the Voluntourism Network Attention as the explanatory variable, the seven indicators including the consumption level of the residents, the population above the college and the total tourism revenue of the region as explained variables. Using the STATA15.0 software, the big data model is determined by a descriptive test, a unit root test, a co-integration test, a variance expansion factor (VIF) test, etc. The regression equation of this study was constructed by the generalized least squares method. The empirical study is conducted to determine the influence mode and degree of each factor on the public interest tourism network attention and analyzes the internal causes. The results show that the educational level of the region, the level of economic development and the level of tourism development have a significant positive impact on the Voluntourism Network Attention, and the regression coefficient of the education level and economic development level is the largest. Both of them are the main factors of network the Voluntourism Network Attention. In view of the empirical results, this paper puts forward the following suggestions: First, it is necessary to create more voluntourism routes suitable for college students. Second, it is essential to increase the marketing and promotion of voluntourism activities in economic and tourism developed areas.Last, there is a need to innovate the connotation and form of the voluntourism.

Distributed matrix computing system for big data

In order to solve the problem of low computing efficiency in big data analysis and model construction, this paper intended to deeply explore the big data analysis programming model, DAG (Directed Acyclic Graph) and other contents, and on this basis, it adopted a distributed matrix computing system Octopus for big data analysis. Octopus is a universal matrix programming framework that provides a programming model based on matrix operations, which can conveniently analyze and process large-scale data. By using Octopus, users can extract functions and data from multiple platforms and operate through a unified matrix operation interface. The distributed matrix representation and storage layer can design data storage formats for distributed file systems. Each computing platform in OctMatrix provides its own matrix library, and it provides a matrix library written in R language for the above users. SymboMatrix provides a matrix interface to OctMatrix that is consistent with OctMatrix. However, SymboMatrix also retains the flow diagram for matrix operations in the process, and it also supports logical and physical optimization of the flow diagram on a DAG. For the DAG computational flow graph generated by SymbolMatrix, this paper divided it into two parts: logical optimization and physical optimization. This paper adopted a distributed file system based on line matrix, and obtained the corresponding platform matrix by reading the documents based on line matrix. In the evaluation of system performance, it was found that the distributed matrix computing system had a high computing efficiency, and the average CPU (central processing unit) usage reached 70%. This system can make full use of computing resources and realize efficient parallel computing.

Economics of diabetes mellitus type 2 control based on big data modeling

Background. It has been done previously that the main expenditures in diabetes mellitus type 2 (DM2T) not related with glycemia correction, but they are spending on the cardiovascular diseases (CVDs) of DM2T and indirect costs of CVDs in work-aged patients.Objective. Modelling of potential clinical and economic results of effective programs for DM2T control realizations and their influence on CVDs and mortality in case of HbA1c targets reached.Materials and methods. The Oxford UKPDS model has been used for epidemiologic and economic benefits in cases of decreased CVDs and premature deaths due to more effective control of DM2T. We created 48 cluster groups of patients based on age (less 40 y. o., 40–60 y. o., 60+ y. o.) and HbA1c levels and CVDs anamnesis. The probability of CVDs over 10 years was calculated for each cluster. Subsequently, DALY, YLL, and YLD have also been defined. Average salary, GDP per capita, etc. were used for the analysis.Results. Successful control of DM2T can lead to saving 1.69 billion YLL, or 115.93 years per 100th of people annually. The prognosis for CVD morbidity and mortality decreasing due to more effective DM2T control can decrease DALY loss by 17 %. The effect on the Russian economy in this case can be evaluated as 197,8 billion RUR over 10 years (based on salary level). The benefit in the GDP per capita metric is higher — 213.6 billion RUR annually.Conclusion. Effective methods of DM2T control can reduce the risk of CVD occurrence and progression; therefore, they are economically justified and can be considered as an additional source of budgetary benefits, leading to lower medical care costs.

A Systematic Review of Big Data Driven Education Evaluation

The rapid development of artificial intelligence has driven the transformation of educational evaluation into big data-driven. This study used a systematic literature review method to analyzed 44 empirical research articles on the evaluation of big data education. Firstly, it has shown an increasing trend year by year, and is mainly published in thematic journals such as educational technology, science education, and language teaching. Chinese and American researchers have made the greatest contributions in this field. Secondly, the algorithmic models for big data education evaluation research are diverse, the text modality is the most popular, the evaluation subjects are mainly college students, with fewer primary and secondary school students, and science is the discipline that most commonly applies big data education evaluation. The evaluation objectives of big data education evaluation mainly focus on five aspects: high-order thinking analysis, learning performance prediction, learning emotion recognition, teaching management decision-making, and evaluation mode optimization, and the text modality is widely used for data collection in high-order thinking analysis; regardless of the evaluation objectives, higher education students are the most widely evaluated objects; the science discipline is the main field of using big data technology to empower teaching evaluation. Thirdly, the current research topics of big data education evaluation mainly focus on online learning behavior and environmental participation evaluation, process assessment of learning motivation and emotional analysis, development and optimization of subject domain big data models, cognitive diagnosis and high-order thinking skills evaluation, and design of learning analysis frameworks based on data mining.

THE POSSIBILITIES OF USING BIG DATA TECHNOLOGIES IN SOLVING PROBLEMS OF PROCESSING DATA ON ATMOSPHERIC AIR POLLUTION

The main objective of the article is to substantiate the possibility of using Big Data technologies in the field of atmospheric air monitoring. In the form of a diagram, a model for processing big data obtained from measuring meteorological gas analysis stations using the PySpark library for further experimental studies is presented. The factors accompanying the use of Big Data in the field of atmospheric air monitoring are derived, and the performance of the Pandas and PySpark libraries is compared. The obtained results will allow us to further rely on the derived factors and use the most optimal data processing technologies to build predictive machine learning models in the field of analyzing the level of atmospheric air pollution. Consistent use of big data and machine learning methods will ensure clean and healthy air for future generations through more effective predictive analytics. This article is valuable for students and specialists in the field of information technology, in particular, in the field of data processing and machine learning.

Application of Machine Learning in Vegetable Classification and Recognition

The rapid development of artificial intelligence plays an important role in People's Daily life today. Vegetable classification is closely related to life. With the continuous advancement of artificial intelligence technology, similar applications in the agricultural and retail industries are expected to play a more significant role. This paper aims to envision a vegetable identification system based on a convolutional neural network to provide intuitive and accurate vegetable classification information for supermarket managers and customers. This study mainly uses the convolutional neural network model as the main research model to carry out the design of a vegetable classification system. In this paper, the usefulness and disadvantages of the CNN model are summarized, how to use the model for deep learning and big data is learned, and the problems that may be encountered after the model is run are also predicted. Furthermore, the future development direction of the project is predicted too. As technology continues to evolve, it is expected that more innovative solutions will emerge, delivering enhanced benefits to society.

With super SDMs (machine learning, open access big data, and the cloud) towards more holistic global squirrel hotspots and coldspots

Species-habitat associations are correlative, can be quantified, and used for powerful inference. Nowadays, Species Distribution Models (SDMs) play a big role, e.g. using Machine Learning and AI algorithms, but their best-available technical opportunities remain still not used for their potential e.g. in the policy sector. Here we present Super SDMs that invoke ML, OA Big Data, and the Cloud with a workflow for the best-possible inference for the 300 + global squirrel species. Such global Big Data models are especially important for the many marginalized squirrel species and the high number of endangered and data-deficient species in the world, specifically in tropical regions. While our work shows common issues with SDMs and the maxent algorithm (‘Shallow Learning'), here we present a multi-species Big Data SDM template for subsequent ensemble models and generic progress to tackle global species hotspot and coldspot assessments for a more inclusive and holistic inference.