Big Data Model Research Articles

The concept and evaluating of big data quality in the semantic environment

Big data refers to large volumes, complex data sets with various autonomous sources, characterized by continuous growth. Data storage and data collection capabilities are now rapidly expanding in all fields of science and technology due to the rapid development of networks. Evaluating the quality of data is a difficult task in the context of big data, because the speed of semantic data reasoning directly depends on its quality. The appropriate strategies are necessary to evaluate and assess data quality according to the huge amount of data and its rapid generation. Managing a large volume of heterogeneous and distributed data requires defining and continuously updating metadata describing various aspects of data semantics and its quality, such as conformance to metadata schema, provenance, reliability, accuracy and other properties. The article examines the problem of evaluating the quality of big data in the semantic environment. The definition of big data and its semantics is given below and there is a short excursion on a theory of quality assessment. The model and its components which allow to form and specify metrics for quality have already been developed. This model includes such components as: quality characteristics; quality metric; quality system; quality policy. A quality model for big data that defines the main components and requirements for data evaluation has already been proposed. In particular, such evaluation components as: accessibility, relevance, popularity, compliance with the standard, consistency, etc. are highlighted. The problem of inference complexity is demonstrated in the article. Approaches to improving fast semantic inference through materialization and division of the knowledge base into two components, which are expressed by different dialects of descriptive logic, are also considered below. The materialization of big data makes it possible to significantly speed up the processing of requests for information extraction. It is demonstrated how the quality of metadata affects materialization. The proposed model of the knowledge base allows increasing the qualitative indicators of the reasoning speed.

An access control model for medical big data based on clustering and risk

Access control has been widely adopted by distributed platforms, and its effectiveness is of great importance to the quality of services provided by such platforms. However, traditional access control is difficult to apply to scenarios where authorization changes frequently and to extremely large-scale datasets with limited resources. This paper proposes an access control model based on spectral clustering (SC) and risk (SC-RBAC), which is more suitable for big data medical scenarios. Based on user history access data, an improved SC algorithm is used to cluster doctor users. Then, the user classification is introduced as a parameter into the information entropy to improve the accuracy of quantifying the user’s access behavior risk. Finally, based on the accurate risk value of access behavior, we assign access rights to users through the access control function constructed in the paper. Experimental results show that in three different situations, the model proposed in this paper can distinguish the two types of doctors well, the accuracy of the model can reach more than 90%, and it outperforms other access control models.

A Panel Data Analysis of Subjective Well-Being Based on Microblog User Information.

Big data modelling using microblogs is applied to acquire nationwide representative panel data on subjective well-being. The analysis directly validates the influence of China's Livelihood Index on subjective well-being. Using panel data on subjective well-being collected for the period from 2010 to 2021 from users of the Weibo (Sina Corporation, Beijing, China) microblogging platform, this study finds Granger causality running from China's Livelihood Index to subjective well-being and that the two are positively correlated. We also find Granger causality running from a life stress indicator to a life satisfaction indicator. The education indicator model is found to be positively correlated with life satisfaction and positive emotions, whereas the life stress indicator and life satisfaction are negatively correlated. Medical and health indicators are positively related to life satisfaction, while a negative correlation is found between the traffic indicator model and life satisfaction. The relationship between economic development and subjective well-being also displays bidirectional Granger causality and a positive correlation. However, in China's provinces and prefecture-level cities with relatively strong economic growth, the correlation between the livelihood index and economic development appears to be weaker. We suggest boosting gross domestic product per capita and absolute per capita income to increase subjective well-being in less developed western China. Bridging the gross domestic product per capita gap nationwide may also positively influence subjective well-being. To achieve this, we suggest measures that include improving medical and health services, alleviating traffic congestion, increasing the teacher-student ratio and improving the education universalisation rate. These steps would improve the equitable and balanced development of China's Livelihood Index across the country's 31 provinces.

Artificial intelligence facilitates tropical infectious disease control and research

Since the global pandemic of coronavirus disease 2019 (COVID-19) in late 2019, artificial intelligence technology has shown increasing values in the research and control of tropical infectious diseases. The introduction of artificial intelligence technology has shown remarkable effectiveness to reduce the diagnosis and treatment burdens, reduce missing diagnosis and misdiagnosis, improve the surveillance and forecast ability and enhance the medicine and vaccine development efficiency. This paper summarizes the current applications of artificial intelligence in tropical infectious disease control and research and discusses the important values of artificial intelligence in disease diagnosis and treatment, disease surveillance and forecast, vaccine and drug discovery, medical and public health services and global health governance. However, artificial intelligence technology suffers from problems of single and inaccurate diagnosis, poor disease surveillance and forecast ability in open environments, limited capability of intelligent system services, big data management and model interpretability. Hereby, we propose suggestions with aims to improve multimodal intelligent diagnosis of multiple tropical infectious diseases, emphasize intelligent surveillance and forecast of vectors and high-risk populations in open environments, accelerate the research and development of intelligent management system, strengthen ethical security, big data management and model interpretability.

Data-driven Seeing Prediction for Optics Telescope: from Statistical Modeling, Machine Learning to Deep Learning Techniques

Predicting seeing of astronomical observations can provide hints of the quality of optical imaging in the near future, and facilitate flexible scheduling of observation tasks to maximize the use of astronomical observatories. Traditional approaches to seeing prediction mostly rely on regional weather models to capture the in-dome optical turbulence patterns. Thanks to the developing of data gathering and aggregation facilities of astronomical observatories in recent years, data-driven approaches are becoming increasingly feasible and attractive to predict astronomical seeing. This paper systematically investigates data-driven approaches to seeing prediction by leveraging various big data techniques, from traditional statistical modeling, machine learning to new emerging deep learning methods, on the monitoring data of the Large sky Area Multi-Object fiber Spectroscopic Telescope (LAMOST). The raw monitoring data are preprocessed to allow for big data modeling. Then we formulate the seeing prediction task under each type of modeling framework and develop seeing prediction models through using representative big data techniques, including ARIMA and Prophet for statistical modeling, MLP and XGBoost for machine learning, and LSTM, GRU and Transformer for deep learning. We perform empirical studies on the developed models with a variety of feature configurations, yielding notable insights into the applicability of big data techniques to the seeing prediction task.

Task Scheduling for Spark Applications With Data Affinity on Heterogeneous Clusters

The Internet of Things (IoT)-enabled applications use sensors and actuators to collect big data, which are processed by big data models, e.g., Spark. Generally, data processing tasks are precedence constrained and the computation results are transmitted to other IoT devices. In this article, we consider the Spark workflow problem of scheduling tasks with data affinity to heterogeneous servers to minimize the maximum completion time. In a Spark instance, jobs are precedence constrained and stages for each job are also precedence constrained. There are a large number of topological stage orders. A balance between task execution times, determined by heterogeneous servers, and transmission times caused by data affinity is difficult to achieve. A scheduling optimization algorithm framework is proposed, which consists of five components: 1) temporal parameter calculation; 2) ready stage adding; 3) task sequencing; 4) resource allocation; and 5) schedule improvement. Strategies for each component are developed. The algorithmic components are statistically calibrated over a comprehensive set of instances. The proposed algorithm is compared to two modified classic algorithms for similar problems on typical scientific workflow instances. The experimental results demonstrate the effectiveness of the proposal for the considered problem.

Data-Efficient Performance Modeling for Configurable Big Data Frameworks by Reducing Information Overlap Between Training Examples

To support the various analysis application of big data, big data processing frameworks are designed to be highly configurable. However, for common users, it is difficult to tailor the configurable frameworks to achieve optimal performance for every application. Recently, many automatic tuning methods are proposed to configure these frameworks. In detail, these methods firstly build a performance prediction model through sampling configurations randomly and measuring the corresponding performance. Then, they conduct heuristic search in the configuration space based on the performance prediction model. For most frameworks, it is too expensive to build the performance model since it needs to measure the performance of large amounts of configurations, which cause too much overhead on data collection. In this paper, we propose a novel data-efficient method to build the performance model with little impact on prediction accuracy. Compared to the traditional methods, the proposed method can reduce the overhead of data collection because it can train the performance model with much less training examples. Specifically, the proposed method can actively sample the important examples according to the dynamic requirement of the performance model during the iterative model updating. Hence, it can make full use of the collected informative data and train the performance model with much less training examples. To sample the important training examples, we employ several virtual performance model to estimate the importance of all candidate configurations efficiently. Experimental results show that our method needs less training examples than traditional methods with little impact on prediction accuracy.

Artificial intelligence in healthcare: Setting new algo RHYTHM in medicine

Artificial intelligence (AI) is a rapidly expanding avenue in science and is often used to describe the use of computers, big data mining, modeling, and network that may simulate intelligent behavior and critical thinking similar to a human being but at an exceedingly fast pace. What lies at the heart of AI are intelligent algorithms that drive the whole process integratively, imparting the decisive ability to machines. The artificial intelligence market is expected to grow at a rate of 39.4% by 2022-2028. British mathematician Alan Turing’s 1950 paper “Computing Machinery and Intelligence " established AI's fundamental goal and vision. Fundamentally, AI is a branch of computer science that aims to answer Turing’s question of replicating human intelligence as machines. Since AI research aims to make smarter algorithms evolve and emulate human-like functioning, the degree to which an AI system can replicate human capabilities is used to determine the types of AI. The volume, magnitude, and complexity of data in healthcare mean that artificial intelligence (AI) will be increasingly applied and exploited, with several types of AI already being employed by researchers, clinicians, and data analysts. AI is a collective term, not just one technology, with an immediate relevance to healthcare. Few AI technologies of high importance to healthcare are: Machine learning, Neural Networks, and Deep Learning Natural Language Processing ( NLP) Physical Robots Rule-based expert systems The AI-associated healthcare market is growing at an exponential pace and is expected to reach USD 6.6 billion by 2021. Precision medicine initiatives can be divided into three types of clinical areas: complex algorithms, digital health applications, and “omics”-based tests. Large genomic datasets and informatics related to demographic data or diagnostic reports are exploited by machine learning algorithms for prediction and prognosis. This is coupled with patient healthcare data, such as treatment protocols, treatment response, response to therapy, and health monitoring, with data input from clinicians, researchers, and the public domain. High throughput big data from population-based studies are used with machine learning algorithms to establish correlations, build predictive models, and provide customized treatment protocols. In Omics based era, data from genomics, proteomics, and metabolomics are exceedingly used to address and identify health-related problems in large study cohorts . This will provide opportunities for translation and transition from bench to bedside through precision medicine.

Connotation, characteristics and framework of coal mine safety big data

With the continuous development of automation and information technology, large amounts of safety data are produced in the processes of coal production. Most enterprises simply focus on statistics and do not conduct systematic big data analyses. Therefore, it is necessary to study the theory of coal mine safety while using big data systematically. This paper expounds on the changes in coal mine safety that have been driven by big data from three aspects: the connotation, characteristics and research framework. First, the connotation of coal mine safety big data (CMSBD) is redefined by changing the safety entities and methods. Second, the advantages and disadvantages of the big data model are compared from the perspective of feature analysis. Finally, the research paradigm and technical framework of CMSBD are designed. The results show that the management connotation of CMSBD focuses on the role of big data in coal mine safety. Compared with coal mine safety small data (CMSSD), CMSBD has both advantages and disadvantages. Therefore, CMSBD must be combined with a small data method. The research paradigm emphasizes the intersection of the research, the relevance of safety thinking, the importance of safety data analysis, and the fusion of big data with traditional small data models.

End-to-End Data Quality Assessment Using Trust for Data Shared IoT Deployments

Continued development of communication technologies has led to widespread Internet-of-Things (IoT) integration into various domains, including health, manufacturing, automotive, and precision agriculture. This has further led to the increased sharing of data among such domains to foster innovation. Most of these IoT deployments, however, are based on heterogeneous, pervasive sensors, which can lead to quality issues in the recorded data. This can lead to sharing of inaccurate or inconsistent data. There is a significant need to assess the quality of the collected data, should it be shared with multiple application domains, as inconsistencies in the data could have financial or health ramifications. This article builds on the recent research on trust metrics and presents a framework to integrate such metrics into the IoT data cycle for real-time data quality assessment. Critically, this article adopts a mechanism to facilitate end-user parameterization of a trust metric tailoring its use in the framework. Trust is a well-established metric that has been used to determine the validity of a piece or source of data in crowd-sourced or other unreliable data collection techniques such as that in IoT. The article further discusses how the trust-based framework eliminates the requirement for a gold standard and provides visibility into data quality assessment throughout the big data model. To qualify the use of trust as a measure of quality, an experiment is conducted using data collected from an IoT deployment of sensors to measure air quality in which low-cost sensors were colocated with a gold standard reference sensor. The calculated trust metric is compared with two well-understood metrics for data quality, root mean square error (RMSE), and mean absolute error (MAE). A strong correlation between the trust metric and the comparison metrics shows that trust may be used as an indicative quality metric for data quality. The metric incorporates the additional benefit of its ability for use in low context scenarios, as opposed to RMSE and MAE, which require a reference for comparison.

Model establishment and risk management model of industrial big data based on neural network

With the maturity of big data, the data hierarchy and data processing methods of big data are changing with each passing day. Big data has begun to occupy an important position in all walks of life, and data is an asset. Industrial risk has always been the focus of data generation and application. In this paper, we use TensorFlow to implement a big data classification model, and add different optimization methods and convolutional neural networks. In the research, the usage scenarios of different optimization methods and the characteristics of self-sufficiency were compared, so that the final classification accuracy of the model reached 99.67%. Using TensorFlow to build different types of neural networks to fit the Fourier transform, and use the trained model for ultrasonic detection of time-domain signals, and obtain better signal processing results. On this basis and analyze the characteristics of different models, as well as the advantages and disadvantages of signal processing.

Modeling and Analysis Method of National Fitness Big Data for Basketball Projects Based on a Multivariate Statistical Model

How to start from the fitness needs of people and effectively improve the precision of the supply of public fitness services everyone is an important issue that needs to be solved first at the current stage. This requires us to proceed from the reality, conduct accurate research, and find a method that can match the current problem. In this paper, taking basketball projects in national fitness as an example, by introducing a proposition about the development of small basketball events, the corresponding big data modeling and analysis methods are studied. The research methods and research objectives involved in this paper are based on the relevant parameters of the multivariate statistical model. First, the article introduces the calculation principle of the multiple linear regression model. We introduce the concept of variance inflation factor involved in this principle and carry out the modeling and analysis of big data based on this variable. In order to illustrate the application effect of big data in this kind of research, this paper introduces three different big data technologies, including immune selection optimization algorithm, particle swarm optimization algorithm, and Elman neural network, to predict and analyze the variance inflation factor (VIF) corresponding to the small basketball project. The analysis results show that the Elman network exhibits certain advantages in terms of computing convergence time. And, as the number of calculation steps increases, the superiority of the Elman network is more obvious. As far as the prediction performance is concerned, the square of the correlation coefficient corresponding to the immune selection optimization algorithm is the largest and the sum of the squares of the residuals is the smallest, showing superior prediction performance.

Big Data Model of Digital Employees of High-Tech Enterprises under the Background of Digital Transformation

This paper firstly conducts a theoretical analysis on the digital transformation of enterprises. Theories such as Marxism and new institutional economics related to the digital transformation of enterprises are sorted out. Secondly, it conducts an empirical analysis on the digital transformation of high-tech enterprises. This paper summarizes the three major transformation paths of the existing production service digitization, marketing model digitization, and industrial digitization. Based on the data analysis of the questionnaire, this paper summarizes the difficulties of digital transformation of high-tech enterprises, mainly due to the lack of transformation ability, resulting in “cannot transfer”; the lack of self-development conditions and digital transformation, resulting in “will not transfer”; digital transformation lacks full guarantee, resulting in “do not dare to turn.” And based on the analytic hierarchy process, the influencing factors of the digital transformation of high-tech enterprises are analyzed, and the relative importance of each factor is obtained.

The Big Data Model for Urban Road Land Use Planning Is Based on a Neural Network Algorithm.

The spatial differentiation of land use induces traffic demand and guides the construction of traffic supply; traffic conditions are an important influencing factor in determining the nature of land use, and there is a close interaction between the two. This study uses a neural network-based approach at the urban grid level to portray representative phenomena of urban development and analyze the interaction between transportation and land use. The results reflect the model's effective simulation of urban laws, and the case study reveals the differences in the laws of different cities, to guide the benign development of cities and transportation. This article firstly conducts a study on the theoretical foundation; compares the development history, planning, and design methods and practical experience of road planning and resilient planning; summarizes the experience of resilient road system design; and analyzes the future development trend, based on the above basic theoretical research, to develop research ideas and methods. Secondly, the scenario analysis method is explicitly applied to analyze various scenarios that may occur in the future development process of simulated urban roads and rank the scenarios based on the probability of occurrence. For the impact of traffic on land use, the concepts of vitality and potential are introduced, and a multidimensional long and short-term memory network (MDLSTM) model is established. The model takes into account land use lags and potential transfer and has relatively higher prediction accuracy. The results show that larger cities with urban dominant industries and tertiary industries also have higher land use potential and the more significantly influenced by traffic.

Construction of Unbalanced Engineering Cost Management and Multivariate Statistics System Based on Random Matrix Weight Algorithm

The informatization of engineering cost plays an important role in the cost management and is also one of the important subjects of the informatization in the engineering field. It is the direction of the future project cost management. There is a trend of exponential increase in the information about project cost, which brings a great challenge to cost informatization. On the basis of the construction quantity and cost composition in the process of analyzing the present situation of the project cost, the overall index of the construction quantity and cost is constructed. For bridges and tunnels and to achieve a reasonable index of roadbed, on the basis of considering project at different stages and the demand for different levels of cost management personnel, adopt the method of statistical analysis and neural network, respectively, and set up the project cost fast estimation method and cost prediction model: intelligent prediction model and index system forecasting model. The error analysis and checking calculation of the two models provide the possibility for correctly estimating the engineering quantity and engineering cost in the preliminary design stage. Aiming at the low accuracy of the traditional unbalanced cost data classification algorithm, the dimension of random matrix weight was introduced into the cost data classification. On the basis of analyzing the spatial and temporal characteristics of WAMS measurement data, a high-dimensional random matrix model of WAMS measurement big data was constructed according to the high-dimensional random matrix theory, and a nonequilibrium cost data classification model based on the weighted algorithm of improved fuzzy rules was established. The model is applied to a unit project cost data classification, which verifies the operability of the model and provides a new idea for other units to carry out similar work. Finally, other common problems that should be paid attention to when using multivariate statistical analysis method are put forward.

Stochastic Modelling of HFT Big Data in Finance Manifesto

His experience with big data in finance, namely, high-frequency and algorithmic trading (HFT) data, including limit order books (LOB), and stochastic modeling of those data

A Systematic Review Towards Big Data Analytics in Social Media

The recent advancement in internet 2.0 creates a scope to connect people worldwide using society 2.0 and web 2.0 technologies. This new era allows the consumer to directly connect with other individuals, business corporations, and the government. People are open to sharing opinions, views, and ideas on any topic in different formats out loud. This creates the opportunity to make the “Big Social Data” handy by implementing machine learning approaches and social data analytics. This study offers an overview of recent works in social media, data science, and machine learning to gain a wide perspective on social media big data analytics. We explain why social media data are significant elements of the improved data-driven decision-making process. We propose and build the “Sunflower Model of Big Data” to define big data and bring it up to date with technology by combining 5 V's and 10 Bigs. We discover the top ten social data analytics to work in the domain of social media platforms. A comprehensive list of relevant statistical/machine learning methods to implement each of these big data analytics is discussed in this work. “Text Analytics” is the most used analytics in social data analysis to date. We create a taxonomy on social media analytics to meet the need and provide a clear understanding. Tools, techniques, and supporting data type are also discussed in this research work. As a result, researchers will have an easier time deciding which social data analytics would best suit their needs.

Research Trends, Hotspots and Frontiers of Ozone Pollution from 1996 to 2021: A Review Based on a Bibliometric Visualization Analysis

A total of 2932 publications in the field of ozone pollution were obtained from the Web of Science and briefly reviewed using a bibliometric analysis and WOS-based citation reports. CiteSpace 5.7.R3 (64 bit) was used to perform a visualization of knowledge mapping by keywords co-words, burst analysis, co-cited reference analysis, and cooperation network analysis. Research topics in this field have mainly focused on three aspects: risk assessment of ozone pollution for both humans and plants under short-term and long-term exposure; ozone pollution characterization and modeling of ozone transport on different scales; and elucidating the mechanism of ozone formation and source apportionment. By clustering the co-cited references using the data from 2016 to 2021, the frontiers are found to be: (1) VOCs’ precursors and ozone transformation mechanism; (2) modeling of source apportionment and source-oriented chemical transport considering meteorological influence to predict ozone concentration at different spatial and temporal scales; and (3) premature mortality and health burden with relation to ozone exposure. It should be mentioned that an emerging research hotspot is the utilization of artificial intelligence (AI) tools (e.g., machine learning, deep learning, etc.) to facilitate the modeling of big data at different scales.

Big Data Model of Higher Education Online Teaching Based on Intelligent Algorithm

With the continuous development of society, the demand for processing large-scale data in many fields is increasing. Traditional processing training techniques have many limitations for big data analysis applications. Therefore, how to transform big data into general-purpose information becomes particularly important. This research mainly discusses the big data model analysis of higher education online teaching based on intelligent algorithms. The process of the experiment is to access how trainers interact or receive information stimulation in videos and courseware and how to cause relatively lasting changes in cognitive behavior. From the experimental research, we discovered the law of practical training and finally provided personalized teaching support services according to the needs and abilities of the trainers. On the other hand, the online training algorithm for big data analysis is studied, the methods needed to solve the big data mining task are discussed, and the online course training is recommended in many ways. Experimental data show that the algorithm of large-scale online training behavior data analysis on the behavior analysis results of online trainers is conducive to the improvement of online trainers’ learning efficiency. The experimental results show that the algorithm of large-scale online training behavior data analysis can show good model analysis performance, which is conducive to the prediction of the training personnel, and the prediction accuracy reaches about 90%. It is found that the algorithm that implements large-scale online training behavior data analysis can effectively categorize the relationship between the trainee’s visits. Through innovative data analysis methods, fast, efficient, and timely analysis of big data streams is realized.

Optimizing the Prognostic Model of Cervical Cancer Based on Artificial Intelligence Algorithm and Data Mining Technology

With the accumulation and development of medical multimodal data as well as the breakthrough in the theory and practice of artificial neural network and deep learning algorithm, the deep integration of multimodal data and artificial intelligence based on the Internet has become an important goal of the Internet of Medical Things. The deep application of the latest technologies in the medical field, such as artificial intelligence, machine learning, multimodal data, and advanced sensors, has a profound impact on the development of medical research. Artificial intelligence can achieve low-consumption and high-efficiency screening of specific markers due to its powerful data integration and processing capabilities, and its advantages are fully demonstrated in the construction of disease-related risk prediction models. In this study, multi-type cloud data were used as research objects to explore the potential of alternative CpG sites and establish a high-quality prognosis model of cervical cancer DNA methylation big data. 14,419 strict differentially methylated CpG sites (DMCs) were identified by ChAMP methylation analysis and presented these distributions based on different genomic regions and relation to island. Further, rbsurv and Cox regression analyses were performed to construct a prognostic model integrating these four methylated CpG sites that could adequately predict the survival of patients ( AUC = 0.833 , P < 0.001 ). The low- and high-risk patient groups, divided by risk score, showed significantly different overall survival (OS) in both the training ( P < 0.001 ) and validation datasets ( P < 0.005 ). Moreover, the model has an independent predictive value for FIGO stage and age and is more suitable for predicting survival time in patients with histological type (SCC) and histologic grade (G2/G3). Finally, the model exhibited much higher predictive accuracy compared to other known models and the corresponding expression of genes. The proposed model provides a novel signature to predict the prognosis, which can serve as a useful guide for increasing the accuracy of predicting overall survival of cervical cancer patients.