Big Data Analytics Platform Research Articles

Probabilistic Statistics-Based Endurance Life Prediction of Bridge Structures.

With the massive construction of bridge infrastructures, bridge health monitoring systems have gradually matured in application and research, but previous research has primarily focused on structural damage detection and bridge safety warnings based on valid data. The structural details of steel bridge panels and structural systems are determined by the coupling effects of many intrinsic and extrinsic uncertainties, such as material properties, structural characteristics, manufacturing processes, and random traffic loads. The evaluation of fatigue is a difficult task. This article first builds a big data platform, utilizing its high-efficiency parallel computing capability and highly fault-tolerant distributed file system to achieve second-level monitoring data processing; ensuring real-time data cleaning, data analysis, and safety warning; and building a big data analysis and processing platform with high reliability, high availability, high storage efficiency, and high scalability of bridge health monitoring. The big data platform chooses HDFS for offline data storage and Spark for data analysis and modelling after comparing and analysing the benefits and drawbacks of various big data technologies. Kafka is used for caching real-time data, and Spark-streaming is used for reading data and real-time processing. Finally, the platform's superiority and reliability in terms of offline computing performance, real-time online performance, scalability, and fault tolerance are confirmed through experimental analysis; the optimal data cleaning method is derived by comparing and analysing monitoring data noise, jump point, and drift phenomena. This part of the research is based on bridge temperature data with stable signals and bridge strain data with fluctuating signals, taking into account the influence of different data types; the corresponding data missing repair algorithms are proposed for different types of data to form a complete and general data patching method process. The probabilistic fracture mechanics theory, in comparison to the traditional deterministic fatigue assessment method, can better reflect the essential uncertainty of fatigue problems and is an effective way to assess the fatigue performance of orthotropic steel bridge decks. The goal of data patching is to ensure data recovery accuracy of over 90%, with no patching repair required for monitoring data with too much missing data. The endurance life of bridge structures is predicted using a big data probabilistic statistics approach based on a variety of factors such as material properties, construction characteristics, manufacturing processes, and random traffic loads.

Treatment of Cancer Gene Changes in Chronic Myeloid Leukemia by Big Data Analysis Platform-Based Dasatinib.

Based on data mining, an innovative big data analysis platform was utilized to discuss the treatment of cancer in chronic myeloid leukemia (CML) by dasatinib, aiming to offer help to the diagnosis and treatment of cancer. An integrated gene expression analysis system (IEAS) was firstly constructed to automatically classify data in the online human Mendelian genetic database using clustering algorithms. At the same time, the gene expression profile was analyzed by principal component analysis (PCA) in the analysis system. In addition, the efficacy of dasatinib in the treatment of patients with advanced CML was then retrospectively analyzed. The results showed that the IEAS system could incorporate the gene expression analysis vectors it contained by JAVA-related technologies, and the generated clustering genes showed similar functions. The clustering algorithm could homogenize data and generate visual clustering heat maps. The analysis results of major elements were diverse under different experimental conditions. The characteristic value of the first major element was the largest. Messenger ribonucleic acid (mRNA) datasets of CML patients were selected from cancer genomic map, including 120 samples and 20,614 mRNA in total. In micro-RNA (miRNA) datasets, there were 202 samples including 1,406 miRNAs. Data were screened by miRNA–mRNA regulation template, and 20 differentially expressed mRNAs were obtained. In conclusion, the proposed IEAS system could mine and analyze the gene expression data. Dasatinib showed good efficacy in the treatment of patients with advanced CML. Besides, it could improve visual queries, and data mining had a broad application prospect in clinical application. Dasatinib was considered to be a good option for patients with advanced CML.

Data Analytics of Electronic Health Records to Enhance Care of Coronary Artery Disease in Younger Women with Avoiding Possible Delay in Treatment.

Early detection plays a key role to enhance the outcome for Coronary Artery Disease. We utilized a big data analytics platform on ∼32,000 patients to trace patients from the first encounter to CAD treatment. There are significant gender-based differences in patients younger than 60 from the time of the first encounter to Coronary Artery Bypass Grafting with a p-value=0.03. This recognition makes significant changes in outcome by avoiding delay in treatment.

Twitter Data Visualization using SVM (Machine Learning Approach)

Abstract: Twitter produces a massive amount of data due to its popularity that is one of the reasons underlying big data problems. One of those problems is the classification of tweets due to use of sophisticated and complex language, which makes the current tools in- sufficient. We present our framework HTwitt, built on top of the Hadoop ecosystem, which consists of a MapReduce algorithm and a set of machine learning techniques embedded within a big data analytics platform to efficiently address the following problems. Keywords: Support Vector Machine (SVM), Machine learning, Big Data, MapReduce

Construction and Application of Psychological Quality Assessment Model for College Students Based on Extensive Data Analysis.

This paper constructs a platform framework for extensive data analysis of college students' psychological quality with the help of the thinking mode of big data and related technologies and proposes the construction principles, data sources, data processing methods, data platform construction, and platform application of big data analysis platform for college students' psychological quality assessment. This paper combines the application methods of big data technology, collects the management data related to the psychological quality assessment of college students, saves them into the system database with certain storage logic, and realizes the function of psychological quality assessment through the design of selected psychological quality assessment data, data management and data resource management and other parts based on the data results of extensive data analysis. This study provides some insights into the psychological quality assessment of college students. The strength of association between the variables of psychological quality assessment of college students changes over time, but the overall psychological structure is more stable. This stable psychological structure characteristic is conducive to constructing the policy of constant psychological education in large universities.

Construction of Game Model between Carbon Emission Minimization and Energy and Resource Economy Maximization Based on Deep Neural Network.

Under this background, this paper tries to find countermeasures and ways for carbon reduction by observing and analyzing the influencing factors of carbon emissions, designing ways to minimize carbon emissions and maximize resources and energy. In view of the above problems, the carbon emission prediction research is closely combined with the research of deep neural network, the carbon emission prediction models based on deep neural network are established, respectively, and the game theory is introduced to maximize the resource economy. Based on the analysis of the cost of energy resources, this paper puts forward a model based on game theory and makes an overall planning of the bidding online auxiliary decision-making system in combination with the actual market demand. Build a big data analysis platform based on the Internet of things, collect the data related to carbon emission for normalization, analyze the influencing factors related to carbon emission by using the principal component analysis method, select the data with higher connection value, and take the time series data as the input of the deep neural network for simulation verification. The simulation results show that the game model of carbon emission minimization and energy resource economic maximization based on deep neural network can effectively improve the economic maximization of energy resources and reduce carbon emissions.

An end-to-end big data analytics platform for IoT-enabled smart factories: A case study of battery module assembly system for electric vehicles

Within the concept of factories of the future, big data analytics systems play a critical role in supporting decision-making at various stages across enterprise processes. However, the design and deployment of industry-ready, lightweight, modular, flexible, and cost efficient big data analytics solutions remains one of the main challenges towards the Industry 4.0 enabled digital transformation. This paper presents an end-to-end IoT-based big data analytics platform that consists of five interconnected layers and several components for data acquisition, integration, storage, analytics and visualisation purposes. The platform architecture benefits from state-of-the-art technologies and integrates them in a systematic and interoperable way with clear information flows. The developed platform has been deployed in an electric vehicle battery module assembly automation system designed by the Automation Systems Group at the University of Warwick, the UK. The developed proof-of-concept solution demonstrates how a wide variety of tools and methods can be orchestrated to work together aiming to support decision-making and to improve both process and product qualities in smart manufacturing environments.

Research on College Students’ Physique Testing Platform Based on Big Data Analysis

With the progress of the times, people’s living standards have significantly improved, gradually put physical health in the first place, college students as the country’s future talent reserve, its physical health development is particularly important. But for a long time, the physique level of college students has failed to reach the standard, and universities are exploring ways to improve the physique level of college students. In view of the existing deficiencies in the monitoring platform of student’s physique, we fully utilize the advanced technology of computer and Internet development and follow the basic principles of economical practicality, scalability, user-friendliness and real-time information exchange to establish monitoring and service platform for college students’ physique. With the aid of the thinking mode of big data and related technology, build the physical health of college students in China big data analytics platform framework, physical health of large numbers of college students in China are put forward According to the analysis platform construction principle, data source, data handling, data platform and application platform, etc., in order to guide the physical health of college students practice in our country, to promote the reform of physical health education for students in colleges and universities. By using big data analysis technology, 65,535 physical health records of all students in a university from 2017 to 2019 were recorded as data sources, and algorithm based on distance was applied to cluster analysis of two groups of data classified by male and female gender, and a series of data were processed, converted, and modeled.

Big data analysis and optimization and platform components

Communication operators are paying more and more attention to the value of data and are demanding more and bigger data technologies. Many companies have started to take advantage of their resources to tap the value of data and develop their own core business. The use of a high-performance, secure, scalable and easy-to-manage big data management system will help companies avoid the tedious system operation and maintenance, especially in the communication business system, and help them focus on their own business development. In this paper, we first investigate existing data management systems and analyze their strengths and weaknesses. In response to the problems of not too light, insufficient timeliness of data migration, and not enough innovation in data analysis, we design a more efficient, convenient and easy-to-use big data management platform. Firstly, the big data management system is designed. According to the process of big data processing, six modules of interface acquisition, program scheduling, data aggregation, platform alerting, marketing analysis and visualization are designed based on the communication big data platform architecture. The management system mainly focuses on data access and data mining analysis, so the main modules of this paper are program scheduling, data aggregation and marketing analysis modules, while other modules are based on the original big data management system of the enterprise with a small amount of improvement. In order to realize the needs of dynamically creating data testing environment and isolating production and experimental environments under the communication application scenario, a mechanism of the big data system for production and the virtualized system for experiments acting together is proposed. Then the corresponding scheduling module architecture process is designed and built, the corresponding scheduling rules and related scheduling information field tables are designed, and the data aggregation storage is improved. The program scheduling module was designed to be more lightweight and easy to use, and the data migration module increased the timeliness of data migration.

Research on the Construction of Enterprise Accounting Data Analysis Platform Based on Cloud Computing

With the emergence of massive data inside and outside the enterprise, paying much attention to the processing and analysis of accounting big data can bring huge value-added value to enterprises, and adapt to complex and changing economic environment. Based on this, by analyzing the related theories of accounting big data, accounting information and cloud computing, we build a cloud computing storage module of big data analysis platform, and apply Apriori data mining algorithm based on association rules to deal with massive data of accounting. A comparative prediction of the financial status of a group shows that the maximum error is less than 8% compared with the actual results, thus verifying the reliability and superiority of the established accounting big data analysis platform based on cloud computing.

Perceptions and Key Attributes of Exercise Among Korean Men and Women During the COVID-19 Pandemic: A Comparative Study Using Big Data Analysis

Background: The COVID-19 pandemic has increased risks to people’s health due to restrictions on movement and access to exercise facilities. This study compares and analyzes perceptions and key attributes of exercise among Korean men and women in the context of the COVID-19 pandemic by using data collected from social media. Methods: Data were collected from January 2020 to June 2021. The search keywords were classified into “COVID-19, men, and exercise” and “COVID-19, women, and exercise”, and data were collected through Textom (The Imc Inc., Daegu, Republic of Korea), a big data collection and analysis platform. Frequency analysis, TF-IDF analysis, and degree centrality analysis were performed using Textom. In addition, a CONCOR (CONvergent and CORrelation) analysis was performed using UCINET 6 (Analytic Technologies Corp., Lexington, KY, USA) to visualize related words and analyze clusters. Results: In total, 10,703 data points of 5.935 MB were collected and analyzed. The top 50 words related to men’s and women’s exercises of COVID-19 were derived through frequency analysis, TF-IDF analysis, and degree centrality analysis. We also derived four common clusters (exercise method, COVID-19 situation, friends, and sports). In addition, three clusters (women’s fitness, children and hashtags) for men’s exercise and two clusters (weight loss and Olympics) for women’s exercise were derived separately. Conclusions: The study results show that it is possible to provide suggestions and health improvement plans for exercise based on gender. In addition, our findings can serve as an academic basis for developing health promotion programs related to exercise during the COVID-19 pandemic, which can contribute to reducing the economic costs and health risks associated with the pandemic.

Design and Application of Sports-Oriented Public Health Big Data Analysis Platform.

People's pursuit of public health continues to improve with the rapid economic development. Physical activity is an important way to achieve public health. Excessive physical activity intensity and uncomfortable forms of physical activity can affect people's physical and mental health. Reasonable physical activity intensity and reasonable physical activity form will be beneficial to public health. People need to choose the corresponding sports mode according to physical function parameters and mental health parameters. However, it is difficult to understand the relationship between physical activity patterns and public health-related parameters, which limits people to establish reasonable exercise patterns. This research uses big data technology to design an intelligent sports-oriented public health data analysis scheme. It mainly uses MLCNN method and LSTM method to extract physical function parameter features, mental health parameter features, and sports parameter features. The research results show that the MLCNN method and LSTM can accurately extract and predict the parametric features related to sports and public health. The largest relative mean error is only 2.52%, which is the predicted value of the physical performance parameter characteristics. The smallest prediction error is also 2.27%, and this part of the relative error comes from the prediction of sports parameters.

Retracted] Classification and Visual Design Analysis of Network Expression Based on Big Data Multimodal Intelligence Technology

The rapid development of the Internet in modern society has promoted the development of many different network platforms. In the context of big data, many types of multimodal data such as pictures, videos, and texts are generated in the platform. Through the analysis of multimodal data, we can provide better services for users. The traditional big data analysis platform cannot achieve a completely stable state for the analysis of multimodal data. The construction of multimodal intelligent platform can achieve efficient analysis of relevant data, so as to create greater economic benefits for the society. This paper mainly studies the historical development trend of big data multimodal intelligence technology and the data processing method of multimodal intelligence technology applied to network expression classification, including data acquisition, storage, and analysis. Finally, it studied the fusion algorithm between multimodal data and visual design, as well as the classification of network expression and the application result analysis of visual design in big data multimodal intelligence technology.

Construction of a Public Health-Oriented Sports Training Big Data Analysis Platform.

Sports health has become a goal pursued by most people, both young and old, which is mainly due to the improvement of people's living standards and the improvement of economic level. Different groups have great differences in the way of physical exercise for public health. The idea of pursuing physical exercise is better but most ignore the factors that affect exercise. Not only will this have a certain negative impact on body function but it also defeats the purpose of physical exercise. Reasonable physical exercise is more urgently needed. However, for public health physical exercise, reasonable methods are also difficult to obtain. This is mainly due to the large differences in the number of groups and hurdles faced by public health. This study designs a public health-oriented sports training platform based on big data technology. It mainly uses the hollow convolutional neural network (A-CNN) and the GRU method to extract the relationship between physical training and physical function, weather factors, and exercise intensity. The research results show that the A-CNN and GRU methods can better map the relationship between sports training parameters and the three characteristics that affect sports health. This kind of sports training platform based on big data technology can better guide young people or the elderly to carry out reasonable physical exercise. A-CNN and GRU techniques have relatively high accuracy in predicting the three characteristics of physical exercise. The smallest error is only 1.43%, and the largest error is also 2.56%.

Big data analytics opportunities for applications in process engineering

Abstract Big data is an expression for massive data sets consisting of both structured and unstructured data that are particularly difficult to store, analyze and visualize. Big data analytics has the potential to help companies or organizations improve operations as well as disclose hidden patterns and secret correlations to make faster and intelligent decisions. This article provides useful information on this emerging and promising field for companies, industries, and researchers to gain a richer and deeper insight into advancements. Initially, an overview of big data content, key characteristics, and related topics are presented. The paper also highlights a systematic review of available big data techniques and analytics. The available big data analytics tools and platforms are categorized. Besides, this article discusses recent applications of big data in chemical industries to increase understanding and encourage its implementation in their engineering processes as much as possible. Finally, by emphasizing the adoption of big data analytics in various areas of process engineering, the aim is to provide a practical vision of big data.

Access Data Analysis Technology and Implementation of Electric Power Big Data Achievement Sharing Platform through Artificial Intelligence

With the rapid development of the information age, the data generated by all walks of life is showing an increasing trend of “blowout”. According to statistics, the total amount of data generated by mankind in the past 40,000 years is less than the total amount of data generated from 2010 to 2013 alone. The global big data reserves reached 8.61ZB in 2015 alone, and the growth of data in the future will reach an unpredictable value, entering the data age in an all-round way. Traditional power data computing technology and intelligent analysis technology are undergoing profound changes, and emerging big data intelligent analysis platforms are gradually emerging. With the in-depth development of power informatization and the concept of smart grid, the power industry data has grown exponentially, and the business demand for intelligent analysis of large amounts of power data is increasing day by day. Therefore, the access data analysis technology and realization of the power big data achievement sharing platform based on AI is of great significance. The AI-based power big data achievement sharing platform constructed in this article is a research on previous data analysis, and it aims to use the current cutting-edge artificial intelligence technology to build a scalable and highly available power big data analysis and processing platform to provide fast and reliable smart data services for the power industry, smart grid and other grid businesses. Research has shown that the overall availability of the ultrasonic partial discharge recognition system of the artificial intelligence-based electric power big data achievement sharing platform is 99.9967%, which meets the high availability index requirements, and verified that the artificial intelligence-based power big data achievement sharing platform provides highly available computing, storage and other services for its applications.

Large scale data analysis using MLlib

Recent advancements in the internet, social media, and internet of things (IoT) devices have significantly increased the amount of data generated in a variety of formats. The data must be converted into formats that is easily handled by the data analysis techniques. It is mathematically and physically expensive to apply machine learning algorithms to big and complicated data sets. It is a resource-intensive process that necessitates a huge amount of logical and physical resources. Machine learning is a sophisticated data analytics technology that has gained in importance as a result of the massive amount of data generated daily that needs to be examined. Apache Spark machine learning library (MLlib) is one of the big data analysis platforms that provides a variety of outstanding functions for various machine learning tasks, spanning from classification to regression and dimension reduction. From a computational standpoint, this research investigated Apache Spark MLlib 2.0 as an open source, autonomous, scalable, and distributed learning library. Several real-world machine learning experiments are carried out in order to evaluate the properties of the platform on a qualitative and quantitative level. Some of the fundamental concepts and approaches for developing a scalable data model in a distributed environment are also discussed.

Application of Big Data Analysis and Cloud Computing in Network Platform Building

With the advent of the cloud era, big data has attracted more and more attention. At present, due to the expansion of the scale of chain educational training institutions and the complexity of management, the information stored in the data of management platform is mostly unstructured data model. The large amount and complexity of unstructured data become the main difficulties in big data analysis and decision-making. This paper analyzes and discusses the application of chain educational training institutions based on the background of big data, and build a platform for big data analysis and decision-making of chain educational training institutions based on cloud computing. The platform greatly improves the ability to process massive data and discovers the internal relationship and rules between big data. It aims to make better use of big data to serve chain educational training institutions, promote the continuous development and progress of chain educational training institutions, and even lay the foundation for the global chain educational training institutions.

Design and Implementation of Hotel Big Data Analysis Platform Based on Hadoop and Spark

In order to provide decision support for user travel and hotel management planning, this paper proposes the design and implementation of a hotel big data analysis platform based on Hadoop and Spark technology, using hotel basic data and user comment data as data sources. First, we design a crawler program to collect basic data and user comment data of city hotels in Qingdao, Shandong Province, and upload them to the hadoop storage platform, use Spark for big data preprocessing, use Hive to build a big data platform data warehouse, and finally use echarts to analyze the results and make decisions perform a visual display.

Research on the Framework of Precision Education Model in Universities Based on Big Data Technology

The era of big data in education has come, data-driven intelligent decision-making has become the development trend of the era of big data. The requirements for the subdivision of education are constantly strengthened, and students prefer to get accurately connected educational courses and practical training. Colleges and universities need to use the existing teaching resources and teacher resources to subdivide the needs of students, and further establish a curriculum and education model in line with the needs of students and personalized characteristics and advantages. Nowadays, precision and personalization have become the key words of education and teaching in the era of big data. This article uses python to search, and uses CiteSpace to define the concept of precision education and analyze and summarize the related literature. Then the article analyzes the application of education big data in precision education, and finally build the overall structure of the precision education big data analysis platform.