Traditional Big Data Research Articles

Research and Application of Big Data Clustering Algorithm based on AI Technology in Cloud Environment

Traditional big data filling algorithms often give inaccurate results due to vulnerabilities to different data types.To solve this problem, this study presents a new big data clustering algorithm powered by AI technology in a cloud environment. The study proposes an advanced Big Data clustering algorithm that leverages AI technology in a cloud environment. It optimizes clustering based on predicted strength using parallel processing. The research focuses on optimizing the clustering algorithm based on the predicted intensity through parallel processing. Experimental results demonstrate that image clustering stability is achieved when the number of clusters exceeds 4, indicating reduced sensitivity to random factors. Although it was not possible to precisely determine the optimal number of clusters, the use of an optimization algorithm showed that at four clusters the prediction intensity reached its peak, ensuring more accurate cluster identification. Through rigorous testing, the optimal number of clusters was determined to be 4. Clustering results show that visitors characterized by certain attributes show higher interest in most columns. This algorithm makes it easier to cluster incomplete large data, improves clustering speed, and improves the accuracy of filling in missing data. Compared to existing methods, this algorithm leverages AI technology in the cloud environment to optimize clustering based on prediction intensity, providing improved accuracy and efficiency during processing in big data management.

Research on Management and Optimization of Big Data Computing Engine Based on Cloud Native Technology IT Architecture

In the wave of digital transformation, we are in a data-driven era. With the explosive growth of data volume, traditional big data processing platforms are facing more and more challenges, including insufficient computational efficiency and resource utilization. At the same time, cloud native technology has emerged as a key factor in filling this gap due to its flexibility, automated management, and cross platform compatibility. Although cloud native technology has broad prospects, its application in the field of big data is still in its early stages, and there are still many problems that need to be solved. Among them, the unity of cluster management, the collaboration between operation and development, and the efficiency of technology architecture transformation are the main challenges currently faced. To address these issues, we will conduct in-depth research around cloud native technology, big data computing engine technology, and containerized services, exploring and implementing a big data computing engine management and optimization solution based on cloud native technology. By proposing and implementing cloud native deployment methods, containerized construction methods, and a one-stop big data application management and development platform, the aim is to provide new ideas and methods for solving many problems in traditional big data computing engine management, and to contribute to the development and innovation of the big data field.

New Teaching Model of Professional Big Data Courses in Universities Based on an Outcome-Oriented Educational Concept

Most traditional Big Data courses focus on the cultivation of students’ professional theoretical knowledge but neglect the training of students’ programming skills. To test the effectiveness of students’ programming learning, the study constructed a classification algorithm based on the TCNN model from the perspective of program identification for identifying programs written by students on their own. To improve the convergence speed and to retain more data features of the programs, the study used the Softplus-Relu combined activation function. To avoid overfitting the model, a dropout strategy was also introduced to optimize the existing TCNN model. The experimental results show that the TCNN model with the Softplus-Relu activation function converges faster, and the classification accuracy obtained is higher than 95%. The loss values and classification accuracies obtained with the TCNN-Dropout model are better than those of the GIST-KNN algorithm. The former has a loss value close to 0 and an accuracy rate of 98.9%. Thus, this indicates that the improved TCNN model proposed in the study has advantages in the identification procedure and can be used as a teaching aid for the training of big data professionals.

A Big Data Grided Organization and Management Method for Cropland Quality Evaluation

A new gridded spatio-temporal big data fusion method is proposed for the organization and management of cropland big data, which could serve the analysis application of cropland quality evaluation and other analyses of geographic big data. Compared with traditional big data fusion methods, this method maps the spatio-temporal and attribute features of multi-source data to grid cells in order to achieve the structural unity and orderly organization of spatio-temporal big data with format differences, semantic ambiguities, and different coordinate projections. Firstly, this paper constructs a dissected cropland big data fusion model and completes the design of a conceptual model and logic model, constructs a cropland data organization model based on DGGS (discrete global grid system) and Hash coding, and realizes the unified management of vector data, raster data and text data by using multilevel grids. Secondly, this paper researches the evaluation methods of grid-scale adaptability, and generates distributed multilevel grid datasets to meet the needs of cropland area quality evaluation. Finally, typical data such as soil organic matter data, road network data, cropland area data, and statistic data in Da’an County, China, were selected to carry out the experiment. The experiment verifies that the method could not only realize the unified organization and efficient management of cultivated land big data with multimodal characteristics, but also support the evaluation of cropland quality.

Understanding the effects of environmental perceptions on walking behavior by integrating big data with small data

Human-perceived landscape and amenities are not equivalent to the objective physical world. Traditional big data research that only uses objectively measured environments based on point-of-interest data has difficulty capturing the effect of human environmental perceptions on walking behavior. We proposed a conceptual framework to examine the effect of human perception on walking behavior by integrating travel trajectories from the “big data” derived from mobile phones and “small data” obtained from a questionnaire survey of residents living in Chengdu, China. Results show that the human subjective perceived environment is inconsistent with the objectively measured built environment. The walking behavior of individuals is completely regulated by their personal perception of the surrounding environment. Moreover, enhancing the environmental design of pathways leading to retail shops, recreational areas, and shopping centers could significantly enhance perceived accessibility and foster a more pedestrian-friendly neighborhood.

The communication path and improvement strategy of symbolic culture of sneaker consumption culture using the big data analysis.

With the emergence of Artificial Intelligence technology and the advancement of science and technology, the current mainstream path of social development is continuously updating and improving various industries using technology. Therefore, in order to promote the development of sneaker consumer culture, this study explores the use of technological means to improve the dissemination effect of symbolic culture in sneaker consumer culture. Firstly, the development concept and mainstream direction of sneaker consumer culture in the era of big data are discussed, and the application principle of big data technology is introduced. Then, a sneaker culture dissemination model based on big data technology is designed. Finally, the model is optimized using a Convolutional Neural Network (CNN), and its effectiveness is evaluated. The results show that the Convolutional Neural Network-Big Data (CNN-BD) model designed in this study has the highest fitting degree of 93% and a lowest fitting degree of 78% in the UT-Zap50K dataset. In the Ai2 dataset, the highest fitting degree of the big data classification model is 94%, and the lowest is 76%. In the Kaggle Women's Shoe dataset, the highest fitting degree of the big data classification model is 92%, and the lowest is 77%. In the Kaggle Men's Shoe dataset, the highest fitting degree of the big data classification model is 94%, and the lowest is 79%. The designed model has the highest accuracy rate of 93% in sneaker classification, while other models have the highest accuracy rate of around 82% in sneaker classification. Compared with traditional big data technology, the designed model has greatly improved and can adapt to more working environments. This study not only provides technical support for the application of big data technology but also contributes to improving the dissemination effect and promoting the comprehensive development of sneaker consumer culture.

Cluster analysis-based big data mining method of e-commerce consumer behaviour

In order to overcome the problems of low precision and long time of data mining in traditional big data mining methods of consumer behaviour, a clustering analysis method for big data mining of e-commerce consumer behaviour is proposed. In this paper, the K-means algorithm is used to calculate the similarity of behaviour clustering nodes of fee payers, determine the clustering process of consumer behaviour data and determine the mining weight of behaviour data. According to the FCM clustering algorithm, the target function for data mining of e-commerce consumer behaviour is constructed. According to Lagrange multiplication, the membership degree of consumer behaviour data is obtained, and the big data mining of consumer behaviour in e-commerce is realised. The experimental results show that with the method proposed in this paper, when the number of consumers is 500, the time for big data mining of consumer behaviour is 15.6s and the accuracy of behaviour big data mining is 95.34%.

Comprehensive Analysis of Various Big Data Classification Techniques: A Challenging Overview

Data over the internet has been increasing everyday, and automatic mining of essential information from an enormous amount of data has become a challenging task today for an organisation with a huge dataset. In recent years, the prominent technology in the domain of Information Technology (IT) is big data, which is unstructured data that solves the computational complexity of classical database systems. The data is fast and big and typically derived from multiple and independent sources. The three main challenges are data accessing, semantics, and domain knowledge for various big data utilisations and complexities raised by big data volumes. One of the major limitations is the classification of big data. This paper introduces well-defined classification methodologies employed for big data classification. This paper reviews 50 research papers based on classification methods of big data, and such methodologies are primarily categorised into six different categories, namely K-Nearest Neighbor (KNN), Support Vector Machine (SVM), Fuzzy-based method, Bayesian-based method, Random Forest, and Decision Tree. In addition, detailed analysis and discussion are carried out by considering classification techniques, dataset utilised, evaluation metrics, semantic similarity measures, and publication year. In addition, research gaps and issues for several traditional big data classification techniques are explained to expand investigators’ works to provide effective big data management.

Research on Data Fusion Method Based on Multisource Data Awareness of Internet of Things

The diversity of big data in Internet of Things is one of the important characteristics that distinguish it from traditional big data. Big data of Internet of Things is often composed of a variety of data with different structural forms. The description of the same thing by these different modal data has certain independence and strong relevance. Accurately and efficiently extracting and processing the hidden fusion information in the big data of the Internet of Things is helpful to solve various multimodal data analysis tasks at present. In this paper, a multimodal interactive function fusion model based on attention mechanism is proposed, which provides more efficient and accurate information for emotion classification tasks. Firstly, a sparse noise reduction self-encoder is used to extract text features, Secondly, features are extracted by encoder. Finally, an interactive fusion module is constructed, which makes text features and image features learn their internal information then the combination function is applied to the emotion classification task.

Application analysis of digital fund prediction model based on neural network

SummaryWith the development of the economy, digital funds have developed rapidly, and traditional big data processing technologies have been unable to effectively solve the update of digital funds information. This research uses neural network to study the digital fund forecasting model based on big data technology and deep learning algorithm. First, on the basis of the feedforward neural network model, the network structure and network parameters are optimized to effectively improve the algorithm's ability to learn the characteristics of digital capital information. Through the first‐order approximation, the number of iterations in the model parameter update is effectively reduced, and the computational efficiency of the algorithm is improved. By improving the model network structure, increasing the number of neurons in the hidden layer, and improving the dynamic change learning ability of the algorithm, the dynamic characteristics of the incremental file information can be effectively obtained. Through the experimental simulation calculation, it can be seen that compared with the traditional algorithm, the algorithm proposed in this study has certain advantages in terms of convergence speed, convergence ability and solution error, and the algorithm has good stability and strong inheritance, and is suitable for digital capital forecasting aspects of research.

Design of Cloud Computing Platform for Large-Scale Multimedia Communication

In order to enlarge the scale of multimedia communication, based on the design of cloud computing platform, this paper makes an in-depth study on the design of multimedia communication platform. Firstly, we study from three methods: extracting high-quality content, broadening multimedia profit channels, and enhancing copyright awareness. Among them, the big data diversion system under the cloud computing platform is designed to optimize data clustering and pattern recognition. The traditional big data shunting system under the cloud computing platform adopts the open-source embedded system design method. The shunting system is connected to the switch of the cloud computing platform through the coupling coil, resulting in missing points and wrong points in the data in the shunting system. Using this method, it is concluded that the cloud computing platform cannot achieve the optimal allocation in terms of SLA violation rate, resource consumption, and power consumption. After experiments and research, the current development trend of media shows a deviation from this phenomenon. The development of media itself, that is, the self-appreciation of media, is gradually becoming the main purpose of media development. A power simulation cloud computing platform architecture with dynamic expansion, efficient computing, and convenient resource scheduling should be given to face large-scale multimedia communication.

Hybrid Elliptic Curve Cryptographic Approach for Data Privacy and Authentication in Secured Map Reduce Layer (SMR) for Optimized CPU Utilization

The traditional Big Data model doesn't explicitly state for both structured and unstructured data sensitivity capture. In addition, the concept of privacy and security needs to be incorporated so that the risk of revealing personal data is subsequently mitigated. This paper proposes Secured Map Reduce Layer (SMR) in the big data structure for optimizing CPU utilization through Hybrid PSO/CS (Particle Swarm Optimization/Cuckoo Search) and Elliptic Curve Cryptographic approaches. This proposed method deals with choosing the private key for Authentication using either particle swarm optimization or cuckoo search algorithm and Provide and encryption and decryption through Key-Value Pair (KVP) of Elliptic Curve Cryptographic. HECC is implemented in a privacy layer called the Secured Map Reduce Layer (SMR). The proposed hybrid model combined with the Particle Swarm Optimization (PSO) that endorses the best value of global fitness for identifying the values and Cuckoo Search (CS) which generates the compiled post-process by finding with efficient authentication. This model provides a promise for privacy and protection in the data. It also solves privacy scalability problems such as CPU and memory optimization retains the trade-off of privacy-utility for big data. There is a remarkable improvement in running time and loss of information over current approaches and optimization of CPU and memory usage. Confidentiality is the process of the sender encrypting through a private key and the public key is utilized to decrypt the document. The protection of the entire algorithm suits a challenge compared with the related techniques through execution speed and accuracy.

Target similarity matching algorithm of big data in remote sensing image based on Henon mapping

In order to overcome the problem of low matching accuracy in traditional big data target similarity matching algorithm of remote sensing image, this paper proposes a new target similarity matching ...

Retracted] Classification and Visual Design Analysis of Network Expression Based on Big Data Multimodal Intelligence Technology

The rapid development of the Internet in modern society has promoted the development of many different network platforms. In the context of big data, many types of multimodal data such as pictures, videos, and texts are generated in the platform. Through the analysis of multimodal data, we can provide better services for users. The traditional big data analysis platform cannot achieve a completely stable state for the analysis of multimodal data. The construction of multimodal intelligent platform can achieve efficient analysis of relevant data, so as to create greater economic benefits for the society. This paper mainly studies the historical development trend of big data multimodal intelligence technology and the data processing method of multimodal intelligence technology applied to network expression classification, including data acquisition, storage, and analysis. Finally, it studied the fusion algorithm between multimodal data and visual design, as well as the classification of network expression and the application result analysis of visual design in big data multimodal intelligence technology.

Dataflow Management in the Internet of Things: Sensing, Control, and Security

The pervasiveness of the smart Internet of Things (IoTs) enables many electric sensors and devices to be connected and generates a large amount of dataflow. Compared with traditional big data, the streaming dataflow is faced with representative challenges, such as high speed, strong variability, rough continuity, and demanding timeliness, which pose severe tests of its efficient management. In this paper, we provide an overall review of IoT dataflow management. We first analyze the key challenges faced with IoT dataflow and initially overview the related techniques in dataflow management, spanning dataflow sensing, mining, control, security, privacy protection, etc. Then, we illustrate and compare representative tools or platforms for IoT dataflow management. In addition, promising application scenarios, such as smart cities, smart transportation, and smart manufacturing, are elaborated, which will provide significant guidance for further research. The management of IoT dataflow is also an important area, which merits in-depth discussions and further study.

RETRACTED ARTICLE: Research on roof waterproof construction of agricultural construction engineering based on big data technology

This article has been retracted from publication in the Taylor & Francis journal, Acta Agriculturae Scandinavica, Section B — Soil & Plant Science. Following publication, concerns were raised by multiple third-parties around the content of the special issue and the decision-making process. Following an investigation by the Taylor & Francis Publishing Ethics & Integrity team in full cooperation with the Editor-in-Chief, it was confirmed that this article included in Special Issue titled “ Envisage Computer Modelling and Statistics for Agriculture”, guest edited by Gunasekaran Manogaran was not peer-reviewed appropriately, in line with the Journal's peer review standards and policy. As the stringency of the peer review process is core to the integrity of the publication process, the Editor and Publisher have decided to retract all of the articles within the above-named Special Issue. The journal has not confirmed if the authors were aware of this compromised peer review process. The journal is committed to correcting the scientific record and will fully cooperate with any institutional investigations into this matter. The authors have been informed of this decision. We have been informed in our decision-making by our policy on publishing ethics and integrity and the COPE guidelines.

SmarT: Machine Learning Approach for Efficient Filtering and Retrieval of Spatial and Temporal Data in Big Data

Spatiotemporal data has always been big data. In these days, big data analytics for spatiotemporal data is receiving considerable attention to allow users to analyze huge amounts of data. Traditional big data platforms cannot handle all the challenges of processing spatio-temporal data. Although some big data platforms have been proposed to process a massive volume of spatiotemporal data, neither is considered a clear winner for all possible scenarios. This paper presents the SmarT query engine, a machine learning-based solution that chooses the best big data platform for processing spatiotemporal queries on the fly. In a detailed experimental evaluation, considering the Apache Spark, Elasticsearch, and SciDB big data platforms, the response time decreased up to 22% when using SmarT.

Big Data Privacy Protection Technology

With the development of Internet technology, we have ushered in the “big data era”. The arrival of big data has made data calculation, acquisition, and statistics more convenient, and has had a significant impact on people’s lives, social management, and national governance. However, because the information unearthed by big data analysis technology may exceed people’s imagination, its privacy issues have also attracted much attention. This paper first briefly introduces the definition and current status of big data and its privacy issues, describes the privacy protection link model, and then starts from the privacy protection technology of big data, introduces the K-anonymization traditional big data privacy protection technology, advantages and disadvantages, Specific algorithms, etc.

Service Oriented Virtual Machine for Maximising Quality of Service in Wireless Networks

A traditional big data network is a wireless multimedia network, as multiple video/audio streaming account for 70 percent of mobile traffic and the fifth generation wireless networks are projected to rise 500 times. However, it does raise various new and transparent challenges for multimedia big data communication over 5G wireless systems because multimedia big data services provide timely, throughput broadcasts through time-sensitive communication networks with restricted wireless resources. In order to address the above issues, we suggest in this paper information-centric virtualization architectures for a statistical latency consistency of content delivery across 5G wireless large media platforms for technology. In particular, three successful nominee strategies to promise the mathematical time limit for interactive large-data communications are implemented in our proposed scheme: information-centered network which extracts an optimum in-network storage location for multimedia big data; a virtualization of the network feature which transforms PHY architecture into many virtualized channels. In our architectures we build the 3 primary computer-generated machine collection besides power distribution plans to collectively simplify the application of NFV and SDN techniques in the ICN architecture: to improve performance for single users, to integrate effective combining capability with equity allocation for all users besides under non-cooperative betting Via models and computational assesses, we prove that our suggested designs and applications greatly perform better other existing systems to enable QoS provisioning with statistical delays over 5G immersive Large Data Wireless communication.

Application Structure Computer Big Data Technology in Internet Learning

The establishment of online learning platform through the Internet is a more popular way of learning. However, the existing Internet learning platform is still not intelligent enough, especially in data integration and analysis. Therefore, this paper proposes the application and research of computer BD(Big Data) based on structure in Internet learning. In this paper, the structure of computer BD and Internet learning methods for in-depth research and analysis, the existing problems in the Internet learning, combined with structural computer BDAT(Big Data Analysis Technology), established a new Internet learning BD analysis method. In view of the complex source of Internet learning data and large amount of invalid information, this paper optimizes and improves the traditional BD analysis data preprocessing method according to the characteristics of Internet students’ learning. Through the optimization and improvement of the method in this paper, the data analysis ability of the system is improved. In order to further verify the practical application effect of structural computer BD in Internet learning, this paper takes the freshmen of computer major of grade 2018 in a university as the research object of this experiment and carries out relevant investigation and research. According to the survey, 90.3% of students think that BD analysis can improve their mastery of knowledge in online learning. The analysis shows that the structure-based computer BD plays a good auxiliary role in students’ Internet learning.