Hadoop Cloud Research Articles

Retraction Note: Application and functional simulation of data mining technology in Hadoop Cloud platform based on improved algorithm

Retraction Note: Application and functional simulation of data mining technology in Hadoop Cloud platform based on improved algorithm

Big data clustering using fuzzy based energy efficient clustering and MobileNet V2

Big data analytics (BDA) is a systematic way to analyze and detect various patterns, relationships, and trends in vast amounts of data. Big data analysis and processing require significant effort, techniques, and equipment. The Hadoop framework software uses the MapReduce approach to do large-scale data analysis using parallel processing in order to generate results as soon as possible. Due to the traditional algorithm’s longer execution time and difficulty in processing big amounts of data, this is one of the main issues. Clusters are highly correlated inside each other but are not highly correlated with one another. The technique of effectively allocating limited resources is known as an optimization algorithm for clustering. For processing large amounts of data with several dimensions, the conventional optimization approach is insufficient. By using a fuzzy method, this can be prevented. In this paper, we proposed Fuzzy based energy efficient clustering approach to enhance the clustering mechanism. In summary, Fuzzy based energy efficient clustering introduces a function that measures the distance between the cluster center and the instance, which aids in improved clustering, and we then present the MobileNet V2 model to improve efficiency and speed up computation. To enhance the method’s performance and reduce its time complexity, the distributed database simulates the shared memory space and parallelizes on the MapReduce framework on the Hadoop cloud computing platform. The proposed approach is evaluated using performance metrics such as Accuracy, Precision, Adjusted Rand Index (ARI), Recall, F1-Score, and Normalized Mutual Information (NMI). The experimental findings indicate that the proposed approach outperforms the existing techniques in terms of clustering accuracy.

Map vector tile construction for arable land spatial connectivity analysis based on the Hadoop cloud platform

The concentrated connection of arable land is one of the important indicators reflecting the quality of cultivated land, and large-scale arable land blocks are more conducive to agricultural mechanization operation, thereby improving the land use efficiency. However, the calculation of farmland connectivity is essentially a large-scale calculation of spatial vector data, especially for the national or global farmland patch data. This article proposes a framework for calculating farmland connectivity based on spatial vector map tiles and parallelizes the algorithm based on the Hadoop cloud platform. The framework is based on the tile pyramid model and uses the Douglas–Peucker algorithm to simplify the data to meet the needs of rapid display of large-scale data under multi-scale. The consistency and integrity of the front display of vector tiles are ensured using the setting tile buffer. Meanwhile, the parallelization of the vector tile construction algorithm is realized based on the MapReduce programming mode. Finally, the effectiveness and usability of this framework were verified through the calculation of patch connectivity on the tillage map. Experiments show that the algorithm can not only meet the rapid construction requirements of large-scale vector tile data but also support the cultivated land spatial connectivity analysis and greatly improve the efficiency of supporting data calculation.

Application of Hadoop cloud platform based on soft computing in financial accounting budget control

Application of Hadoop cloud platform based on soft computing in financial accounting budget control

REVIEW: BIG DATA V’S MODELS, CHALLENGES, HADOOP ECOSYSTEM, ISSUES, USES, BENEFITS AND APPLICATIONS

Big Data, encompassing colossal volumes of archival and consequential information, stands as the greatest invaluable treasure for any establishment, holding the potential to fortify business decisions grounded on factual evidence rather than mere perceptions. The most sophisticated method of defining big data is 56V's, provided in the paper. In this work, we have defined extensive data using the V technique, beginning with the 3V's and moving on to the latest definitions with the 5V's, 7V's, 10V's, and 14V's. Undoubtedly, forthcoming technological and corporate efficiency contests will amalgamate into extensive information exploration. The Hadoop cloud computing platform, which is open source and developed by the Apache Foundation, is also included in this study. It features a distributed system known as HDFS and MapReduce software programming platform. The leading big data technologies include Hadoop, Map Reduce, YARN, Hive, Flume, Apache Spark, and No SQL. The handling of massive data can significantly benefit from these technologies. This document also summarizes these tools' capabilities, advantages, and disadvantages. The concluding section of this research paper also delves into utilizing large-scale data in diverse sectors such as banking, finance, education, healthcare, and agriculture.

RETRACTED ARTICLE: Application and functional simulation of data mining technology in Hadoop cloud platform based on improved algorithm

RETRACTED ARTICLE: Application and functional simulation of data mining technology in Hadoop cloud platform based on improved algorithm

IoT sensor network data processing using the TWLGA Scheduling Algorithm and the Hadoop Cloud Platform

Monitoring environmental conditions can be done effectively with the help of the Internet of Things (IOT) sensor network. Massive data generated by IOT sensor networks presents technological hurdles in terms of storage, processing, and querying. A Hadoop cloud platform is suggested as a fix for the issue. The data processing platform makes it possible for one node's work to be shared with others employing the time and workload genetic algorithm (TWLGA), which lowers the risk of software and hardware compatibility while simultaneously increasing the efficiency of a single node. For the experiment, a Hadoop cluster platform employing the TWLGA scheduling algorithm is built, and its performance is assessed. The outcomes demonstrate that processing huge volumes of data from the IOT sensor network is acceptable for the Hadoop cloud platform .

A Hadoop cloud-based surrogate modelling framework for approximating complex hydrological models

Abstract Hydrological simulation has long been a challenge because of the computationally intensive and expensive nature of complex hydrological models. In this paper, a surrogate modelling (SM) framework is presented based on the Hadoop cloud for approximating complex hydrological models. The substantial model runs required by the design of the experiment (DOE) of SM were solved using the Hadoop cloud. Polynomial chaos expansion (PCE) was fitted and verified using the high-fidelity model DOE and was then used as a case study to investigate the approximation capability in a Soil and Water Assessment Tool (SWAT) surrogate model with regard to the accuracy, fidelity, and efficiency. In experiments, the Hadoop cloud reduced the computation time by approximately 86% when used in a global sensitivity analysis. PCE achieved results equivalent to those of the standard Monte Carlo approach, with a flow variance coefficient of determination of 0.92. Moreover, PCE proved to be as reliable as the Monte Carlo approach but significantly more efficient. The proposed framework greatly decreases the computational costs through cloud computing and surrogate modelling, making it ideal for complex hydrological model simulation and optimization.

Design and Development of a Data Structure Visualisation System Using the Ant Colony Algorithm

Aim: A data structure visualisation system uses object-oriented thinking and COM technology to dynamically and interactively simulate and track data structure algorithms and realize the dynamic synchronisation and visualisation of abstract data structures and algorithms using the data modelling function and self-test function. Background: Teaching data structures and algorithms is difficult because of their abstraction and dynamics; the use of icons in classroom teaching can be partially abstracted into intuition, but analysing the instantaneous dynamic characteristics of the object and the dynamic execution process of the algorithm is difficult. Objective: A data structure visualisation system employs a data modelling function, visual data structures, a user-friendly and flexible interface, and multipath features for multilevel users. Such systems can be designed more effectively by using the ant colony algorithm. Methods: The more the ants pass by a certain path, the higher the concentration of residual pheromone, and the higher the probability of the subsequent ant selecting that path. Therefore, the individuals in an ant colony communicate messages and cooperate with each other for foraging. Results: The resulting speedup ratio indicates that the speedup is smaller when the number of nodes is 100 or more; the acceleration is higher when the node reaches a certain scale, and the speedup ratio does not change considerably. In this study, the traffic simulation software VISSIM was used to generate road network data; the generated traffic data were analysed and used to design a traffic network data structure. Conclusion: The traffic network data model oriented to the ant colony algorithm was established through abstraction. Accordingly, a parallel ant colony algorithm based on cloud computing was implemented. Finally, a Hadoop cloud computing platform was established and used to run and test the parallel ant colony algorithm program; several experiments were conducted, and the experimental results were analysed concurrently.

Research on logistics service recommendation model and application under mobile cloud environment

Today's personal computer applications are mainly cloud computing services, but it is usually inevitable to apply cloud computing to mobile cloud environments. Computing in the mobile cloud environment has the following advantages: more resources, free access to the "cloud" through any device anytime and anywhere, and provides powerful functions for computer storage and computing, while meeting the needs of more convenient software services. It has become the focus of attention in the last two years. Logistics service quality is an important indicator to measure service quality and reflects customer satisfaction with service. This is very important for service users and logistics providers. However, at present, there is a lack of unified judgment standards for the measurement of logistics service quality, and different scoring systems have different points of emphasis, so users can't get effective recommendation when choosing products. Therefore, this paper applies the principle of mobile cloud environment, combines the evaluation of logistics service quality with the big data system, selects the appropriate logistics service according to the needs of users, and recommends it to users, so as to provide more powerful assistance for users to choose products and improve the efficiency of product selection.This article introduces the recommendation model design, analyzes the complete update problem of the traditional recommendation model, proposes the incremental update recommendation model design, and implements it in a distributed manner on the basis of the Hadoop cloud computing platform.

Application of Hadoop-Based Cloud Computing in Teaching Platform Research

This article explains the knowledge properties for characterization and research outcomes. It deals with developing and integrating the Hadoop cloud computing platform. The platform utilized in this paper is a one-piece learning algorithm, a statistical model and a cloud-based selection model. Hadoop supports this model, which is suitable for data size computing. We develop Cloud Computing integrated Autocorrelation Function (CCiACF) computation model based on Hadoop learning and implement an efficient teaching platform research data processing and correlation system. Multiple simulations are conducted on the Hadoop platform under various operating conditions to check the training ability’s exactness and characteristics. The Spark Structure of this research is to successfully and effectively develop computational system performance and improve teaching platform research models using Hadoop based cloud computing. Various experimental studies have been performed, and findings indicate that the method proposed is highly successful in data collection, governance and analysis.

Reflections on the Innovation of University Scientific Research Management in the Era of Big Data

In China, universities are important centers for SR (scientific research) and innovation, and the quality of SR management has a significant impact on university innovation. The informatization of SR management is a critical component of university development in the big data environment. As a result, it is crucial to figure out how to improve SR management. As a result, this paper builds a four-tier B/W/D/C (Browser/Web/Database/Client) university SR management innovation information system based on big data technology and thoroughly examines the system’s hardware and software configuration. The SVM-WNB (Support Vector Machine-Weighted NB) classification algorithm is proposed, and the improved algorithm runs in parallel on the Hadoop cloud computing platform, allowing the algorithm to process large amounts of data efficiently. The optimization strategy proposed in this paper can effectively optimize the execution of scientific big data applications according to a large number of simulation experiments and real-world multidata center environment experiments.

Research on interactive data packet storage algorithm for Hadoop cloud computing platform

Research on interactive data packet storage algorithm for Hadoop cloud computing platform

Realizing dynamic resource orchestration on cloud systems in the cloud-to-edge continuum

Cloud computing has been widely utilized to handle the huge volume of data from many cutting-edge research areas such as Big Data and Internet of Things (IoT). The fast growing of edge devices makes it difficult for cloud systems to process all data and jobs originating from edge devices, which leads to the development of edge computing by completing jobs on edges instead of clouds. Unfortunately, edge devices generally possess only limited computing power. Therefore, jobs demanding heavy computation under strict time constraints could have more difficulties to successfully complete their work on edges than on clouds in the Cloud-to-Edge continuum. If cloud systems could dynamically orchestrate cloud resources to expedite the execution of those jobs, not only their timely execution could be assured, also the loading of edge devices could be reduced. The Apache Hadoop is considered one of the most popular cloud systems in industry and academia. However, it does not support dynamic resource allocation. Previously we proposed and implemented a new model which can dynamically adjust the computing resources assigned to given jobs in the Hadoop cloud system to speed up their execution. Like other computer software, cloud systems completely rely on their underlying operating systems to access hardware components such as CPUs and hard drives. In this paper, we report our efforts to improve our model to collaborate with the Linux operating system to accelerate the execution of jobs with high priority to a greater extent. Compared with what our original model achieved, experiments show that our ameliorated model could further quicken the execution of prioritized jobs in Hadoop by up to around 21%. As a result, jobs from edges that require substantial computing resources promptly could have better chances to get accomplished on cloud systems.

Analysis of English Teaching Mode in Big Data Environment Based on Hadoop Cloud Platform

With the continuous deepening of the application of big data technology, the education field is also paying attention to the impact of big data. “Internet +” education has become a hot topic in the current education field. This article relies on the big data Hadoop cloud platform technology through integration lines Online and offline resources and a variety of teaching techniques and methods are used to analyze the business English teaching model, and use the mixed teaching model to promote the development from extensive to standardization, aiming to enhance the richness, interaction and effectiveness of business English teaching.

Retraction: Analysis of English Teaching Mode in Big Data Environment Based on Hadoop Cloud Platform (J. Phys.: Conf. Ser. 1992 022165)

This article has been retracted by IOP Publishing following an allegation that raises concerns this article may have been created, manipulated, and/or sold by a commercial entity. In addition, IOP Publishing has seen no evidence that reliable peer review was conducted on this article, despite the clear standards expected of and communicated to conference organisers. The authors of the article have been given opportunity to present evidence that they were the original and genuine creators of the work, however at the time of publication of this notice, IOP Publishing has not received any response. IOP Publishing has analysed the article and agrees there are enough indicators to cause serious doubts over the legitimacy of the work and agree this article should be retracted. The authors are encouraged to contact IOP Publishing Limited if they have any comments on this retraction. Retraction published: 9 September 2022

RETRACTED ARTICLE: Urban ecological environment investigation based on a cloud computing platform and optimization of computer neural network algorithm

The rapid development of Internet technology has contributed to the development of the Internet. The rapid expansion of the industrial application scale and the sharp increase of data generated by an industrial application and cloud computing have promoted the birth and development of cloud computing platform technology. Data analysis has become an important support for company decision-making; how to quickly and effectively extract useful information from large data sets is very important. Among them, clustering analysis and computer neural network algorithms are the most important methods of data mining. Due to the limitations of computer performance and programming models, traditional data mining methods cannot meet the efficiency and computing needs of a large amount of information. The development of cloud computing technology has opened up a new research field of cloud mining clustering analysis and neural network algorithm formation. The first research is the deployment of the Hadoop cluster based on Linux, and the novelty lies in the parallelization of the MapReduce cluster algorithm. Because there are many clustering algorithms, this paper starts with the K-means clustering algorithm and optimizes the earlier algorithm according to the MapReduce programming model. Through the text clustering process of wine data set in the UCI database, the parallel algorithm is applied to the Hadoop cloud computing platform. Experiments show that the parallel K-means clustering algorithm in MapReduce can significantly improve the execution speed. This paper applies it to the urban ecological environment survey in China, clarifies the relationship between various factors of urban environmental governance, expands the content of urban environmental governance in China, and improves the ecological environment of Chinese cities.

A Hadoop-Based Online Teaching Model of "VisibleBody"

It is the responsibility of nursing educators to cultivate community nursing professionals who can adapt to the post and meet social needs. However, there exist some problems in current community nursing curriculum education such as inconsistent teaching content and different levels of teachers’ lecturing. In this study, a good educational resource ecosystem was established in virtue of the mighty flexibility and expansibility of hadoop technology, the resources in community nursing teaching courses are stored in the hadoop cloud resource library. At the same time, community nursing online course resources were newly designed and improved through HdoopYarn resource scheduling algorithm, thereby realizing the establishment of a virtual experimental environment for nursing major. VisibleBody virtual medical software was applied to the internship and practical training, specifically in 12 knowledge units of 5 modules according to the characteristics of this course, demonstrating a deep-level human body structure and accurate nursing operation. An objective structured clinical examination mode was adopted in the teaching method, which could create a simulated community nursing situation during teaching, so that students could complete standardized patient assessment and examination within the given time, and accept testers’ questions and acquire results. It was found from the teaching practice that the application of teaching model is conductive to enhancing students' academic performance and improving community nursing capabilities, including problem-solving competence, communication skills, teamwork ability, etc.

Medical Cloud Computing Data Processing to Optimize the Effect of Drugs.

In recent years, cloud computing technology is maturing in the process of growing. Hadoop originated from Apache Nutch and is an open-source cloud computing platform. Moreover, the platform is characterized by large scale, virtualization, strong stability, strong versatility, and support for scalability. It is necessary and far-reaching, based on the characteristics of unstructured medical images, to combine content-based medical image retrieval with the Hadoop cloud platform to conduct research. This study combines the impact mechanism of senile dementia vascular endothelial cells with cloud computing to construct a corresponding data retrieval platform of the cloud computing image set. Moreover, this study uses Hadoop's core framework distributed file system HDFS to upload images, store the images in the HDFS and image feature vectors in HBase, and use MapReduce programming mode to perform parallel retrieval, and each of the nodes cooperates with each other. The results show that the proposed method has certain effects and can be applied to medical research.

Sports Big Data Analysis Based on Cloud Platform and Its Impact on Sports Economy

The service information system is constantly transforming to a networked information model, and domestic hardware equipment is constantly updated. Independent controllability has also become the basic requirement of the new information age. With the development of the information age and the new era of independent control, more and more services and applications will also be deployed on autonomous and controllable cloud platforms. With the rapid development of Internet technology in the information age and the resulting changes in productivity, people can record, store, and transmit more and more information. When information becomes recordable, storage, and easy to transmit, information becomes modern meaning nowadays, an era of information explosion characterized by massive, volatile, timely transmission, and diverse forms has truly come, forming what is now called the “big data era”. This article mainly introduces the analysis of sports big data based on the cloud platform and the research on the impact on the sports economy and intends to provide ideas and directions for the analysis of sports big data and the research on the impact on the sports economy. This paper proposes a cloud platform-based sports big data analysis and research methods for its impact on the sports economy, including the use of Hadoop cloud platform big data processing systems and support vector regression algorithms for cloud platform-based sports big data analysis and sports economy. The experimental results of this paper show that the average correlation between sports big data analysis and sports economic development is 0.5155, and appropriate cloud platform-based sports big data analysis plays a positive role in promoting sports economic development.