Abstract

In recent years, with the development of big data technologies and applications, more and more enterprises realize that big data will play an important role in enterprises, which leads to a significant increase of big data platforms adopted by enterprises. Intelligent operation and maintenance could complete system state evaluation, abnormal diagnosis, alarm and prediction by means of data analysis, so as to reduce the complexity and workload of system operation and maintenance. In order to simplify the operation and maintenance of big data platform, intelligent operation and maintenance of big data platform has become a trend. In this context, the intelligent operation and maintenance system for big data platform is designed and implemented. The system realizes the collection, aggregation and storage, analysis with query and visualization of the runtime data of the big data platform. By providing a series of intelligent operation and maintenance functions, the complexity and workload of operation and maintenance personnel are reduced. In this thesis, the unified scoring model of big data platform is proposed; the HDFS capacity prediction method is proposed to manage the storage capacity of big data platform more actively; the task scheduling method based on platform free time analysis is proposed to optimize the platform efficiency.

Talk to us

Join us for a 30 min session where you can share your feedback and ask us any queries you have

Schedule a call

Disclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.