Data Management Platform of Forest Ecological Station Based on Internet of Things and Big Data Sensor

Abstract

In order to solve the problems of low data storage efficiency and poor retrieval performance in forest ecological station, a method of a forest ecological station data management platform based on Internet of Things and big data sensor is proposed. The framework method designs the prepartition algorithm to ensure the uniform distribution of data in the cluster. According to the characteristics of ecological data, the RowKey is scientifically designed to realize the rapid retrieval of ecological data. The Elasticsearch index fragment placement strategy based on index data and server performance evaluation is designed, and the packaging and merging strategy based on data site and time correlation is proposed to improve the storage efficiency. The results are as follows: when the scale of structured data is 108, the retrieval time of the system is 1.045 s, which is 3.99 times faster than that of the original HBase. When the scale of unstructured data is 107, the packaging small image strategy based on data site and time correlation is 1.15 times higher than that of the sequence file and 1.79 times higher than that of the original HBase. In the case of 104 concurrent users, the number of queries per second after optimization is 1.88 times higher than the original, the throughput per second is 1.74 times higher than that before optimization, and the system response time is 69.5% lower than that before optimization. The results show that the proposed scheme has significantly improved the performance in the aspects of cluster load balancing and massive structured and unstructured data retrieval efficiency and system throughput, and provide the necessary theoretical basis and technical implementation for the storage and management of forest ecological data.