Research on Collection and Batch Processing Technology of Ethnic Cultural Resources Based on Web Crawler

Abstract

This paper takes China's intangible cultural heritage as the main research object. Through studying the standardized classification system of culture and using web crawler technology, the fragmented intangible cultural heritage is collected and sorted, and finally a large-scale digital cultural sharing platform is built. To promote cultural prosperity and development. In the subject of Technology Integration and Application Demonstration for the Inheritance and Development and Utilization of Ethnic and Folk Cultural Resources, experimental methods were used to build a digital resource platform from three levels of methods, procedures and tools. Through Python crawler technology and lightweight data exchange format JSON, the direct classification and collection of cultural resources is realized. And realized the use of self-developed 51cst tool to directly upload the acquired resources to the database according to their attributes. Through the study of cultural classification system, the 51Cst collection tool developed in combination with related technologies can effectively control the labeling, collection and quantity of data. According to the established social labeling system, the collection, classification and classification of cultural resources based on known classifications can be realized. One-click operation of sorting. At present, the project of the research group has built a national folk cultural resource material pool, which has collected more than 640,000 pieces of opera, dance, folk tales and other resources, providing important support for the development of China's cultural tourism industry.