Abstract
E-commerce websites has abundant commercial data. Some very beneficial information to the analysis and prediction of the market can be discovered from these data by applying data mining techniques. The topic-focused web crawler can crawl and gather the subject-related web pages as soon as possible. This thesis has designed and realized the topic-focused crawler based on Scrapy. It firstly introduces the design idea of the crawler and highlights the functions of Scrapys every part. Then, it uses this topic-focused crawler to realize the capture of information from the C2C e-commerce platform, for example TaoBao. At last, it obtains the running result and comparisons of crawling performance between Scrapy based crawler and general crawler.
Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
