SEED SELECTION BASED WEB CRAWLER FOR WEB PAGE CLASSIFICATION: A SURVEY

Abstract

A web search engine is a three-phase method of which the first phase is Web crawling. Web crawler works to collect data for search engines. Web crawler Collects pages through hyperlinks exist on web pages; these pages are elements of Seed URLs set that defined in the initial stage of the web crawling process. The search engine performs the ranking algorithm on a set of web pages that are covered through crawlers. Therefore the result of the search engine is typically based on web crawler coverage. Most research is already done in the web crawler area. Different web crawling strategies, resulting from different ways of ordering the URLs in the frontier, can explore the web in different ways. Researchers have also studied the different issues and challenges that web crawlers faced. As per literature web crawlers performance depends on the Seed URLs. This paper gives a bird eye over Web crawling methods, recent research over it and possible research gap.