Abstract

Now the public traffics make the life more and more convenient. The amount of vehicles in large or medium sized cities is also in the rapid growth. In order to take full advantage of social resources and protect the environment, regional end-to-end public transport services are established by analyzing online travel data. The usage of computer programs for processing of the web page is necessary for accessing to a large number of the carpool data.

Highlights

  • Developing public transport is a development strategy of China in the field of urban transportation

  • In order to download these data, search engine programs need to be developed based on web crawlers

  • For webpage mining [10], in this paper explore the role of web crawlers in webpage mining and explore how to construct a theoretical web crawler framework for web mining

Read more

Summary

INTRODUCTION

Developing public transport is a development strategy of China in the field of urban transportation. Many scholars have been exploring the application of data mining in traffic traveling. The second method is to collect the traffic data of cities manually. The third method is to access the online traffic data through the web crawler. The third method is used to design a dedicated web crawler for a particular site to collect data. In order to download these data, search engine programs need to be developed based on web crawlers. The crawler is a search engine component It accesses portions of the Web tree based on certain policies and collects the retrieved objects in the local repository. Through the above three steps of the web crawler, data can be downloaded from some simple web pages. That are more complex or have protective measures, web crawlers need to be designed based on specific structures

Research Status of Web Crawlers
Description of the Problem
Initialization of URL
Page Request
Page Analysis
Page Storage
Theoretical Analysis and Comparison
THE PROGRAM ARCHITECTURE
Experimental Results
Results Analysis
CONCLUSIONS
Full Text
Published version (Free)

Talk to us

Join us for a 30 min session where you can share your feedback and ask us any queries you have

Schedule a call