Web Page Classification Research Articles

Web page Classification by Using PCA and Neural Network

Web page Classification by Using PCA and Neural Network

Classifying Web Pages by Aimed Nation Using Machine Learning

Classifying web pages is to automatically assign predefined class to them. It is one of the main applications of web mining. The authors' aim is to detect the targeted nation based on the web pages content. It is an original application. In this paper, the authors propose different web mining approaches using machine learning algorithms such as Naïve Bayes and Support Vector Machine in order classify them. They present detailed stages of the procedure. The best experimental result based on an original corpus created by their own means shows a very attention grabbing f-score of 85%.

New use of the HITS algorithm for fast web page classification

The immense number of documents published on the web requires the utilization of automatic classifiers that allow organizing and obtaining information from these large resources. Typically, automatic web pages classifiers handle millions of web pages, tens of thousands of features, and hundreds of categories. Most of the classifiers use the vector space model to represent the dataset of web pages. The components of each vector are computed using the term frequency inversed document frequency (TFIDF) scheme. Unfortunately, TFIDF-based classifiers face the problem of the large-scale size of input data that leads to a long processing time and an increase in resource requests. Therefore, there is an increasing demand to alleviate these problems by reducing the size of the input data without influencing the classification results. In this paper, we propose a novel approach that improves web page classifiers by reducing the size of the input data (i.e. web pages and feature reduction) by using the hypertext induced topic search (HITS) algorithm. We employ HITS results for weighting remaining features. We evaluate the performance of the proposed approach by comparing it with the TFIDF-based classifier. We demonstrate that our approach significantly reduces the time needed for classification.

An insight into campus network user behaviour analysis decision system

Through research tools such as data warehouse and data mining in combination with abundant data peculiar to network users, this paper provides an insight into campus network user behaviour analysis and decision systems. Furthermore, the authors attempt to base webpage classification criteria on the open directory project (ODP) with a view to exploring specific patterns and rules found in network user behaviours. On the basis of those pioneering studies, the authors have put forward some ideas, procedures, and a framework concerning behaviour analysis and decision systems.

An insight into campus network user behaviour analysis decision system

Through research tools such as data warehouse and data mining in combination with abundant data peculiar to network users, this paper provides an insight into campus network user behaviour analysis and decision systems. Furthermore, the authors attempt to base webpage classification criteria on the open directory project (ODP) with a view to exploring specific patterns and rules found in network user behaviours. On the basis of those pioneering studies, the authors have put forward some ideas, procedures, and a framework concerning behaviour analysis and decision systems.

A Method of Web Page Classification Based on Feature Dimension Reduction

Text classification technology to quickly retrieve information pages, information filtering and data mining provides an important foundation. Due to the diversity and complexity of web page format, the problem of web page classification is more difficult to deal with than text classification. This paper on web page classification method research, through the improvement of bloom filter, after pretreatment of the text content of pages, the improved bloom filter used in filtering feature set, greatly reducing the feature dimension, then according to the characteristics of web page, the feature weight algorithm has been improved. Finally, using the naive Bayesian classifier can verify the effectiveness of the algorithm.

A semantic based Web page classification strategy using multi-layered domain ontology

World Wide Web is a continuously growing giant, and within the next few years, Web contents will surely increase tremendously. Hence, there is a great requirement to have algorithms that could accurately classify Web pages. Automatic Web page classification is significantly different from traditional text classification because of the presence of additional information, provided by the HTML structure. Recently, several techniques have been arisen from combinations of artificial intelligence and statistical approaches. However, it is not a simple matter to find an optimal classification technique for Web pages. This paper introduces a novel strategy for vertical Web page classification, which is called Classification using Multi-layered Domain Ontology (CMDO). It employs several Web mining techniques, and depends mainly on proposed multi-layered domain ontology. In order to promote the classification accuracy, CMDO implies a distiller to reject pages related to other domains. CMDO also employs a novel classification technique, which is called Graph Based Classification (GBC). The proposed GBC has pioneering features that other techniques do not have, such as outlier rejection and pruning. Experimental results have shown that CMDO outperforms recent techniques as it introduces better precision, recall, and classification accuracy.

Toward automated assessment of health Web page quality using the DISCERN instrument

As the Internet becomes the number one destination for obtaining health-related information, there is an increasing need to identify health Web pages that convey an accurate and current view of medical knowledge. In response, the research community has created multicriteria instruments for reliably assessing online medical information quality. One such instrument is DISCERN, which measures health Web page quality by assessing an array of features. In order to scale up use of the instrument, there is interest in automating the quality evaluation process by building machine learning (ML)-based DISCERN Web page classifiers. The paper addresses 2 key issues that are essential before constructing automated DISCERN classifiers: (1) generation of a robust DISCERN training corpus useful for training classification algorithms, and (2) assessment of the usefulness of the current DISCERN scoring schema as a metric for evaluating the performance of these algorithms. Using DISCERN, 272 Web pages discussing treatment options in breast cancer, arthritis, and depression were evaluated and rated by trained coders. First, different consensus models were compared to obtain a robust aggregated rating among the coders, suitable for a DISCERN ML training corpus. Second, a new DISCERN scoring criterion was proposed (features-based score) as an ML performance metric that is more reflective of the score distribution across different DISCERN quality criteria. First, we found that a probabilistic consensus model applied to the DISCERN instrument was robust against noise (random ratings) and superior to other approaches for building a training corpus. Second, we found that the established DISCERN scoring schema (overall score) is ill-suited to measure ML performance for automated classifiers. Use of a probabilistic consensus model is advantageous for building a training corpus for the DISCERN instrument, and use of a features-based score is an appropriate ML metric for automated DISCERN classifiers. The code for the probabilistic consensus model is available at https://bitbucket.org/A_2/em_dawid/ .

Characteristic Selection with Rough Sets for Web Page Ranking

Objective: The objective is to classify web pages and assign ranking to web pages using feature selection with rough sets and TF_IDF methodology. Proposed Method: Web page ranking is a process to assign position at a particular site appears in the result of web page. A site is said to have a high page ranking when it appear at or near the top of the list of web result. A challenge in web page ranking is to provide relevant information to the user according to query. To finding relevant information from the result set is a tedious process. To obtain a refined result set that contains the URL’s more relevant to the user’s query, so it is essential to rank. For classification purpose, we are using feature reduction method based Rough Set Theory (RST). Application: Feature selection is most essential technique in rough sets as well as the data mining. Attribute selection is a main challenge for expanding the theory and making use of rough set. Findings: The proposed method emphases on the removal of the unnecessary attributes as a way to sort the effective reduct set and framing the core of the attribute set. After successful classification procedure, we have to applying TF_IDF methodology for assign the ranking to the documents.Keywords: Core, Data Preprocessing, Data Mining, Feature Selection, Rough Sets Theory (RST), Reduct, Tf-IDF, Text Mining

Revealing connectivity structural patterns among web objects based on co-clustering of bipartite request dependency graph

Web objects are the entities retrieved from websites by users to compose the web pages. Therefore, exploring the relationships among web objects has theoretical and practical significance for many important applications, such as content recommendation, web page classification, and network security. In this paper, we propose a graph model named Bipartite Request Dependency Graph (BRDG) to investigate the relationships among web objects. To build the BRDG from massive network traffic data, we design and implement a parallel algorithm by leveraging the MapReduce programming model. Based on the study of a number of BRDGs derived from real wireless network traffic datasets, we find that the BRDG is large, sparse and complex, implying that it is very hard to derive the structural characteristics of the BRDG. Towards this end, we propose a co-clustering algorithm to decompose and extract coherent co-clusters from the BRDG. The co-clustering results of the experimental dataset reveal a number of interesting and interpretable connectivity structural patterns among web objects, which are useful for more comprehensive understanding of web page architecture and provide valuable data for e-commerce, social networking, search engine, etc.

An Empirical Comparison Of Feature Selection Methods In Problem Transformation Multi-label Classification

Multi-label classification (MLC) is a supervised learning problem in which a particular example can be associated with a set of labels instead of a single one as in traditional classification. Many real-world applications, such as Web page classification or resource tagging on the Social Web, are challenging for existing MLC algorithms, because the label space grows exponentially as instance space increases. Under the problem transformation approach, the most common alternative for MLC, multi-label problems are transformed into several single label problems, whose outputs are then aggregated into a prediction to the whole classification problem. Feature selection techniques become crucial in large-scale MLC problems to help reducing dimensionality. However, the impact of feature selection in multi-label setting has not been as extensively studied as in the case of single-label data. In this paper, we present an empirical evaluation of feature selection techniques in the context of the three main problem transformation MLC methods: Binary Relevance, Pair-wise and Label power-set. Experimentation was performed across a number of benchmark datasets for multi-label classification exhibiting varied characteristics, which allows observing the behavior of techniques and assessing their impact according to multiple metrics.

An Improved Optimized Web Page Classification using Firefly Algorithm with NB Classifier (WPCNB)

web is a huge repository of information which needs for accurate automated classifiers for Web pages to maintain Web directories and to increase search engines performance. In web page classification problem each term in each HTML/XML tag of each Web page can be taken as a feature, an efficient methods to select best features to reduce feature space of the Web page classification problem derived here. Classification of Web page content is essential to many tasks in Web information retrieval such as maintaining, web directories and focused crawling. The uncontrolled nature of Web content presents additional challenges to Web page classification as compared to traditional text classification, but the interconnected nature of hypertext also provides features that can assist the process. As in derived work reviewed in Web page classification, the importance of these Web-specific features and algorithms, describe state-of-the-art practices, and track the underlying assumptions behind the use of information from neighboring pages. This work, our aimed to optimize best features selection for Web page classification problem. Since Firefly Algorithm (FA) is a recent nature inspired optimization algorithm, that simulates the flash pattern and characteristics of fireflies. Clustering is a popular data analysis technique to identify homogeneous groups of objects based on the values of their attributes. Here FA is used for clustering on benchmark problems which is being found more suitable than Artificial Bee Colony (ABC), Particle Swarm Optimization (PSO), and other nine methods used. The web page optimization using Naive Bayes classifier (WPCNB) is an improved optimized web page classification using firefly algorithm with NB classifier. this work is tested on research banking data set where firefly algorithm used for web optimization and Naive Bayes (NB) classifier used for classification of pages in contrast to selected pages with reference to different fireflies. The entitled work is being found better in terms of feature measure(FM),accuracy, precision etc. parameters with respect to existing key concepts.it is also an search optimization approach and can be enhanced by different genetic algorithm(GA)based classifiers use in future.

Large-scale hierarchical classification with rare categories and inconsistencies

Structuring data is crucial for managing massive amount of available data. Hierarchy (taxonomy) provides a natural and convenient way to organize the information. It has been extensively used in several domains, such as gene taxonomy for organizing gene sequences, international patent hierarchy for easy browsing and retrieval of patent documents, DMOZ taxonomy for web-pages categorization, and ImageNet database for indexing millions of images according to WordNet hierarchy. Given, a hierarchy containing thousands of classes (categories) and millions of instances (examples), there is an essential need to develop an efficient and automated approaches to categorize unlabeled test instances. This problem is referred to as Hierarchical Classification (HC) task. HC is an important machine learning problem that has been researched and explored extensively in the past few years (Silla Jr & Freitas, 2011). The popularity of large-scale HC problem is evident from various HC competitions organized in the past few years such as LSHTC 1 , BioASQ 2 and ILSVRC 3 . HC poses several challenges due to the following reasons: (i) Data imbalance with large number of classes having very few positive examples for training (rare categories), (ii) Multi-label classification, (iii) Feature selection, (iv) Inconsistent hierarchy due to domain experts manual design, and (v) Scalability due to large number of examples, features and classes. Several approaches that address these issues individually (or multiple issues together) have been developed over the years (Gopal & Yang, 2013; Babbar et al., 2013), however there are many possibilities of improving the existing methods. Specifically, we have developed the methods for handling rare categories and inconsistent hierarchy problem.

Big Data, Big Noise

In this article, we focus on noise in the sense of irrelevant information in a data set as a specific methodological challenge of web research in the era of big data. We empirically evaluate several methods for filtering hyperlink networks in order to reconstruct networks that contain only webpages that deal with a particular issue. The test corpus of webpages was collected from hyperlink networks on the issue of food safety in the United States and Germany. We applied three filtering strategies and evaluated their performance to exclude irrelevant content from the networks: keyword filtering, automated document classification with a machine-learning algorithm, and extraction of core networks with network-analytical measures. Keyword filtering and automated classification of webpages were the most effective methods for reducing noise, whereas extracting a core network did not yield satisfying results for this case.

An efficient scheme for automatic web pages categorization using the support vector machine

ABSTRACTIn the past few years, with an evolution of the Internet and related technologies, the number of the Internet users grows exponentially. These users demand access to relevant web pages from the Internet within fraction of seconds. To achieve this goal, there is a requirement of an efficient categorization of web page contents. Manual categorization of these billions of web pages to achieve high accuracy is a challenging task. Most of the existing techniques reported in the literature are semi-automatic. Using these techniques, higher level of accuracy cannot be achieved. To achieve these goals, this paper proposes an automatic web pages categorization into the domain category. The proposed scheme is based on the identification of specific and relevant features of the web pages. In the proposed scheme, first extraction and evaluation of features are done followed by filtering the feature set for categorization of domain web pages. A feature extraction tool based on the HTML document object model of the web page is developed in the proposed scheme. Feature extraction and weight assignment are based on the collection of domain-specific keyword list developed by considering various domain pages. Moreover, the keyword list is reduced on the basis of ids of keywords in keyword list. Also, stemming of keywords and tag text is done to achieve a higher accuracy. An extensive feature set is generated to develop a robust classification technique. The proposed scheme was evaluated using a machine learning method in combination with feature extraction and statistical analysis using support vector machine kernel as the classification tool. The results obtained confirm the effectiveness of the proposed scheme in terms of its accuracy in different categories of web pages.

Enterprise information integration

Integrating a web application into an automated business process requires to design wrappers that get user queries as input and map them onto the search forms that the application provides. Such wrappers build on automatic navigators which are responsible for navigating to the pages that provide the information required to answer the original user queries. A navigator relies on a web page classifier that discerns which pages provide the information and which do not. In the literature, there are many proposals to classify web pages, but none of them fulfills the requirements for a web page classifier in a navigator context. We address the problem of designing an unsupervised web page classifier that builds solely on the information provided by the URLs and does not require extensive crawling of the site being analysed. Our contribution is CALA, a new automated proposal to generate URL-based web page classifiers. Its salient features are that it does not need to previously crawl the complete web site, it is unsupervised, it does not require to download a page before classifying it, and it is computationally tractable. It has been validated by a number of experiments using real-world, top-visited web sites.

Multi-Label Genre Classification of Web Pages Using an Adaptive Centroid-Based Classifier

This paper proposes an adaptive centroid-based classifier (ACC) for multi-label classification of web pages. Using a set of multi-genre training dataset, ACC constructs a centroid for each genre. To deal with the rapid evolution of web genres, ACC implements an adaptive classification method where web pages are classified one by one. For each web page, ACC calculated its similarity with all genre centroids. Based on this similarity, ACC either adjusts the genre centroid by including the new web page or discards it. A web page is a complex object that contains different sections belonging to different genres. To handle this complexity, ACC implements a multi-label classification where a web page can be assigned to multiple genres at the same time. To improve the performance of genre classification, we propose to aggregate the classifications produced using character n-grams extracted from URL, title, headings and anchors. Experiments conducted using a known multi-label dataset show that ACC outperforms many other multi-label classifiers and has the lowest computational complexity.

CALA: ClAssifying Links Automatically based on their URL

Web page classification refers to the problem of automatically assigning a web page to one or more classes after analysing its features. Automated web page classifiers have many applications, and many researchers have proposed techniques and tools to perform web page classification. Unfortunately, the existing tools have a number of drawbacks that makes them unappealing for real-world scenarios, namely: they require a previous extensive crawling, they are supervised, they need to download a page before classifying it, or they are site-, language-, or domain-dependent. In this article, we propose CALA, a tool for URL-based web page classification. The strongest features of our tool are that it does not require a previous extensive crawling to achieve good classification results, it is unsupervised, it is based exclusively on URL features, which means that pages can be classified without downloading them, and it is site-, language-, and domain-independent, which makes it generally applicable. We have validated our tool with 22 real-world web sites from multiple domains and languages, and our conclusion is that CALA is very effective and efficient in practice.

A Segment-based Weighting Technique for URL-based Genre Classification of Web Pages

We propose a segment-based weighting technique for genre classification of web pages. This technique exploits character n-grams extracted from the URL of the web page rather than its textual content. The main idea of our technique is to segment the URL and assigns a weight for each segment. Experiments conducted on three known genre datasets show that our method achieves encouraging results.

An Efficient Multiclass Classifier Using On-Page Positive Personality Features for Web Page Classification for the Next Generation Wireless Communication Networks

Over the years, wireless communication networks have been widely used by a large community of users in wide variety of applications such as intelligent transportation systems, energy management, safety, and security etc. But, during this era, due to large number of user's request, there may be a performance bottleneck in some part of the network with respect to various QoS parameters such as congestion and network delay. Hence, there is a requirement of an efficient classification technique to reduce congestion in the network so that throughput of various applications can be increased. Classification helps in searching, sorting, retrieval, and querying of a document for the wireless networks. World Wide Web (WWW) contains huge repository of information in the form of web pages. However, size of Internet is growing day-by-day. The huge repository of information poses challenge to collect and process the relevant related information of a particular domain. So, traditional text classification techniques are difficult to apply on the rapidly growing web-based contents. Hence, novel approaches and techniques need to be devised to reduce the manual efforts in web page classification. Keeping focus on these points, this paper proposes a novel approach for multiclass classifier based on unique personality features of the web page of particular domain category for the next generation wireless networks. Personality features are collected and assigned weights in the proposed scheme. Then, the proposed classifier is trained based on these special features. Results obtained depict that proposed classifier successfully classified news domain pages, education, resume, online shopping, and research web pages from large database repository. Accuracy of the proposed classifier is found to be satisfactory from a large data set of different categories. Also, there is a 10---15 % overall performance gain using the proposed scheme in comparison to the other existing schemes.