Web Page Classification Research Articles

Detecting and classifying online dark visual propaganda

The staggering increase in the amount of information on the World Wide Web (referred to as Web) has made Web page classification essential to retrieve useful information while filtering out unwanted, futile, or harmful contents. This massive information-sharing platform is occasionally abused for propagating extreme and radical ideologies and posing threats to national security and citizens. Detecting the so called dark material has gained more impetus following the recent outbreak of extremist groups and radical ideologies across the Web. The goal of this project, being the first of its own, is to surveil online social networks (OSN) and Web for real-time detection of visual propaganda by violent extremist organizations (VEOs). This is valuable not only for flagging and removing such content from OSN and Web, but also to provide military insight and narrative context inside VEOs. Visual propaganda by VEOs are not only detected, but also further classified based on the type of VEO and focus or intent of the image into hard propaganda, soft propaganda, symbolic propaganda, landscape, and organizational communications. Over 1.2 million images were automatically collected from suspicious OSN accounts and Web pages over a course of four years. Out of which, 120,000 images were manually classified to provide the training data for a convolutional neural network. An overall generalization accuracy of 97.02% and F1 of 97.89% were achieved for a binary classification or mere detection of visual VEO propaganda and an overall generalization accuracy of 86.08% and F1¯ of 85.76% for an eight-way classification based on the intent of the image.

Advantages of Giraph over Hadoop in Graph Processing

This article presents a comparison of the computing performance of the MapReduce tool Hadoop and Giraph on large-scale graphs. The main ideas of MapReduce and bulk synchronous parallel (BSP) are reviewed as big data computing approaches to highlight their applicability in large-scale graph processing. This paper reviews the execution performance of Hadoop and Giraph on the PageRank algorithm to classify web pages according to their relevance, and on a few other algorithms to find the minimum spanning tree in a graph with the primary goal of finding the most efficient computing approach to work on large-scale graphs. Experimental results show that the use of Giraph for processing large-size graphs reduces the execution time by 25% in comparison with the results obtained using the Hadoop for the same experiments. Giraph represents the optimal option thanks to its in-memory computing approach that avoids secondary memory direct interaction.

Optimized Focused Web Crawler with Natural Language Processing Based Relevance Measure in Bioinformatics Web Sources

Abstract In the fast growing of digital technologies, crawlers and search engines face unpredictable challenges. Focused web-crawlers are essential for mining the boundless data available on the internet. Web-Crawlers face indeterminate latency problem due to differences in their response time. The proposed work attempts to optimize the designing and implementation of Focused Web-Crawlers using Master-Slave architecture for Bioinformatics web sources. Focused Crawlers ideally should crawl only relevant pages, but the relevance of the page can only be estimated after crawling the genomics pages. A solution for predicting the page relevance, which is based on Natural Language Processing, is proposed in the paper. The frequency of the keywords on the top ranked sentences of the page determines the relevance of the pages within genomics sources. The proposed solution uses a TextRank algorithm to rank the sentences, as well as ensuring the correct classification of Bioinformatics web page. Finally, the model is validated by being compared with a breadth first search web-crawler. The comparison shows significant reduction in run time for the same harvest rate.

Application of improved distributed naive Bayesian algorithms in text classification

The naive Bayes classifier is a widely used text classification method that applies statistical theory to text classification. Due to the particularity of the text, related feature items may generate new semantic information, which may be lost when the traditional vector space model represents text. This paper mainly studies the construction and improvement of distributed naive Bayes automatic classification system. The application of Hadoop cloud computing in web page classification is one of the focuses of this article. Firstly, the text classification system and Bayesian classification model are analyzed and discussed, including the representation and extraction of text information, text classification methods and Bayesian text classification methods. Then, in view of the shortcomings of the above-mentioned naive Bayesian text classification method, when training text, we use the mutual information method to check the correlation between the feature sets generated after feature selection, and then combine the features with higher correlation degree appropriately. Through a series of tests, the experimental data show that the improved text classification system can achieve better classification results.

An improved SVM web page classification algorithm

This study explored an indepth support vector machine(SVM) classification algorithm suitable for high-dimension computing. An improved SVM algorithm was proposed aiming at two kinds of classical kernel functions of support vector machine. The global kernel function and the local kernel function were weighted into a new mixed kernel function, and the genetic algorithm was used to optimize the function parameters. The improved algorithm avoided the limitation of a single kernel function, taking into account the generalization and learning abilities of the text classification algorithm. Simulation experiments of the collected web pages confirmed the effectiveness of improved SVM algorithm.

Performance Comparison between Keyword-based and Semantic-based Web Page Classification Systems

Performance Comparison between Keyword-based and Semantic-based Web Page Classification Systems

Survey on Optimization Algorithms Used for Feature Selection Techniques in Web Page Classification

Web page classification refers to one of the significant research are in the web mining domain. Enormous quantity of data existing in the web demands the essential development of various effective and robust techniques to undergo web mining task that involves the process to categorizing the web page based on the data labels. It also includes various other tasks such as web crawling, analysis of web links and contextual advertising process. Existing machine learning and data mining techniques are being efficiently used for various web mining processes which include classification of web pages. Using of multiple classifier techniques are most promising research area while considering machine learning that works on the base of merging various classifiers with difference in base classifier and/or dataset distribution. With this several classification models are constructed that is highly robust in nature. This review paper, comparison has been done between FA, PSO, ACO, GA and IWT, to evaluate best fit algorithm for classifying web pages.

A New Technique of Web Page Classification and Optimization

International Journal of Computer Sciences and Engineering (A UGC Approved and indexed with DOI, ICI and Approved, DPI Digital Library) is one of the leading and growing open access, peer-reviewed, monthly, and scientific research journal for scientists, engineers, research scholars, and academicians, which gains a foothold in Asia and opens to the world, aims to publish original, theoretical and practical advances in Computer Science,Information Technology, Engineering (Software, Mechanical, Civil, Electronics & Electrical), and all interdisciplinary streams of Computing Sciences. It intends to disseminate original, scientific, theoretical or applied research in the field of Computer Sciences and allied fields. It provides a platform for publishing results and research with a strong empirical component. It aims to bridge the significant gap between research and practice by promoting the publication of original, novel, industry-relevant research.

Visual content-based web page categorization with deep transfer learning and metric learning

The growing amounts of online multimedia content challenge the current search, recommendation and information retrieval systems. Information in the form of visual elements is highly valuable in a range of web mining tasks. However, the mining of these resources is a difficult task due to the complexity and variability of images, and the cost of collecting big enough datasets to successfully train accurate deep learning models. This paper proposes a novel framework for the categorization of web pages on the basis of their visual content. This is achieved by exploring the joint application of a transfer learning strategy and metric learning techniques to build a Deep Convolutional Neural Network (DCNN) for feature extraction, even when training data is scarce. The obtained experimental results evidence that the proposed approach outperforms the state-of-the-art handcrafted image descriptors and achieves a high categorization accuracy. In addition, we address the problem of over-time learning, so the proposed framework can learn to identify new web page categories as new labeled images are provided at test time. As a result, prior knowledge of the complete set of possible web categories is not necessary in the initial training phase.

Web Page Classification Using RNN

Web page classification is an information retrieval application that provides useful information that can be a basis for many different application domains. In this study, a deep learning-based system has been developed for the classification of web pages. The meta tag information contained in the web page is used to classify a web page. The meta tags used are title, description and keywords. RNN based deep learning architecture was used during the tests. Transfer learning is the name given to the approach to building a machine learning model with the use of pre-trained parameters to solve a problem. The effect of using transfer learning on the system has also been examined. According to the results obtained, success rate of web page classification system is approximately 85%. It is not observed that transfer learning has significant contribution to the success rates. However, the use of transfer learning has reduced the consumed system resources.

Feature Selection for Partially Labeled Data Based on Neighborhood Granulation Measures

As an effective feature selection technique, rough set theory plays an important part in machine learning. However, it is only applicable to labeled data. In reality, there are massive partially labeled data in machine learning tasks, such as webpage classification, speech recognition, and text categorization. To effectively remove redundant features of partially labeled data, the neighborhood granulation measures based on a neighborhood rough set model are put forward in this paper, which can be used to evaluate the discernibility ability of feature subsets under both information systems and decision systems. Moreover, a new definition of significance is introduced. Based on that, a semisupervised reduction algorithm is presented for the feature selection of partially labeled data. Several datasets are chosen to verify its effectiveness. The comparative experiments show that our proposed method is more effective and applicable to the feature selection of partially labeled data.

Multi-Label Classification using an Ontology

During these last few years, the problem of multi-label classification (ML) has been studied in several domains, such as text categorization. Multi-label classification is a main challenging task because each instance can be assigned to multiple classes simultaneously. This paper studies the problem of Multi-label classification in the context of web pages categorization. The categories are defined in an ontology. Among the weakness of the multi-label classification methods, exist the number of positive and negative examples used to build the training dataset of a specific label. So the challenge comes from the huge number of labels combinations that grows exponentially. In this paper, we present an ontology-based Multi-label classification which exploit dependence between the labels. In addition, our approach uses the ontology to take into account relationships between labels and to give the selection of positive and negative examples in the learning phase. In the prediction phase, if a label is not predicted, the ontology is used to prune the set of descendant labels. The results of experimental evaluation show the effectiveness of our approaches.

The Technique of Multi-aspect Evaluation and Categorization of Malicious Information Objects on the Internet

Under the influence of rapid development in the sphere of information technologies, rises the challenge related to detection of malicious information sources on the Internet. To solve this we can use machine learning methods as one of the most popular and powerful tools designed to identify dependencies between input (observed) data and output (desired) results. This article presents a methodology which is aimed at multi-level processing of input data about malicious information objects on the Internet and providing their multi-aspect assessment and categorization using machine learning methods. The purpose of the investigation is to improve the efficiency of the detecting process of malicious information on the Internet using the examples of Web-pages classification.

Bayesian methods for classification inappropriate web pages

The incursion of the Internet has created new forms of information and communication, but it can also carry great dangers, when its use is related to inappropriate content, such as, access to harmful contents and the rise of new kinds of crimes. In this situation, automatic filtering systems identify improper Internet content. This paper describes the use of an algorithm, to automatically filter out inappropriate Web pages. To accomplish this (automatic filtering) task implementation method TAN (Tree Augmented Naive Bayes) is plasma. Data mining algorithms and computational learning for the extraction process, representation and classification of web pages are implemented.

Indexing Based Feature Selection by Applying Ant Colony Optimization Method for Improving Web Page Classification

In this information age many research work are carried out in web page classification to acquire the relevant and appropriate information. To be more specific, for enhancing the web page classification to obtain the optimized feature sets are chosen by utilizing the evolutionary algorithms. Normally, these algorithms are designed by the heuristic principles stimulated by natural evolution. After analyzing the significance of the various evolutionary algorithms deployed by several researchers in this domain so far, this work also intended to apply them to acquire the best solutions (enhanced features). In general, applying the evolutionary algorithms the fittest genes are generated and determined by the fitness function. Once the fittest genes are decided picking up the fittest individual genomes from a population for taking them to the next generations is the challenging task. In this article a novel approach is proposed to choose the best solutions.

Performance of Using Tag-based Feature Sets in Web Page Classification

As the Web is a large collection of data growing daily, an automatic Web page classification mechanism is needed to effectively reach to useful information. Majority of the Web pages are in the form of HTML documents, therefore the aim of this study is to explore the effect of HTML tags on classification process, and try to determine the most valuable HTML tags for feature extraction of the classification task. To achieve this goal, we employ 13 different datasets, and use 5 popular classifiers that are SVM, naive bayes (NB), kNN, C4.5, and OneR. The statistical analysis shows that, the features extracted by using solely the anchor, or tags can be used as an alternative to the features extracted from the whole Web page. SVM is the best among the classifiers used in this study. Using the HTML tags for feature extraction improves classification accuracy.

Multi-View Missing Data Completion

A growing number of multi-view data arises naturally in many scenarios, including medical diagnosis, webpage classification, and multimedia analysis. A challenge in learning from multi-view data is that not all instances are fully represented in all views, resulting in missing view data. In this paper, we focus on feature-level completion for missing view of multi-view data. Aiming at capturing both semantic complementarity and identical distribution among different views, an Isomorphic Linear Correlation Analysis (ILCA) method is proposed to linearly map multi-view data to a feature-isomorphic subspace through learning a set of excellent isomorphic features, thereby unfolding the shared information from different views. Meanwhile, we assume that missing view obeys normal distribution. Then, the missing view data matrix can be modeled as a low-rank component plus a sparse contribution. Thus, to accomplish missing view completion, an Identical Distribution Pursuit Completion (IDPC) model based on the learned features is proposed, in which the identical distribution constraint of missing view to the other available one in the feature-isomorphic subspace is fully exploited. Comprehensive experiments on several multi-view datasets demonstrate that our proposed framework yields promising results.

The Use of Support Vector Machines When Designing a User-Defined Niche Search Engine

<div>This study presents the construction of a niche search engine, whose search topic domain is to be user-defined.  The specific focus of this study is the investigation of the role that a Support Vector Machine plays when classifying textual data from web pages. Furthermore, the aim is to establish whether this niche search engine can return results that are more relevant to a user than when compared to those returned by a commercial search engine Through the conduction of various experiments across a number of appropriate datasets, the suitability of the SVM to classify web pages has been proven to meet the needs of a niche search engine. A subset of the most useful webpage-specific features has been discovered, with the best performing feature being a web pages’ Text & Title component. The user defined niche search engine was successfully designed and an experiment showed that it returned more relevant results than a commercial search engine.<div> </div></div>

Research on Web Page Classification Method Based on Query Log

Web page classification is an important application in many fields of Internet information retrieval, such as providing directory classification and vertical search. Methods based on query log which is a light weight version of Web page classification can avoid Web content crawling, making it relatively high in efficiency, but the sparsity of user click data makes it difficult to be used directly for constructing a classifier. To solve this problem, we explore the semantic relations among different queries through word embedding, and propose three improved graph structure classification algorithms. To reflect the semantic relevance between queries, we map the user query into the low-dimensional space according to its query vector in the first step. Then, we calculate the uniform resource locator (URL) vector according to the relationship between the query and URL. Finally, we use the improved label propagation algorithm (LPA) and the bipartite graph expansion algorithm to classify the unlabeled Web pages. Experiments show that our methods make about 20% more increase in F1-value than other Web page classification methods based on query log.

Comparative Study of Web Page Classification Approaches

Comparative Study of Web Page Classification Approaches