Irrelevant Pages Research Articles

Mining relevant solutions for programming tasks from search engine results

AbstractOfficial documentation of software development technologies, for example, APIs, may not be sufficient for all developer needs, so searching on the Internet is a usual practice. Nonetheless, finding useful information may be challenging because the best solutions are not always among the first ranked pages. Developers need to read and discard irrelevant pages, that is, those without code examples or those that have content with little focus on the desired solution. This work aims at proposing an approach to mine relevant solutions for programming tasks from search engine results by removing irrelevant pages. The authors evaluated the top‐20 pages returned by the Google search engine, for 10 different queries, and observed that only 31% of the evaluated pages are relevant to developers. Then, the authors proposed and evaluated three different approaches to mine the relevant pages returned by the search engine. Google's search engine has been used as a baseline, and authors’ results have shown that it returns a reasonable number of irrelevant pages for developers, and the authors could establish an effective approach to remove irrelevant pages, suggesting that developers could benefit from a customised web search filter for development content.

Automatic Web Page Classification System with Improved Accuracy

Nowadays, the Internet contain s a wide variety of online documents, making finding useful information about a given subject impossible, as well as retrieving irrelevant pages. Web document and page recognition software is useful in a variety of fields, including news, medicine, and fitness, research, and information technology. To enhance search capability, a large number of web page classification methods have been proposed, especially for news web pages. Furthermore existing classification approaches seek to distinguish news web pages while still reducing the high dimensionality of features derived from these pages. Due to the lack of automated classification methods, this paper focuses on the classification of news web pages based on their scarcity and importance. This work will establish different models for the identification and classification of the web pages. The data sets used in this paper were collected from popular news websites. In the research work we have used BBC dataset that has five predefined categories. Initially the input source can be preprocessed and the errors can be eliminated. Then the features can be extracted depend upon the web page reviews using Term frequency-inverse document frequency vectorization. In the work 2225 documents are represented with the 15286 features, which represents the tf-idf score for different unigrams and bigrams. This type of the representation is not only used for classification task also helpful to analyze the dataset. Feature selection is done by using the chi-squared test which will be in the task of finding the terms that are most correlated with each of the categories. Then the pointed features can be selected using chi-squared test. Finally depend upon the classifier the web page can be classified. The results showed that list has obtained the highest percentage, which reflect its effectiveness on the classification of web pages.

An Evolutionary-based Random Weight Networks with Taguchi Method for Arabic Web Pages Classification

Nowadays, a huge number of web documents are available on the Internet, which makes the retrieval process of a specific topic very difficult, where some irrelevant pages may be retrieved as well. The automatic classification of web documents and pages has an essential application in different domains such as medicine, health, science, and information technology. A large number of web pages classification methods have been proposed to improve the search capabilities, especially in English language. In addition, the current classification methods attempt to classify the English web pages, and at the same time to reduce the high dimensionality of features extracted from these web pages. Due to the lack of classification methods for other languages, this paper focuses on Arabic web pages classification according to its scarcity as well as the importance of the Arabic language. In particular, we propose an evolutionary model based on binary particle swarm optimization (BPSO) combined with random weight networks (RWNs) as an induction algorithm to reduce the high dimensionality of features in the Arabic web pages and to perform document classification automatically. The datasets used in this paper were collected from popular Arabic websites. We collected three different datasets relating to three different fields, namely Computer Science, Science, and Health. Further, Taguchi method is incorporated to locate the best parameters of the proposed algorithm. The experimental results showed that the proposed model gives better performance results for Arabic web pages classification. In addition, an analysis study was conducted to identify the most important features learned from the proposed model as well as the most important tags. The results showed that list tag has obtained the highest percentage, which reflect its effectiveness on the classification of Arabic web pages.

Developing an automated mechanism to identify medical articles from wikipedia for knowledge extraction

Wikipedia contains rich biomedical information that can support medical informatics studies and applications. Identifying the subset of medical articles of Wikipedia has many benefits, such as facilitating medical knowledge extraction, serving as a corpus for language modeling, or simply making the size of data easy to work with. However, due to the extremely low prevalence of medical articles in the entire Wikipedia, articles identified by generic text classifiers would be bloated by irrelevant pages. To control the false discovery rate while maintaining a high recall, we developed a mechanism that leverages the rich page elements and the connected nature of Wikipedia and uses a crawling classification strategy to achieve accurate classification. Structured assertional knowledge in Infoboxes and Wikidata items associated with the identified medical articles were also extracted. This automatic mechanism is aimed to run periodically to update the results and share them with the informatics community.

Identify the Relevant Pages of Book to be Indexed Using Naive Bayes Classification Method

The back-of-book index is a component that is often found in non-fiction books. The back-of-book index contains important terms and page numbers where the terms appear in the book, it is useful for helping readers find certain term directly, without having to search on every page. Usually, the back-of-book index is compiled by the author or a professional indexer, although currently there are several automatic indexing applications available. To determine a term on a page worth to be indexed requires knowledge and expertise of the author whom better understand the book’s context. Hence, generating a good back-of-book index is a task that requires great effort, knowledge and cost. Therefore, the back-of-book index is very possible containing indexed terms that refer to irrelevant page numbers due to human error or the weakness of the indexing application. This study aims to identify the relevance of the pages to be indexed in the back-of-book index, using Naive Bayes Classification. The testing result shows that the approach with Naive Bayes Classification produces an average precision value of 74.02 percent and 100 percent for the recall value. The average precision value that more than 50 percent indicates that the naive bayes classification approach is capable to identify the relevant and irrelevant pages.

Improved Memetic Algorithm Enabled Intelligent Multi Agent (IMAEIMA) System for Web Mining

Wider web space, the searching of a relevant data is the most curious problem for the common people accessing the web. For retreving the relevant information the user request is given to search engine. The relevant pages combined with irrelevant pages are returned to the user. The proposed work emphasizes an Improved Memetic Algorithm Enabled Intelligent Multi Agent (IMAEIMA) for searching the most appropriate pages when submitting complex queries. Improved Memetic algorithm is the traditional genetic algorithm combined with local search and random selection. In this proposed system Improved Memetic algorithm additionally enhanced with logarithmic weight function for more accuracy. Intelligent Agents are introduced in this IMAEIMA to improve its performance and accuracy by reacting intelligently based on feedback and previous experience. This system helps to retrieve relevant pages from web databases with high precision and recall. The derived architecture reveals greater precision and recall overriding the conventional search algorithms.

Role of navigational ability in website visit duration

PurposeOnline search effort is routinely measured by the duration of visit at the website as obtained from clicksream data or surveys. Measuring search effort by the time spent at a website assumes that all consumers who search for the same duration obtain the same amount of information. This would be acceptable if all consumers possessed the same navigational ability. In practice, different consumers have different levels of ability to navigate a website. The purpose of this study is to find whether an individual’s navigational ability has an influence on visit duration and purchase likelihood.Design/methodology/approachThe authors use visit duration data from a real website which makes it possible to partition the visit duration into the times spent on relevant and irrelevant pages. The data were collected through an experimental study. The authors develop an empirical model, comprising hazard and choice models, to assess the relationship between navigational ability and elements of website usage.FindingsA consumer with poor navigational ability spends more time searching on the Web and has lower purchase probability compared to a consumer with superior ability.Research limitations/implicationsThe study is limited to one data.Practical implicationsThis research has managerial implications for website design, such as link-structure, appearance, size and the number of graphics.Originality/valueThis is the first study to research navigational ability’s influence on online consumer behavior.

Search Result Enhancement For Arabic Datasets Using Modified Chicken Swarm

The need for information web-searching is needed by many users nowadays. They use the search engines to input their query or question and wait for the answer or best search results. As results to user query the search engines many times may be return irrelevant pages or not related to information need. This paper presents a proposed model to provide the user with efficient and effective result through search engine, based on modified chicken swarm algorithm and cosine similarity to eliminate and delete irrelevant pages(outliers) from the ranked list results, and to improve the results of the user's query . The proposed model is applied to Arabic dataset and use the ZAD corpus dataset for 27300 document. The experimental result shows that the proposed model improves the precision, recall, and accuracy. Thus the result produced by this method improves accuracy.

Water fluoridation and the quality of information available online

The Internet has transformed the way in which people approach their health care, with online resources becoming a primary source of health information. Little work has assessed the quality of online information regarding community water fluoridation. This study sought to assess the information available to individuals searching online for information, with emphasis on the credibility and quality of websites. We identified the top 10 web pages returned from different search engines, using common fluoridation search terms (identified in Google Trends). Web pages were scored using a credibility, quality and health literacy tool based on Global Advisory Committee on Vaccine Safety (GAVCS) and Center for Disease Control and Prevention (CDC) criteria. Scores were compared according to their fluoridation stance and domain type, then ranked by quality. The functionality of the scoring tool was analysed via a Bland-Altman plot of inter-rater reliability. Five-hundred web pages were returned, of which 55 were scored following removal of duplicates and irrelevant pages. Of these, 28 (51%) were pro-fluoridation, 16 (29%) were neutral and 11 (20%) were anti-fluoridation. Pro, neutral and anti-fluoridation pages scored well against health literacy standards (0.91, 0.90 and 0.81/1 respectively). Neutral and pro-fluoridation web pages showed strong credibility, with mean scores of 0.80 and 0.85 respectively, while anti-fluoridation scored 0.62/1. Most pages scored poorly for content quality, providing a moderate amount of superficial information. Those seeking online information regarding water fluoridation are faced with comprehensible, yet poorly referenced, superficial information. Sites were credible and user friendly; however, our results suggest that online resources need to focus on providing more transparent information with appropriate figures to consolidate the information.

A Novel Approach on Focused Crawling With Anchor Text

A novel approach with focused crawling for various anchor texts is discussed in this paper. Most of the search engines search the web with the anchor text to retrieve the relevant pages and answer the queries given by the users. The crawler usually searches the web pages and filters the unnecessary pages which can be done through focused crawling. A focused crawler generates its boundary to crawl the relevant pages based on the link and ignores the irrelevant pages on the web. In this paper, an effective focused crawling method is implemented to improve the quality of the search. Here, three learning phases are considered namely, content-based, link-based and sibling-based learning are undergone to improve the navigation of the search. In this approach, the crawler crawls through the relevant pages efficiently and more relevant pages are retrieved in an effective way. It is proved experimentally that more number of relevant pages are retrieved for different anchor texts with three learning phases using focused crawling.

Instruction of Digital Reading Strategies Based on Eye-Movements Modeling Examples

During the last decade, several studies have proposed and tested different instructional methods for teaching digital reading strategies to young students. In this study, we have tested the effectiveness of a program combining eye-movements modeling examples (EMMEs) and contrasting cases to instruct ninth-grade students how to plan, evaluate, and monitor their digital reading. EMMEs are videos that display a dot representing the eye movements of a model and an oral transcription of her thoughts while answering a specific question in a hypertext. Students in the EMME condition obtain higher comprehension scores in a posttest performed 1 week after the instruction, as compared with a control group that have received a control instruction using written case examples. Students working with EMMEs also spend more time reading the main digital document, but they do not differ in terms of visits and time to relevant and irrelevant pages. Our study suggests that EMMEs can be used to foster literacy strategy instruction.

Removing non-relevant links from top search results using feature score computation

To promote website in search engine rankings in order to get better visibility and more traffic, search engine optimizers have to follow legal ways but some search engine optimizers manipulate the web pages and boost the irrelevant pages in search results and this leads to practice the problem of web spam. These malicious pages reduce the performance of the search engine. We innovates a method to override the irrelevant search results. In this paper a new approach is designed and developed to move non-deserving pages downward from list of results returned by search engine. The first step was to determine the importance of different page features used for the ranking of a page in search results. Based on this information the devised system identified the features in a page and assigned some weight to each feature. The user query i.e. search keyword, precision and ranking factor are used to calculate precision score of a page. This score is used to predict ranking position for each page. A comparison was made between original ranking positions and predicted ranking position of irrelevant web pages. From analysis of results of Google, accuracy of search results improved from 88% to 99% for the corpus compiled now.

Research on Improved Algorithm of PageRank Based on Vector Space

The traditional PageRank algorithm, which is based on the link analysis algorithm, considers the randomness of user access behavior, but the relevance of the query topic is poor. In order to improve the relevance of the page data search and acquisition, this paper proposes a PageRank algorithm based on the lucent vector space scoring model. The algorithm builds a vector space model based on the web content characteristics, calculates the similarity of the subject content, and combines the original PageRank algorithm, the new PR value is obtained after weighted fusion. Experiments show that the improved algorithm reduces the number of irrelevant pages, and the PR value can better reflect the relevance of the topic.

A Novel Approach on Focused Crawling with Anchor Text

Title: A novel approach with focused crawling for various anchor texts is discussed in this paper. Background: Most of the search engines search the web with the anchor text to retrieve the relevant pages and answer the queries given by the users. The crawler usually searches the web pages and filters the unnecessary pages which can be done through focused crawling. A focused crawler generates its boundary to crawl the relevant pages based on the link and ignores the irrelevant pages on the web. Methods and findings: In this paper, an effective focused crawling method is implemented to improve the quality of the search. Here, three learning phases are considered namely, content-based, link-based and sibling-based learning are undergone to improve the navigation of the search. In this approach, the crawler crawls through the relevant pages efficiently and more relevant pages are retrieved in an effective way. Conclusion: It is proved experimentally that more number of relevant pages are retrieved for different anchor texts with three learning phases using focused crawling.

Survey of Web Crawling Algorithms

The World Wide Web is the largest collection of data today and it continues increasing day by day. A web crawler is a program from the huge downloading of web pages from World Wide Web and this process is called Web crawling. To collect the web pages from www a search engine uses web crawler and the web crawler collects this by web crawling. Due to limitations of network bandwidth, time-consuming and hardware's a Web crawler cannot download all the pages, it is important to select the most important ones as early as possible during the crawling process and avoid downloading and visiting many irrelevant pages. This paper reviews help the researches on web crawling methods used for searching.

¿Acaso la navegación siempre predice el desempeño? Los efectos de la navegación en la lectura digital son moderados por las habilidades de comprensión

<p>El presente estudio investiga los efectos interactivos de la navegación y las habilidades de comprensión en el desempeño de la lectura digital. Se consideraron como indicadores de la navegación la selección de páginas relevantes y la selección de páginas irrelevantes. Se encontró que en 533 secundarias españolas los estudiantes de 11 a 17 años muestran efectos positivos en la lectura de comprensión fuera de línea y la selección de páginas relevantes para el desempeño de la lectura digital, mientras que la selección de páginas irrelevantes tuvo un efecto negativo. Además, se halló una relación entre la selección de páginas relevantes y la habilidad de comprensión fuera de línea. Mientras que el efecto de la selección de páginas relevantes fue bueno en aquellos alumnos que mostraron buenas habilidades de lectura fuera de línea, se redujo en quienes mostraron una comprensión pobre. El efecto de la habilidad de comprensión fuera de línea fue importante en los estudiantes que muestran altos índices de selección de páginas relevantes, mientras que era pobre e insignificante en los estudiantes que mostraron bajos índices de selección de páginas relevantes.</p><p>--</p><p><strong>Este artículo ha sido reproducido de la Revista <em>The International Review of Research in Open and Distributed Learning (IRRODL). </em>Volumen 17, Número 1, Número especial: Documentos de investigación sobre rendimiento y comportamiento del aprendizaje en línea; utilizando una <em>licencia Creative Commons Attribution International 4.0</em>. © 2016. Autores: Johannes Naumann, Ladislao Salmerón.</strong></p>

Does Navigation Always Predict Performance? Effects of Navigation on Digital Reading are Moderated by Comprehension Skills

<p align="left">This study investigated interactive effects of navigation and offline comprehension skill on digital reading performance. As indicators of navigation relevant page selection and irrelevant page selection were considered. In 533 Spanish high school students aged 11-17 positive effects of offline comprehension skill and relevant page selection on digital reading performance were found, while irrelevant page selection had a negative effect. In addition, an interaction between relevant page selection and offline comprehension skill was found. While the effect of relevant page selection was strong in good offline comprehenders, it was significantly reduced in weak offline comprehenders. The effect of offline comprehension skill was strong in students showing high rates of relevant page selection, while it was weak and insignificant in students showing low rates of relevant page selection.</p>

Detecting spam web pages using content and link-based techniques

Web spam is a technique through which the irrelevant pages get higher rank than relevant pages in the search engine’s results. Spam pages are generally insufficient and inappropriate results for user. Many researchers are working in this area to detect the spam pages. However, there is no universal efficient technique developed so far which can detect all spam pages. This paper is an effort in that direction, where we propose a combined approach of content and link-based techniques to identify the spam pages. The content-based approach uses term density and Part of Speech (POS) ratio test and in the link-based approach, we explore the collaborative detection using personalized page ranking to classify the Web page as spam or non-spam. For experimental purpose, WEBSPAM-UK2006 dataset has been used. The results have been compared with some of the existing approaches. A good and promising F-measure of 75.2% demonstrates the applicability and efficiency of our approach.

Spam web page detection using combined content and link features

Web spamming refers to actions that have intentions to mislead search engines by ranking some irrelevant web pages higher in the search results than they deserve. It is thus a roadblock in obtaining high-quality information retrieval from the web. Spam web pages are often littered with irrelevant and meaningless content. Therefore, spam detection methods have been proposed as a solution for web spam in order to minimise the adverse effects of spam web pages. There has been no single defining profile that can encompass all types of spam websites. As such, this makes spam web page detection extremely difficult. In this paper, the proposed technique combines the content and link-based features of web pages to classify them as spam or non-spam. For experimental purpose, WEBSPAM-UK2006 dataset has been used. The results of the proposed approach were compared with the existing approaches and it has been found that the F-measure of the proposed approach outperformed the others.

Improving Researcher Homepage Classification with Unlabeled Data

A classifier that determines if a webpage is relevant to a specified set of topics comprises a key component for focused crawling. Can a classifier that is tuned to perform well on training datasets continue to filter out irrelevant pages in the face of changing content on the Web? We investigate this question in the context of identifying researcher homepages. We show experimentally that classifiers trained on existing datasets of academic homepages underperform on “non-homepages” present on current-day academic websites. As an alternative to obtaining labeled datasets to retrain classifiers for the new content, in this article we ask the following question: “How can we effectively use the unlabeled data readily available from academic websites to improve researcher homepage classification?” We design novel URL-based features and use them in conjunction with content-based features for representing homepages. Within the co-training framework, these sets of features can be treated as complementary views enabling us to effectively use unlabeled data and obtain remarkable improvements in homepage identification on the current-day academic websites. We also propose a novel technique for “learning a conforming pair of classifiers” that mimics co-training. Our algorithm seeks to minimize a loss (objective) function quantifying the difference in predictions from the two views afforded by co-training. We argue that this loss formulation provides insights for understanding co-training and can be used even in the absence of a validation dataset. Our next set of findings pertains to the evaluation of other state-of-the-art techniques for classifying homepages. First, we apply feature selection (FS) and feature hashing (FH) techniques independently and in conjunction with co-training to academic homepages. FS is a well-known technique for removing redundant and unnecessary features from the data representation, whereas FH is a technique that uses hash functions for efficient encoding of features. We show that FS can be effectively combined with co-training to obtain further improvements in identifying homepages. However, using hashed feature representations, a performance degradation is observed possibly due to feature collisions. Finally, we evaluate other semisupervised algorithms for homepage classification. We show that although several algorithms are effective in using information from the unlabeled instances, co-training that explicitly harnesses the feature split in the underlying instances outperforms approaches that combine content and URL features into a single view.