Web Structure Mining Research Articles

DATA AND WEB MINING

Data and web mining involves the use of data mining techniques to extract information from the web in order to gain insights into online behavior, customer preferences, and trends. The process requires data preprocessing to clean and prepare the data for analysis, and pattern recognition to identify relationships and patterns within the data. The benefits of data and web mining include improved decision-making and business growth, while challenges include the large volume of data and ensuring data quality. Web mining can be divided into three main categories: web content mining, web structure mining, and web usage mining. Web content mining involves extracting information from the content of web pages, such as text, images, and videos. Web structure mining involves analyzing the links between web pages to identify patterns and relationships. Web usage mining involves analyzing user behavior on the web, such as clickstream data and session logs, to extract useful information. KEYWORDS: data mining, web mining, information extraction, data preprocessing, pattern recognition, online behavior, customer preferences,

IMPLEMENTATION AND RESEARCH OF INTELLIGENT INFORMATION RETRIEVAL AND DATA ANALYSIS IN WEB APPLICATIONS

The main stages of Web-Mining were described, the main difference between information extraction and information retrieval was highlighted in the article. The contents of Web Content Mining (extraction of Web content), Web Structure Mining (extraction of Web structures), and Web Usage Mining (analysis of the use of Web resources) have been clarified separately. In the section entitled “Information systems for Data Mining and processes in Web Applications”, it was shown that when using Web Mining, developers face several types of challenges. Also, the main approaches to conducting intelligent search of information and data analysis in web applications were discussed in the article. The components of the Data Mining model and the stages of its application was considered. As a practical example of the implementation of a software product for research, Russian development - the program PolyAnalyst was considered. It was also noted that the professional version of the system - PolyAnalyst Pro allows us to analyze various data formats and optimize business processes. The system performs a full range of data analysis tasks: loading, combining, cleaning and transforming data, deep text analysis, extracting information, visualizing results and creating reports. In the conclusion part it was noted that to improve the efficiency of work in real time, the creation of new software tools using Web Mining can bring maximum benefits. Building such intelligent analyzers can help analyze log files efficiently. Keywords: Web Mining, Data Mining, Web applications, information retrieval, Web resources.

Automatic Clustering of User Behaviour Profiles for Web Recommendation System

Web usage mining, content mining, and structure mining comprise the web mining process. Web-Page Recommendation (WPR) development by incorporating Data Mining Techniques (DMT) did not include end-users with improved performance in the obtained filtering results. The cluster user profile-based clustering process is delayed when it has a low precision rate. Markov Chain Monte Carlo-Dynamic Clustering (MC2-DC) is based on the User Behavior Profile (UBP) model group’s similar user behavior on a dynamic update of UBP. The Reversible-Jump Concept (RJC) reviews the history with updated UBP and moves to appropriate clusters. Hamilton’s Filtering Framework (HFF) is designed to filter user data based on personalised information on automatically updated UBP through the Search Engine (SE). The Hamilton Filtered Regime Switching User Query Probability (HFRSUQP) works forward the updated UBP for easy and accurate filtering of users’ interests and improves WPR. A Probabilistic User Result Feature Ranking based on Gaussian Distribution (PURFR-GD) has been developed to user rank results in a web mining process. PURFR-GD decreases the delay time in the end-to-end workflow for SE personalization in various methods by using the Gaussian Distribution Function (GDF). The theoretical analysis and experiment results of the proposed MC2-DC method automatically increase the updated UBP accuracy by 18.78%. HFRSUQP enabled extensive Maximize Log-Likelihood (ML-L) increases to 15.28% of User Personalized Information Search Retrieval Rate (UPISRT). For feature ranking, the PURFR-GD model defines higher Classification Accuracy (CA) and Precision Ratio (PR) while utilising minimum Execution Time (ET). Furthermore, UPISRT's ranking performance has improved by 20%.

Web Page Ranking Using Web Mining Techniques: A Comprehensive Survey

Due to the exponential growth of Internet users and traffic, information seekers depend highly on search engines to extract relevant information. Due to the accessibility of a large amount of textual, audio, video etc., contents, the responsibility of search engines has increased. The search engine provides relevant information to Internet users concerning their query, based on content, link structure, etc. However, it does not guarantee the correctness of the information. The performance of a search engine is highly dependent upon the ranking module. The performance of the ranking module is dependent upon the link structure of web pages, which analyze through Web structure mining (WSM) and their content, which analyzes through Web content mining (WCM). Web mining plays a vital role in computing the rank of web pages. This article presents web mining types, techniques, tools, algorithms, and their challenges. Further, it provides a critical comprehensive survey for the researchers by presenting different features of web pages, which are essential to check their quality. In this work, authors presented different approaches/techniques, algorithms and evaluation approaches in previous researches and identified some critical issues in page ranking and web mining, which provide future directions for the researchers working in the area.

A Semantically Enhanced Deep Neural Network Framework for Reputation System in Web Mining for Covid-19 Twitter Dataset

With the web containing a huge amount of information, the extraction of application oriented understandable data has become easier with web mining. Web Mining is the area that is derived from data mining. Unlike data mining, web mining is used to extract interesting patterns from the information available on the web. When used with deep learning, the pattern recognition becomes much easier. Deep learning works in the same way, how a human brain works in terms of predicting the outcomes when a bulk of information is provided. It deals with mathematical models that recognize the patterns efficiently. The different types of web mining techniques, namely: web content mining (WCM), web structure mining (WSM) and web usage mining (WUM) persists. Researchers and economists around the globe are keen in knowing the impact of the pandemic on the society’s economic status; this work helps find the same using reputation system. As twitter is a hub of different opinions of public, we work with covid- 19 data set from twitter. A reputation system helps finding the socio economic status of the tweets regarding covid-19 dataset. This paper has proposed a framework in which the web mining is implemented using a semantic enhanced deep neural network technique for the reputation system.

Web Pages Ranking Algorithms: A Survey

Due to the daily expansion of the web, the amount of information has increased significantly. Thus, the need for retrieving relevant information has also increased. In order to explore the internet, users depend on various search engines. Search engines face a significant challenge in returning the most relevant results for a user's query. The search engine's performance is determined by the algorithm used to rank web pages, which prioritizes the pages with the most relevancy to appear at the top of the result page. In this paper, various web page ranking algorithms such as Page Rank, Time Rank, EigenRumor, Distance Rank, SimRank, etc. are analyzed and compared based on some parameters, including the mining technique to which the algorithm belongs (for instance, Web Content Mining, Web Structure Mining, and Web Usage Mining), the methodology used for ranking web pages, time complexity (amount of time to run an algorithm), input parameters (parameters utilized in the ranking process such as InLink, OutLink, Tag name, Keyword, etc.), and the result relevancy to the user query.

Exploring the Potentialities of Automatic Extraction of University Webometric Information

Abstract Purpose The main objective of this work is to show the potentialities of recently developed approaches for automatic knowledge extraction directly from the universities’ websites. The information automatically extracted can be potentially updated with a frequency higher than once per year, and be safe from manipulations or misinterpretations. Moreover, this approach allows us flexibility in collecting indicators about the efficiency of universities’ websites and their effectiveness in disseminating key contents. These new indicators can complement traditional indicators of scientific research (e.g. number of articles and number of citations) and teaching (e.g. number of students and graduates) by introducing further dimensions to allow new insights for “profiling” the analyzed universities. Design/methodology/approach Webometrics relies on web mining methods and techniques to perform quantitative analyses of the web. This study implements an advanced application of the webometric approach, exploiting all the three categories of web mining: web content mining; web structure mining; web usage mining. The information to compute our indicators has been extracted from the universities’ websites by using web scraping and text mining techniques. The scraped information has been stored in a NoSQL DB according to a semi-structured form to allow for retrieving information efficiently by text mining techniques. This provides increased flexibility in the design of new indicators, opening the door to new types of analyses. Some data have also been collected by means of batch interrogations of search engines (Bing, www.bing.com) or from a leading provider of Web analytics (SimilarWeb, http://www.similarweb.com). The information extracted from the Web has been combined with the University structural information taken from the European Tertiary Education Register (https://eter.joanneum.at/#/home), a database collecting information on Higher Education Institutions (HEIs) at European level. All the above was used to perform a clusterization of 79 Italian universities based on structural and digital indicators. Findings The main findings of this study concern the evaluation of the potential in digitalization of universities, in particular by presenting techniques for the automatic extraction of information from the web to build indicators of quality and impact of universities’ websites. These indicators can complement traditional indicators and can be used to identify groups of universities with common features using clustering techniques working with the above indicators. Research limitations The results reported in this study refers to Italian universities only, but the approach could be extended to other university systems abroad. Practical implications The approach proposed in this study and its illustration on Italian universities show the usefulness of recently introduced automatic data extraction and web scraping approaches and its practical relevance for characterizing and profiling the activities of universities on the basis of their websites. The approach could be applied to other university systems. Originality/value This work applies for the first time to university websites some recently introduced techniques for automatic knowledge extraction based on web scraping, optical character recognition and nontrivial text mining operations (Bruni & Bianchi, 2020).

Performance Comparison of Data Mining Classifiers on Web Log Data

Web usage mining is the branch of web mining that deals with mining of data over the web. Web mining can be categorized as web content mining, web structure mining, web usage mining. In this paper, we have summarized the web usage mining results executed over the user tool WMOT (web mining optimized tool) based on the WEKA tool that has been used to apply various classification algorithms such as Naïve Bayes, KNN, SVM and tree based algorithms. Authors summarized the results of classification algorithms on WMOT tool and compared the results on the basis of classified instances and identify the algorithms that gives better instances accuracy.

Extraction of Knowledge from Web Server Logs Using Web Usage Mining

Information on internet increases rapidly from day to day and the usage of the web also increases, thus there is the need to discover interesting patterns from web. The process used to extract and mine useful information from web documents by using Data Mining Techniques is called Web Mining. Web Mining is broadly classified in to three types namely Web Content Mining, Web Structure Mining and Web Usage Mining. In this paper our focus is mainly on Web Usage Mining, where we are applying the data mining techniques to analyse and discover interesting knowledge from the Web Usage data. The activities of the user are captured and stored at different levels such as server level, proxy level and user level called as Web Usage Data and the usage data stored at server side is Web Server Log, where it records the browsing behavior of users and their requests based on the user clicks. Web server Log is a primary source to perform Web Usage Mining. This paper also brings in to discussion of various existing pre-processing techniques and analysis of web log files and how clustering is applied to group the users based on the browsing behavior of users on their interested contents.

Improved Weighted Page Ranking Algorithm Based on Principal Component Analysis and Map Reduce Frame work for Web Access

In general the World Wide Web become the most useful information resource used for information retrievals and knowledge discoveries. But the Information on Web to be expand in size and density. The retrieval of the required information on the web is efficiently and effectively to be challenge one. For the tremendous growth of the web has created challenges for the search engine technology. Web mining is an area in which applies data mining techniques to deal the requirements. The following are the popular Web Mining algorithms, such as PageRanking (PR), Weighted PageRanking (WPR) and Hyperlink-Induced Topic Search (HITS), are quite commonly used algorithm to sort out and rank the search results. In among the page ranking algorithm uses web structure mining and web content mining to estimate the relevancy of a web site and not to deal the scalability problem and also visits of inlinks and outlinks of the pages. In recent days to access fast and efficient page ranking algorithm for webpage retrieval remains as a challenging. This paper proposed a new improved WPR algorithm which uses a Principal Component Analysis technique called (PWPR) based on mean value of page ranks. The proposed PWPR algorithm takes into account the importance of both the number of visits of inlinks and outlinks of the pages and distributes rank scores based on the popularity of the pages. The weight values of the pages is computed from the inlinks and outlinks with their mean values. But in PWPR method new data and updates are constantly arriving, the results of data mining applications become stale and obsolete over time. To solve this problem is a MapReduce (MR) framework is promising approach to refreshing mining results for mining big data .The proposed MR algorithm reduces the time complexity of the PWPR algorithm by reducing the number of iterations to reach a convergence point.

A Novel Approach of Development of Web Pattern by Focusing on Web Structure Mining Techniques

The World Wide Web is a very useful and interactive resource of information like hypertext, multimedia etc. When we search any information on the Google, there are many URL’s has been opened. The bulk amount of information becomes very difficult for the users to find, extract and filter the relevant information, so that some techniques are used to solve these problems. The objective of current manuscript is focus on processing of structured and unstructured data mining. With the tremendous growth in website, web portal to provide downloaded data to the user. The semantic web is about machine-understandable web pages to make the web more intelligent and able to provide useful services to the users. The data structure definition and recognition is to estimate the accurate page ranking and to produce better result while searching operation with web data.

Web Site Ranking Feedback System

We all search on google for something and get the results in the form of different websites with some description. We generally click on first or second website links if results are not found we go down on google page. The website ranking is given by search engines by different criteria. Last website won’t be seen by none of the people in most cases and first website will be having a great market compared to last website. So we need to help the last website to be moved up in results and help in generating revenue and have good rank on searching by giving feedback. This system will provide the difference between the first website and last website on the google results and will provide the feedback to the last website like content, links, images used by first website which helps the last website to be used in his webpage. This system is user friendly which is built on HTML as front end and Python flask as back end and used python package beautiful soup to parse HTML data and to automate browser behaviour with python. This system is done on web mining which has three categories firstly web content mining in which we scan the web pages and get to know the links, text, images used. Secondly web usage mining in which reports are generated after analysis which contain the details of text, images, links. Finally the web structure mining states that structural summary of website

Website Structure Improvement Based on the Combination of Selected Web Structure and Web Usage Mining Methods

The different web mining methods and techniques can help to solve some typical issues of the contemporary websites, contribute to more effective personalization, improve a website structure and reorganize its web pages. However, only several papers tried to combine web structure and web usage mining (WUM) methods with this aim. The paper researches if and how the combination of selected web structure and WUM methods can identify misplaced web pages and how they can contribute to improving the website structure. The paper analyzes the relationship between the estimated importance of the web page from the web page creator’s point of view using the web structure mining method based on PageRank and visitors’ real perception of the importance of that individual web page using the WUM method based on sequence patterns analysis, which eliminates the problem with repeated visits of the same web page during one session. The results prove that the expected probability of accesses to the individual web page correlates with the observed visit rate obtained from the log files using the WUM method. Furthermore, the website can be improved based on the consequent application of the residual analysis on the obtained results. The applicability of the proposed combination of the web structure and WUM methods is presented on two case studies from different application domains of the contemporary web. As a result, the web pages, which are underestimated or overestimated by the web page creators, are successfully identified in both cases.

Analysis Web Mining Technology to Predict the User Behavior in Multi-Media

The fast growth of the Internet, we have entered a period of information explosion, there is a considerable measure of repetitive data in the Network. How to extract a useful part of this information from the massive information resources, dissecting the huge measure of data lastly get the potential information we need to extracted. Web mining innovation appeared and spared out the human from the data sea. This paper will break down the acknowledgment of Web content mining and Web structure mining, their essential algorithms standards, and their application areas.

Optimizing webpage relevancy using page ranking and content based ranking

Systems for web information mining can be isolated into a few classifications as indicated by a sort of mined data and objectives that specif-ic classifications set: Web structure mining, Web utilization mining, and Web Content Mining. This paper proposes another Web Content Mining system for page significance positioning taking into account the page content investigation. The strategy, we call it Page Content Rank (PCR) in the paper, consolidates various heuristics that appear to be critical for breaking down the substance of Web pages. The page significance is resolved on the base of the significance of terms which the page contains. The significance of a term is determined concern-ing a given inquiry q and it depends on its measurable and linguistic elements. As a source set of pages for mining we utilize an arrangement of pages reacted by a web search tool to the question q. PCR utilizes a neural system as its inward order structure. We depict a usage of the proposed strategy and an examination of its outcomes with the other existing characterization framework –page rank algorithm.

Clickstream data mining and market segmentation

Clickstream data is one of the most important sources of information in websites USAge and customers' behavior in Banks e-services. A number of web USAge mining scenarios are possible depending on the available information. While simple traffic analysis based on clickstream data may easily be performed to improve the e-banks services. The banks need data mining techniques to substantially improve Banks e-services activities. The relationships between data mining techniques and the Web USAge mining are studied. Web structure mining has three types of these types are web USAge structure, mining data streams, and web content. The integration between the Web USAge mining and data mining techniques are presented for processes at different stages, including the pattern discovery phases, and introduces banks cases, that have analytical mining technique. A general framework for fully integrating domain Web USAge mining and data mining techniques are represented for processes at different stages. Data Mining techniques can be very helpful to the banks for better performance, acquiring new customers, fraud detection in real time, providing segment based products, and analysis of the customers purchase patterns over time. And in Market Segmentation The importance of data mining techniques for market segmentation is becoming indispensable in the field of marketing research. This is the first identified academic literature review of the available data mining techniques related to market segmentation. This research paper provides surveys of the available literature on data mining techniques in market segmentation. Eight online journal databases were used for searching, and finally, 103 articles were selected and categorized into 13 groups based on data mining techniques. The utility of data mining techniques and suggestions are also discussed. The findings of this study show that neural networks is the most used method, and kernel-based method is the most promising data mining techniques. Our research work provides a comprehensive understanding of past, present as well as future research trend on data mining techniques in market segmentation.

A Survey on Web Mining Techniques and Applications

Information on Internet and especially on Web sites increasing rapidly day by day, Web sites play an important role in this manner where a lot of Web users are always upload, download and brows a lot of contents based on their needs. Web server provide a way to browse these contents that scattered in many Web sites that reside in it by assigning an Internet Protocol (IP) address or Domain Name System (DNS) to be accessed around the world. Server log the requests that made by users to access their interesting content then store this information in the form of log file. Log file size can be growth from some kilobytes to several megabytes in few days depending on data traffic and the popularity of Web sites. With the fast growth of the data and information in Web environment made a necessity to use sophisticated techniques that have never used in other domains to extract knowledge and significant Web patterns. Web Mining is an extension of Data Mining that integrated various technology in research fields including Artificial Intelligence (AI), statistics, informatics, knowledge discovery and computational linguistics. The aim of Web Mining is to provide a algorithm or technique to make data accesses more efficient and convenient. Web Mining techniques are categorized into three classes depend on which part to be mined which are: Web Content Mining (WCM), Web Structure Mining (WSM) and Web Usage Mining (WSM), in this paper a survey of Web Mining techniques and application are discussed briefly.

An I/O-Efficient Buffer Batch Replacement Policy for Update-Intensive Graph Databases

With the proliferation of graph-based applications, such as social network management and Web structure mining, update-intensive graph databases have become an important component of today’s data management platforms. Several techniques have been recently proposed to exploit locality on both data organization and computational model in graph databases. However, little investigation has been conducted on buffer management of graph databases. To the best of our knowledge, current buffer managers of graph databases suffer performance loss caused by unnecessary random I/O access. To solve this problem, we develop a novel batch replacement policy for buffer management. This policy enables us to maximally exploit sequential I/O to improve the performance of graph database. However, trivial solution produces impractical maintenance for replacement plan with maximal sequential I/O. To enable the policy, we first devise a segment tree-based buffer manager to efficiently maintain a optimal replacement plan. Unfortunately, segment tree-based solution becomes bottleneck in multi-core environment. To remedy this weakness, a B-tree-based buffer manager is further proposed. Extensive experiments on real-world and synthetic datasets demonstrate the superiority of our method.

English

English

Analysis of Web Visit Histories, Part I: Distance-Based Visualization of Sequence Rules

This paper constitutes Part I of the contribution to the analysis of web visit histories through a new methodological framework. Firstly, web usage and web structure mining are considered as an unique mining process to detect the latent structure of the web navigation across the web sections of a single portal. We extend association rules theory to web data defining new concepts of web (patterns) association and preference matrices, as well as of (indirect and direct) sequence rules. We identify the most significant rules, according to a multiple testing procedure. In the literature, web usage patterns can be visualized in no-distance-based graphs describing the navigation behavior across web pages with sequential arrows. In the following, we introduce a geometrical visualization of sequence rules at any click of the web navigation. In particular, we provide two distance-based visualization methods for the static analysis of all data tout court and the dynamic analysis to discover the most significant web paths click by click. A real world case study is considered throughout the methodological description.