Web Access Logs Research Articles

EWAP-mine: enhanced mining algorithm to mine web access pattern from WAP-tree

As the information available on the World Wide Web is increasing day-by-day, access to the websites is also increasing which results in huge amount of web log data (also called web usage data). Discovery and analysis of useful information from these web logs become a practical necessity. Frequent access pattern, which is the sequence of accesses pursued by users frequently, is one of the interesting and useful knowledge in practice. Web access pattern tree (WAP-tree) mining is a frequent pattern mining technique for web log access sequences, which first stores the original web access sequence database on a prefix tree for storing non-sequential data. WAP-tree algorithm then, mines the frequent sequences from the WAP-tree by recursively reconstructing intermediate trees, starting with suffix sequences and ending with prefix sequences. In this paper, we propose a more efficient algorithm named eWAP-mine (enhanced web access pattern mining algorithm), which is based directly on the initial conditional web access sequence base (1-CWASD) of each frequent event and eliminates the need for reconstructing intermediate conditional WAP-trees.

Finding Related Search Engine Queries by Web Community Based Query Enrichment

The conventional approaches of finding related search engine queries rely on the common terms shared by two queries to measure their relatedness. However, search engine queries are usually short and the term overlap between two queries is very small. Using query terms as a feature space cannot accurately estimate relatedness. Alternative feature spaces are needed to enrich the term based search queries. In this paper, given a search query, first we extract the Web pages accessed by users from Japanese Web access logs which store the users individual and collective behavior. From these accessed Web pages we usually can get two kinds of feature spaces, i.e, content-sensitive (e.g., nouns) and content-ignorant (e.g., URLs), to enrich the expressions of search queries. Then, the relatedness between search queries can be estimated on their enriched expressions. Our experimental results show that the URL feature space produces much lower precision scores than the noun feature space which, however, is not applicable in non-text pages, dynamic pages and so on. It is crucial to improve the quality of the URL (content-ignorant) feature space since it is generally available in all types of Web pages. We propose a novel content-ignorant feature space, called Web community which is created from a Japanese Web page archive by exploiting link analysis. Experimental results show that the proposed Web community feature space generates much better results than the URL feature space.

Evolution patterns and gradual trends

Nowadays, many databases record ordered or temporally annotated data, such as Web access logs or genomic sequences. Therefore, sequence mining has become an important research area. Among these data mining approaches, sequential patterns aim at describing frequent behaviors. In the access data of a commercial Web site, one may, for instance, discover that “35% of customers successively buy a PSP then a memory stick and PSP games”. To provide more complete information, fuzzy sequential patterns were designed, including quantitative values within the mining task. Such patterns, considering the previous example, would be “35% of customers buy a PSP, then they buy few games many times, and then they buy a high-capacity memory stick once.” However, symbolic or fuzzy sequential patterns, in their current form, do not allow to extract temporal tendencies that are typical of sequential data. By means of temporal tendency mining, one may discover in the same access data that “An increasing number of purchases of PSP games during a very short period is frequently followed by a purchase of a high-capacity memory stick a few days later.” It would be easy to conclude that the users either quickly succeed in registering or make several attempts before they look at the help page within a few seconds. To the best of our knowledge, no method has been designed for discovering this kind of patterns. Therefore, we propose, in this paper, two approaches that extract pattern-expressing trends or evolution. First, we define evolution patterns that summarize the evolution of the quantities in the data. We explain how they can be obtained from a quantitative sequence database. Second, we define gradual trends in fuzzy sequential data. These trends describe variations in the fulfillment of fuzzy properties according to time. For both kinds of patterns, we developed algorithms that were implemented and tested on real data. © 2009 Wiley Periodicals, Inc.

Binary Coded Web Access Pattern Tree in Education Domain

Web Access Pattern (WAP), which is the sequence of accesses pursued by users frequently, is a kind of interesting and useful knowledge in practice. Sequential Pattern mining is the process of applying data mining techniques to a sequential database for the purposes of discovering the correlation relationships that exist among an ordered list of events. WAP tree mining is a sequential pattern mining technique for web log access sequences, which first stores the original web access sequence database on a prefix tree. WAP-tree algorithm then, mines the frequent sequences from the WAP-tree by recursively re-constructing intermediate trees. In this paper, we propose efficient sequential pattern techniques called BC-WAP (Binary Coded WAP). The proposed algorithm uses Kongu Arts and Science College web logs for sequential pattern mining. It eliminates recursively reconstructing intermediate WAP trees during the mining by assigning the binary codes to each node in the WAP tree. The results of the experiments show the efficiency of the improved algorithm.

Web Access Pattern Algorithms in Education Domain

Sequential pattern mining discovers frequent user access patterns from web logs. Apriori-like sequential pattern mining techniques requires expensive multiple scans of database. So, now days, WAP (Web Access Pattern) tree based algorithm is used. It is faster than traditional techniques. However, the use of conditional search strategies in WAP-tree based mining algorithms requires re-construction of large numbers of intermediate conditional WAP-trees, which is also very costly.In this paper, Kongu Arts and Science College (KASC) web logs are taken for mining. Here, we propose an efficient sequential pattern mining techniques for KASC web log access sequences known as CS-WAP Tree. This proposed algorithm modifies the WAP tree approach for improving efficiency. The proposed algorithm totally eliminates the need to engage in numerous reconstructions of intermediate WAP trees and considerable reduces execution time. The results of experiments show the efficiency of the improved algorithm. The next key aim is to compare WAP algorithms.

Virtual Evidence

This paper presents a study of Web Crawler activities based upon Web access logs from the Web site of an academic library. It further compares crawler behavior with that of regular human visitors. The results provide practical insights and foster a culture of evidence-based practice for better managing network-based resources and maintaining a reliable IT infrastructure.

Lessons learned from evaluation of the use of the National electronic Library of Infection

The National electronic Library of Infection (NeLI: http://www.neli.org.uk) in the UK is a freely available portal to key evidence and guidelines in the infectious disease field. This paper discusses 5 years of evaluation of the pilot library and how this evaluation informed design of the new library website. The importance of combining qualitative and quantitative evaluation is highlighted and the results of web access logs analysis, free text search query analysis and an online user survey are compared. The paper concludes with a discussion of lessons learned for future development and evaluation of this Internet digital library.

A Hierarchical Fuzzy Clustering-based System to Create User Profiles

The rapid development of the World Wide Web as a medium of commerce and information dissemination has generated a growing interest of web portal managers in systems able to identify user profiles from the web access logs. The interpretation of these profiles can help re-organize the web portal, e.g., by restructuring the site's content more efficiently, or even to build adaptive web portals, i.e., portals whose organization and presentation change depending on the specific visitor's needs. In this paper, we assume that the pages of the web portal have been prearranged in a number of different categories. We introduce a systematic approach to determine a hierarchy of user profiles from the history of users' accesses to the categories. First, we filter the access log by removing both occasional users and categories of poor interest. Then, we apply an Unsupervised Fuzzy Divisive Hierarchical Clustering (UFDHC) algorithm to cluster the users of the web portal into a hierarchy of fuzzy groups characterized by a set of common interests and each represented by a prototype, which defines the profile of the group typical member. To identify the profile a specific user belongs to, we propose a novel classification method which completely exploits the information contained in the hierarchy. To prove the effectiveness of our approach, we apply the UFDHC algorithm to access log data collected over a period of 15 days and use the classification method to associate a profile with the users defined by access log data collected during subsequent 60 days. Finally, we highlight the good characteristics of our system by comparing our results with the ones obtained by applying a profiling system based on a modified version of the fuzzy C-means.

Just how international is my Web site? Estimating reach through analysis of hourly demand

The increased commercialisation of Internet domain sales created the unanticipated side effect that domain extensions no longer signify the residence of the domain user. As a result, the analysis of the domain attributes in the Web access logs no longer provides accurate information on the origin of the users and thus of the geographical ‘reach’ of a given site. This study provides an alternative method to assess the geographical ‘reach’ by calculating the average demand for Web pages in hourly intervals originating from each time zone. The resulting analysis tool, which relates to Greenwich Mean Time, is location independent and can be applied to Web sites world wide.

A SCALABLE CLUSTERING METHOD FOR CATEGORICAL SEQUENCE DATA

Clustering of sequences is relatively less explored but it is becoming increasingly important in data mining applications such as web usage mining and bioinformatics. The web user segmentation problem uses web access log files to partition a set of users into clusters such that users within one cluster are more similar to one another than to the users in other clusters. Similarly, grouping protein sequences that share a similar structure can help to identify sequences with similar functions. However, few clustering algorithms consider sequentiality. In this paper, we study how to cluster sequence datasets. Due to the high computational complexity of hierarchical clustering algorithms for clustering large datasets, a new clustering method is required. Therefore, we propose a new scalable clustering method using sampling and a k-nearest-neighbor method. Using a splice dataset and a synthetic dataset, we show that the quality of clusters generated by our proposed approach is better than that of clusters produced by traditional algorithms.

Using Markov chain Monte Carlo and dynamic programming for event sequence data

Sequences of events are a common type of data in various scientific and business applications, e.g. telecommunication network management, study of web access logs, biostatistics and epidemiology. A...

Using Markov chain Monte Carlo and dynamic programming for event sequence data

Sequences of events are a common type of data in various scientific and business applications, e.g. telecommunication network management, study of web access logs, biostatistics and epidemiology. A natural approach to modelling event sequences is using time-dependent intensity functions, indicating the expected number of events per time unit. In Bayesian modelling, piecewise constant functions can be utilized to model continuous intensities, if the number of segments is a model parameter. The reversible jump Markov chain Monte Carlo (RJMCMC) methods can be exploited in the data analysis. With very large quantities, these approaches may be too slow. We study dynamic programming algorithms for finding the best fitting piecewise constant intensity function, given a number of pieces. We introduce simple heuristics for pruning the number of the potential change points of the functions. Empirical evidence from trials on real and artificial data sets is provided, showing that the developed methods yield high performance and they can be applied to very large data sets. We also compare the RJMCMC and dynamic programming approaches and show that the results correspond closely. The methods are applied to fault-alarm sequences produced by large telecommunication networks.

2A1-S-068 ウェブ・アクセスログに基づくインターネットユーザの興味遷移パターンの抽出(進化・学習とロボティクス4,生活を支援するロボメカ技術のメガインテグレーション)

2A1-S-068 ウェブ・アクセスログに基づくインターネットユーザの興味遷移パターンの抽出(進化・学習とロボティクス4,生活を支援するロボメカ技術のメガインテグレーション)

Mining Web Log Sequential Patterns with Position Coded Pre-Order Linked WAP-Tree

Sequential mining is the process of applying data mining techniques to a sequential database for the purposes of discovering the correlation relationships that exist among an ordered list of events. An important application of sequential mining techniques is web usage mining, for mining web log accesses, where the sequences of web page accesses made by different web users over a period of time, through a server, are recorded. Web access pattern tree (WAP-tree) mining is a sequential pattern mining technique for web log access sequences, which first stores the original web access sequence database on a prefix tree, similar to the frequent pattern tree (FP-tree) for storing non-sequential data. WAP-tree algorithm then, mines the frequent sequences from the WAP-tree by recursively re-constructing intermediate trees, starting with suffix sequences and ending with prefix sequences. This paper proposes a more efficient approach for using the WAP-tree to mine frequent sequences, which totally eliminates the need to engage in numerous re-construction of intermediate WAP-trees during mining. The proposed algorithm builds the frequent header node links of the original WAP-tree in a pre-order fashion and uses the position code of each node to identify the ancestor/descendant relationships between nodes of the tree. It then, finds each frequent sequential pattern, through progressive prefix sequence search, starting with its first prefix subsequence event. Experiments show huge performance gain over the WAP-tree technique.

Clustering web transactions using rough approximation

Web usage mining is the application of data mining techniques to web log data repositories. Discovering user access patterns from web access log is increasing the importance of information to build up adaptive web server according to the individual user's behavior. In general, the discovered knowledge or any unexpected rules are likely to be imprecise or incomplete, which requires a framework with soft computing techniques like rough sets. In this paper, we present a rough approximation-based clustering to cluster web transactions from web access logs. Using this approach, users can effectively mine web log records to discover web page access patterns.

A rough fuzzy approach to web usage categorization

This paper introduces a novel clustering scheme employing a combination of rough set theory and fuzzy set theory to generate meaningful abstractions from web access logs. Our experimental results show that the proposed scheme is capable of capturing the semantics involved in web access logs at an acceptable computational expense.

Online Analytical Mining of Path Traversal Patterns for Web Measurement

The WWW and its associated distributed information services provide rich world-wide online information services, where objects are linked together to facilitate interactive access. Users seeking information of Internet traverse from one object via links to another. It is important to analyze user access patterns which will help improve Web page design by providing an efficient access between highly correlated objects, and also assist in better marketing decisions by placing advertisements in frequently visited document. We need to study the user surfing behavior by examining the Web access log, browsing frequency of Web pages and computing the average duration time of visit. This paper offers an architecture to store the derived web user access paths in a data warehouse, and facilitates its view maintainability by use of a metadata. The system will update the user access paths pattern with the data warehouse by the data operation functions in the metadata. Whenever a new user access path occurs, the view maintainability is triggered by a constraint class in the metadata. The data warehouse can be analyzed on the frequent pattern tree of user access paths on the Website within a period and duration. The result is an online analytical mining path traversal pattern. Our experimental and performance studies have demonstrated the effectiveness and efficiency of our system with the following contributions: an architecture of online analytical mining (OLAM) using frame model metadata, a methodology (stepwise procedure) of implementing OLAM and the resultant cluster of web pages frequently visited by users for marketing use.

Web usage mining approaches to page recommendation and restructuring

AbstractAs an increasing number of Web sites such as e‐businesses consist of an increasing number of pages, users find it more difficult to rapidly reach their own target pages. Ill‐structured design of Web sites also prevents the users from rapidly accessing the target pages. In this paper, we describe two complementary approaches to Web usage mining as a key solution to these issues. First, we describe an adaptable recommendation system called the L‐R system, which constructs user models by classifying the Web access logs and by extracting access patterns based on the transition probability of page accesses and recommends the relevant pages to the users based on both the user models and the Web structures. We have evaluated the prototype system and have obtained the positive effects. Second, we describe another approach to constructing user models, which clusters Web access logs based on access patterns. The user models also help to discover unexpected access paths corresponding to ill‐formed Web site design. Copyright © 2003 John Wiley & Sons, Ltd.

KMシステムへのWebマイニング技術の応用

KM (Knowledge Management) systems have recently been adopted within the realm of enterprise management. On the other hand, data mining technology is widely acknowledged within Information systems' R&D Divisions. Specially, acquisition of meaningful information from Web usage data has become one of the most exciting eras. In this paper, we employ a Web based KM system and propose a framework for applying Web Usage Mining technology to KM data. As it turns out, task duration varies according to different user operations such as referencing a table-of-contents page, down-loading a target file, and writing to a bulletin board. This in turn makes it possible to easily predict the purpose of the user's task. By taking these observations into account, we segmented access log data manually. These results were compared with results abstained by applying the constant interval method. Next, we obtained a segmentation rule of Web access logs by applying a machine-learning algorithm to manually segmented access logs as training data. Then, the newly obtained segmentation rule was compared with other known methods including the time interval method by evaluating their segmentation results in terms of recall and precision rates and it was shown that our rule attained the best results in both measures. Furthermore, the segmented data were fed to an association rule miner and the obtained association rules were utilized to modify the Web structure.

データ宝石箱を用いたウエブアクセスログの視覚化

This paper presents a tool for visualizing Web access logs, which has two views. The left view represents Web sitemaps by a set of webpage icons. It places the icons onto display spaces by data jewelry box algorithm. The right view represents Web access statistics by a bar chart. It aggregates the accesses according to user-specified attributes, such as time, client, and status. When a user clicks a bar in the right view, the left view highlights icons that have corresponding accesses. When a user clicks an icon in the left view, the right view shows the access statistics of the clicked webpage. These combinations of the two views help users to find problems of websites, and discover trends of accesses.