Web Log Mining Research Articles

An Improved Web Log Mining and Online Navigational Pattern Prediction

The aim of this study is to improve web log mining and online navigation pattern prediction. Web mining is an active and wide area which incorporates several usages for the web site design, providing personalization server and other business making decisions etc. Efficient web log mining results and online navigational pattern prediction is a tough process due to vast development in web. It includes the process such as data cleaning, session identification and clustering of web logs generally. In this study initially the web log data is preprocessed and sessions are identified using refined time-out based heuristic for session identification. Then for pattern discovery a density based clustering algorithm is used. Finally for online navigation pattern prediction a new technique of SVM classification is used, which rectifies time complexity with increased prediction accuracy.

Research on the Model of Personalized Recommendation System Based on Multi Agent

There are some lacks of intelligence, self-adaptability, initiative and processing power limitations in the traditional recommendation system. Using the multi-agent technology and the web log mining technology, this paper converts the function modules of traditional personalized recommendation system into an agent. This paper proposes an architecture model based on multi-agent e-commerce personalized recommendation system (MAPRS), and discusses the function of each component of the model and the system's running processes.

Web Log Mining to Enhance Surfing Experience

Analyzing navigation history of a web site and that of a user reported in log files and web click stream helps in determining the components that shall be needed in the near future and fetch them in advance. Identifying the frequently visited websites from such data enhances the process. Processing and analyzing the vast amounts of click stream data and web logs that are generated at very high rates in a smart and cost-efficient way is a daunting challenge to the data mining community. Mining frequent itemsets plays an important role in analyzing such data streams. In spite of the existence of many such mining algorithms, more time and space efficient algorithms for mining frequent patterns are the need of the hour and are attracting wide attention in web click-stream analysis in recent years. In this paper an effective algorithm for mining frequent itemsets from a time-sensitive sliding window is proposed. It is a one-pass algorithm that uses a circular queue to implement the sliding window. The time-sensitive sliding window stores the web click stream data of various sessions of the web users. The proposed one-pass algorithm, FIM_CQTimeSWin has three phases: representation of time unit, maintenance of the sliding window and generation of frequent itemsets in the current sliding window.

Web Log Mining Based-on Improved Double-Points Crossover Genetic Algorithm

Web log files have become important data source for discoveries of user behaviors. Analyzing web log files is one of the significant research fields of web mining. This paper proposes an improved double-points crossover genetic algorithm for mining user access patterns from web log files. Our work contains three different components. First, we design a coding rule according to pre-processed web log data. Second, a fitness function is presented by analyzing user sessions. Finally, a new genetic algorithm based on double-points crossover genetic algorithm is designed. In comparison with simple genetic algorithm, double-points crossover genetic algorithm demonstrates better convergence than the other, and it is more suitable for web log mining. We conducted an experiment to verify the effectiveness of the proposed algorithm. The results show that the proposed algorithm helps the website to easily gain access patterns.

Web Log Data Analysis by Enhanced Fuzzy C Means Clustering

World Wide Webis a huge repository of information and there is a tremendous increase in the volume of information daily. The number of users are also increasing day by day. To reduce users browsing time lot of research is taken place. Web Usage Mining is a type of web mining in which mining techniques are applied in log data to extract the behaviour of users.Clustering plays an important role in a broad range of applications like Web analysis, CRM, marketing, me dical diagnostics, computational biology, and many others. Clustering is the grouping of similar instances or objects. The key factor for clustering is some sort of measure that can determine whether two objects are similar or dissimilar . In this paper a novel clustering method to partition user sessions into accurate clusters is discussed. The accuracy and various performance measures of the proposed algorithm shows that the proposed method is a better method for web log mining.

Web Usage Mining for Discovery and Evaluation of Online Navigation Pattern Prediction

Web mining is combination of two activated research areas Data Mining and World Wide Web. Web mining is used for mining the interested knowledge from World Wide Web. Web usage mining is used to discover the user access patterns from web server log files. The first step of web usage mining called as data pre-processing used for gaining an accurate web log mining results and good quality input data. The session identification is accomplished by time oriented heuristics. The focus is on referrer log which will which contain information about referrer page of the current page. An efficient approach for discovery of navigation pattern can be done by density based clustering algorithm. An online navigation pattern prediction is proposed by use of K nearest neighbor algorithm along with inverted index concept. The prediction accuracy of patterns can be increased by modifying TF-IDF values to include time spent on page.

A New Algorithm for Web Log Mining

The enormous content of information on the World Wide Web makes it obvious candidate for data mining research. Data Mining Technique application is used to the World Wide Web referred as Web mining where this term has been used in three distinct ways; , Web Structure Mining, Web Content Mining and Web Usage Mining. Web Log Mining is one of the Web based application where it will facing with large amount of log data. In order to produce the web log through portal usage patterns and user behaviors, this research work implements the high level process of Web Log mining technique using basic rules. Web Log Mining consists of three main phases, namely Data Preprocessing, Pattern filtering and Pattern Analysis. As we know that server log files become a set of raw data where it’s must go through with all the Web Log Mining phases to producing the final results. Here, Web Log mining, approach has been combining with the basic rules, to optimize the total execution time. Finally, this work wills present an overview of results analysis and Web Log Mining can use the findings for the suitable valuable actions.

Algorithm for Tracing Visitors' On-Line Behaviors for Effective Web Usage Mining

behavior identification is an important task in web usage mining. Web usage mining is also called as web log mining. The web logs are mainly used to identify the user behavior. There are so many pattern mining methods which enable this user behavior identification. The preprocessing techniques will maximize the accurate and quality of pattern mining methodologies. In existing algorithms, the preprocessing concepts are applied to calculate the unique user's count, to minimize the log file size and to identify the sessions. The newly proposed algorithm is Visitors' Online Behavior (VOB) which identifies user behavior, creates user cluster and page cluster, and tells the most popular web page and least popular web page. This paper brings into discussion about the basic concepts of web mining, web usage mining, general data preprocessing, how to preprocess the web data, what are the various existing preprocessing techniques and the proposed VOB algorithm. Keywordspreprocessing methods, Web mining, Web usage mining, Web usage data, Web log.

A Survey on Approaches for Mining Frequent Itemsets

Data mining is gaining importance due to huge amount of data available. Retrieving information from the warehouse is not only tedious but also difficult in some cases. The most important usage of data mining is customer segmentation in marketing, shopping cart analyzes, management of customer relationship, campaign management, Web usage mining, text mining, player tracking and so on. In data mining, association rule mining is one of the important techniques for discovering meaningful patterns from large collection of data. Discovering frequent itemsets play an important role in mining association rules, sequence rules, web log mining and many other interesting patterns among complex data. This paper presents a literature review on different techniques for mining frequent itemsets.

Research on Cluster Analysis Method of E-government Public Hotspot Information Based on Web Log Analysis

To master the hotspot information of E-government public opinion macroscopically, and so that the government can provide more target-oriented information and services for the public. A method for calculating the public concern degree of E-government public opinion is proposed in this paper. In this method, two indices that can reflect the hotspot information of public opinion to a certain degree are calculated through the web log mining, the attention degree of the information through the weighting computation to find the hotspot problems that the public cared about in a certain period of time. A fuzzy cluster analysis method that can find hotspot problems is presented in this paper. And an illustrative example is given to describe the clustering process.

Visualisation techniques and its uses in representation of web usage patterns: a survey

Web usage mining, the analysis of user navigation paths through websites, is a common technique for evaluating site designs or adaptive websites. Web usage mining results in usage patterns which can help website designers, analysts and business executives in proactive decision making. Different visualisation techniques are used to represent mined patterns so that it can be easily understood by the executives. It is found that the visualisation can significantly increase the quality and usefulness of web log mining results. In this paper, we tried to find out how many different visualisation techniques are utilised for web usage mining.

Resource management scheme based on ubiquitous data analysis.

Resource management of the main memory and process handler is critical to enhancing the system performance of a web server. Owing to the transaction delay time that affects incoming requests from web clients, web server systems utilize several web processes to anticipate future requests. This procedure is able to decrease the web generation time because there are enough processes to handle the incoming requests from web browsers. However, inefficient process management results in low service quality for the web server system. Proper pregenerated process mechanisms are required for dealing with the clients' requests. Unfortunately, it is difficult to predict how many requests a web server system is going to receive. If a web server system builds too many web processes, it wastes a considerable amount of memory space, and thus performance is reduced. We propose an adaptive web process manager scheme based on the analysis of web log mining. In the proposed scheme, the number of web processes is controlled through prediction of incoming requests, and accordingly, the web process management scheme consumes the least possible web transaction resources. In experiments, real web trace data were used to prove the improved performance of the proposed scheme.

Modelling of domestic and foreign visitors' behaviour at commercial bank website during the recent financial crisis

The paper focuses on modelling of commercial bank website visitors’ behaviour. The authors analyse domestic and foreign market participants’ interests in mandatory financial information disclosure of a commercial bank during the recent financial crisis and try to answer the question whether the purposes of Basel 2 regulations under the Pillar 3 – Market discipline, publishing financial information, have been fulfilled. The authors analyse bank website logs files using web log mining methods to better understand the rate of using of web pages, where mandatory financial information about Basel 2 is published. After data pre-processing the authors use association rule analysis to identify the association among content categories of the website. The results show that there is small interest in mandatory financial information disclosure by the commercial bank in general. The foreign website visitors take more concern in mandatory financial information disclosure, and they take less interest in general information about bank than domestic ones.

Verification of the fulfilment of the purposes of Basel II, Pillar 3 through application of the web log mining methods

The objective of the paper is the verification of the fulfilment of the purposes of Basel II, Pillar 3 – market discipline during the recent financial crisis. The objective of the paper is to describe the current state of the working out of the project that is focused on the analysis of the market participants’ interest in mandatory disclosure of financial information by a commercial bank by means of advanced methods of web log mining. The output of the realized project will be the verification of the assumptions related to the purposes of Basel III by means of the web mining methods, the recommendations for possible reduction of mandatory disclosure of information under Basel II and III, the proposal of the methodology for data preparation for web log mining in this application domain and the generalised procedure for users’ behaviour modelling dependent on time. The schedule of the project has been divided into three phases. The paper deals with its first phase that is focusing on the data pre-processing, analysis and evaluation of the required information under Basel II, Pillar 3 since 2008 and its disclosure into the web site of a commercial bank. The authors introduce the methodologies for data preparation and known heuristic methods for path completion into web log files with respect to the particularity of investigated application domain. They propose scientific methods for modelling users’ behaviour of the webpages related to Pillar 3 with respect to time.

A Survey on Hash based A-priori Algorithm for Web Log Analysis

This paper attempts to signify the importance of newest variation of data mining in form of web mining. The paper also discusses about some of the existing web log analysis algorithms. The previous some web mining algorithm working with different concepts gives effective result for the web mining. Presently the user want to execs the web pages or other web related applications very fast and very easily. Then arises a strong need for performing web log analysis for increases the performance the web log mining. However there exist many systems for performing the web log analysis, the results they generate are far from the expectations. In this paper present a new hash based Apriori algorithm which work very fast on the web server and give the effective result in comparison to previous algorithms.

Data pre-processing for web log mining: Case study of commercial bank website usage analysis

We use data cleaning, integration, reduction and data conversion methods in the pre-processing level of data analysis. Data processing techniques improve the overall quality of the patterns mined. The paper describes using of standard pre-processing methods for preparing data of the commercial bank website in the form of the log file obtained from the web server. Data cleaning, as the simplest step of data pre-processing, is non–trivial as the analysed content is highly specific. We had to deal with the problem of frequent changes of the content and even frequent changes of the structure. Regular changes in the structure make use of the sitemap impossible. We presented approaches how to deal with this problem. We were able to create the sitemap dynamically just based on the content of the log file. In this case study, we also examined just the one part of the website over the standard analysis of an entire website, as we did not have access to all log files for the security reason. As the result, the traditional practices had to be adapted for this special case. Analysing just the small fraction of the website resulted in the short session time of regular visitors. We were not able to use recommended methods to determine the optimal value of session time. Therefore, we proposed new methods based on outliers identification for raising the accuracy of the session length in this paper.

Implicit Measures of User Interests through Country and Predicting Users' Future Requests in WWW

has become the center of attraction for business transactions and hence e-commerce due to its ease of use and speed. its ability from tracking the browsing behaviour of any user to even the mouse clicks of any individual has brought the vendor and end customer closer than ever before.WWW has made it possible for vendors to advertize their products i.e. they are personalizing their product messages for individual customers at a very large scale such a phenomenon is termed as mass communication such a utility is not only applicable for e-commerce but also such personalization is aiding several web browsing activities. Any action that tailors the web experience to any individual or several is termed as web personalization. Web personalization is that the method of customizing an internet website to the wants of specific users, taking advantage of the information noninheritable from the analysis of the user's guidance behaviour(Weblog data) in correlation with alternative data collected within the web context, namely, structure, content and user profile information. The domain of web personalization has gained importance both in the area of research and commerce. In this paper we proposed a framework of web log mining to implicit measures of user interests through Country and predicting users future requests in WWW.

A cross‐language personalized recommendation model in digital libraries

PurposeThe purpose of this paper is to develop a cross‐language personalized recommendation model based on web log mining, which can recommend academic articles, in different languages, to users according to their demands.Design/methodology/approachThe proposed model takes advantage of web log data archived in digital libraries and learns user profiles by means of integration analysis of a user's multiple online behaviors. Moreover, keyword translation was carried out to eliminate language dissimilarity between user and item profiles. Finally, article recommendation can be achieved using various existing algorithms.FindingsThe proposed model can recommend articles in different languages to users according to their demands, and the integration analysis of multiple online behaviors can help to better understand a user's interests.Practical implicationsThis study has a significant implication for digital libraries in non‐English countries, since English is the most popular language in current academic articles and it is a very common phenomenon for users in these countries to obtain literatures presented by more than one language. Furthermore, this approach is also useful for other text‐based item recommendation systems.Originality/valueA lot of research work has been done in the personalized recommendation area, but few works have discussed the recommendation problem under multiple linguistic circumstances. This paper deals with cross‐language recommendation and, moreover, the proposed model puts forward an integration analysis method based on multiple online behaviors to understand users' interests, which can provide references for other recommendation systems in the digital age.

Effective web log mining and online navigational pattern prediction

Accurate web log mining results and efficient online navigational pattern prediction are undeniably crucial for tuning up websites and consequently helping in visitors’ retention. Like any other data mining task, web log mining starts with data cleaning and preparation and it ends up discovering some hidden knowledge which cannot be extracted using conventional methods. In order for this process to yield good results it has to rely on some good quality input data. Therefore, more focus in this process should be on data cleaning and pre-processing. On the other hand, one of the challenges facing online prediction is scalability. As a result any improvement in the efficiency of online prediction solutions is more than necessary. As a response to the aforementioned concerns we are proposing an enhancement to the web log mining process and to the online navigational pattern prediction. Our contribution contains three different components. First, we are proposing a refined time-out based heuristic for session identification. Second, we are suggesting the usage of a specific density based algorithm for navigational pattern discovery. Finally, a new approach for efficient online prediction is also suggested. The conducted experiments demonstrate the applicability and effectiveness of the proposed approach.

A Forecasting Model for the Pages Crawled by Search Engine Crawlers at a Web Site

World Wide Web is exploding in terms of the number of web sites and users. Without search engines the web sites will not be visible to the users. Different search engine crawlers behave in different ways while they access a web site. The number of visits and pages crawled by search engines could be helpful in identifying their behavior and also the server load. A forecasting model in time series has been proposed for predicting the number of pages crawled by search engines. This model was compared with the actual values and it was found feasible. General Terms Web log mining, Web analytics