Search Engine Transaction Logs Research Articles

The impact of big data on research methods in information science

Social media platforms, search engine transaction logs, Web portal transaction logs, large text corpora, and other data sources offer users a variety of invaluable data sources. Data generated based on social media platforms and other data sources have become important sources of big data for researchers in information science. Big data provide not only challenges but also opportunities for information science. Emerging big data trends inevitably have an impact on research methods in information science. The authors of this paper discuss the impact of big data on research methods in information science.This paper addresses these challenges and opportunities through the lens of research methods, ranging from data processing, to sampling, to information visualization, to temporal analysis, to sentiment analysis, to correlation, to cause-effect relationship, to data accessibility, to data privacy, and data ethics issues.The discussions on related aspects of research methods provoke a healthy reflection and debate for the information science research community, which can help researchers in the field produce sound research designs for big data-oriented research studies and assist information practitioners in solving problems they face in the big data age.Research methods are fundamental and essential for any research studies. Big data analysis has a natural relationship with research methods. The paper discusses an emerging and important research topic big data from the unique angle of research methods.

Neural network applications for automatic new topic identification of FAST and Excite search engine transaction logs

Abstract: Content analysis of search engine user queries is an important task, since successful exploitation of the content of queries can result in the design of efficient information retrieval algorithms for more efficient search engines. Identification of topic changes within a user search session is a key issue in content analysis of search engine user queries. This study proposes an artificial neural network application in the area of search engine research to automatically identify topic changes in a user session by using statistical characteristics of queries, such as time intervals and query reformulation patterns. Sample data logs from the FAST and Excite search engines are selected to train the neural network and then the neural network is used to identify topic changes in the data log. As a result, almost all the performance measures yielded favourable results.

THE FUTURE OF JOURNALISM

Click to increase image sizeClick to decrease image size Notes 1. This is a modified and extended version of a plenary address given to the Future of Journalism Conference, Cardiff School of Journalism, in 2009. 2. See, for example, US News and World Report, 29 September 1975; Washington Post, 5 June 1978. 3. Reported in the Boston Globe, 7 November 1993. 4. For example, British media companies made this argument to the Office of Fair Trading (2009), only for it to be rejected. 5. See, for instance, Preston (2009 Preston , Peter 2009 “Reasons to Be Cheerful, Part 1: recession didn't stop the press” , Observer , 27 December . [Google Scholar]), among many others. 6. Information supplied by the Newspaper Society. 7. One of a series of wonderful Rowson covers, it was commissioned by Routledge for the cover of Curran and Seaton (2010). 8. A substantial number of articles in “The Future of Journalism” special issue of Journalism, 10(3), 2009 veer towards the position, prompting me to label it the “liberal educator” approach. 9. See the reports of the Internet Advertising Bureau (United States) and (United Kingdom), IREP (France), and Interactive Advertising Bureau of Canada, among others. 10. The Pew figure for the United States is higher partly because respondents are able to cite more than one primary news source in its survey. 11. Nielsen NetRatings Website Ranking, January 2009. My thanks to David Cowling (BBC) for supplying this information. 12. “Reach and percentage growth for UK News websites” (table derived from Nielsen, September 2008), BBC News Online: 2008's highlights. My thanks again to David Cowling. 13. This matters because most Internet users in Europe and the United States do not go beyond the first page (Jansen and Spink, 2005 Jansen , Bernard and Spink , Amanda 2005 “An Analysis of Web Searching by European AlltheWeb.com users” , Information Processing Management 41 2 , pp. 361 91 .[Crossref], [Web of Science ®] , [Google Scholar], 2006 Jansen , Bernard and Spink , Amanda 2006 “How Are We Searching the World Wide Web? A comparison of nine search engine transaction logs” , Information Processing Management 42 1 , pp. 248 63 .[Crossref], [Web of Science ®] , [Google Scholar]). 14. See Note 11. 15. Information derived from Ina Zwerger at the Osterreichischer Rundfunk.

Identifying the optimal set of parameters for new topic identification through experimental design

Users are interested in multiple topics during a search session, and identifying the boundaries of search sessions is an important task. This study proposes to use neural networks for defining the topic boundaries in search engine transaction logs, and is a part of ongoing research on automatic new topic identification. The objective of the study is to determine the best set of parameters for neural networks that are designed to perform automatic new topic identification. Sample data logs from FAST (currently owned by Yahoo) and Excite (currently owned by IAC Search & Media) search engines were analyzed. The findings show that neural networks are fairly successful in identifying topic continuations and shifts in search engine transaction logs. The choice of the neural network structure depends on which performance measure is more important to the user. For a certain performance measure, there is a set of parameters of neural networks that will increase the performance of new topic identification in search engine transaction logs. In addition, the threshold value of the output level of neural networks is the most influential parameter on the performance of new topic identification.

DEFINING TOPIC BOUNDARIES IN SEARCH ENGINE TRANSACTION LOGS USING GENETIC ALGORITHMS

This study proposes to use genetic algorithms for defining the topic boundaries in search of engine transaction logs. Users are interested in multiple topics during a search session, and genetic algorithms are used in this study to determine whether a search engine user has changed topics during a session. Sample data logs from FAST and Excite search engines were analyzed. The findings show that genetic algorithms are fairly successful in identifying topic continuations and shifts in search engine transaction logs.

Markovian analysis for automatic new topic identification in search engine transaction logs

AbstractTopic analysis of search engine user queries is an important task, since successful exploitation of the topic of queries can result in the design of new information retrieval algorithms for more efficient search engines. Identification of topic changes within a user search session is a key issue in analysis of search engine user queries. This study presents an application of Markov chains in the area of search engine research to automatically identify topic changes in a user session by using statistical characteristics of queries, such as time intervals, query reformulation patterns and the continuation/shift status of the previous query. The findings show that Markov chains provide fairly successful results for automatic new topic identification with a high level of estimation for topic continuations and shifts. Copyright © 2009 John Wiley & Sons, Ltd.

Analyzing the results of automatic new topic identification

PurposeIdentification of topic changes within a user search session is a key issue in content analysis of search engine user queries. Recently, various studies have focused on new topic identification/session identification of search engine transaction logs, and several problems regarding the estimation of topic shifts and continuations were observed in these studies. This study aims to analyze the reasons for the problems that were encountered as a result of applying automatic new topic identification.Design/methodology/approachMeasures, such as cleaning the data of common words and analyzing the errors of automatic new topic identification, are applied to eliminate the problems in estimating topic shifts and continuations.FindingsThe findings show that the resulting errors of automatic new topic identification have a pattern, and further research is required to improve the performance of automatic new topic identification.Originality/valueImproving the performance of automatic new topic identification would be valuable to search engine designers, so that they can develop new clustering and query recommendation algorithms, as well as custom‐tailored graphical user interfaces for search engine users.

A Monte-Carlo simulation application for automatic new topic identification of search engine transaction logs

One of the most important dimensions of Web user information seeking behavior and search engine research is content-based behavior, and limited research has focused on content-based behavior of search engine users. The purpose of this study is to perform automatic new topic identification in search engine transaction logs using Monte-Carlo simulation. Sample data logs from FAST and Excite are used in the study. Findings show that Monte-Carlo simulation for new topic identification yields satisfactory results in terms of identifying topic continuations; however, the performance measures regarding topic shifts should be improved.

Cross‐validation of neural network applications for automatic new topic identification

AbstractThe purpose of this study is to provide results from experiments designed to investigate the cross‐validation of an artificial neural network application to automatically identify topic changes in Web search engine user sessions by using data logs of different Web search engines for training and testing the neural network. Sample data logs from the FAST and Excite search engines are used in this study. The results of the study show that identification of topic shifts and continuations on a particular Web search engine user session can be achieved with neural networks that are trained on a different Web search engine data log. Although FAST and Excite search engine users differ with respect to some user characteristics (e.g., number of queries per session, number of topics per session), the results of this study demonstrate that both search engine users display similar characteristics as they shift from one topic to another during a single search session. The key finding of this study is that a neural network that is trained on a selected data log could be universal; that is, it can be applicable on all Web search engine transaction logs regardless of the source of the training data log.

Using conditional probabilities for automatic new topic identification

PurposeOne of the most important dimensions of search engine user information seeking behaviour is content‐based behaviour. One of the main elements in developing a personalised intelligent search engine is new topic identification. The purpose of this study is to perform automatic new topic identification in search engine transaction logs using conditional probabilities of new topic arrivals.Design/methodology/approachSample data logs from FAST (currently owned by Yahoo!) and Excite (currently owned by IAC Search & Media) are used in the study. Conditional probabilities of new topic arrivals and topic continuations given query category are used to estimate new topic arrivals.FindingsThe findings of this study show that the conditional probability approach reduced overestimation of topic shifts, increasing some performance measures to their highest ever value compared to previous studies. A straightforward procedure such as the conditional probability approach can be as successful as, and for some measures more successful than, more complex methods applied in previous automatic new topic identification studies.Originality/valueA straightforward procedure that can enable fast automatic new topic identification, a problem not yet solved, and an important step towards personalised search engines.

Study of Search Engine Transaction Logs Shows Little Change in How Users use Search Engines

A review of:
 
 Jansen, Bernard J., and Amanda Spink. “How Are We Searching the World Wide Web? A Comparison of Nine Search Engine Transaction Logs.” Information Processing & Management 42.1 (2006): 248-263.
 
 Objective – To examine the interactions between users and search engines, and how they have changed over time. 
 
 Design – Comparative analysis of search engine transaction logs.
 
 Setting – Nine major analyses of search engine transaction logs.
 
 Subjects – Nine web search engine studies (4 European, 5 American) over a seven-year period, covering the search engines Excite, Fireball, AltaVista, BWIE and AllTheWeb.
 
 Methods – The results from individual studies are compared by year of study for percentages of single query sessions, one-term queries, operator (and, or, not, etc.) usage and single result page viewing. As well, the authors group the search queries into eleven different topical categories and compare how the breakdown has changed over time.
 
 Main Results – Based on the percentage of single query sessions, it does not appear that the complexity of interactions has changed significantly for either the U.S.-based or the European-based search engines. As well, there was little change observed in the percentage of one-term queries over the years of study for either the U.S.-based or the European-based search engines. Few users (generally less than 20%) use Boolean or other operators in their queries, and these percentages have remained relatively stable. One area of noticeable change is in the percentage of users viewing only one results page, which has increased over the years of study. Based on the studies of the U.S.-based search engines, the topical categories of ‘People, Place or Things’ and ‘Commerce, Travel, Employment or Economy’ are becoming more popular, while the categories of ‘Sex and Pornography’ and ‘Entertainment or Recreation’ are declining.
 
 Conclusions – The percentage of users viewing only one results page increased during the years of the study, while the percentages of single query sessions, one-term sessions and operator usage remained stable. The increase in single result page viewing implies that users are tending to view fewer results per web query. There was also a significant difference in the percentage of queries using Boolean operators between the US-based and the European-based search engines. One of the study’s findings was that results from a study of a particular search engine cannot necessarily be applied to all search engines. Finally, web search topics show a trend towards information or commerce searching rather than entertainment. 

Automatic new topic identification in search engine transaction logs

PurposeContent analysis of search engine user queries is an important task, since successful exploitation of the content of queries can result in the design of efficient information retrieval algorithms of search engines, which can offer custom‐tailored services to the web user. Identification of topic changes within a user search session is a key issue in content analysis of search engine user queries. The purpose of this study is to address these issues.Design/methodology/approachThis study applies genetic algorithms and Dempster‐Shafer theory, proposed by He et al., to automatically identify topic changes in a user session by using statistical characteristics of queries, such as time intervals and query reformulation patterns. A sample data log from the Norwegian search engine FAST (currently owned by overture) is selected to apply Dempster‐Shafer theory and genetic algorithms for identifying topic changes in the data log.FindingsAs a result, 97.7 percent of topic shifts and 87.2 percent of topic continuations were estimated correctly. The findings are consistent with the previous application of the Dempster‐Shafer theory and genetic algorithms on a different search engine data log. This finding could be implied as an indication that content‐ignorant topic identification, using query patterns and time intervals, is a promising line of research.Originality/valueStudies an important dimension of user behavior in information retrieval.

How are we searching the World Wide Web? A comparison of nine search engine transaction logs

The Web and especially major Web search engines are essential tools in the quest to locate online information for many people. This paper reports results from research that examines characteristics...

How are we searching the World Wide Web? A comparison of nine search engine transaction logs

The Web and especially major Web search engines are essential tools in the quest to locate online information for many people. This paper reports results from research that examines characteristics and changes in Web searching from nine studies of five Web search engines based in the US and Europe. We compare interactions occurring between users and Web search engines from the perspectives of session length, query length, query complexity, and content viewed among the Web search engines. The results of our research shows (1) users are viewing fewer result pages, (2) searchers on US-based Web search engines use more query operators than searchers on European-based search engines, (3) there are statistically significant differences in the use of Boolean operators and result pages viewed, and (4) one cannot necessary apply results from studies of one particular Web search engine to another Web search engine. The wide spread use of Web search engines, employment of simple queries, and decreased viewing of result pages may have resulted from algorithmic enhancements by Web search engine companies. We discuss the implications of the findings for the development of Web search engines and design of online content.