Transaction Log Research Articles

Assessing used content across five digital health information services using transaction log files

A digital service, like a web site, may contain a lot of information but we often do not know if it is used, relevant or valuable. Transaction log files generated by digital information services do record the pages (topics or content) viewed by users and this is perhaps the most interesting aspect of the logs. However, analysing these pages poses plenty of problems for researchers, especially when comparing content coverage of various related services. It is quite normal, even for digital services of the same organization, to adopt different page naming conventions for each service. This is even truer about digital services run by different organizations. What all this means is that there is no easy way to compare topic use as revealed by access behaviour. This paper looks at the problems of describing and comparing the content usage of digital information services, covering three digital platforms operating in the health field. This paper discusses problems posed in making health content comparisons based on page names listed in the transaction log files and between very large data sets. It reviews the impact that system architecture might have as well as the time the service has been available online and the impact due to outlet differences. However, the main focus of the article is a comparison of five sources of health information through their log files. It makes use of cluster analysis and applies procedures normally used to define species diversity to research content coverage. In all, two million page views were analysed, covering more than 5000 unique health pages.

Micro-Mining and Segmented Log File Analysis: A Method for Enriching the Data Yield from Internet Log Files

The authors propose improved ways of analysing web server log files. Traditionally web site statistics focus on giving a big (and shallow) picture analysis based on all transaction log entries. The pictures are, however, distorted because of the problems associated with resolving Internet protocol (IP) numbers to a single user and cross-border IP registration. The authors argue that analysing extracted sub-groups and categories presents a more accurate picture of the data and that the analysis of the online behaviour of selected individuals (rather than of very large groups) can add much to our understanding of how people use web sites and, indeed, any digital information source. The analysis is labelled `micro' to distinguish it from traditional macro, big picture transactional log analysis. The methods are illustrated with recourse to the logs of the Surgery Door (www.surgerydoor.co.uk) consumer health web site. It was found that use attributed to academic users gave a better approximation of the sites' geographical distribution of users than an analysis based on all users. This occurs as academic institutions, unlike other user types, register in their host country. Selecting log entries where each user is allocated a unique IP number can be particularly beneficial, especially to analyses of returnees. Finally the paper tracks the online behaviour of a small number of IP numbers, in an example of the application of microanalysis,

Multitasking Web searching and implications for design

AbstractThis paper presents findings from a study of users multitasking searches on Web search engines. A user's single session with a Web search engine may consist of seeking information on single or multiple topics. Limited research has focused on multitasking search and the implications for Web design. Incidence of multitasking search by AlltheWeb.com and Excite Web search engine users were filtered from transaction logs. Findings include: (1) multitasking Web searches are a noticeable user behavior, one tenth of Excite users and one third of AlltheWeb.com users conducted multitasking searches, (2) multitasking search sessions are longer than regular search sessions in terms of queries per session and duration, (3) both Excite and AlltheWeb.com users search for about three topics per multitasking session and submit about 4–5 queries per topic, and (4) there is a broad variety of search topics in multitasking search sessions. The implications of our findings for Web design and further research are discussed.

Coverage, relevance, and ranking

Research has reported that about 10% of Web searchers utilize advanced query operators, with the other 90% using extremely simple queries. It is often assumed that the use of query operators, such as Boolean operators and phrase searching, improves the effectiveness of Web searching. We test this assumption by examining the effects of query operators on the performance of three major Web search engines. We selected one hundred queries from the transaction log of a Web search service. Each of these original queries contained query operators such as AND, OR, MUST APPEAR (+), or PHRASE (" "). We then removed the operators from these one hundred advanced queries. We submitted both the original and modified queries to three major Web search engines; a total of 600 queries were submitted and 5,748 documents evaluated. We compared the results from the original queries with the operators to the results from the modified queries without the operators. We examined the results for changes in coverage, relative precision, and ranking of relevant documents. The use of most query operators had no significant effect on coverage, relative precision, or ranking, although the effect varied depending on the search engine. We discuss implications for the effectiveness of searching techniques as currently taught, for future information retrieval system design, and for future research.

Understanding the brevity of Web queries

Introduction Internet search engines are the most common resource for people seeking answers to questions and access to webbased information, and in most search engines, keyword queries are the primary means of retrieving information. Yet, web queries tend to be remarkably brief and general. Although they have grown slightly in length over time, web queries are still on average about two words in length, which is significantly shorter and less complex than queries used in earlier types of information systems (Jansen & Pooch, 2000; Wang, Berry & Yang, 2003). Popular queries listed on the Google site, such as "weather", "moon," and "nature" serve as very coarse filters for the two billion indexed web pages, and return millions of hits. Given this scenario, it is not surprising that people are often frustrated by web searching, and that search success rates of about 50% are reported (Nielsen, 2001).

Designing, implementing, and evaluating an educational digital library resource

This paper presents the current status in the development of the ongoing project now known as Project i‐DLR. The content of this “pointer site” includes resources pertaining to digital libraries organised using an educational framework for access. The paper describes the five‐stage evaluation of that educational digital library resource (www.coe.missouri.edu/ rafee/idigital libraryR/index.php). The focus of this particular effort is on the continued development and refinement based on the recent evaluations of this resource by end users seeking to access digital library resources. The five evaluation methods are presented and described, beginning with focus group reviews, Web log analysis, database transaction logs, a Web survey, and most recently, a remote usability evaluation. As the resource continues to grow in both breadth and depth, such analyses are critical to continued refinement of the interface, the sources themselves, and the manner in which they are organised and presented.

Non use of health information kiosks examined in an information needs context.

Whilst great emphasis has rightly been placed on the increased availability and use of health information, little research has been undertaken into the non use of such material, particularly with regard to electronic sources. Computer transactional log data from health information touch screen kiosks, collected as part of an ongoing Department of Health-funded study being carried out by City University, showed that females in the 55-74 age group were particularly under-represented as users. To explore reasons for this, in-depth interviews were carried out with 13 non-kiosk-using, primary-care out-patients fitting this profile, at a surgery which had a kiosk that was being monitored. Subjects were interviewed at length about their information needs and information-seeking behaviour, to determine reasons (if any) they might have had for non-kiosk use. The study utilized an interview schedule and technique loosely based on the 'Sense-Making' methodology of Brenda Dervin (Talk Presented at the International Communication Association Annual Meeting, May 1983, Dallas, Texas, USA. Available from: http://communicationsbsohio-stateedu/sense-making/art/artdervin83html.). Findings elicited many factors accounting for non use of the system. The first and major source of information remained the doctor, with written or other sources only being consulted where recommended or provided. There was evidence that patients wanted little more than the minimum information or instructions required to deal with their condition. Many appeared unaware of the presence of the kiosk and others assumed either that it was not for patient use or that it would not serve their needs. The methodology proved itself, with certain caveats, to be an appropriate vehicle for this kind of exploratory work.

Information Retrieval in Domain-specific Databases: An Analysis to Improve the User Interface of the Alcohol Studies Database

Academic libraries are becoming more directly involved in the design and publishing of electronic information resources, including bibliographic databases, electronic journals, and digital archives. As a result, librarians are dealing with many user interface design issues that computer scientists and information specialists in other fields have encountered. Transaction log analysis can provide a rich source of information on user behavior and insights as to how user interfaces can be improved. This article describes the methodology and results of the log analysis for the Alcohol Studies Database (ASDB), a domain-specific database supported by the Center of Alcohol Studies and Rutgers University Libraries (RUL). The goals of this study were to better understand user search behavior, to analyze failure rates, and to develop approaches for improving the user interface.

Students' conceptual structure, search process, and outcome while preparing a research proposal: A longitudinal case study

AbstractThis article focuses on analysing students' information needs in terms of conceptual understanding of the topic they propose to study and its consequences for the search process and outcome. The research subjects were 22 undergraduates of psychology attending a seminar for preparing a research proposal for a small empirical study. They were asked to make searches in the PsycINFO database for their task in the beginning and end of the seminar. A pre‐ and postsearch interview was conducted in both sessions. The students were asked to think aloud in the sessions. This was recorded, as were the transaction logs. The results show that during the preparation of research proposals different features of the students' conceptual structure were connected to the search success. Students' ability to cover their conceptual construct by query terms was the major feature affecting search success during the whole process. In the beginning also the number of concepts and the proportion of subconcepts in the construct contributed indirectly via search tactics to retrieving partly useful references. Students' ability to extract new query terms from retrieved items improved search results.

User studies informing E-table interfaces

In this paper, we describe a series of user studies that were used to advance understanding of how people use electronic tables (E-tables) and inform the development of a web-based statistical table browser for use by non-specialists. Interviews and focus groups with providers, intermediaries, and non-specialist end users; transaction log analysis; and e-mail content analysis were used to develop a user-task taxonomy for government statistical data. These studies were the basis for a prototype web-based interface for browsing statistical tables. Two usability studies with 23 subjects and two eye-tracking studies with 21 subjects were conducted with this interface and paper, PDF, and spreadsheet interfaces. The results of the needs assessment, prototype development, and user studies provide a foundation for understanding E-tables in general and guiding continued design of interfaces for E-tables.

An evaluation of the use of NHS touch‐screen health kiosks: a national study

Provides the first exploratory analysis of the use of NHS touch‐screen information kiosks located throughout the UK. It examines, through a one month snapshot of the transactional logs of the kiosks, their use and makes comparisons between the types of organisation in which the kiosks are housed. Details of over 120 kiosks and nearly 47,000 user sessions and 310,000 page views were used for the comparison. The kiosks are proving popular, although some use appears to be of a cursory and seemingly unproductive kind. Surprisingly significant differences between kiosk locations were found. This early research reported here is part of an ongoing study of how users obtain health information by interacting with different digital platforms: kiosks, the Internet, and digital television. Comparisons are made between the results of this study and a similar study conducted by the authors on commercial health kiosks, those of InTouch With Health.

Co‐evolution of user and organizational interfaces: A longitudinal case study of WWW dissemination of national statistics

AbstractThe data systems, policies and procedures, corporate culture, and public face of an agency or institution make up its organizational interface. This case study describes how user interfaces for the Bureau of Labor Statistics web site evolved over a 5‐year period along with the larger organizational interface and how this co‐evolution has influenced the institution itself. Interviews with BLS staff and transaction log analysis are the foci in this analysis that also included user information‐seeking studies and user interface prototyping and testing. The results are organized into a model of organizational interface change and related to the information life cycle.

Evaluating interest profiles based on users' judgment, interest change, and class specificity in the context of filtering medical documents

AbstractQuality of profiles is critical for the performance of information filtering systems. A series of experiments were conducted to investigate factors that can impact the profile acquisition process. Profiles in MedSIFTER, the system used in the study, are represented as degrees of interests over a set of fixed classes and for this research were based on established MeSH categories. We collected data from 12 subjects, who used the system in a mode where profile acquisition is exclusively based on feedback provision. User‐provided profiles, machine‐built profiles, transaction log records and written statements of subjects were the main empirical data. Profile similarities and consistency of feedback with explicit preferences for classes were measured. Results suggested that the system captured user preferences better when these preferences involved specific classes and when the user preferences didn't change significantly. Feedback assessments were more conducive to the representation of user preferences when these preferences involved specific classes. Discrepancies between what the subjects stated, when explicitly expressing their preferences, and what they actually did, when providing feedback, pointed to key factors related to the representation and modeling of the documents and profiles.

An exploration of the effectiveness for the citizen of Web-based systems of communicating UK parliamentary and devolved assembly information

This article reports on an exploratory study, funded by the UK Economic and Social Research Council, which investigated the impact of technology on the communication of parliamentary information to the general public. This study tested the application of a new data collection tool: an interactive, electronically assisted interview that was taken out across the UK as part of a pilot roadshow to organizations such as public libraries, community centers, sheltered accommodation, and universities. Here, members of the public were invited to explore and respond to the Web sites of the UK Parliament, the Scottish Parliament, and the National Assembly for Wales. During these sessions, online activities were recorded in a data transaction log, while search behavior and verbal responses were coded in a protocol analysis approach. The article provides full details of the results of these interactive interviews. They appear to indicate that the availability of information in readily accessible electronic form is insufficient alone to encourage citizen participation. Other motivators and forms of support are required to encourage and enable people to access, use, and apply that information and to encourage them to use technology to interact with democracy.

Loglinear and multidimensional scaling models of digital library navigation

Computer transaction log analysis comprises the modeling of patterns of machine-user transactions. Due to the web-like structure of the Internet, Interactions can be represented in a three-dimensional, origin-destination-time flow/transaction matrix. Loglinear models are used to model these transactions in terms of their time, origin and destination components and to detect the signature and stability of patterns of navigation through an Internet-based digital library. Through the use of metric multidimensional scaling, transaction patterns are then visualized and interpreted. The methods are applied to a 12-month time Series of transaction logs taken from the Alexandria Digital Library, a web-based digital library of maps and other cartographic materials. Among other things, results show that transaction patterns do not change significantly despite changes in the user interface and that user training has a significant impact on people's navigation of the library.

Comparing two digital consumer health television services using transaction log analysis

Use is an important characteristic in determining the success or otherwise of any digital information service, and in making comparisons between services. The source of most use data is the server logs that record user activity on a real-time and continuous basis. There is much demand from sponsors, channel owners and marketing departments for this information. The authors evaluate the performance of use metrics, including reach, in order to make comparisons between two services and discuss the methodological problems associated with making such comparisons. The two services were: Living Health, managed by Flextech and distributed by Telewest, and NHS Direct Digital, managed by Communicopia Data and distributed by Kingston Interactive Television. The data were collected over the period August 2001 to February 2002. During this period, the two sites were visited by approximately 20 000 people who recorded more than three-quarters of a million page views.

Modeling Information Needs in Engineering Databases Using Tacit Knowledge

Online resources of engineering design information are a critical resource for practicing engineers. These online resources often contain references and content associated with technical memos, journal articles and “white papers” of prior engineering projects. However, filtering this stream of information to find the right information appropriate to an engineering issue and the engineer is a time-consuming task. The focus of this research lies in ascertaining tacit knowledge to model the information needs of the users of an engineering information system. It is proposed that the combination of reading time and the semantics of documents accessed by users reflect their tacit knowledge. By combining the computational text analysis tool of Latent Semantic Analysis with analyses of on-line user transaction logs, we introduce the technique of Latent Interest Analysis (LIA) to model information needs based on tacit knowledge. Information needs are modeled by a vector equation consisting of a linear combination of the user’s queries and prior documents downloaded, scaled by the reading time of each document to measure the degree of relevance. A validation study of the LIA model revealed a higher correlation between predicted and actual information needs for our model in comparison to models lacking scaling by reading time and a representation of the semantics of prior accessed documents. The technique was incorporated into a digital library to recommend engineering education materials to users.

Loglinear and multidimensional scaling models of digital library navigation

Computer transaction log analysis comprises the modeling of patterns of machine–user transactions. Due to the web-like structure of the Internet, Interactions can be represented in a three-dimensional, origin–destination–time flow/transaction matrix. Loglinear models are used to model these transactions in terms of their time, origin and destination components and to detect the signature and stability of patterns of navigation through an Internet-based digital library. Through the use of metric multidimensional scaling, transaction patterns are then visualized and interpreted. The methods are applied to a 12-month time series of transaction logs taken from the Alexandria Digital Library, a web-based digital library of maps and other cartographic materials. Among other things, results show that transaction patterns do not change significantly despite changes in the user interface and that user training has a significant impact on people's navigation of the library.

The impact of location on the use of information systems

Touch‐screen kiosks are situated in a variety of locations to provide the public with ready access to health information. This paper examines their use, via the transactional logs, and makes comparisons between the types of organisation in which the kiosks are housed. Twenty‐one kiosks were selected and categorised into four groups – pharmacy, hospital, information centre and surgery. A small case study features a supermarket kiosk. Details of nearly 90,000 user sessions and 750,000 page views were used for the comparison. Comparisons between sites were made in terms of number of users, their age and gender, trends over time, the number of sessions conducted, page view time, session duration, pages viewed, site penetration, number of pages printed and health topics viewed. There were considerable differences between the kiosk locations. This early research provides the quantitative foundation for a fuller study of kiosk location and the differences in perceptions of the quality/authority of kiosk data.

The characteristics of users and non-users of a digital interactive television service - case study: the Living Health channel

In the Birmingham area, 35 000 households were provided with access to the Living Health television channel for a pilot period of six months. The transactional logs showed that over the period 13 718 people used the system and that an estimated third of subscribers (34%) chose to view the channel. A questionnaire survey (n = 723) in the third month of service showed that nearly a quarter (23%) of respondents had used the service. a The analysis seeks to reveal what types of people used the service during this period and why.