Previous User Studies Research Articles

Determining Key Parameters with Data-Assisted Analysis of Conditionally Automated Driving

In conditionally automated driving, a vehicle issues a take-over request when it reaches the functional limits of self-driving, and the driver must take control. The key driving parameters affecting the quality of the take-over (TO) process have yet to be determined and are the motivation for our work. To determine these parameters, we used a dataset of 41 driving and non-driving parameters from a previous user study with 216 TOs while performing a non-driving-related task on a handheld device in a driving simulator. Eight take-over quality aspects, grouped into pre-TO predictors (attention), during-TO predictors (reaction time, solution suitability), and safety performance (off-road drive, braking, lateral acceleration, time to collision, success), were modeled using multiple linear regression, support vector machines, M5’, 1R, logistic regression, and J48. We interpreted the best-suited models by highlighting the most influential parameters that affect the overall quality of a TO. The results show that these are primarily maximal acceleration (88.6% accurate prediction of collisions) and the TOR-to-first-brake interval. Gradual braking, neither too hard nor too soft, as fast as possible seems to be the strategy that maximizes the overall TO quality. The position of the handheld device and the way it was held prior to TO did not affect TO quality. However, handling the device during TO did affect driver attention when shorter attention times were observed and drivers held their mobile phones in only one hand. In the future, automatic gradual braking maneuvers could be considered instead of immediate full TOs.

Sixteen Years of Phishing User Studies: What Have We Learned?

Several previous studies have investigated user susceptibility to phishing attacks. A thorough meta-analysis or systematic review is required to gain a better understanding of these findings and to assess the strength of evidence for phishing susceptibility of a subpopulation, e.g., older users. We aim to determine whether an effect exists; another aim is to determine whether the effect is positive or negative and to obtain a single summary estimate of the effect. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">OBJECTIVES:</i> We systematically review the results of previous user studies on phishing susceptibility and conduct a meta-analysis. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">METHOD:</i> We searched four online databases for English studies on phishing. We included all user studies in phishing detection and prevention, whether they proposed new training techniques or analyzed users’ vulnerability. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">FINDINGS:</i> A careful analysis reveals some discrepancies between the findings. More than half of the studies that analyzed the effect of <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">age</i> reported no statistically significant relationship between age and users’ performance. Some studies reported older people performed better while some reported the opposite. A similar finding holds for the gender difference. The meta-analysis shows: 1) a significant relationship between participants’ age and their susceptibility 2) females are more susceptible than males 3) users training significantly improves their detection ability.

An Explanatory Study on User Behavior in Discovering Aggregated Multimedia Web Content

The recent advancements in the web allow users to generate multimedia content, resulting in multimedia information proliferation. Existing search engines provide access to multimedia content via a disjoint assembly of media-specific results called verticals. However, this decentralized assembly of media contents requires manual aggregation and synthesizing efforts at the user&#x2019;s end, hindering the information exploration process and subsequently may cause cognitive overload, hence, demanding innovative tools to discover multimedia content. The researchers have devised numerous state-of-the-art approaches; however, analysis to confirm the efficacy has little emphasis. This study investigates users&#x2019; complex multimedia information-seeking behavior over state-of-the-art web search systems to unveil the user&#x2019;s information-seeking issues. Our research employs between-subjects study and post hoc analysis strategies to analyze participants&#x2019; information-seeking characteristics. The study design adopted statistical hypothesis testing to consolidate previous user behavioral studies, confirm existing strategies, and present recommended practices for future general-purpose web search engines. The participants were assigned Google and an advanced discovery search system using the same multimedia dataset to ensure the obtained results&#x2019; credibility. The primary behavioral parameters include search efforts, multimedia content exploration, search user interface (SUI), information management and presentation, and user cognition. This study uncovers several inadequacies of the search engines in meeting users&#x2019; complex discovery needs, including 29.6% less user engagement, 43% system and searching dissatisfaction, and 32% less knowledge acquisition with 63.9% increased clicking effort on traditional search engines. The results confirmed previous user studies and suggest novel research recommendations statistically significant in multimedia information exploration-related endeavors.

A Systematic Review of User Acceptance in Industrial Augmented Reality

In the industrial work context, Augmented Reality (AR) can support work processes and employees’ cognitive relief through the location-specific and context-related superimposition of real objects with virtual information. The AR-based support of industrial work processes ranges over product development, manufacturing, assembly, maintenance, and training. In all these areas, numerous location-based AR support functions are being prototypically implemented, aiming to improve work efficiency, communication in mobile work situations, or employee qualification in the work process. In contrast to the increasing number of developed AR solutions in recent years, there is no widespread use of these solutions in industrial practice. AR systems’ successful introduction is closely related to user acceptance, which has not been comprehensively considered over the system development process. In addition to improving AR hardware ergonomic features, usability or user interface design play an essential role in user acceptance. Particularly in the context of employee qualification, increasing employee engagement can be named as a success factor. Previous user studies of industrial AR systems only include individual user acceptance aspects. The use of game elements has not been widely addressed in connection with manual tasks in production environments, including AR-based assistance systems. This paper aims to examine user acceptance of industrial AR systems and the relevant factors for investigating user acceptance, e.g., ease of use or enjoyment, based on a systematic literature review. An analysis of existing review articles on industrial AR systems elaborates the current state of the art and identifies the research gap. This review of 109 scientific articles from 2011 to 2020 provides an overview of the current state of research on the inclusion of user acceptance in industrial AR systems. The identified papers from the scientific databases, Scopus, Web of Science, IEEE Xplore Digital Library, ACM Digital Library, and Science Direct, are evaluated for their relevance and selected for further analysis based on inclusion and exclusion criteria, e.g., year of publication. This review presents the current challenges regarding user acceptance of industrial AR systems and future possibilities for the comprehensive integration of user acceptance factors into the development, evaluation, and implementation process.

3D Reconstructions as Research Hubs: Geospatial Interfaces for Real-Time Data Exploration of Seventeenth-Century Amsterdam Domestic Interiors

Abstract This paper presents our ongoing work in the Virtual Interiors project, which aims to develop 3D reconstructions as geospatial interfaces to structure and explore historical data of seventeenth-century Amsterdam. We take the reconstruction of the entrance hall of the house of the patrician Pieter de Graeff (1638–1707) as our case study and use it to illustrate the iterative process of knowledge creation, sharing, and discovery that unfolds while creating, exploring and experiencing the 3D models in a prototype research environment. During this work, an interdisciplinary dataset was collected, various metadata and paradata were created to document both the sources and the reasoning process, and rich contextual links were added. These data were used as the basis for creating a user interface for an online research environment, taking design principles and previous user studies into account. Knowledge is shared by visualizing the 3D reconstructions along with the related complexities and uncertainties, while the integration of various underlying data and Linked Data makes it possible to discover contextual knowledge by exploring associated resources. Moreover, we outline how users of the research environment can add annotations and rearrange objects in the scene, facilitating further knowledge discovery and creation.

Mixed Assessment of Virtual Serious Games Applied in Architectural and Urban Design Education.

The creation and usage of serious games on virtual reality (VR) and/or interactive platforms for the teaching of architecture, construction, urban planning, and other derived areas, such as security and risk prevention, require design processes, studies, and research that lead to further consolidation expansion. In that sense, this paper presents two main aims developed: the improvement of a virtual navigation system through the results of previous user studies and mixed research (quantitative and qualitative) improved based on the user perception for educational and professional uses. The VR system used is based on Unreal Engine programming of the HTC Vive sensor. This study is related to the GAME4City 3.0 and a broader project focused on gamified visualization and its educational uses in architectural and urban projects. The results reflect great interest, good usability, and high motivation for further usage for all types of users. However, an apparent resistance to deepen its use continues to be perceived in academia. Based on the research results, weak points of educational gamified systems have been identified, and the main differences and needs in user profiles’ function. With these data, progress regarding implementing this kind of system at the teaching and professional levels must be pursued.

Semi-Autonomous Robotic Arm Reaching With Hybrid Gaze-Brain Machine Interface.

Recent developments in the non-muscular human–robot interface (HRI) and shared control strategies have shown potential for controlling the assistive robotic arm by people with no residual movement or muscular activity in upper limbs. However, most non-muscular HRIs only produce discrete-valued commands, resulting in non-intuitive and less effective control of the dexterous assistive robotic arm. Furthermore, the user commands and the robot autonomy commands usually switch in the shared control strategies of such applications. This characteristic has been found to yield a reduced sense of agency as well as frustration for the user according to previous user studies. In this study, we firstly propose an intuitive and easy-to-learn-and-use hybrid HRI by combing the Brain–machine interface (BMI) and the gaze-tracking interface. For the proposed hybrid gaze-BMI, the continuous modulation of the movement speed via the motor intention occurs seamlessly and simultaneously to the unconstrained movement direction control with the gaze signals. We then propose a shared control paradigm that always combines user input and the autonomy with the dynamic combination regulation. The proposed hybrid gaze-BMI and shared control paradigm were validated for a robotic arm reaching task performed with healthy subjects. All the users were able to employ the hybrid gaze-BMI for moving the end-effector sequentially to reach the target across the horizontal plane while also avoiding collisions with obstacles. The shared control paradigm maintained as much volitional control as possible, while providing the assistance for the most difficult parts of the task. The presented semi-autonomous robotic system yielded continuous, smooth, and collision-free motion trajectories for the end effector approaching the target. Compared to a system without assistances from robot autonomy, it significantly reduces the rate of failure as well as the time and effort spent by the user to complete the tasks.

Non-Uniform Information Access in Collaborative Information Retrieval

Although there has been a great deal of research into Collaborative Information Retrieval/Seeking (CIR/S), the majority has assumed that team members in a collaborative search team always have the same level of unrestricted access to information. However, observations from different domains such as healthcare, business, government, etc. have suggested that collaboration sometimes involves people with differing levels of access to information. This type of scenario has been referred to as Multi-Level Collaborative Information Retrieval (MLCIR). As well as a support for collaboration, MLCIR must consider information flow, security and shareability between collaborators. Therefore, existing CIR/S interface components and concepts (e.g. awareness, division of labour, persistence, etc.) may not be directly applicable to MLCIR. To the best of our knowledge, there has not been any systematic evaluations to investigate the impact of MLCIR on search outcomes and to provide user-centred design recommendations for MLCIR interfaces. To address this gap, a number of studies were conducted using real-world MLCIR examples as scenarios. This thesis highlights four research objectives; these are: 1) to investigate the impact of different collaborative search strategies and MLCIR scenarios on search outcomes using a user simulation approach, 2) to develop a preliminary MLCIR interface and investigate the impact of MLCIR scenarios on search outcomes using a user study, 3) to investigate the impact of existing CIR concepts on MLCIR search outcomes using a user study, and 4) to develop and evaluate a working prototype of an MLCIR interface that supports users for asynchronous collaboration. These investigations followed an iterative approach until a final MLCIR interface could be produced. The first study was a user simulated study which allowed us to test a large number of variables and to pick prominent ones for further user studies. It was found that when relevance feedback search strategies were used, search performance of non-full access (where team members had 90% access to collection) outperformed full access (where team members had 100% access to collection). This simulated study also fulfilled the first research objective. Next, a preliminary user study was conducted using variables selected based on the findings of the simulated study. This study uncovered a number of interface components, and awareness concepts such as query awareness, result awareness and team awareness that could be used to support users in MLCIR scenarios. This user study addressed the second research objective. Based on findings from this user study, two further user studies were conducted where we evaluated a number of interface components implemented to support query awareness, result awareness and team awareness. Results suggested that having access to query properties and result properties in MLCIR scenarios can leverage team members? search performance. These two user studies addressed the third research objective. Finally, a working MLCIR interface was built based on the findings elicited from our previous user studies. The interface was then evaluated using the web and non-uniform information access scenarios. The results showed that blacklisting search terms had a significant negative impact on search outcome than blacklisting documents (f-measure: 0.04 vs 0.197). This study fulfilled the final research objective. All five studies presented in this thesis provided us with an increasing understanding of how to develop an MLCIR interface that is usable and helpful for users. As a final product, a working MLCIR interface was produced which could provide collaborating users with useful awareness information without disclosing any sensitive data. Overall, this thesis provided important new design suggestions and showed reusability of CIR components for MLCIR interfaces, something that has never been demonstrated before. The interfaces, experimental designs and findings including design suggestions that are presented in this thesis serve as novel guidelines for further investigations into MLCIR.

Targeted Search Engines for Children

Children are a fast growing user group on the Internet. Among different online activities, children use web search engines in order to gather information related to their personal interests and school activities. Children's knowledge, cognitive abilities and fine motor skills are different from those of adults. Therefore, they may experience difficulties with search engines that are built using standard information retrieval algorithms and search interfaces for adults. Special or targeted search engines for children are essential in order to better support children in their search tasks. Therefore, the goal of this thesis is to design appropriate search engines for children with a focus on the search user interface. However, this is not an easy task to accomplish. Not only are children's abilities different from the abilities of adults, children also undergo relatively fast changes in their abilities. The specific and dynamically changing characteristics of young users pose a great challenge. In order to address this challenge, first, the specifics of information retrieval for young users are analysed. Second, open issues are identified in user studies with children using logfile analysis and eye-tracking. The conceptual challenges in the design of user interfaces regarding search engines for children are derived based on the findings of one's own and previous user studies as well as theories of human development. Third, user interfaces of search engines that address these conceptual challenges are designed, prototypically implemented and evaluated in user studies with children following a user-centered design. Specifically, the proposed user interfaces of the search engine address the changing characteristics of the users by providing a means of adaptation. Furthermore, a novel type of search result visualisation for children with cartoon style characters is developed which takes the children's preference for visual information into account. Both approaches were very positively received by children during evaluation. Children rated different user interface aspects of the search user interface prototypes as good, e.g., the adaptation of the search user interface towards user wishes and the helpfulness of the cartoon style characters during search. Finally, this thesis provides criteria and guidelines on how to design user interfaces of search engines for children.

Using Facebook out of habit

This article investigates the uses and gratifications of the popular social networking site Facebook. In the exploratory stage, 70 users generated phrases to describe the manner they used Facebook. Interestingly, some users not only described the uses, but also mentioned how they perceive these uses. These phrases were coded into 14 items and clustered into four factors. The principal component analysis that was conducted in the third stage of the study, which was addressed to 222 Facebook users, verified the validity of the four factors: Social Connection, Social Network Surfing, Wasting Time and Using Applications. Previous user studies on Facebook have examined the immediate social effects of this popular social networking site, but they have not regarded emerging uses of the platform, such as gaming and applications, which do have a social component as a feature and not as a core principle. The ‘Wasting Time’ factor and the growth of ‘Using Applications’ factor indicate that Facebook has already become an integral part of daily computing routine, alongside with the rest of the entertainment desktop and web applications.

Specifics of information retrieval for young users: A survey

In this paper, we present the state of the art in the field of information retrieval that is relevant for understanding how to design information retrieval systems for children. We describe basic theories of human development to explain the specifics of young users, i.e., their cognitive skills, fine motor skills, knowledge, memory and emotional states in so far as they differ from those of adults. We derive the implications these differences have on the design of information retrieval systems for children. Furthermore, we summarize the main findings about children’s search behavior from multiple user studies. These findings are important to understand children’s information needs, their search strategies and usage of information retrieval systems. We also identify several weaknesses of previous user studies about children’s information-seeking behavior. Guided by the findings of these user studies, we describe challenges for the design of information retrieval systems for young users. We give an overview of algorithms and user interface concepts. We also describe existing information retrieval systems for children, in specific web search engines and digital libraries. We conclude with a discussion of open issues and directions for further research. The survey provided in this paper is important both for designers of information retrieval systems for young users as well as for researchers who start working in this field.

Term Familiarity to indicate Perceived and Actual Difficulty of Text in Medical Digital Libraries.

With increasing text digitization, digital libraries can personalize materials for individuals with different education levels and language skills. To this end, documents need meta-information describing their difficulty level. Previous attempts at such labeling used readability formulas but the formulas have not been validated with modern texts and their outcome is seldom associated with actual difficulty. We focus on medical texts and are developing new, evidence-based meta-tags that are associated with perceived and actual text difficulty. This work describes a first tag, term familiarity , which is based on term frequency in the Google corpus. We evaluated its feasibility to serve as a tag by looking at a document corpus (N=1,073) and found that terms in blogs or journal articles displayed unexpected but significantly different scores. Term familiarity was then applied to texts and results from a previous user study (N=86) and could better explain differences for perceived and actual difficulty.

Socially Consistent Characters in Player-Specific Stories

In the context of interactive, virtual experiences, the use of personality models to maintain consistent character behaviour is becoming more widespread in both industry and academia. Most current techniques, however, are limited in one of three ways: either they overly restrict user actions, have a high cost for creating varied content, or rely on a representation that prohibits conveying complex content to the user. Toward addressing these issues, we introduce Socially Consistent Role Passing, a mechanism for ensuring consistent character behaviour that leverages the design of PaSSAGE, an existing system for generating adaptive, interactive stories. While results from previous human user studies have shown that PaSSAGE improves the enjoyment of players with little gaming experience, we present results from a new study showing that PaSSAGE's adaptive stories, augmented with Socially Consistent Role Passing, improve the enjoyment of all players versus a set of fixed-structure alternatives.

More Appropriate Information Systems and Services for the Social Scientist: Time to Put Our Findings to Work

Objective - The study reported in this article was conceived in order to answer a question of very large scope: What are the information systems and services requirements of social scientists? Inherent in this question was the correlative question: How do social scientists tend to use such systems and services, and what resources

Assisting the searcher: utilizing software agents for Web search systems

Much previous research on improving information retrieval applications has focused on developing entirely new systems with advanced searching features. Unfortunately, most users seldom utilize these advanced features. This research explores the use of a software agent that assists the user during the search process. The agent was developed as a separate, stand‐alone component to be integrated with existing information retrieval systems. The performance of an information retrieval system with the integrated agent was subjected to an evaluation with 30 test subjects. The results indicate that agents developed using both results from previous user studies and rapidly modeling user information needs can result in an improvement in precision. Implications for information retrieval system design and directions for future research are outlined.