Comparative User Study Research Articles

Estimating depth information of vascular models: A comparative user study between a virtual reality and a desktop application

Vascular structures are assessed, e.g., in tumor surgery to understand the influence of a planned resection on the vascular supply and venous drainage. The understanding of complex branching vascular structures may benefit from immersive virtual reality (VR) visualization. Therefore, the estimation of distance, depth and shape information is a crucial task to support diagnosis and therapy decisions. Depending on the visualization techniques used, perceptual issues can influence this process and may thus lead to false conclusions. Many studies were carried out to study depth perception for different variants of vessel visualization. However, these studies are restricted to desktop applications. Since VR exhibits specific perceptual problems, we aim at an understanding of the appropriateness of vessel visualization techniques in VR. Therefore, this paper presents a user study that investigates the effects of three commonly used visualization techniques on depth perception. The set of visualization techniques comprises Phong shading, pseudo-chromadepth and fog shading. An immersive VR setup of the study using a head-mounted display (HMD) was compared to a traditional desktop setup. Results suggest that depth judgments are less error-prone and more certain in VR than in desktop environments. Moreover, depth-enhancing visualization techniques had greater effects in the desktop study compared to the VR study.

Applied comparison of large\u2010scale propensity score matching and cardinality matching for causal inference in observational research

BackgroundCardinality matching (CM), a novel matching technique, finds the largest matched sample meeting prespecified balance criteria thereby overcoming limitations of propensity score matching (PSM) associated with limited covariate overlap, which are especially pronounced in studies with small sample sizes. The current study proposes a framework for large-scale CM (LS-CM); and compares large-scale PSM (LS-PSM) and LS-CM in terms of post-match sample size, covariate balance and residual confounding at progressively smaller sample sizes.MethodsEvaluation of LS-PSM and LS-CM within a comparative cohort study of new users of angiotensin-converting enzyme inhibitor (ACEI) and thiazide or thiazide-like diuretic monotherapy identified from a U.S. insurance claims database. Candidate covariates included patient demographics, and all observed prior conditions, drug exposures and procedures. Propensity scores were calculated using LASSO regression, and candidate covariates with non-zero beta coefficients in the propensity model were defined as matching covariates for use in LS-CM. One-to-one matching was performed using progressively tighter parameter settings. Covariate balance was assessed using standardized mean differences. Hazard ratios for negative control outcomes perceived as unassociated with treatment (i.e., true hazard ratio of 1) were estimated using unconditional Cox models. Residual confounding was assessed using the expected systematic error of the empirical null distribution of negative control effect estimates compared to the ground truth. To simulate diverse research conditions, analyses were repeated within 10 %, 1 and 0.5 % subsample groups with increasingly limited covariate overlap.ResultsA total of 172,117 patients (ACEI: 129,078; thiazide: 43,039) met the study criteria. As compared to LS-PSM, LS-CM was associated with increased sample retention. Although LS-PSM achieved balance across all matching covariates within the full study population, substantial matching covariate imbalance was observed within the 1 and 0.5 % subsample groups. Meanwhile, LS-CM achieved matching covariate balance across all analyses. LS-PSM was associated with better candidate covariate balance within the full study population. Otherwise, both matching techniques achieved comparable candidate covariate balance and expected systematic error.ConclusionsLS-CM found the largest matched sample meeting prespecified balance criteria while achieving comparable candidate covariate balance and residual confounding. We recommend LS-CM as an alternative to LS-PSM in studies with small sample sizes or limited covariate overlap.

Synthesizing Indoor Scene Layouts in Complicated Architecture Using Dynamic Convolution Networks

Synthesizing indoor scene layouts is challenging and critical, especially for digital design and gaming entertainment. Although there has been significant research on the indoor layout synthesis of rectangular-shaped or L-shaped architecture, there is little known about synthesizing plausible layouts for more complicated indoor architecture with both geometric and semantic information of indoor architecture being fully considered. In this paper, we propose an effective and novel framework to synthesize plausible indoor layouts in various and complicated architecture. The given indoor architecture is first encoded to our proposed representation, called InAiR, based on its geometric and semantic information. The indoor objects are grouped and then arranged by functional blocks, represented by oriented bounding boxes, using dynamic convolution networks based on their functionality and human activities. Through comparisons with other approaches as well as comparative user studies, we find that our generated indoor scene layouts for diverse, complicated indoor architecture are visually indistinguishable, which reach state-of-the-art performance.

The Vicarios Virtual Reality Interface for Remote Robotic Teleoperation

Intuitive interaction is the cornerstone of accurate and effective performance in remote robotic teleoperation. It requires high-fidelity in control actions as well as perception (vision, haptic, and other sensory feedback) of the remote environment. This paper presents Vicarios, a Virtual Reality (VR) based interface with the aim of facilitating intuitive real-time remote teleoperation, while utilizing the inherent benefits of VR, including immersive visualization, freedom of user viewpoint selection, and fluidity of interaction through natural action interfaces. Vicarios aims to enhance the situational awareness, using the concept of viewpoint-independent mapping between the operator and the remote scene, thereby giving the operator better control in the perception-action loop. The article describes the overall system of Vicarios, with its software, hardware, and communication framework. A comparative user study quantifies the impact of the interface and its features, including immersion and instantaneous user viewpoint changes, termed “teleporting”, on users’ performance. The results show that users’ performance with the VR-based interface was either similar to or better than the baseline condition of traditional stereo video feedback, approving the realistic nature of the Vicarios interface. Furthermore, including the teleporting feature in VR significantly improved participants’ performance and their appreciation for it, which was evident in the post-questionnaire results. Vicarios capitalizes on the intuitiveness and flexibility of VR to improve accuracy in remote teleoperation.

Communicative functions of emoji sequences in the context of self-presentation: A comparative study of Weibo and Twitter users

Focusing on Weibo (a Chinese Microblogging site) and Twitter, this study adopts computer-mediated discourse analysis to examine how influencers use emoji sequences when engaging in self-presentation. It identified a variety of text-based speech acts, emoji functions, and functional relations by conducting speech act and pragmatic function analysis. ‘Claim’ is the most common text-based speech act accompanying with emoji sequences in both data groups; however, the former had a higher percentage than the later. Moreover, emoji functioning as a combination of ‘stance and action’ in sequences comprise the most prominent category in the Twitter data, whereas the ‘concept’ function accounts for the largest percentage within the Weibo data. Finally, emoji sequences serving as ‘emphasis on text’ is most employed in connection with accompanying texts in both data. This study also observed how a lack of user-desired emoji may be compensated for through collocation of emoji sequences, a device which involves significant creativity and intricacy.

Comparing Alternative Route Planning Techniques: A Comparative User Study on Melbourne, Dhaka and Copenhagen Road Networks

Many modern navigation systems and map-based services do not only provide the fastest route from a source location s to a target location t but also provide a few alternative routes to the users as more options to choose from. Consequently, computing alternative paths has received significant research attention. However, it is unclear which of the existing approaches generates alternative routes of better quality because the quality of these alternatives is mostly subjective. Motivated by this, in this paper, we present a user study conducted on the road networks of Melbourne, Dhaka and Copenhagen that compares the quality (as perceived by the users) of the alternative routes generated by four of the most popular existing approaches including the routes provided by Google Maps. We also present a web-based demo system that can be accessed using any internet-enabled device and allows users to see the alternative routes generated by the four approaches for any pair of selected source and target. We report the average ratings received by the four approaches and our statistical analysis shows that there is no credible evidence that the four approaches receive different ratings on average. We also discuss the limitations of this user study and recommend the readers to interpret these results with caution because certain factors may have affected the participants' ratings.

Interactive Optimization of Generative Image Modelling using Sequential Subspace Search and Content‐based Guidance

AbstractGenerative image modeling techniques such as GAN demonstrate highly convincing image generation result. However, user interaction is often necessary to obtain desired results. Existing attempts add interactivity but require either tailored architectures or extra data. We present a human‐in‐the‐optimization method that allows users to directly explore and search the latent vector space of generative image modelling. Our system provides multiple candidates by sampling the latent vector space, and the user selects the best blending weights within the subspace using multiple sliders. In addition, the user can express their intention through image editing tools. The system samples latent vectors based on inputs and presents new candidates to the user iteratively. An advantage of our formulation is that one can apply our method to arbitrary pre‐trained model without developing specialized architecture or data. We demonstrate our method with various generative image modelling applications, and show superior performance in a comparative user study with prior art iGAN [ZKSE16].

Developing a scenario-based video game generation framework for computer and virtual reality environments: a comparative usability study

Serious games—games that have additional purposes rather than only entertainment—aim to educate people, solve, and plan several real-life tasks and circumstances in an interactive, efficient, and user-friendly way. Emergency training and planning provide structured curricula, rule-based action items, and interdisciplinary collaborative entities to imitate and teach real-life tasks. This rule-based structure enables the curricula to be transferred into other systematic learning platforms. Although emergency training includes these highly structured and repetitive action responses, a general framework to map the training scenarios’ actions, roles, and collaborative structures to serious games’ game mechanics and game dialogues, is still not available. To address this issue, in this study, a scenario-based game generator, which maps domain-oriented tasks to game rules and game mechanics, was developed. Also, two serious games (i.e., Hospital game and BioGarden game) addressing the training mechanisms of Chemical, Biological, Radiological, Nuclear, and Explosives (CBRNe) domain, were developed by both the game developers and the scenario-based game generator for comparative analysis. Finally, the outcomes of these games were mapped to the virtual reality environment to provide a thorough training program. To test the usability, immersion, presence, and technology acceptance aspects of the proposed game generator’s outcomes, 15 game developer participants tested a complete set of games and answered the questionnaires of the corresponding phenomenon. The results show that although the game generator has higher CPU time and memory usage, it highly outperforms the game development pipeline performance of the game developers and provides usable and immersive games. Thus, this study provides a promising game generator which bridges the CBRNe practitioners and game developers to transform real-life training scenarios into video games efficiently and quickly.

A Comparative Usability Study of Bare Hand Three-Dimensional Object Selection Techniques in Virtual Environment

Object selection is the basis of natural user–computer interaction (NUI) in a virtual environment (VE). Among the three-dimensional object selection techniques employed in virtual reality (VR), bare hand-based finger clicking interaction and ray-casting are two convenient approaches with a high level of acceptance. This study involved 14 participants, constructed a virtual laboratory environment in VR, and compared the above two finger-based interaction techniques in terms of aspects of the task performance, including the success rate, total reaction time, operational deviation, and accuracy, at different spatial positions. The results indicated that the applicable distance range of finger clicking interaction and finger ray-casting was 0.2 to 1.4 m and over 0.4 m, respectively. Within the shared applicable distance, the finger clicking interaction achieved a shorter total reaction time and higher clicking accuracy. The performance of finger clicking interaction varied remarkably at the center and edge of the horizontal field of view, while no significant difference was found among ray-casting at various horizontal azimuths. The current findings could be directly applied to the application of bare-hand interaction in VR environments.

Using Augmented Reality Application to Reduce Time Completion and Error Rate in PC Assembly

In the present context of globalization, the demand for assembly skill has increased and play an essential role in today’s industry. The traditional assembly instruction, face-to-face and manual instruction, may contain unimportant information that can result in misinterpretation, which in turn may increase the number of error and takes longer time to complete the task. A new technology (AR) claims to increase the efficiency of assembly task by directly visualizing computer generated 3D information in the real environment. Therefore, this study aims to determine the impact of AR on the time of task completion and the number of error made during the assembly task. The comparative user study was quantitative involving 18 users divided into either AR group or traditional group performing a pc assembly task. Statistical analysis revealed that the time of completion and error rate for two different group is statistically significant. The findings showed that the use of AR application has resulted in decreasing the number of error made and shorten the time to complete the task than the traditional instructional manual in assemble a pc. Considering these result, it can conclude that augmented reality application is an effective and beneficial tool to be applied in assembly and education.

PolyFit

Raster clip-art images, which consist of distinctly colored regions separated by sharp boundaries typically allow for a clear mental vector interpretation. Converting these images into vector format can facilitate compact lossless storage and enable numerous processing operations. Despite recent progress, existing vectorization methods that target such data frequently produce vectorizations that fail to meet viewer expectations. We present PolyFit , a new clip-art vectorization method that produces vectorizations well aligned with human preferences. Since segmentation of such inputs into regions had been addressed successfully, we specifically focus on fitting piecewise smooth vector curves to the raster input region boundaries, a task prior methods are particularly prone to fail on. While perceptual studies suggest the criteria humans are likely to use during mental boundary vectorization, they provide no guidance as to the exact interaction between them; learning these interactions directly is problematic due to the large size of the solution space. To obtain the desired solution, we first approximate the raster region boundaries with coarse intermediate polygons leveraging a combination of perceptual cues with observations from studies of human preferences. We then use these intermediate polygons as auxiliary inputs for computing piecewise smooth vectorizations of raster inputs. We define a finite set of potential polygon to curve primitive maps, and learn the mapping from the polygons to their best fitting primitive configurations from human annotations, arriving at a compact set of local raster and polygon properties whose combinations reliably predict human-expected primitive choices. We use these primitives to obtain a final globally consistent spline vectorization. Extensive comparative user studies show that our method outperforms state-of-the-art approaches on a wide range of data, where our results are preferred three times as often as those of the closest competitor across multiple types of inputs with various resolutions.

A comparative user study of visualization techniques for cluster analysis of multidimensional data sets

This article presents an empirical user study that compares eight multidimensional projection techniques for supporting the estimation of the number of clusters, [Formula: see text], embedded in six multidimensional data sets. The selection of the techniques was based on their intended design, or use, for visually encoding data structures, that is, neighborhood relations between data points or groups of data points in a data set. Concretely, we study: the difference between the estimates of [Formula: see text] as given by participants when using different multidimensional projections; the accuracy of user estimations with respect to the number of labels in the data sets; the perceived usability of each multidimensional projection; whether user estimates disagree with [Formula: see text] values given by a set of cluster quality measures; and whether there is a difference between experienced and novice users in terms of estimates and perceived usability. The results show that: dendrograms (from Ward’s hierarchical clustering) are likely to lead to estimates of [Formula: see text] that are different from those given with other multidimensional projections, while Star Coordinates and Radial Visualizations are likely to lead to similar estimates; t-Stochastic Neighbor Embedding is likely to lead to estimates which are closer to the number of labels in a data set; cluster quality measures are likely to produce estimates which are different from those given by users using Ward and t-Stochastic Neighbor Embedding; U-Matrices and reachability plots will likely have a low perceived usability; and there is no statistically significant difference between the answers of experienced and novice users. Moreover, as data dimensionality increases, cluster quality measures are likely to produce estimates which are different from those perceived by users using any of the assessed multidimensional projections. It is also apparent that the inherent complexity of a data set, as well as the capability of each visual technique to disclose such complexity, has an influence on the perceived usability.

Reconstructing Reflection Maps Using a Stacked-CNN for Mixed Reality Rendering.

Corresponding lighting and reflectance between real and virtual objects is important for spatial presence in augmented and mixed reality (AR and MR) applications. We present a method to reconstruct real-world environmental lighting, encoded as a reflection map (RM), from a conventional photograph. To achieve this, we propose a stacked convolutional neural network (SCNN) that predicts high dynamic range (HDR) 360 ° RMs with varying roughness from a limited field of view, low dynamic range photograph. The SCNN is progressively trained from high to low roughness to predict RMs at varying roughness levels, where each roughness level corresponds to a virtual object's roughness (from diffuse to glossy) for rendering. The predicted RM provides high-fidelity rendering of virtual objects to match with the background photograph. We illustrate the use of our method with indoor and outdoor scenes trained on separate indoor/outdoor SCNNs showing plausible rendering and composition of virtual objects in AR/MR. We show that our method has improved quality over previous methods with a comparative user study and error metrics.

Evaluating the user experience of omnidirectional VR walking simulators

Omnidirectional treadmills (ODTs) have been traditionally proposed as a promising solution for users’ navigation in large-scale virtual environments. These mechanical devices enable users to perform locomotive motion with 360-deg freedom, while keeping their position fixed in the physical world. However, most locomotion approaches based on omnidirectional treadmills have presented either high acquisition or maintenance costs, being the capabilities of the general public, or a limited reliability. In this paper, we present a comparative usability and acceptance study with real users evaluating the two most common approaches for the development of this type of Virtual Reality (VR) walking simulators, whether flat-based or bowl-based omnidirectional treadmills. The results of our study indicate that both design alternatives accomplish a similar performance and can be considered as personal, low-cost walking simulators for navigation in virtual environments. Nevertheless, when compared between themselves, the users preferred the bowl-based omnidirectional VR treadmill when following three different selection criteria: overall preference, recommendation, and usefulness. Last, a correlation analysis showed statically significant correspondences among the parameters concerning to the simulation presence, which in turn, are related to the global scores and responses given by the participants of the Virtual Reality experiment.

A definition-by-example approach and visual language for activity patterns in engineering disciplines.

Modeling tools are well established in software development. A model is the result of a series of modeling activities. The ability to recognize when a user is working on a certain modeling activity opens up a range of possibilities for context-sensitive support. One possible way to support the user is offering the auto-completion of the current task. The recognition of modeling activities is typically carried out by matching event patterns against events emitted by a user’s editing operations. A user that intends to add or customize auto-completions must be able to easily understand and create activity definitions. However, defining the currently required complex event patterns is a challenging and error-prone task even for a person with an intensive knowledge of event-processing languages. In this paper, we propose the visual definition language VisPaRec accompanied by a method that allows creating activity definitions in a semi-automated and graphical way. We evaluate our visual definition language in a comparative user study against the generic event-processing language Rapide. We found that the proposed visual representation increases comprehensibility while reducing time for constructing and modifying activity definitions significantly.

A Deep Learning-Based Model for Head and Eye Motion Generation in Three-party Conversations

In this paper we propose a novel deep learning based approach to generate realistic three-party head and eye motions based on novel acoustic speech input together with speaker marking (i.e., speaking time for each interlocutor). Specifically, we first acquire a high quality, three-party conversational motion dataset. Then, based on the acquired dataset, we train a deep learning based framework to automatically predict the dynamic directions of both the eyes and heads of all the interlocutors based on speech signal input. Via the combination of existing lip-sync and speech-driven hand/body gesture generation algorithms, we can generate realistic three-party conversational animations. Through many experiments and comparative user studies, we demonstrate that our approach can generate realistic three-party head-and-eye motions based on novel speech recorded on new subjects with different genders and ethnicities.

SurgeryCuts: Embedding Additional Information in Maps without Occluding Features

AbstractVisualizing contextual information to a map often comes at the expense of overplotting issues. Especially for use cases with relevant map features in the immediate vicinity of an information to add, occlusion of the relevant map context should be avoided. We present SurgeryCuts, a map manipulation technique for the creation of additional canvas area for contextual visualizations on maps. SurgeryCuts is occlusion‐free and does not shift, zoom or alter the map viewport. Instead, relevant parts of the map can be cut apart. The affected area is controlledly distorted using a parameterizable warping function fading out the map distortion depending on the distance to the cut. We define extended metrics for our approach and compare to related approaches. As well, we demonstrate the applicability of our approach at the example of tangible use cases and a comparative user study.

Comparing Human Factors for Augmented Reality Supported Single-User and Collaborative Repair Operations of Industrial Robots.

In order to support the decision-making process of industry on how to implement Augmented Reality (AR) in production, this article wants to provide guidance through a set of comparative user studies. The results are obtained from the feedback of 160 participants who performed the same repair task on a switch cabinet of an industrial robot. The studies compare several AR instruction applications on different display devices (head-mounted display, handheld tablet PC and projection-based spatial AR) with baseline conditions (paper instructions and phone support), both in a single-user and a collaborative setting. Next to insights on the performance of the individual device types for the single mode operation, the study is able to show significant indications on AR techniques are being especially helpful in a collaborative setting.

A Deep Learning-Based Framework for Intersectional Traffic Simulation and Editing.

Most of existing traffic simulation methods have been focused on simulating vehicles on freeways or city-scale urban networks. However, relatively little research has been done to simulate intersectional traffic to date despite its broad potential applications. In this paper, we propose a novel deep learning-based framework to simulate and edit intersectional traffic. Specifically, based on an in-house collected intersectional traffic dataset, we employ the combination of convolution network (CNN) and recurrent network (RNN) to learn the patterns of vehicle trajectories in intersectional traffic. Besides simulating novel intersectional traffic, our method can be used to edit existing intersectional traffic. Through many experiments as well as comparative user studies, we demonstrate that the results by our method are visually indistinguishable from ground truth, and our method can outperform existing methods.

An Exploration of a Website Design Process and Results of a Usability Study

A transition of content management systems provided Kornhauser Health Sciences Library with an opportunity to redesign the library’s website to be more user-friendly, efficient, and visually appealing. A multistage approach was taken: (1) informal interviews of various stakeholders from the library, (2) a redesign using information gained from those stakeholders, and (3) a retrospective comparative usability study. This study was conducted with the goal to inform library staff whether the redesigned website improved accuracy and efficiency of information retrieval through the completion of timed tasks. In addition, user satisfaction was measured through guided interview questions.