Marking Menus Research Articles

Breadth and orientation of pie menus for mid-air interaction: effects on upper extremity biomechanics, performance, and subjective assessment

ABSTRACT The number and layout of options within pie menus are crucial for determining the direction of radial swiping in mid-air interactions, influencing the physical demands imposed on users. This research examined six distinct option quantities, referred to as menu breadth, and investigated two layouts, designated as menu orientations. These orientations encompass landscape and portrait formats, with the former positioning a top option centred parallel to the horizontal axis, while the latter situates two top options on either side of the vertical axis. The study employed comprehensive metrics, including muscle activity, wrist movement, user performance, and subjective assessments. Our results indicated that the breadth should be controlled to no more than six to ensure high efficiency and align with users’ interactive experience, while breadth 2 should be avoided to eliminate the potential risk of wrist musculoskeletal disorders. Besides, the portrait orientation for breadth 3 and the landscape orientation for breadth 6 were recommended for their lower physiological loads and higher efficiency. Also, the landscape orientation for breadth 4 should be favoured as it outperformed portrait orientation in both wrist movement dimensions. These findings could provide scientific design principles for using pie menus in mid-air interaction within future HCI applications.

Dual-Thumb pointing and command selection techniques for tablets: Enhancing Distant Interaction with Large Displays using a Two-Handed Tablet

Tablets are often used as a distant input device to interact with large displays. Users can hold the tablet with two hands when interacting for a long time to avoid fatigue or dropping it. Still, most previous distant interaction techniques consider only one-handed tablets. This paper presents novel interaction techniques to perform distant pointing and command selection tasks using both thumbs while holding the device with two hands. In a first step, we design and evaluate six interaction techniques for distant pointing, based on using one or two thumbs, absolute or relative cursor control and single- or dual-precision pointing. Our results show that a combination of dual-thumb and relative pointing is preferred and more efficient than any other dual-thumb or single-thumb technique we designed. In a second step, we focus on command selection using two thumbs. We designed seven dual-thumb menu techniques based on quarter-pie or half-pie layouts, offering from 64 to 900 items. Our results show that using quarter-pie menu techniques is more efficient, usable and appreciated than using a linear or half-pie menu. Furthermore, we show that augmenting the number of items for quarter-pie and half-pie menu techniques has a low impact on command selection performance. In a third step, we compare our quarter-pie menus to bimanual marking menus. Our results show that quarter-pie menus are more efficient and less error-prone. We finally provide a set of lessons for designing efficient dual-thumb interaction techniques for two-handed tablets.

Effects of spatial constraints and ages on children’s upper limb performance in mid-air gesture interaction

Through two controlled experiments, including a pie menu study and a target acquisition study, this paper investigates children’s performance on mid-air gesture interactions in different spatial constraints (i.e., in different orientations/distances), as well as the age effect on such interaction scenarios. The first experiment recorded children’s speeds and accuracies following certain directions under menus with different numbers of items, while the second evaluated the speed–accuracy trade-off (SAT) of children’s arm movements. We also compared performance differences between two age-related groups (i.e., 6–8 years old and 9–12 years old). Based on these experiments, we propose an improved design for UI menus based on mid-air gesture interaction for children. The improved design provides suggestions for setting appropriate directions and difficulty indexes, which makes it much easier and quicker for children to use the menus with mid-air interaction.

THE EVALUATION OF CASCADING AND RADIAL MENUS FOR AUGMENTED REALITY IN A SMARTPHONE

The article describes an evaluation of cascading menu and radial menu in augmented reality (AR) on a smartphone. To evaluate the menus, we measured time to finish the tasks (in seconds) and usability (learnability, efficiency, memorability, errors, and satisfaction) using the System Usability Scale Questionnaire. Data obtained is then analyzed using paired-samples t-test with repeated measures and Wilcoxon signed-rank test to find the statistically significant difference. Overall results showed a preference for the cascading menu type over the radial menu in terms of usability, different from other studies. Statistically, a significant difference was found performance-wise, which indicates that participants are more familiar with cascading menus which contributes to better task time. However, the performance improved the more prolonged the participants got used to the menu when it comes to the radial menu. For errors rates, it is found that the radial menu is less error-prone than cascading menu. This evaluation can help interface designers decide whether they should use the cascading menu or radial menu when designing an interface for AR on a smartphone. The research looks into filling the evaluation gap on the radial and cascading menu, specifically in augmented reality on a smartphone. While other studies have studied and evaluated the cascading menu and radial menu in other environments, this research, on the other hand, tested and evaluated the menu in an augmented reality environment instead of the more traditional desktop environment.

An Exploration of Freehand Crossing Selection in Head-Mounted Augmented Reality

Crossing, or goal crossing, has proven useful in various selection scenarios, including pen, mouse, touch, and virtual reality (VR). However, crossing has not been exploited for freehand selection using augmented reality head-mounted displays (AR HMDs). Using the HoloLens, we explore freehand crossing for selection and compare it to the state-of-the-art “gaze and commit” (head gaze) method. We report on three studies investigating freehand crossing in multiple use cases. The first study shows that crossing outperforms head gaze in selection scenarios of varying target arrangements. The second explores crossing, head gaze, and hand pointing in radial menu and dynamic interface scenarios. The third explores crossing as a function carrier for a variety of basic interaction functions in a drawing application. This work builds on existing knowledge on the goal-crossing paradigm by demonstrating its potential as a useful interaction method in 3D AR HMD interfaces.

The Effect of Dwell Time on Swipe-based Pie-Menu Navigation Efficiency

In this paper we present and analyze pie-menu implementation for touchscreen mobile devices which supports swipe-based navigation through deep hierarchical menu configurations. We propose a solution that synergistically encompasses important design features for a given context: radial menu visualization, semi-transparency, manual repositioning, occlusion awareness, and marking menu concept. From the menu navigation efficiency standpoint, dwell time represents one of the most influential factors in such a design. Reducing the time needed to automatically activate submenus could potentially provide faster navigation through the pie-menu hierarchy; however, short dwell times could make swipe-based interaction more error-prone, as unintentional item selections are then more probable. Therefore, we specifically investigate the effect of dwell time on navigation efficiency when using the proposed solution. A remotely administered experiment involving 36 participants was carried out in order to empirically evaluate different dwell time values. The interaction style, i.e. the way of holding a smartphone device, was also taken into account as an independent variable. We present the results of this empirical research, additionally tackling the interaction workload, usability attributes, and overall design considerations of the proposed solution. According to the obtained results, we argue that highly adaptable pie-menus could provide interaction benefits in the mobile touchscreen domain.

Longer Delays in Rehearsal-based Interfaces Increase Expert Use

Rehearsal-based interfaces are designed to encourage a transition from novice to expert, but many users fail to make this transition. Most of these interfaces activate novice mode after a short delay, between 150 and 500 ms. We investigate the impact of delay time on expert usage and learning in three crowdsourced experiments. The first experiment examines an 8-item marking menu with delay times from 200 ms to 2 s. Results show longer delays increase successful expert selections. The second and third experiments generalise this result to a different rehearsal-based menu, a desktop clone of FastTap with 8 items and 15 items. Together, our results show that expert use correlates positively with increased delay time, but can increase errors since users are less risk averse. We also find imperceptible delays of 200 ms can harm long-term retention of menu items. Designers should consider longer delays in rehearsal-based interfaces to encourage a transition to expert usage.

Depth and Breadth of Pie Menus for Mid-air Gesture Interaction

ABSTRACT Mid-air gesture interaction has become one of the most promising human–computer interaction manners. However, the principles of designing pie menus for mid-air gesture interaction are largely lacking. Currently, no study has addressed the breadth and depth of the pie menu for mid-air gesture interaction. Here, in two experiments, we found that the breadth and depth of a pie menu had distinct influences on the operation performance in mid-air gestural interaction: breadth affects both the accuracy and reaction time of the operation, while depth only affects reaction time. Overall, when operation accuracy is the main concern, the breadth of the pie menu will be the key factor, and it should not exceed eight items (Experiments 1 and 2). However, when the operation also emphasizes efficiency, the depth of the pie menu should be considered: fewer layers lead to faster responses, with two layers yielding the best performance (Experiment 2).

ArmMenu: command input on distant displays with proprioception based lateral arm movements

ABSTRACT In this paper, we present ArmMenu, a command input approach for distant displays. ArmMenu has a circular interface like pie menus and menu selection is performed by proprioception-based lateral arm movements. We implemented ArmMenu with an off-the-shelf body tracking device (Kinect) and conducted two experiments to validate its efficacy. In the first experiment, we explored the design space of ArmMenu by varying the number of menu items, with exposed or hidden menu modes. Users can operate up to 8-item menus with high selection accuracy (>98%). ArmMenu was fast and accurate even with the hidden menu mode. The second experiment compared the performance of ArmMenu and touchless marking menus. While having similar selection accuracy, ArmMenu was faster and more preferable by users. Our studies consequently demonstrate ArmMenu's effectiveness for command input on distant displays.

Relative Pointing Interface: A gesture interaction method based on the ability to divide space

A new type of 3D gesture interface called Relative Pointing Interface (RelPoInt) is suggested in this study. RelPoInt allows users to select a virtual button by pointing to a relative position in a virtual 3D menu. This approach makes use of the distance or angle between the relative point and the reference point. Users can set the reference point freely and operate all the functions on the menu with a simple gesture. Microsoft Kinect was used to implement the RelPoInt in this study. To suggest a radial menu suitable for the RelPoInt, we conducted an experiment in which 20 participants were asked to point to virtual buttons, which were divided into four to eight regions. Errors and subjective assessment of easiness were measured at each treatment condition. RelPoInt can make users to manipulate functions without memorizing complicated gesture vocabularies, which is among the most serious pain points of using a gesture interface. Furthermore, RelPoInt does not require expensive gesture recognition equipment compared with finger recognition interfaces because the required resolution of interaction recognition is less sophisticated. It can be useful in applying 3D gesture interfaces to a variety of devices for smart home, virtual or augmented reality. Relevance to industry The proposed interface can be applied to various environments with low cost and easy to implement. Therefore, this interface is highly worthy of use in areas where gestures are useful to interact, such as IoT (Internet of Things) and VR (Virtual Reality)/AR (Augmented Reality).

A mechanism based on finger-sliding behavior for designing radial menus

Human behavior about sliding their fingers on touch screens needs to be understood for the design of radial menus. For this purpose, data about the angles of finger sliding in performing radial pointing to eight predefined directions were collected. The optimal angle ranges and boundaries for these directions were then determined by the proposed mechanism. Consistent with previous research, results showed that the deviations of slide angles for the four diagonal directions were larger than that for the four orthogonal directions. Therefore, to reduce overall pointing errors, the angle ranges for the diagonal directions should be wider than that for the orthogonal directions. Based on the proposed mechanism, the total probability of pointing errors could be limited to 0.05%, compared to the error of 2.00% if the eight angle ranges were evenly divided by the radii of a regular octagon. Besides the angles, the lengths and starting points of the slides were reported and discussed along with the oblique effect, judder effect, and screen orientation. Possible applications of the proposed mechanism for dynamic and personalized radial menus are addressed. Further research would be worthwhile to examine the effects of screen orientation and the number of predefined directions on pointing performances. Moreover, extending the research to mid-air sliding gestures would be of interest and value in designing three-dimensional radial menus. Relevance to industryThe proposed mechanism could effectively enhance the accuracy of the selections through radial menus, and could be applied to the remote controls via smart handheld devices.

Comparison of Radial and Panel Menus in Virtual Reality

Although selection menus are widely used for interaction, their use on 3D virtual reality applications needs to be objectively assessed. The focus of this study is to evaluate a traditional panel and a radial menu in two distinct virtual environment placements (i.e. fixed on the wall and following the users' hands). Fifty-one participants used two different menus of the four possible combinations. To evaluate the menus' effectiveness and efficiency, we measured usability (System Usability Scale Questionnaire), user satisfaction (After-Scenario Questionnaire), time to finish the tasks (in seconds) and the number of unnecessary steps (errors) performed by the users. Overall results showed a clear preference for the traditional panel menu type and the fixed wall placement of the menu. We conclude that all menu types perform well, despite different user preferences, and that fixing the menu to the wall gives users a better overview of both the menu and the virtual environment, improving their ability to perceive their actions on the menu.

Designing motion marking menus for people with visual impairments

Abstract Current smartphone accessibility for people with visual impairments relies largely on screen readers and voice commands. However, voice commands and screen readers are often not ideal because users with visual impairments rely mostly on hearing ambient sound from the environment for their safety in mobile situations. Recent research has shown that marking menus in mobile devices provide fast and eyes-free access for sighted users Francone et al., 2010; Oakley and Park, 2007, 2009. However, the literature is lacking design implications and adaptations that meet the needs of users with visual impairments. This paper investigates the capabilities of visually impaired people to invoke smartphone functions using marking menus via 3D motions. We explore and present the optimal numbers of menu items (breadth) and menu levels (depth) for marking menus that people with visual impairments can successfully adopt. We also compared a marking menu prototype to TalkBack™ which is an accessibility menu system in Android smartphones. The experimental results show that our participants could perform menu selections using marking menus faster than when using TalkBack. Based on the study results, we provide implications and guidelines for designing marking menus and motion gesture interfaces for people with visual impairments.

Performance of smartphone users with half-pie and linear menus

ABSTRACTIt will not be easy for smartphone users to get accustomed to one-hand operation using smartphones with larger screens. Coupled with the massive data on the handset screen, the menu design has become extremely important. This study is to apply the pie menu into the smartphones which is used in previous PC user interface. Specifically, how efficient are the users when using pie menu on 5″ handsets? The experiment is divided into two phases. In Phase 1, behavioural analysis, 36 participants used their right hand to operate the Left-Side-Linear Menu (LSLM), Right-Side-Linear Menu (RSLM), and Half-Pie Menu (HPM) while walking. After the experiment we asked them to fill in the NASA-TLX (NASA Task Load Index Standard Workload Test). In Phase 2, eight participants with different thumb lengths and diverse operating experience were invited to operate three menu techniques while standing. Results showed that participants with shorter thumbs are more efficient in operating the three menu techniques than participants with longer thumbs. The reason for this difference is related to how the users grip their phones. In addition, LSLM has the highest error rate and RSLM has the lowest among the three types. The efficiency of HPM depends on continuous learning for five minutes and the result closely matches RSLM. However, there was no significant difference between the HPM and the LSLM in walking speed. In addition, 137 participants took the improved version of the test and made ‘Good’ comments in the System Usability Scale.

PP-Menus: Localizing Pie Menus by Pressure

We present PP-Menus, a novel technique for enhancing the performance of pen-based interfaces. PP-Menus are pressure-based pie menus, which are implemented by using pen input pressure values to localize traditional pie menus. PP-Menus possess two states, i.e. stationary state and tracking state. A variation of pen input pressure can switch PP-Menus between the two states. A quantitative experiment shows that PP-Menus significantly outperformed tracking menus on task speed, and there was no significant difference between the two types of menus on accuracy.

マルチタッチとパイメニューを用いた電子黒板インタフェースの開発と評価( 学習・教育支援のための技術開発)

マルチタッチとパイメニューを用いた電子黒板インタフェースの開発と評価( 学習・教育支援のための技術開発)

Evaluation of a Low-cost Eye Tracking System for Computer Input

Eye tracking technology has become one of the most popular techniques within the human and computer interaction (HCI) era, this is especially important research for people which have difficulty with speech and movement disabilities. The primary function of this technology is based on a device that tracks the movement of the eye to identify a position or scan a display. Suitable devices for eye movement can then be integrated in concordance with the requirements of the organization. Currently, eye tracking devices are becoming increasingly cheaper which make them an interesting resource for research. Although numerous studies have been conducted involving applications of eye tracking with a low-cost device, few studies have compared the actual eye tracking systems themselves. This paper empirically evaluated the performance of three independent low-cost eye tracking devices, Gazepoint’s GP3, EyeTribe, and DIY (Do-It-Yourselt). The performance evaluation used a multi-directional point-select task based on visual searching and selection to change the ground colour of a circular menu conforming to ISO 9241-9 standards for computer pointing devices. Results indicated that the spatial accuracy and speed were a good reflection of targeted tracking of errors, completion time and throughput. Consequently, the experiment with the target task showed that all devices can be a potentially valuable resource for human computer interaction research. Additionally, this basic result will be used to develop an advanced system for Thai text entry to aid communication for handicapped users.

Informatics and Computing.

Informatics and Computing.

RegionalSliding: Facilitating small target selection with marking menu for one-handed thumb use on touchscreen-based mobile devices

Small target selection on mobile devices with touchscreens usually brings trouble to users due to the occlusion of the target by the user’s finger, as well as the ambiguity about which part of the finger generates the contact point. In this paper, we propose a novel technique to enhance target selection on touchscreen-based mobile devices, called RegionalSliding, which if necessary renders the initially selected target as well as its “surrounding” targets as a marking menu in a non-occluded area when users press down on the screen without lifting, and enables users to complete the selection with natural sliding gestures according to the visual feedback from the rendered area. We recognized the range of unintentional displacement and users’ sliding habits through two pilot studies, which indicated significant parameters of the marking menu for our design. We then conducted a user study that compared RegionalSliding with Shift (Vogel and Baudisch, 2007) and TapTap (Roudaut et al., 2008). Experimental results showed that RegionalSliding not only performed at the lowest mean error rate and at a shorter task completion time than Shift, but was also less affected by other factors, such as the target size, than the other two target selection techniques when acquiring small square targets.

WallTop: Managing Overflowing Windows on a Large Display

With the ever increasing amount of digital information, users desire more screen real estate to process daily desktop computing work, and might well benefit from using a large high-resolution display for information management. Unfortunately, we know very little about users' behaviors when using such a display for daily computing, and current user interfaces are mainly designed for normal-sized desktop monitors, which might not well suit a large display. In this article, we first present a longitudinal study that investigates how users manage overflowing digital information on a wall-sized display in a personal desktop computing context by comparing it with single and dual desktop monitors. Results showed users' unanimous preferences of working on a large display and revealed large-display users' unique activity patterns of managing windows. Guided by the study results, we designed a set of interaction techniques that provide greater flexibility in managing multiple windows. They include facile methods for selecting, moving, and resizing multiple windows using an active window boundary called Fringe, rearranging selected windows using multi- and single-window marking menus, packing/unpacking the selected windows using easily activated icons, and freely adjusting the order of overlapping windows with a Jab-to-Lift operation. We coherently integrated these techniques with traditional operations in a large-display window management prototype called WallTop. Two rounds of usability testing showed that users can quickly and easily learn the new interaction techniques and apply them to realistic window management tasks on a large display with increased efficiency.