Gesture Selection Research Articles

Politisi dengan tingkat elektabilitas rendah dan strategi branding politik: analisis multimodalitas akun instagram puan maharani

Politicians potentially nominated by political party as a candidate for running in the elections commonly expected to get endorsement from a substantial number of electoral markets. Only few of them have a chance to do so however. In consideration of this, most of these politicians select and use particular semiotics codes strategically as an element to establish political branding strategy they carry out through media platforms, including social media. Having adopted Aaker’s (1997) concept of brand personality, this article examines how did Puan Maharani exploit semiotic code resources communicated through his Instagram account (@puanmaharaniri) as a political branding strategy. 3 materials posted in his Instagram account with high level engagement were selected, extracted and analyzed using multimodal analysis. The findings are follows. Puan deployed various kinds of semiotic code resources strategically. This includes strategic selection of represented participants, gestures, image composition, clothing colors, and hair styling. She kept exploiting all dimensions of brand personality, as posited by Aaker (1997), but she highlighted following three dimensions of this brand personality, which are competence, sophistication, and ruggedness much more and did it consistently. Political branding strategy she carried out was directed to create her personal political image as a populist politician; though this image belongs to her toughest rival from the party she associated with, which is Ganjar Pranowo

Surgical gestures as a method to quantify surgical performance and predict patient outcomes

How well a surgery is performed impacts a patient’s outcomes; however, objective quantification of performance remains an unsolved challenge. Deconstructing a procedure into discrete instrument-tissue “gestures” is a emerging way to understand surgery. To establish this paradigm in a procedure where performance is the most important factor for patient outcomes, we identify 34,323 individual gestures performed in 80 nerve-sparing robot-assisted radical prostatectomies from two international medical centers. Gestures are classified into nine distinct dissection gestures (e.g., hot cut) and four supporting gestures (e.g., retraction). Our primary outcome is to identify factors impacting a patient’s 1-year erectile function (EF) recovery after radical prostatectomy. We find that less use of hot cut and more use of peel/push are statistically associated with better chance of 1-year EF recovery. Our results also show interactions between surgeon experience and gesture types—similar gesture selection resulted in different EF recovery rates dependent on surgeon experience. To further validate this framework, two teams independently constructe distinct machine learning models using gesture sequences vs. traditional clinical features to predict 1-year EF. In both models, gesture sequences are able to better predict 1-year EF (Team 1: AUC 0.77, 95% CI 0.73–0.81; Team 2: AUC 0.68, 95% CI 0.66–0.70) than traditional clinical features (Team 1: AUC 0.69, 95% CI 0.65–0.73; Team 2: AUC 0.65, 95% CI 0.62–0.68). Our results suggest that gestures provide a granular method to objectively indicate surgical performance and outcomes. Application of this methodology to other surgeries may lead to discoveries on methods to improve surgery.

Freehand Gestural Selection with Haptic Feedback in Wearable Optical See-Through Augmented Reality

Augmented reality (AR) technologies can blend digital and physical space and serve a variety of applications intuitively and effectively. Specifically, wearable AR enabled by optical see-through (OST) AR head-mounted displays (HMDs) might provide users with a direct view of the physical environment containing digital objects. Besides, users could directly interact with three-dimensional (3D) digital artefacts using freehand gestures captured by OST HMD sensors. However, as an emerging user interaction paradigm, freehand interaction with OST AR still requires further investigation to improve user performance and satisfaction. Thus, we conducted two studies to investigate various freehand selection design aspects in OST AR, including target placement, size, distance, position, and haptic feedback on the hand and body. The user evaluation results indicated that 40 cm might be an appropriate target distance for freehand gestural selection. A large target size might lower the selection time and error rate, and a small target size could minimise selection effort. The targets positioned in the centre are the easiest to select, while those in the corners require extra time and effort. Furthermore, we discovered that haptic feedback on the body could lead to high user preference and satisfaction. Based on the research findings, we conclude with design recommendations for effective and comfortable freehand gestural interaction in OST AR.

Robot Dialog System in the Context of Hospital Receptionist and its Demonstration

Task-oriented dialogue systems play an important role in human-robot interaction. Task-oriented dialogue systems help users achieve specific goals, and robots with systems are very effective in terms of reducing time and effort when replacing human workers. The conventional pipeline method, which has disadvantages such as a large amount of cost and time for development, has been applied to reception robots. For example, developers manually define responses that correspond to user input. Also, they use predefined robotic actions such as gestures and facial expressions. Recently, end-to-end learning of Recurrent Neural Networks is an attractive solution for the dialogue system. Based on the strengths of RNNs, we propose a social robot system in the context of hospital receptionists. We utilize Hybrid Code Network as an end-to-end dialogue system and extend it to select both response and gesture using RNN-based gesture selector. A user study is conducted comparing our proposed system with one of the existing methods, a rule-based approach. The empirical results show that the proposed method has an advantage in terms of dialog efficiency, which indicates how efficiently the user performed a given task with the help of a robot. In addition, there is no significant difference in experimental results between the proposed RNN based gesture selection and the rule-based gesture selection.

Assessing the efficacy of dissection gestures in robotic surgery.

Our group previously defined a dissection gesture classification system that deconstructs robotic tissue dissection into its most elemental yet meaningful movements. The purpose of this study was to expand upon this framework by adding an assessment of gesture efficacy (ineffective, effective, or erroneous) and analyze dissection patterns between groups of surgeons of varying experience. We defined three possible gesture efficacies as ineffective (no meaningful effect on the tissue), effective (intended effect on the tissue), and erroneous (unintended disruption of the tissue). Novices (0 prior robotic cases), intermediates (1-99 cases), and experts (≥ 100 cases) completed a robotic dissection task in a dry-lab training environment. Video recordings were reviewed to classify each gesture and determine its efficacy, then dissection patterns between groups were analyzed. 23 participants completed the task, with 9 novices, 8 intermediates with median caseload 60 (IQR 41-80), and 6 experts with median caseload 525 (IQR 413-900). For gesture selection, we found increasing experience associated with increasing proportion of overall dissection gestures (p = 0.009) and decreasing proportion of retraction gestures (p = 0.009). For gesture efficacy, novices performed the greatest proportion of ineffective gestures (9.8%, p < 0.001), intermediates commit the greatest proportion of erroneous gestures (26.8%, p < 0.001), and the three groups performed similar proportions of overall effective gestures, though experts performed the greatest proportion of effective retraction gestures (85.6%, p < 0.001). Between groups of experience, we found significant differences in gesture selection and gesture efficacy. These relationships may provide insight into further improving surgical training.

Development and validation of an objective scoring tool to evaluate surgical dissection: Dissection Assessment for Robotic Technique (DART).

Evaluation of surgical competency has important implications for training new surgeons, accreditation, and improving patient outcomes. A method to specifically evaluate dissection performance does not yet exist. This project aimed to design a tool to assess surgical dissection quality. Delphi method was used to validate structure and content of the dissection evaluation. A multi-institutional and multi-disciplinary panel of 14 expert surgeons systematically evaluated each element of the dissection tool. Ten blinded reviewers evaluated 46 de-identified videos of pelvic lymph node and seminal vesicle dissections during the robot-assisted radical prostatectomy. Inter-rater variability was calculated using prevalence-adjusted and bias-adjusted kappa. The area under the curve from receiver operating characteristic curve was used to assess discrimination power for overall DART scores as well as domains in discriminating trainees (≤100 robotic cases) from experts (>100). Four rounds of Delphi method achieved language and content validity in 27/28 elements. Use of 3- or 5-point scale remained contested; thus, both scales were evaluated during validation. The 3-point scale showed improved kappa for each domain. Experts demonstrated significantly greater total scores on both scales (3-point, p< 0.001; 5-point, p< 0.001). The ability to distinguish experience was equivalent for total score on both scales (3-point AUC= 0.92, CI 0.82-1.00, 5-point AUC= 0.92, CI 0.83-1.00). We present the development and validation of Dissection Assessment for Robotic Technique (DART), an objective and reproducible 3-point surgical assessment to evaluate tissue dissection. DART can effectively differentiate levels of surgeon experience and can be used in multiple surgical steps.

ExpressGesture: Expressive gesture generation from speech through database matching

AbstractCo‐speech gestures are a vital ingredient in making virtual agents more human‐like and engaging. Automatically generated gestures based on speech‐input often lack realistic and defined gesture form. We present a database‐driven approach guaranteeing defined gesture form. We built a large corpus of over 23,000 motion‐captured co‐speech gestures and select individual gestures based on expressive gesture characteristics that can be estimated from speech audio. The expressive parameters are gesture velocity and acceleration, gesture size, arm swivel, and finger extension. Individual, parameter‐matched gestures are then combined into animated sequences. We evaluate our gesture generation system in two perceptual studies. The first study compares our method to the ground truth gestures as well as mismatched gestures. The second study compares our method to five current generative machine learning models. Our method outperformed mismatched gesture selection in the first study and showed competitive performance in the second.

An Evaluation of SVM in Hand Gesture Detection Using IMU-Based Smartwatches for Smart Lighting Control

Hand gesture detection with a smartwatch can be used as a smart lighting control on the internet of things (IoT) environment using machine learning techniques such as support vector machine (SVM). However, several parameters affect the SVM model's performance and need to be evaluated. This study evaluates the parameters in building an SVM model for hand gesture detection in intelligent lighting control. In this study, eight gestures were defined to turn on and off four different lights, and then the data were collected through a smartwatch with an Inertial Measurement Unit (IMU) sensor. Feature selection using Pearson Correlation is then carried out on 36 features extracted from each gesture data. Finally, two sets of gestures were compared to evaluate the effect of gesture selection on model performance. The first set of gestures show that the accuracy of 10 features compared to the accuracy of 36 features is 94% compared to 71%, respectively. Furthermore, the second set of gestures has an accuracy lower than the first set of gestures, which is 64%. Results show that the lower the number of features, the better the accuracy. Then, the set of gestures that are not too distinctive show lower accuracy than the highly distinctive gesture sets. The conclusion is, in implementing gesture detection with SVM, low data dimensions need to be maintained through feature selection methods, and a distinctive set of gesture selection is required for a model with good performance.

Phonetic detail encoding in explaining boundary-modulated coarticulation

This study examines the acoustic outputs in boundary coarticulation and attempts to explain the native and nonnative production differences in relation to speech planning. English consonant clusters across words produced by English (L1) and Mandarin (L2) speakers are analyzed. This study focuses on two segmental features (place of articulation and voicing) and computes three acoustic-based measurements (internal release, closure durations and closure shortening). The results find a lesser coarticulation degree in L2 production, and differing acoustic deviations varying with speakers’ proficiency level. The results also suggest that cross-word planning involving place of articulation is robust in both L1 and L2 production, while cross-word voicing encoding is not available in L2 speech. The asymmetry in L2 encoding is compatible with a competitive gestural selection analysis, and the implications of L1 production have been interpreted as in favor of a richer mental representation and finer-level execution than the syllable-sized programming. The preliminary results support the core assumption that L1 and L2 coarticulation differences arise from phonological and phonetic encoding. This study adds cross-linguistic output-based evidence on speech planning and encourages speech models to further specify the very late stage of phonetic encoding.

Context-Specific Separable Gesture Selection For Control of a Robotic Manufacturing Assistant

Abstract This work aims to facilitate robotic-assisted limited-access manufacturing, where a mechanic manually performs work in an enclosed space while a robot guides a camera to allow the mechanic to observe visually-occluded operations. Gesture control allows the mechanic to adjust their camera view without requiring contact-based interaction through button presses or manual positioning of the robot, which would require setting down tools or exiting the enclosed space. Because the only input to the robotic platform’s gesture recognition system is a continuously tracked hand, there is potential for confusion between the motions of work and motions intended to communicate control. The main contribution of this article is to develop a separability metric Ψ for systematically selecting a set of control gestures that can be easily distinguished from a given set of work gestures. The context-specific control gesture selection process is implemented on a benchmark dataset to validate the method. The process is then implemented on a hand-following robot, and experimental results show that the proposed metric-based selection of the control gesture set results in the improvement of online gesture recognition across eight subjects even in the presence of application-specific distortions in the gestures.

Gesture selection study for a maternal healthcare information system in rural Assam, India

This paper presents a case study aimed at selecting suitable body gestures to represent actions that viewers of a televised maternal health information program will recognize and understand. This program is designed for pregnant women in rural Assam in India. We observed the gestures of 24 pregnant women to determine how to present the following seven computational functions that were used in the health information program: Select, Pause, Resume, Help, Activate Menu, Next, and Previous. The participants belonged to the low socio-economic strata in rural Assam, India and most had poor literacy levels. A participatory approach through a user-generated gesture collection method was used for this study. This study produced a total of 49 different gestures that participants performed to represent the seven computational functions. We selected seven body gestures based on the frequency of use, logical suitability to represent functions, the decreased possibility that the gestures would be accidently performed (false positives), and the ease of detection for the chosen technology (technical limitations).

Selection of suitable hand gestures for reliable myoelectric human computer interface

BackgroundMyoelectric controlled prosthetic hand requires machine based identification of hand gestures using surface electromyogram (sEMG) recorded from the forearm muscles. This study has observed that a sub-set of the hand gestures have to be selected for an accurate automated hand gesture recognition, and reports a method to select these gestures to maximize the sensitivity and specificity.MethodsExperiments were conducted where sEMG was recorded from the muscles of the forearm while subjects performed hand gestures and then was classified off-line. The performances of ten gestures were ranked using the proposed Positive–Negative Performance Measurement Index (PNM), generated by a series of confusion matrices.ResultsWhen using all the ten gestures, the sensitivity and specificity was 80.0% and 97.8%. After ranking the gestures using the PNM, six gestures were selected and these gave sensitivity and specificity greater than 95% (96.5% and 99.3%); Hand open, Hand close, Little finger flexion, Ring finger flexion, Middle finger flexion and Thumb flexion.ConclusionThis work has shown that reliable myoelectric based human computer interface systems require careful selection of the gestures that have to be recognized and without such selection, the reliability is poor.

Trace select: acquiring remote targets with gesture guides generated from glyphs

When selecting targets in a Smart TV service, the user is frequently required to select remote targets such as menu items, but is unable to touch the targets directly or use a mouse. Solutions include rapid selection techniques using gesture guides. However, such techniques create difficulties, in that they tend to hide the original information. This study proposes a novel technique for selecting remote targets, referred to as Trace Select, which dynamically generates gesture guides from text target characters. The guides do not conceal the original targets. As a result, the guides can be continuously displayed. The technique does not require immediate activation of the gesture-guide displays, and users can identify and draw the gesture of the desired target at any time. Experiments show that Trace Select outperforms both cursor-based selection enhanced by acceleration and conventional activation-required gestural selection.

Selection and coordination of articulatory gestures in temporally constrained production

A variety of factors such as articulatory context, prosodic structure, and rate of articulation are known to induce compression or expansion of articulatory movements and their relative timing. Another potentially important factor is the amount of time available for planning and executing a movement. The current study employs a novel cued insertion task which controls the amount of time available for planning a single articulatory movement produced in the context of others. The study tests the hypothesis that movements associated with onset consonants and coda consonants will respond differently to temporal constraints on planning, due to differences in how they are controlled. It is observed that compression of movement timing is greater in onset consonantal gestures than in coda consonantal gestures. These findings are argued to support a split-control model, in which onset consonants are co-selected and coordinated with vowels, while coda consonants are competitively selected but not coordinated with vowels.

Selection of Touch Gestures for Children’s Applications: Repeated Experiment to Increase Reliability

This paper discusses the selection of touch gestures for children’s applications. This research investigates the gestures that children aged between 2 to 4 years old can manage on the iPad device. Two experiments were conducted for this research. The first experiment was carried out in United Kingdom. The second experiment was carried out in Malaysia. The two similar experiments were carried out to increase the reliability and refine the result. This study shows that children aged 4 years have no problem using the 7 common gestures found in iPad applications. Some children aged 3 years have problem with two of the gestures. A high percentage of children aged 2 years struggled with the free rotate, drag & drop, pinch and spread gestures. This paper also discusses the Additional Criteria for the use of Gestures, Interface Design Components and Research on Children using iPad and Applications.

Evaluating Sensor Placement and Gesture Selection for Mobile Devices

Evaluating Sensor Placement and Gesture Selection for Mobile Devices

3D selection with freehand gesture

The use of 3D computer graphics is important in a very wide range of applications. However, user interaction with 3D applications is still challenging and often does not lend itself to established techniques that have been developed primarily for 2D desktop interaction. Meanwhile, 3D user interfaces that rely on tracking hand-held devices or fiducial markers attached to the user are cumbersome or entirely inappropriate in some situations. These challenges may be addressed by refining and building on the increasing use of freehand gestural input, i.e. without markers or hand-held devices, to extend the fluidity and immediacy of today's 2D touch-based interactions. In this paper, we analyze the characteristics of freehand gestural 3D interaction, and report a set of 3 related evaluation studies focused on the fundamental user interface task of object selection. We found that interaction design requiring a high accuracy single action are not appropriate for freehand gestural selection, while separating it into several connected low demand operations could be a potential solution; that our Reach technique is accurate and potentially useful for option selection tasks with freehand gesture; and that strong directional effects influence performance and usability of both 2D and 3D option selection. We propose guidelines for designers of 3D freehand gestural interaction based on our evaluation results.

Evaluating Sensor Placement and Gesture Selection for Mobile Devices

Evaluating Sensor Placement and Gesture Selection for Mobile Devices

Gestures, vocalizations, and memory in language origins.

This article discusses the possible homologies between the human language networks and comparable auditory projection systems in the macaque brain, in an attempt to reconcile two existing views on language evolution: one that emphasizes hand control and gestures, and the other that emphasizes auditory–vocal mechanisms. The capacity for language is based on relatively well defined neural substrates whose rudiments have been traced in the non-human primate brain. At its core, this circuit constitutes an auditory–vocal sensorimotor circuit with two main components, a “ventral pathway” connecting anterior auditory regions with anterior ventrolateral prefrontal areas, and a “dorsal pathway” connecting auditory areas with parietal areas and with posterior ventrolateral prefrontal areas via the arcuate fasciculus and the superior longitudinal fasciculus. In humans, the dorsal circuit is especially important for phonological processing and phonological working memory, capacities that are critical for language acquisition and for complex syntax processing. In the macaque, the homolog of the dorsal circuit overlaps with an inferior parietal–premotor network for hand and gesture selection that is under voluntary control, while vocalizations are largely fixed and involuntary. The recruitment of the dorsal component for vocalization behavior in the human lineage, together with a direct cortical control of the subcortical vocalizing system, are proposed to represent a fundamental innovation in human evolution, generating an inflection point that permitted the explosion of vocal language and human communication. In this context, vocal communication and gesturing have a common history in primate communication.

Gesturing Meaning: Non-action Words Activate the Motor System

Across cultures, speakers produce iconic gestures, which add – through the movement of the speakers’ hands – a pictorial dimension to the speakers’ message. These gestures capture not only the motor content but also the visuospatial content of the message. Here, we provide first evidence for a direct link between the representation of perceptual information and the motor system that can account for these observations. Across four experiments, participants’ hand movements captured both shapes that were directly perceived, and shapes that were only implicitly activated by unrelated semantic judgments of object words. These results were obtained even though the objects were not associated with any motor behaviors that would match the gestures the participants had to produce. Moreover, implied shape affected not only gesture selection processes but also their actual execution – as measured by the shape of hand motion through space – revealing intimate links between implied shape representation and motor output. The results are discussed in terms of ideomotor theories of action and perception, and provide one avenue for explaining the ubiquitous phenomenon of iconic gestures.