Gesture Duration Research Articles

Bioinspired Monopolar Controlled Ionic Hydrogels for Flexible Non‐Contact Human–Machine Interfaces

AbstractMost flexible human–machine interfaces emulate the tactile system of the skin, which has the risk of contact damage. Additionally, contact deformation often leads to a hysteresis response. Non‐contact interaction can address these problems. Inspired by the electroreception capabilities of the elephantnose fish, this study introduces a non‐contact sensing model employing monopolar controlled ionic hydrogel. Compared to most existing mutual capacitive non‐contact sensing models, this model not only boosts responsivity by over 3.5 times but also streamlines the sensing architecture. Utilizing this sensing model, a flexible non‐contact human–machine interface is developed by organizing three differently shaped hydrogels into an asymmetric configuration. This device reliably discerns six non‐contact gestures using machine learning algorithms and supports at least eleven interactive functions by detecting the duration of gestures, enabling continuous real‐time control over external devices. This advancement heralds a more liberated paradigm of human–machine interaction with promising implications for the Internet of Things.

Continuous hand gesture segmentation and acknowledgement of hand gesture path for innovative effort interfaces

Human-computer interaction (HCI) has revolutionized the way we interact with computers, making it more intuitive and user-friendly. It is a dynamic field that has found it is applications in various industries, including multimedia and gaming, where hand gestures are at the forefront. The advent of ubiquitous computing has further heightened the interest in using hand gestures as input. However, recognizing continuous hand gestures presents a set of challenges, primarily stemming from the variable duration of gestures and the lack of clear starting and ending points. Our main objective is to propose a solution: the framework for “continuous palm motion analysis and retrieval” based on “Spatial-temporal and path knowledge”. Framework harnesses the power of cognitive deep learning networks (DLN), offering a significant advancement in the continuous hand gesture recognition domain. we conducted rigorous experiments using a diverse video dataset capturing hand gestures for boasting an impressive F-score of up to 0.99. The potential of our framework to significantly enhance the accuracy and reliability of hand gesture recognition in real-world applications.

Novel Digital Biomarkers for Fine Motor Skills Assessment in Psoriatic Arthritis: The DaktylAct Touch-based Serious Game Approach.

Psoriatic Arthritis (PsA) is a chronic, inflammatory disease affecting joints, substantially impacting patients' quality of life, with European guidelines for managing PsA emphasizing the importance of assessing hand function. Here, we present a set of novel digital biomarkers (dBMs) derived from a touchscreen-based serious game approach, DaktylAct, intended as a proxy, gamified, objective assessment of hand impairment, with emphasis on fine motor skills, caused by PsA. This is achieved by its design, where the user controls a cannon to aim at and hit targets using two finger pinch-in/out and wrist rotation gestures. In-game metrics (targets hit and score) and statistical features (mean, standard deviation) of gameplay actions (duration of gestures, applied pressure, and wrist rotation angle) produced during gameplay serve as informative dBMs. DaktylAct was tested on a cohort comprising 16 clinically verified PsA patients and nine healthy controls (HC). Correlation analysis demonstrated a positive correlation between average pinch-in duration and disease activity (DA) and a negative correlation between standard deviation of applied pressure during wrist rotation and joint inflammation. Logistic regression models achieved 83% and 91% classification performance discriminating HC from PsA patients with low DA (LDA) and PsA patients with and without joint inflammation, respectively. Results presented here are promising and create a proof-of-concept, paving the way for further validation in larger cohorts.

«Обними меня»: три эпохи русского жеста «объятие» (по данным корпуса МУРКО)

The article examines the formal characteristics of the Russian gesture of embrace (Russian: ob”yatie, объятие) as depicted in feature films. The study material consists of video episodes from Soviet and post-Soviet films of the 20th century, represented in the Multimodal Russian Corpus (MURCO) of the Russian National Corpus. The focus is on the duration of the gesture with the emphasis on its beginning and ending phases. The determination of gesture duration necessitates a scientific justification of its elementary structure, delineating crucial phases (preparation—stroke—retraction), and exploring its potential as a social tactile gesture. The research also aims to analyze the cinema aesthetics of three epochs exemplified by films from the 1930s, 1960s, and 1990s. Different approaches to film montage, sociocultural and political contexts, meaning genesis in the film, as well as the perspectives of the director, actor, and viewer significantly influence what we see in feature films. Consequently, the question arises of whether a valid study of gestures can be conducted using film material, and it is resolved affirmatively. The results indicate that the realization of the gesture ob”yatie varies. The preparation phase is often a manual movement, while the retraction phase is characterized by the departure of one participant from the situation, ending physical contact at a certain point of the stroke. The proposed method of defining the beginning and ending of the gesture helped to reveal the shortest embrace in the 1936 film Circus (Russian: Цирк) and the longest gesture in the 1961 film My Friend, Kolka! (Russian: Друг мой, Колька!). On average, the duration of the embrace ranges from 6 to 13 seconds.

RimSense

Smart eyewear's interaction mode has attracted significant research attention. While most commercial devices have adopted touch panels situated on the temple front of eyeglasses for interaction, this paper identifies a drawback stemming from the unparalleled plane between the touch panel and the display, which disrupts the direct mapping between gestures and the manipulated objects on display. Therefore, this paper proposes RimSense, a proof-of-concept design for smart eyewear, to introduce an alternative realm for interaction - touch gestures on eyewear rim. RimSense leverages piezoelectric (PZT) transducers to convert the eyeglass rim into a touch-sensitive surface. When users touch the rim, the alteration in the eyeglass's structural signal manifests its effect into a channel frequency response (CFR). This allows RimSense to recognize the executed touch gestures based on the collected CFR patterns. Technically, we employ a buffered chirp as the probe signal to fulfil the sensing granularity and noise resistance requirements. Additionally, we present a deep learning-based gesture recognition framework tailored for fine-grained time sequence prediction and further integrated with a Finite-State Machine (FSM) algorithm for event-level prediction to suit the interaction experience for gestures of varying durations. We implement a functional eyewear prototype with two commercial PZT transducers. RimSense can recognize eight touch gestures on the eyeglass rim and estimate gesture durations simultaneously, allowing gestures of varying lengths to serve as distinct inputs. We evaluate the performance of RimSense on 30 subjects and show that it can sense eight gestures and an additional negative class with an F1-score of 0.95 and a relative duration estimation error of 11%. We further make the system work in real-time and conduct a user study on 14 subjects to assess the practicability of RimSense through interactions with two demo applications. The user study demonstrates RimSense's good performance, high usability, learnability and enjoyability. Additionally, we conduct interviews with the subjects, and their comments provide valuable insight for future eyewear design.

Effects of activity time limitation on gesture elicitation for form creation

Cognitive processing employed during design includes both time critical and time-consuming types of thinking. The ability to match the pace of design generation or modification with the designers thinking processes can be particularly important with gesture-based interfaces for form creation, especially where representation modes of input and response may influence the choice of activities performed. Particularly in gesture elicitation studies, time-consuming design activities can shift the focus on forming the analogies between problem at hand and prior knowledge and experiences, rather than intuitive gesture suggestions that would be the best fit for the given representation mode. However, design methodologies do not prescribe or discuss time limitations and their use in this context. In this paper, time limitation is explored during a gesture elicitation study for three-dimensional object creation, modification and manipulation, by comparing two study parts, one where time limitation was imposed and one where time was unlimited. Resulting gesture durations in both parts were comparable and elicited gestures were similar in nature and employing same elements of hand motion, supporting the hypothesis that time limitation can be a useful methodological approach when gestures are used for interaction with 3D objects and representation and interaction modalities are matched.

Variable expression of linguistic laws in ape gesture: a case study from chimpanzee sexual solicitation.

Two language laws have been identified as consistent patterns shaping animal behaviour, both acting on the organizational level of communicative systems. Zipf's law of brevity describes a negative relationship between behavioural length and frequency. Menzerath's law defines a negative correlation between the number of behaviours in a sequence and average length of the behaviour composing it. Both laws have been linked with the information-theoretic principle of compression, which tends to minimize code length. We investigated their presence in a case study of male chimpanzee sexual solicitation gesture. We failed to find evidence supporting Zipf's law of brevity, but solicitation gestures followed Menzerath's law: longer sequences had shorter average gesture duration. Our results extend previous findings suggesting gesturing may be limited by individual energetic constraints. However, such patterns may only emerge in sufficiently large datasets. Chimpanzee gestural repertoires do not appear to manifest a consistent principle of compression previously described in many other close-range systems of communication. Importantly, the same signallers and signals were previously shown to adhere to these laws in subsets of the repertoire when used in play; highlighting that, in addition to selection on the signal repertoire, ape gestural expression appears shaped by factors in the immediate socio-ecological context.

Given-New Effects on the Duration of Gestures and of Words in Face-to-Face Dialogue

ABSTRACT The given-new contract entails that speakers must distinguish for their addressee whether references are new or already part of their dialogue. Past research had found that, in a monologue to a listener, speakers shortened repeated words. However, the notion of the given-new contract is inherently dialogic, with an addressee and the availability of co-speech gestures. Here, two face-to-face dialogue experiments tested whether gesture duration also follows the given-new contract. In Experiment 1, four experimental sequences confirmed that when speakers repeated their gestures, they shortened the duration significantly. Experiment 2 replicated the effect with spontaneous gestures in a different task. This experiment also extended earlier results with words, confirming that speakers shortened their repeated words significantly in a multimodal dialogue setting, the basic form of language use. Because words and gestures were not necessarily redundant, these results offer another instance in which gestures and words independently serve pragmatic requirements of dialogue.

Phonetics and phonology of nasality in Ecuadorian Siona

This study describes the nasal system in Ecuadorian Siona, an endangered Western Tukanoan language spoken in the Ecuadorian province of Sucumbíos, using the Earbuds Method to analyze nasal events acoustically. This method provides a visual representation of the timing and duration of velum gestures through intensity (dB) and amplitude (Pa) fluctuations in the nasal and oral cavities. The studied events include nasal spreading (nasal harmony), triggers, targets, blockers, and transparent segments. Meanwhile, differences between nasal phonemes and nasal allophones are also identified along with the effects of morpheme boundaries during nasal spreading events. Results reveal that, unlike many other Tukanoan languages, /m/ and /n/ function as individual phonemes independent of their oral counterparts (/p̰/ & /t̰/). In addition, nasal harmony was identified as predominantly rightward spreading apart from syllable-delimited leftward spreading to vocoid segments. Moreover, suffixes responsible for blocking nasal spreading appear to be reminiscent of oral suffixes in Eastern Tukanoan languages. Finally, more blockers were identified in Ecuadorian Siona than in most Eastern Tukanoan languages.

Self-organizing jaw gestures with speech rate increase

This article examines how speech rate increase acts to articulatorily change the movements of the jaw gesture. The EMA and acoustic experiments with two subjects have corroborated some previous findings, as follows: a) acceleration duration decrease from slow to fast rate; b) peak/valley velocity (modulus) decrease from slow to fast rate; c) gestural duration decrease from slow to fast rate; and d) constant proportional time-to-peak/valley-velocity from slow to fast rate. The hypotheses of y-extremum decrease from slow to fast rate and constriction displacement decrease from slow to fast rate were not statistically significant, and, therefore, need further experiments.

Dynamic model of speech

This article explores the relationship between intonational patterns and its relationship with speech rhythm and discourse, according to the dynamic systems research program. The study of these relationships were based on Barbosa’s (2006) Dynamic Model of Speech Rhythm; on Dato intonational annotation system proposed by Lucente (2008); and on the Computational Model of the Structure of Discourse, proposed by Grosz & Sidner (1986). The Dynamic Model of Rhythm suggests that the speech rhythm is the result of the action of two oscillators – accentual and syllabic - which receive as input linguistic and gestural information, and give the output as gestural duration. This article hypothesis is that in addition to these oscillators, a glottal oscillator can act controlling the intonation patterns of speech. These patterns, or intonational cycles, which organize the BP intonation, emerge when related to the spontaneous discourse segmentation. For each discourse segment classified as spontaneous according to a criteria proposed in this article, the speech is segmented into the DaTo system in linguistically structured units, which contains the purposes of communication and attention. Each of these segments is aligned to the speech intonation pattern delimitated by a rising contour (LH or> HL) at the beginning and by a falling contour (LHL), or a boundary level (L), at the end. The speech rhythm is also aligned to the pattern formed between intonation and discourse. By the inclusion of a new layer for the stress groups segmentation into DaTo system was possible to observe that the alignment between the stress groups segmentation and the intonational annotation coincide with discourse segments boundaries. The alignment between intonation, rhythm and discourse, having the stress groups as attractors, allowed us to propose the insertion of a glottal oscillator into the Dynamic Model of Rhythm.

Characterizing spoken responses in masked-onset priming of reading aloud using articulography.

A key method for studying articulatory planning at different levels of phonological organization is masked-onset priming. In previous work using that paradigm the dependent variable has been acoustic response time (RT). We used electromagnetic articulography to measure articulatory RTs and the articulatory properties of speech gestures in non-word production in a masked-onset priming experiment. Initiation of articulation preceded acoustic response onset by 199 ms, but the acoustic lag varied by up to 63 ms, depending on the phonological structure of the target. Onset priming affected articulatory response latency, but had no effect on gestural duration, inter-gestural coordination, or articulatory velocity. This is consistent with an account of the masked-onset priming effect in which the computation from orthography of an abstract phonological representation of the target is initiated earlier in the primed than in the unprimed condition. We discuss the implications of these findings for models of speech production and the scope of articulatory planning and execution.

A study of the drug secukinumab in the treatment of palmoplantar psoriasis

Palmoplantar psoriasis is a chronic (long term) and difficult‐to‐treat form of psoriasis that affects the palms of the hands and the soles of the feet. The disease causes reddish, scaly plaques on the palms and/or soles with or without the presence of such scales elsewhere on the body. Since palms and soles are affected, the patient can have serious issues in carrying out tasks of daily life such as walking, bathing and eating. This can lead to severe disturbance of their quality of life, and such patients may withdraw socially or suffer from depression and lack of confidence. A drug called secukinumab has been shown to offer rapid and long‐lasting effectiveness in treating patients with plaque psoriasis (psoriasis with scales on back, trunk, hands and legs); however, its effect on palmoplantar psoriasis had not been exclusively studied. A randomized control trial is a type of clinical trial in which patients are randomly assigned to a treatment group, and there is also a control group who do not receive any of the treatments being studied. GESTURE, the largest and longest duration randomized controlled study, was conducted to assess the efficacy and safety of secukinumab in people with palmoplantar psoriasis. Both the immediate (16 weeks) and long‐term (2.5 years) results were evaluated. Treatment with secukinumab resulted in quick improvements (starting as early as Week 2) in these patients and the Week 16 results proved the efficacy of secukinumab in palmoplantar psoriasis. The patients were further treated and followed‐up until 2.5 years to understand if the effects are continued or if the disease returns (known as relapse). The 2.5‐year results confirmed that secukinumab treatment resulted in long‐lasting effectiveness in patients with palmoplantar psoriasis. Patients experienced improved hand and feet functioning and therefore improved quality of life. No safety issues were identified. This is a summary of the study: Sustained efficacy of secukinumab in patients with moderate‐to‐severe palmoplantar psoriasis: 2.5‐year results from GESTURE, a randomized, double‐blind, placebo‐controlled trial

BRICS Message From South Africa

The paper features the data obtained from the analysis of a video strip with the help of ELAN 5.4, the free software developed by the experts from Max Planck Institute for Psycholinguistics, the Language Archive, Nijmegen, the Netherlands. The software enables to annotate video and audio strips, describing pauses, the duration of utterances, gestures, pronunciation and other linguistic and extralinguistic factors. The speaker in the video – South African President Cyril Ramaphosa – delivers his official address to the leaders of the 10th BRICS leadership summit in Sandton, Johannesburg on July 26, 2018. BRICS is a powerful link of a global financial architecture. Its main targets are to mobilize resources for sustainable development projects of BRICS and to facilitate the global growth of multilateral and regional financial, educational and industrial institutions. The material and the speaker for the analysis belong to the domain of BRICS top level politics. South Africa was the main host of the leadership summit in 2018. South African President Cyril Ramaphosa in his speech stressed the significance of the fourth industrial revolution highlighted by Professor Klaus Schawb at the World Economic Forum in Davos in 2016. The notion of the revolution appeared in the South African leader's address 7 times. Nevertheless, the authors of the paper see more messages hidden between the lines of the South African President's address. In the paper it is argued that BRICS architecture has a right to be interpreted as an attempt of keeping the world away from further plunging into environmental degradation, the development of critical thinking and innovation among BRICS citizens. The purpose of the paper is to demonstrate the method of pauses analysis to reveal a more complex mixture of speakers' visions. Long pauses are meaningful and extremely informative for discourse analysis. The data may be relevant for discourse analysis experts, political journalists, educators and copywriters.

Linguistic laws in chimpanzee gestural communication.

Studies testing linguistic laws outside language have provided important insights into the organization of biological systems. For example, patterns consistent with Zipf's law of abbreviation (which predicts a negative relationship between word length and frequency of use) have been found in the vocal and non-vocal behaviour of a range of animals, and patterns consistent with Menzerath's law (according to which longer sequences are made up of shorter constituents) have been found in primate vocal sequences, and in genes, proteins and genomes. Both laws have been linked to compression-the information theoretic principle of minimizing code length. Here, we present the first test of these laws in animal gestural communication. We initially did not find the negative relationship between gesture duration and frequency of use predicted by Zipf's law of abbreviation, but this relationship was seen in specific subsets of the repertoire. Furthermore, a pattern opposite to that predicted was seen in one subset of gestures-whole body signals. We found a negative correlation between number and mean duration of gestures in sequences, in line with Menzerath's law. These results provide the first evidence that compression underpins animal gestural communication, and highlight an important commonality between primate gesturing and language.

A multifactorial investigation of captive gorillas’ intraspecific gestural laterality

ABSTRACTMultifactorial investigations of intraspecific laterality of primates’ gestural communication aim to shed light on factors that underlie the evolutionary origins of human handedness and language. This study assesses gorillas’ intraspecific gestural laterality considering the effect of various factors related to gestural characteristics, interactional context and sociodemographic characteristics of signaller and recipient. Our question was: which factors influence gorillas’ gestural laterality? We studied laterality in three captive groups of gorillas (N = 35) focusing on their most frequent gesture types (N = 16). We show that signallers used predominantly their hand ipsilateral to the recipient for tactile and visual gestures, whatever the emotional context, gesture duration, recipient’s sex or the kin relationship between both interactants, and whether or not a communication tool was used. Signallers’ contralateral hand was not preferentially used in any situation. Signallers’ right-hand use was more pronounced in negative contexts, in short gestures, when signallers were females and its use increased with age. Our findings showed that gorillas’ gestural laterality could be influenced by different types of social pressures thus supporting the theory of the evolution of laterality at the population level. Our study also evidenced that some particular gesture categories are better markers than others of the left-hemisphere language specialization.

Context-dependent articulation of consonant gemination in Estonian

The three-way quantity system is a well-known phonological feature of Estonian. In a number of studies it has been shown that quantity is realized in a disyllabic foot by the stressed-to-unstressed syllable rhyme duration ratio and also by pitch movement as the secondary cue. The stressed syllable rhyme duration is achieved by combining the length of the vowel and the coda consonant, which enables minimal septets of CVCV-sequences based on segmental duration. In this study we analyze articulatory (EMA) recordings from four native Estonian speakers producing all possible quantity combinations of intervocalic bilabial stops in two vocalic contexts (/ɑ-i/ vs. /i-ɑ/). The analysis shows that kinematic characteristics (gesture duration, spatial extent, and peak velocity) are primarily affected by quantity on the segmental level: Phonologically longer segments are produced by longer and larger lip closing gestures and, in reverse, shorter and smaller lip opening movements. Tongue transition gesture is consistently lengthened and slowed down by increasing consonant quantity. In general, both kinematic characteristics and inter-gestural coordination are influenced by non-linear interactions between segmental quantity levels as well as vocalic context.

Continuous hand gesture recognition based on trajectory shape information

In this paper, we propose a continuous hand gesture recognition method based on trajectory shape information. A key issue in recognizing continuous gestures is that performance of conventional recognition algorithms may be lowered by such factors as, unknown start and end points of a gesture or variations in gesture duration. These issues become particularly difficult for those methods that rely on temporal information. To alleviate the issues of continuous gesture recognition, we propose a framework that simultaneously performs both segmentation and recognition. Each component of the framework applies shape-based information to ensure robust performance for gestures with large temporal variation. A gesture trajectory is divided by a set of key frames by thresholding its tangential angular change. Variable-sized trajectory segments are then generated using the selected key frames. For recognition, these trajectory segments are examined to determine whether the segment belongs to a class among intended gestures or a non-gesture class based on fusion of shape information and temporal features. In order to assess performance, the proposed algorithm was evaluated with a database of digit hand gestures. The experimental results indicate that the proposed algorithm has a high recognition rate while maintaining its performance in the presence of continuous gestures.

Beating time: How ensemble musicians' cueing gestures communicate beat position and tempo.

Ensemble musicians typically exchange visual cues to coordinate piece entrances. “Cueing-in” gestures indicate when to begin playing and at what tempo. This study investigated how timing information is encoded in musicians’ cueing-in gestures. Gesture acceleration patterns were expected to indicate beat position, while gesture periodicity, duration, and peak gesture velocity were expected to indicate tempo. Same-instrument ensembles (e.g., piano–piano) were expected to synchronize more successfully than mixed-instrument ensembles (e.g., piano–violin). Duos performed short passages as their head and (for violinists) bowing hand movements were tracked with accelerometers and Kinect sensors. Performers alternated between leader/follower roles; leaders heard a tempo via headphones and cued their partner in nonverbally. Violin duos synchronized more successfully than either piano duos or piano–violin duos, possibly because violinists were more experienced in ensemble playing than pianists. Peak acceleration indicated beat position in leaders’ head-nodding gestures. Gesture duration and periodicity in leaders’ head and bowing hand gestures indicated tempo. The results show that the spatio-temporal characteristics of cueing-in gestures guide beat perception, enabling synchronization with visual gestures that follow a range of spatial trajectories.

Earbuds: A Method for Analyzing Nasality in the Field

Existing methods for collecting and analyzing nasality data are problematic for linguistic fieldworkers: aerodynamic equipment can be expensive and difficult to transport, and acoustic analyses require large amounts of optimally-recorded data. In this paper, a highly mobile and low-cost method is proposed. By connecting low impedance earbuds into a microphone jack of a recording device and placing one earbud immediately below one nostril while keeping the other earbud by the mouth, it is possible to capture the relative intensity of sound exiting the nasal and oral cavities. The two channels can then be normalized to assess the relative prominence of nasality and orality in a given speech sound. This method can not only be used to establish whether nasality is present in a speech signal, but it can also provide information about the timing and duration of nasal gestures. As such, it is an ideal tool for collecting high-quality nasality data in the field.