Single Gesture Research Articles

Machine Learning-Assisted Gesture Sensor Made with Graphene/Carbon Nanotubes for Sign Language Recognition.

Gesture sensors are essential to collect human movements for human-computer interfaces, but their application is normally hampered by the difficulties in achieving high sensitivity and an ultrawide response range simultaneously. In this article, inspired by the spider silk structure in nature, a novel gesture sensor with a core-shell structure is proposed. The sensor offers a high gauge factor of up to 340 and a wide response range of 60%. Moreover, the sensor combining with a deep learning technique creates a system for precise gesture recognition. The system demonstrated an impressive 99% accuracy in single gesture recognition tests. Meanwhile, by using the sliding window technology and large language model, a high performance of 97% accuracy is achieved in continuous sentence recognition. In summary, the proposed high-performance sensor significantly improves the sensitivity and response range of the gesture recognition sensor. Meanwhile, the neural network technology is combined to further improve the way of daily communication by sign language users.

An Embedded Neural Network Approach for Reinforcing Deep Learning: Advancing Hand Gesture Recognition

Deep neural networks (DNNs) can face limitations during training for recognition, motivating this study to improve recognition capabilities by optimizing deep learning features for hand gesture image recognition. We propose a novel approach that enhances features from well-trained DNNs using an improved radial basis function (RBF) neural network, targeting recognition within individual gesture categories. We achieve this by clustering images with a self-organizing map (SOM) network to identify optimal centers for RBF training. Our enhanced SOM, employing the Hassanat distance metric, outperforms the traditional K-Means method across a comparative analysis of various distance functions and the expanded number of cluster centers, accurately identifying hand gestures in images. Our training pipeline learns from hand gesture videos and static images, addressing the growing need for machines to interact with gestures. Despite challenges posed by gesture videos, such as sensitivity to hand pose sequences within a single gesture category and overlapping hand poses due to the high similarities and repetitions, our pipeline achieved significant enhancement without requiring time-related training data. We also improve the recognition of static hand pose images within the same category. Our work advances DNNs by integrating deep learning features and incorporating SOM for RBF training.

Quantized CNN-based efficient hardware architecture for real-time hand gesture recognition

Nowadays, Convolutional Neural Networks (CNN) have been widely adopted for vision-based hand gesture recognition. Several existing CNN architectures designed for gesture classification perform well with high accuracy but require a high memory footprint and processing when deployed on low-power embedded devices. To address this issue, we present a quantized CNN-based efficient framework for meeting real-life hand gesture recognition challenges. The proposed quantized CNN is designed and implemented using the FINN-based pipelined streaming architecture on an FPGA. Moreover, hardware-based optimizations are used to minimize the resources needed and to achieve fast memory access. Experimental results demonstrate that the developed recognition system achieves an average accuracy of 92% on the numeral database of Indian Sign Language (ISL). Additionally, our optimized design attains an inference latency of 0.85ms for real-time single gesture prediction on the PYNQ Zynq Ultrascale (ZU) FPGA, consuming only 3.63 W of power. The proposed design achieves a better trade-off between hardware resource utilization and speed performance, over previous designs.

Men Becoming Gods in “Style”

Men Becoming Gods in “Style”

UltrasonicGS: A Highly Robust Gesture and Sign Language Recognition Method Based on Ultrasonic Signals.

With the global spread of the novel coronavirus, avoiding human-to-human contact has become an effective way to cut off the spread of the virus. Therefore, contactless gesture recognition becomes an effective means to reduce the risk of contact infection in outbreak prevention and control. However, the recognition of everyday behavioral sign language of a certain population of deaf people presents a challenge to sensing technology. Ubiquitous acoustics offer new ideas on how to perceive everyday behavior. The advantages of a low sampling rate, slow propagation speed, and easy access to the equipment have led to the widespread use of acoustic signal-based gesture recognition sensing technology. Therefore, this paper proposed a contactless gesture and sign language behavior sensing method based on ultrasonic signals-UltrasonicGS. The method used Generative Adversarial Network (GAN)-based data augmentation techniques to expand the dataset without human intervention and improve the performance of the behavior recognition model. In addition, to solve the problem of inconsistent length and difficult alignment of input and output sequences of continuous gestures and sign language gestures, we added the Connectionist Temporal Classification (CTC) algorithm after the CRNN network. Additionally, the architecture can achieve better recognition of sign language behaviors of certain people, filling the gap of acoustic-based perception of Chinese sign language. We have conducted extensive experiments and evaluations of UltrasonicGS in a variety of real scenarios. The experimental results showed that UltrasonicGS achieved a combined recognition rate of 98.8% for 15 single gestures and an average correct recognition rate of 92.4% and 86.3% for six sets of continuous gestures and sign language gestures, respectively. As a result, our proposed method provided a low-cost and highly robust solution for avoiding human-to-human contact.

A Novel Machine Learning–Based Hand Gesture Recognition Using HCI on IoT Assisted Cloud Platform

Machine learning is a technique for analyzing data that aids the construction of mathematical models. Because of the growth of the Internet of Things (IoT) and wearable sensor devices, gesture interfaces are becoming a more natural and expedient human-machine interaction method. This type of artificial intelligence that requires minimal or no direct human intervention in decision-making is predicated on the ability of intelligent systems to self-train and detect patterns. The rise of touch-free applications and the number of deaf people have increased the significance of hand gesture recognition. Potential applications of hand gesture recognition research span from online gaming to surgical robotics. The location of the hands, the alignment of the fingers, and the hand-to-body posture are the fundamental components of hierarchical emotions in gestures. Linguistic gestures may be difficult to distinguish from nonsensical motions in the field of gesture recognition. Linguistic gestures may be difficult to distinguish from nonsensical motions in the field of gesture recognition. In this scenario, it may be difficult to overcome segmentation uncertainty caused by accidental hand motions or trembling. When a user performs the same dynamic gesture, the hand shapes and speeds of each user, as well as those often generated by the same user, vary. A machine-learning-based Gesture Recognition Framework (ML-GRF) for recognizing the beginning and end of a gesture sequence in a continuous stream of data is suggested to solve the problem of distinguishing between meaningful dynamic gestures and scattered generation. We have recommended using a similarity matching-based gesture classification approach to reduce the overall computing cost associated with identifying actions, and we have shown how an efficient feature extraction method can be used to reduce the thousands of single gesture information to four binary digit gesture codes. The findings from the simulation support the accuracy, precision, gesture recognition, sensitivity, and efficiency rates. The Machine Learning-based Gesture Recognition Framework (ML-GRF) had an accuracy rate of 98.97%, a precision rate of 97.65%, a gesture recognition rate of 98.04%, a sensitivity rate of 96.99%, and an efficiency rate of 95.12%.

Learning to express causal events in Mandarin Chinese: A multimodal perspective.

Previous research has shown language-specific features play a guiding role in how children develop expression of events with speech and gestures. This study adopts a multimodal approach and examines Mandarin Chinese, a language that features context use and verb serializations. Forty children (four-to-seven years old) and ten adults were asked to describe fourteen video stimuli depicting different types of causal events involving location/state changes. Participants' speech was segmented into clauses and co-occurring gestures were analyzed in relation to causation. The results show that the older the children, the greater the use of contextual clauses which contribute meaning to event descriptions. It is not until the age of six that children used adult-like structures - namely, using single gestures representing causing actions and aligning them with verb serializations in single clauses. We discuss the implications of these findings for the guiding role of language specificity in multimodal language development.

중국 근대] 밀려나는 근대, 떠오르는 왕조

Compared with the results in 2012-2020, the papers published in 2020-21 did not stand out in terms of special issues, the number of papers, and research subjects. There are cases in which the papers used new materials or dealt with uncommon topics, giving them a fresh feeling, but it can be said that these two years were calm and ordinary.BR Even in calm and ordinary times, researchers’ brains work actively and research develops. These research results are gathered to form a single gesture. That gesture was saying that when we talked about the years between 1840 and 1911 the modern era was pushed aside and the dynasty came to mind. Studies on the traditional themes were rare. The vacancy was filled with various other themes.BR What does it mean that there were many articles looking at and analyzing 1840-1911 from the point of view of the dynasty rather than modern times? This indicates considering that the Qing Dynasty was an important factor in understanding the era. Now, it is time to discuss classifying studies of 1840-1911 as part of the dynasty rather than that of modern history.

Wi-ID: WiFi-Based Identification System Using Rock-Paper-Scissors Hand Gestures

This paper proposes using a WiFi-based identification system, Wi-ID, to identify users from their unique hand gestures. Hand gestures from the popular game rock-paper-scissors are utilized for the system’s user authentication commands. The whole feature of three hand gestures is extracted instead of the single gesture feature extracted by the existing methods. Dynamic time warping (DTW) is utilized to analyze the amplitude information in the time domain based on linear discriminant analysis (LDA), while extract amplitude kurtosis (AP-KU) and shape skewness (SP-SK) are utilized to analyze the Wi-Fi signals energy distribution in the frequency domain. Based on the contributions of the extracted features, the random forests algorithm is utilized for weight inputs in the long short-term memory (LSTM) network. The experiment is conducted on a computer installed with an Intel 5300 wireless networking card to evaluate the effectiveness and robustness of the Wi-ID system. The experiment results showed the accuracy of the proposed Wi-ID system has a personal differentiation accuracy rate over 92%, and with an average accuracy of 96%. Authorized persons who performed incomplete hand gestures are identified with an accuracy of 92% and hostile intruders can be identified with a probability of 90%. Such performance demonstrates that the Wi-ID system achieved the aim of user authentication.

How one gesture curbed ethnic discrimination

AbstractMembers of ethnic and racial minorities across North America and Europe continue to face discrimination, for instance, when applying for jobs or seeking housing. Such unequal treatment can occur because societies categorize people into groups along social, cultural, or ethnic and racial lines that seemingly rationalize differential treatment. Research suggests that it may take generations for such differences to decline, if they change at all. Here, we show that a single gesture by international soccer players at the World Cup 2018 – followed by an extensive public debate – led to a measurable and lasting decline in discrimination. Immediately after the galvanizing event, invitation rates to view apartments increased by 6 percentage points for the migrant group represented by the players, while responses to the native population did not change noticeably. We demonstrate that anti‐immigrant behaviour can disband rapidly when the public receives messages challenging the nature of ethnic and racial categories while sharing a common cause.

Sign Language to Sentence Formation: A Real Time Solution for Deaf People

Communication is a basic need of every human being to exchange thoughts and interact with the society. Acute peoples usually confab through different spoken languages, whereas deaf people cannot do so. Therefore, the Sign Language (SL) is the communication medium of such people for their conversation and interaction with the society. The SL is expressed in terms of specific gesture for every word and a gesture is consisted in a sequence of performed signs. The acute people normally observe these signs to understand the difference between single and multiple gestures for singular and plural words respectively. The signs for singular words such as I, eat, drink, home are unalike the plural words as school, cars, players. A special training is required to gain the sufficient knowledge and practice so that people can differentiate and understand every gesture/sign appropriately. Innumerable researches have been performed to articulate the computer-based solution to understand the single gesture with the help of a single hand enumeration. The complete understanding of such communications are possible only with the help of this differentiation of gestures in computer-based solution of SL to cope with the real world environment. Hence, there is still a demand for specific environment to automate such a communication solution to interact with such type of special people. This research focuses on facilitating the deaf community by capturing the gestures in video format and then mapping and differentiating as single or multiple gestures used in words. Finally, these are converted into the respective words/sentences within a reasonable time. This provide a real time solution for the deaf people to communicate and interact with the society.

Cultural Considerations to the Life Participation Approach in Aphasia:A Filipino Case Study

Stroke is among the common causes of chronic disability (Feigin, 2014). Around one-third of stroke survivors are affected by aphasia, a communication disorder affecting the ability to comprehend and express oneself (Dickey, et. al., 2010). Culture is essential to understanding aphasia and providing person-centered care. Philippine cultural identity is reflected via respect for older persons, collectivism in family and community, and devotion to religion (Pe Pua & Protacio-Marcelino, 2000). The Filipino family is a primary support system, and cultural values directly influence caregiving approaches in chronic disability. This single case study examines the life of a Filipino man who has successfully lived with aphasia for over 25 years. Having finished a doctorate from Harvard University, served as the youngest University president, and member of the Philippine government, he suddenly had a stroke and was able to communicate only via single words, gestures, and facial expressions. His life is discussed in the context of the unique, multi-modal communication system which developed through the years with his family. Music and symbolism via watercolor paintings also define his aphasia journey. The value of religion in Filipino culture (Cruz, et. al., 2019) and its role in fostering positivity in his aphasia journey is examined. This study also highlights Filipino collectivism through the support of family and community in addressing communication needs and facilitating meaningful relationships at various stages in life. Cultural values deeply rooted in Filipino caregiving, such as debt of gratitude and filial devotion to parents (Enriquez, 1992), are discussed.

‘Address and command’: Two-handed mid-air interactions with multiple home devices

Mid-air gestural interaction with multiple targets (or devices) in the physical space presents several challenges: gesture design gets more complex in the attempt to avoid gesture conflicts and accidental activations; the potential commands are many but the gesture set must be kept small to enable memorability; and, from a design-process perspective, there are not readily available technologies for implementation and prototyping. In this paper, we present the Address and Command (A&C) interaction model that enables two-handed mid-air interactions with multiple remote devices, in a research approach that includes gesture elicitation, prototyping, and evaluation of the user experience. The A&C model requires that users employ the non-dominant hand to address a device (address gestures) and the dominant hand to provide a command to it (command gestures). This approach affords interactions that simultaneously address and command multiple devices in ubiquitous environments that respond to mid-air input. A&C interactions also afford the design of a single gesture for the same command when applicable for multiple devices, thus decreasing the total number of gestures to be memorized and promoting end-user learning of the gesture set. We have (a) conducted an elicitation study (n = 18) to define A&C gestures for seven smart home devices and twelve commands; (b) developed a spatial augmented reality prototype that responds to mid-air gesture commands exploiting MS Kinect sensor and Visual Gesture Builder, Unity 3D and projection-mapping technologies onto foam mockups; (c) tested the approach and the gesture vocabulary in terms of usability, memorability and user experience at a scenario of 36 tasks among devices (n = 17). We found that A&C interactions are feasible, fast, error-free, easy to learn and remember, and are highly valued in terms of user experience.

AI enabled sign language recognition and VR space bidirectional communication using triboelectric smart glove

Sign language recognition, especially the sentence recognition, is of great significance for lowering the communication barrier between the hearing/speech impaired and the non-signers. The general glove solutions, which are employed to detect motions of our dexterous hands, only achieve recognizing discrete single gestures (i.e., numbers, letters, or words) instead of sentences, far from satisfying the meet of the signers’ daily communication. Here, we propose an artificial intelligence enabled sign language recognition and communication system comprising sensing gloves, deep learning block, and virtual reality interface. Non-segmentation and segmentation assisted deep learning model achieves the recognition of 50 words and 20 sentences. Significantly, the segmentation approach splits entire sentence signals into word units. Then the deep learning model recognizes all word elements and reversely reconstructs and recognizes sentences. Furthermore, new/never-seen sentences created by new-order word elements recombination can be recognized with an average correct rate of 86.67%. Finally, the sign language recognition results are projected into virtual space and translated into text and audio, allowing the remote and bidirectional communication between signers and non-signers.

Urban cloverleaves: origin, aesthetics, and contradictions of the Slussen, Stockholm

On 11 October 1935, the inauguration of the Slussen urban cloverleaf took place in front of the excited citizens of Stockholm. It had the attributes of a pure traffic machine taken from the most advanced traffic engineering publications, and it expressed the optimistic cultural modernism that five years ago the Stockholm International Exhibition had promoted.1 This urban cloverleaf was made of translucent glass, reinforced concrete, metallic handrails, and reflective tiles and was meant to solve, in one single gesture, the complex urban link between the Lake Mälaren and the Baltic Sea, between Gamla Stan – the historic city centre – and Södermalm – the southern district built on top of the 35-metre-high plateau [1]. The solution made difficult urban compromises between the foothills of the Brunkeberg topography, the smooth water surfaces of the Stockholm archipelago, the architecture of the historic urban tissue, and the demands of a complex articulated mobility. Boats, goods, suburban trains, subways, trams – later buses – pedestrians, cyclists, and automobiles finally converged on this place at different levels, completing the intricacies of a threedimensional geometry which, for the first time in history, was inserted into a compact city.

November, 1494: Politics in Motion in the Streets of Florence

Abstract The spectacular entry of Charles VIII into Florence in 1494 initiated a series of political negotiations and maneuvers in which the French monarch, the local government, and the embattled Medici family vied for control over the city. With the threat of violence so present and reliable information so scarce, Florentines had to perform subtle interpretations of the movements of these actors in order to determine what was happening to their city. These eye-witness accounts reveal that the most elaborately staged ritual and the most improvised single gesture were part of a moving tableau whose meaning was contingent on understanding the relationship between the paths taken, the places visited, and the manner in which the protagonists moved through the city. As a result, we can learn how early modern cities were constituted by the ceaseless exchanges between the ephemeral movements of communities and the solidity of the built environment.

Your Knock Is My Command: Binary Hand Gesture Recognition on Smartphone with Accelerometer

Motion-based hand gesture is an important scheme to allow users to invoke commands on their smartphones in an eyes-free manner. However, the existing scheme is facing some problems. On the one hand, the expression ability of one single gesture is limited. As a result, a gesture set consisting of multiple gestures is typically adopted to represent different commands. Users must memorize all gestures in order to make interaction successfully. On the other hand, the design of gestures needs to be complicated to express diverse intensions. However, complex gestures are difficult to learn and remember. In addition, complex gestures set a high recognition barrier to smart APPs. This leads to an imbalance problem. Different gestures have different recognition accuracy levels, which may result in instability of recognition precision in practical applications. To address these problems, this paper proposes a novel scheme using binary motion gestures. Only two simple gestures are required to express bit “0” and “1,” and rich information can be expressed through the permutation and combination of the two binary gestures. Firstly, four kinds of candidate binary gestures are evaluated for eyes-free interactions. Then, an online signal cutting and merging algorithm is designed to split accelerometer signals sequence into multiple separate gesture signal segments. Next, five algorithms, including Dynamic Time Warping (DTW), Naive Bayes, Decision Tree, Support Vector Machine (SVM), and Bidirectional Long Short-Term Memory (BLSTM) Network, are adopted to recognize these segments of knock gestures. The BLSTM achieves the top performance in terms of both recognition accuracy and recognition imbalance. Finally, an Android application is developed to illustrate the usability of the proposed binary gestures. As binary gestures are much simpler than traditional hand gestures, they are more efficient and user-friendly. Our scheme eliminates the imbalance problem and achieves high recognition accuracy.

American Sign Language Recognition Using an FMCW Wireless Sensor (Student Abstract)

In today's digital world, rapid technological advancements continue to lessen the burden of tasks for individuals. Among these tasks is communication across perceived language barriers. Indeed, increased attention has been drawn to American Sign Language (ASL) recognition in recent years. Camera-based and motion detection-based methods have been researched extensively; however, there remains a divide in communication between ASL users and non-users. Therefore, this research team proposes the use of a novel wireless sensor (Frequency-Modulated Continuous-Wave Radar) to help bridge the gap in communication. In short, this device sends out signals that detect the user's body positioning in space. These signals then reflect off the body and back to the sensor, developing thousands of cloud points per second, indicating where the body is positioned in space. These cloud points can then be examined for movement over multiple consecutive time frames using a cell division algorithm, ultimately showing how the body moves through space as it completes a single gesture or sentence. At the end of the project, 95% accuracy was achieved in one-object prediction as well as 80% accuracy on cross-object prediction with 30% other objects' data introduced on 19 commonly used gestures. There are 30 samples for each gesture per person from three persons.

Bimanual gesture recognition based on convolution neural network

Gesture recognition is a key research field in the human-computer interaction. At present, most of researchers focus on one-handed gesture recognition, but do not pay much attention to bimanual (two hands) gesture recognition. This paper presents a deep learning-based solution to tackle the self-occlusion and self-similarity. To solve this problem, this paper uses Kinect to collect many colour and depth images of different gestures, and each gesture contains multiple sample individuals. Colour images and depth images are used to train the recognition model of bimanual gesture respectively, and then the colour image and depth image are fused, and the bimanual gesture recognition model is trained based on colour image and depth image fusion. Then, the bimanual recognition effects of the three models are compared. The experimental results show that, regardless of the single gesture precision or the mean average precision, the bimanual gesture recognition effect of the fused model is better than the gesture recognition models based on either colour image or depth image.

Types of iconicity and combinatorial strategies distinguish semantic categories in silent gesture across cultures

abstractIn this study we explore whether different types of iconic gestures (i.e., acting, drawing, representing) and their combinations are used systematically to distinguish between different semantic categories in production and comprehension. In Study 1, we elicited silent gestures from Mexican and Dutch participants to represent concepts from three semantic categories: actions, manipulable objects, and non-manipulable objects. Both groups favoured the acting strategy to represent actions and manipulable objects; while non-manipulable objects were represented through the drawing strategy. Actions elicited primarily single gestures whereas objects elicited combinations of different types of iconic gestures as well as pointing. In Study 2, a different group of participants were shown gestures from Study 1 and were asked to guess their meaning. Single-gesture depictions for actions were more accurately guessed than for objects. Objects represented through two-gesture combinations (e.g., acting + drawing) were more accurately guessed than objects represented with a single gesture. We suggest iconicity is exploited to make direct links with a referent, but when it lends itself to ambiguity, individuals resort to combinatorial structures to clarify the intended referent. Iconicity and the need to communicate a clear signal shape the structure of silent gestures and this in turn supports comprehension.