Motion Pattern Recognition Research Articles

Self-Repairing and Energy-Harvesting Triboelectric Sensor for Tracking Limb Motion and Identifying Breathing Patterns.

The increasing prevalence of health problems stemming from sedentary lifestyles and evolving workplace cultures has placed a substantial burden on healthcare systems. Consequently, remote health wearable monitoring systems have emerged as essential tools to track individuals' health and well-being. Self-powered triboelectric nanogenerators (TENGs) have exhibited significant potential for use as emerging detection devices capable of recognizing body movements and monitoring breathing patterns. However, several challenges remain to be addressed in order to fulfill the requirements for self-healing ability, air permeability, energy harvesting, and suitable sensing materials. These materials must possess high flexibility, be lightweight, and have excellent triboelectric charging effects in both electropositive and electronegative layers. In this work, we investigated self-healable electrospun polybutadiene-based urethane (PBU) as a positive triboelectric layer and titanium carbide (Ti3C2Tx) MXene as a negative triboelectric layer for the fabrication of an energy-harvesting TENG device. PBU consists of maleimide and furfuryl components as well as hydrogen bonds that trigger the Diels-Alder reaction, contributing to its self-healing properties. Moreover, this urethane incorporates a multitude of carbonyl and amine groups, which create dipole moments in both the stiff and the flexible segments of the polymer. This characteristic positively influences the triboelectric qualities of PBU by facilitating electron transfer between contacting materials, ultimately resulting in high output performance. We employed this device for sensing applications to monitor human motion and breathing pattern recognition. The soft and fibrous-structured TENG generates a high and stable open-circuit voltage of up to 30 V and a short-circuit current of 4 μA at an operation frequency of 4.0 Hz, demonstrating remarkable cyclic stability. A significant feature of our TENG is its self-healing ability, which allows for the restoration of its functionality and performance after sustaining damage. This characteristic has been achieved through the utilization of the self-healable PBU fibers, which can be repaired via a simple vapor solvent method. This innovative approach enables the TENG device to maintain optimal performance and continue functioning effectively even after multiple uses. After integration with a rectifier, the TENG can charge various capacitors and power 120 LEDs. Moreover, we employed the TENG as a self-powered active motion sensor, attaching it to the human body to monitor various body movements for energy-harvesting and sensing purposes. Additionally, the device demonstrates the capability to recognize breathing patterns in real time, offering valuable insights into an individual's respiratory health.

Multimodal integrated flexible electronic skin for physiological perception and contactless kinematics pattern recognition

Multimodal integrated flexible electronic skin offers exceptional processing capability and versatility in various application scenarios. Researchers are devoting to developing nanomaterials or nano-devices with multidisciplinary functions. Herein, a trinary-mode integrated flexible electronic skin (E-Skin) was designed for physiological perception and contactless kinematics pattern recognition, which serves as real-time health monitor, electrode patch and contactless sensor. The composite film of liquid metal particles and thermoplastic polyurethane elastomers (LMPs@TPU) was prepared by a combination of electrostatic spinning, electrostatic spraying, and knife coating, which can be stretched up to 404.7 % at stress of 1Mpa. The film was endowed three functional modes through different forms of encapsulation, which allows for (1) detecting human arthrosis movements, muscle stretching, and inspiration, (2) achieving a real-time capture of Electrocardiogram (ECG) and surface electromyography (sEMG), and a Human-Machine Interaction (HMI) system for game control based on sEMG signals was also developed, (3) a contactless sensor combining with a machine learning model which realizes the recognition of human motion patterns with a receiver operating characteristic curve and area under curve score (ROC-AUC-score) of 0.99. The three models of this E-Skin highlight the significant potential for industrialization in various fields, such as HMI, physiological perception, virtual reality, and motion analysis.

Concept and Pictogram-Based User-Interface Design of a Helper Tool for People with Aphasia.

Aphasia describes the lack of the already gained ability to use language in a common way. "Language" here covers all variations of forming or understanding messages. The APH-Alarm project aims to develop a service concept that provides alternative communication options for people with Aphasia to trigger timely help when needed. It considers that a typical user may not be familiar with modern technologies and offers several simple and intuitive options. The approach is based on event detection of gestures (during daytime or in bed), movement pattern recognition in bed, and an easy-to-use pictogram-based smartphone app. Agile evaluation of the smartphone app showed a promising outcome. The idea of a versatile and comprehensive solution for aphasic people to easily contact private or public helpers based on their actions or automatic detection is promising and will be further investigated in an upcoming field trial.

Research on eye movement data classification using support vector machine with improved whale optimization algorithm

When performing eye movement pattern classification for different tasks, support vector machines are greatly affected by parameters. To address this problem, we propose an algorithm based on the improved whale algorithm to optimize support vector machines to enhance the performance of eye movement data classification. According to the characteristics of eye movement data, this study first extracts 57 features related to fixation and saccade, then uses the ReliefF algorithm for feature selection. To address the problems of low convergence accuracy and easy falling into local minima of the whale algorithm, we introduce inertia weights to balance local search and global search to accelerate the convergence speed of the algorithm and also use the differential variation strategy to increase individual diversity to jump out of local optimum. In this paper, experiments are conducted on eight test functions, and the results show that the improved whale algorithm has the best convergence accuracy and convergence speed. Finally, this paper applies the optimized support vector machine model of the improved whale algorithm to the task of classifying eye movement data in autism, and the experimental results on the public dataset show that the accuracy of the eye movement data classification of this paper is greatly improved compared with that of the traditional support vector machine method. Compared with the standard whale algorithm and other optimization algorithms, the optimized model proposed in this paper has higher recognition accuracy and provides a new idea and method for eye movement pattern recognition. In the future, eye movement data can be obtained by combining it with eye trackers to assist in medical diagnosis.

Evaluation of CNN algorithm at locomotion mode identification for a knee-assisted exoskeleton

In this study, a knee-assisted exoskeleton that can assist the wearer's knee joints at the appropriate phase of the gait cycle is designed. A convolutional neural network (CNN) model is used in the gait control to identify the locomotion modes of the exoskeleton. In order to verify the CNN-based locomotion mode identification method, an offline CNN model was created in MATLAB. The experiment was conducted and seven healthy subjects’ hip, knee, and ankle joint angles were captured and six common locomotion modes were recognized in the experiments. The recorded data were segmented with overlapping sliding windows of varying sizes and were tested using a stratified 5-fold cross-validation technique. The experimental findings showed that the appropriate window size was 1.25T (T stands for the normal adult gait cycle). The lowest and the highest accuracies of the test sets at the window size of 1.25T were 96.47% ± 0.83% and 99.67% ± 0.49% on different subjects. It also showed that the model has strong generalization. Thus, the algorithm had proved useful to extract the characteristics of lower limb joint angles in various locomotion modes and the approach in this paper can be extended to the recognition of motion patterns in lower limb exoskeletons.

A low-cost framework for the recognition of human motion gait phases and patterns based on multi-source perception fusion

A low-cost framework for the recognition of human motion gait phases and patterns based on multi-source perception fusion

3D Dynamic Image Modeling Based on Machine Learning in Film and Television Animation

With the deep integration and rapid development of computer technology and film and television animation in recent years, computer animation technology has gradually created objective practical value. 3D animation technology also plays a key role in film and television special effects and advertising special effects. As a new type of multimedia data, human motion capture data can be used for 3D human model modeling and human motion simulation because of its high fidelity. A human motion pattern recognition method based on long short-term memory network (LSTM) is proposed. The model uses a deep learning neural network composed of a 2-layer LSTM network to automatically extract features from the collected human body feature information. Then, the multi-class motion patterns are modeled in time series to quickly identify different motion patterns in real time. To evaluate the performance of this model, the effectiveness of this method in identifying six different motion modes is validated using an open dataset. At the same time, this method is compared with four methods based on in-depth learning model. Experimental results verify the effectiveness of the method. It provides a feasible method for human motion recognition and modeling based on capture data in video animation.

CNN-LSTM Model for Recognizing Video-Recorded Actions Performed in a Traditional Chinese Exercise.

Identifying human actions from video data is an important problem in the fields of intelligent rehabilitation assessment. Motion feature extraction and pattern recognition are the two key procedures to achieve such goals. Traditional action recognition models are usually based on the geometric features manually extracted from video frames, which are however difficult to adapt to complex scenarios and cannot achieve high-precision recognition and robustness. We investigate a motion recognition model and apply it to recognize the sequence of complicated actions of a traditional Chinese exercise (ie, Baduanjin). We first developed a combined convolutional neural network (CNN) and long short-term memory (LSTM) model for recognizing the sequence of actions captured in video frames, and applied it to recognize the actions of Baduanjin. Moreover, this method has been compared with the traditional action recognition model based on geometric motion features in which Openpose is used to identify the joint positions in the skeletons. Its performance of high recognition accuracy has been verified on the testing video dataset, containing the video clips from 18 different practicers. The CNN-LSTM recognition model achieved 96.43% accuracy on the testing set; while those manually extracted features in the traditional action recognition model were only able to achieve 66.07% classification accuracy on the testing video dataset. The abstract image features extracted by the CNN module are more effective on improving the classification accuracy of the LSTM model. The proposed CNN-LSTM based method can be a useful tool in recognizing the complicated actions.

Semantic Recognition of Ship Motion Patterns Entering and Leaving Port Based on Topic Model

Recognition and understanding of ship motion patterns have excellent application value for ship navigation and maritime supervision, i.e., route planning and maritime risk assessment. This paper proposes a semantic recognition method for ship motion patterns entering and leavingport based on a probabilistic topic model. The method enables the discovery of ship motion patterns from a large amount of trajectory data in an unsupervised manner and makes the results more interpretable. The method includes three modules: trajectory preprocessing, semantic process, and knowledge discovery. Firstly, based on the activity types and characteristics of ships in the harbor waters, we propose a multi-criteria ship motion state recognition and voyage division algorithm (McSMSRVD), and ship trajectory is divided into three sub-trajectories: hoteling, maneuvering, and normal-speed sailing. Secondly, considering the influence of port traffic rules on ship motion, the semantic transformation and enrichment of port traffic rules and ship location, course, and speed are combined to construct the trajectory text document. Ship motion patterns hidden in the trajectory document set are recognized using the Latent Dirichlet allocation (LDA) topic model. Meanwhile, topic coherence and topic correlation metrics are introduced to optimize the number of topics. Thirdly, a visualization platform based on ArcGIS and Electronic Navigational Charts (ENCs) is designed to analyze the knowledge of ship motion patterns. Finally, the Tianjin port in northern China is used as the experimental object, and the results show that the method is able to identify 17 representative inbound and outbound motion patterns from AIS data and discover the ship motion details in each pattern.

Facile Approach to Fabricate Oriented Porous PDMS Composites for Movements Monitoring and Identifying Motion Patterns.

The facile and rapid fabrication of oriented porous polymers is crucial for flexible pressure sensors. Herein, a pressure sensor is developed based on oriented porous polydimethylsiloxane (PDMS) composites for detecting human motion and identifying joint motion patterns. The oriented porous PDMS composite is first constructed through thiol-ene click chemistry and directional freezing within only 30min, then fabricated by interfacial in situ polymerization of dopamine and pyrrole to generate robust interfaces. As a result, the as-prepared oriented porous PDMS composite is assembled into a pressure sensor that shows potential applications in pressure and human motion detection. Interestingly, a sensor assembled by orthogonally stacking the PDMS composites can be used for joint motion pattern recognition with potential monitoring of football motion due to their directional structures. This facile strategy coupled with the oriented porous structure is expected to help design advanced wearable electronic devices.

Involuntary Breathing Movement Pattern Recognition and Classification via Force-Based Sensors

The study presents a novel scheme that recognizes and classifies different sub-phases within the involuntary breathing movement (IBM) phase during breath-holding (BH). We collected force data from eight recreational divers until the conventional breakpoint (CB). They were in supine positions on force plates. We segmented their data into no-movement (NM) phases, i.e., the easy phase (EP) and IBM phase (comprising several events or sub-phases of IBM). Acceleration and jerk were estimated from the data to quantify the IBMs, and phase portraits were developed to select and extract specific features. K means clustering was performed on these features to recognize different sub-phases within the IBM phase. We found five–six optimal clusters separating different sub-phases within the IBM phase. These clusters separating different sub-phases have physiological relevance to internal struggles and were labeled as classes for classification using support vector machine (SVM), naive Bayes (NB), decision tree (DT), and K-nearest neighbor (K-NN). In comparison with no feature selection and extraction, we found that our phase portrait method of feature selection and extraction had low computational costs and high robustness of 96–99% accuracy.

Identification of phantom movements with an ensemble learning approach

Phantom limb pain after amputation is a debilitating condition that negatively affects activities of daily life and the quality of life of amputees. Most amputees are able to control the movement of the missing limb, which is called the phantom limb movement. Recognition of these movements is crucial for both technology-based amputee rehabilitation and prosthetic control. The aim of the current study is to classify and recognize the phantom movements in four different amputation levels of the upper and lower extremities. In the current study, we utilized ensemble learning algorithms for the recognition and classification of phantom movements of the different amputation levels of the upper and lower extremity. In this context, sEMG signals obtained from 38 amputees and 25 healthy individuals were collected and the dataset was created. Studies of processing sEMG signals in amputees are rather limited, and studies are generally on the classification of upper extremity and hand movements. Our study demonstrated that the ensemble learning-based models resulted in higher accuracy in the detection of phantom movements. The ensemble learning-based approaches outperformed the SVM, Decision tree, and kNN methods. The accuracy of the movement pattern recognition in healthy people was up to 96.33%, this was at most 79.16% in amputees.

High real-time locomotion recognition algorithm using inertial measurement units

Human-machine interaction devices (HMI), such as assisted exoskeletons and assisted prostheses, that improve human motion behaviours are playing an increasingly important role in various fields. In order for these HMI devices to provide accurate and appropriate assistance to wearers, the real-time recognition of locomotion with accuracy is among the key issues to be solved. This paper proposes a high real-time motion pattern recognition algorithm using inertial measurement units (IMU) that can accurately and quickly identify the current motion state by collecting various motion parameters of the lower limb during human walking. The algorithm uses a trajectory solving and motion direction discrimination method based on the shank zero velocity point and a terrain classification method based on an F-score improved support vector machine (F-score-SVM). In this paper, experimental data was used to verify the performance of the algorithm. Results show that the algorithm performs better than traditional recognition algorithms in complex environments.

A new approach to dominant motion pattern recognition at the macroscopic crowd level

Automatic analysis and the recognition and prediction of the behaviour of large-scale crowds in video-surveillance data is a research field of paramount importance for the security of modern societies. It serves to predict and help prevent disasters in public places where crowds of people gather. The paper proposes a novel method for generating meta-tracklets and recognition of dominant motion patterns as a basis for automatic crowd behaviour analysis at the macroscopic level, where a crowd is treated as an entity. The basic characteristic of macroscopic crowd scenes is that it is impossible to detect and track individuals in the scene. The idea of the method proposed in this paper is to recognize dominant crowd motion patterns, by avoiding time-consuming and error-sensitive crowd segmentation, crowd tracking and detection of regions of interest. Thus, the process of determining dominant motion patterns and recognizing crowd behaviour is accelerated. The method is inspired by a quantum mechanical approach. It combines a set of particles, which are considered as particles in quantum mechanics, tracklets of particles’ advection in a video clip, and the interaction of wave functions spread out from particle positions. A wave function is expressed in the form of an asymmetric potential function. Peaks of the wave field define the most probable particle flow, which defines a meta-tracklet. Dominant motion patterns are recognized by applying the functions of fuzzy predicates, which represent a combination of common-sense and human expert knowledge about crowd motions, to the meta-tracklets. The experimental results of the proposed method are presented for a subset of UCF dataset and AGORASET crowd simulation videos and have shown promising results in dominant motion pattern recognition.

Deep Learning Assisted Body Area Triboelectric Hydrogel Sensor Network for Infant Care (Adv. Funct. Mater. 35/2022)

Triboelectric Hydrogel Sensor Networks In article number 2204803, Hulin Zhang, Jun Chen, and co-workers construct a body area sensor network based on edible triboelectric hydrogel sensors, which can realize the recognition of multiple motion patterns of infants with the assistance of deep learning algorithms. Moreover, an application is also developed to wirelessly transmit the motion signals to a mobile phone, providing a promising paradigm for infant care.

Human Motion Pattern Recognition and Feature Extraction: An Approach Using Multi-Information Fusion.

An exoskeleton is a kind of intelligent wearable device with bioelectronics and biomechanics. To realize its effective assistance to the human body, an exoskeleton needs to recognize the real time movement pattern of the human body in order to make corresponding movements at the right time. However, it is of great difficulty for an exoskeleton to fully identify human motion patterns, which are mainly manifested as incomplete acquisition of lower limb motion information, poor feature extraction ability, and complicated steps. Aiming at the above consideration, the motion mechanisms of human lower limbs have been analyzed in this paper, and a set of wearable bioelectronics devices are introduced based on an electromyography (EMG) sensor and inertial measurement unit (IMU), which help to obtain biological and kinematic information of the lower limb. Then, the Dual Stream convolutional neural network (CNN)-ReliefF was presented to extract features from the fusion sensors’ data, which were input into four different classifiers to obtain the recognition accuracy of human motion patterns. Compared with a single sensor (EMG or IMU) and single stream CNN or manual designed feature extraction methods, the feature extraction based on Dual Stream CNN-ReliefF shows better performance in terms of visualization performance and recognition accuracy. This method was used to extract features from EMG and IMU data of six subjects and input these features into four different classifiers. The motion pattern recognition accuracy of each subject under the four classifiers is above 97%, with the highest average recognition accuracy reaching 99.12%. It can be concluded that the wearable bioelectronics device and Dual Stream CNN-ReliefF feature extraction method proposed in this paper enhanced an exoskeleton’s ability to capture human movement patterns, thus providing optimal assistance to the human body at the appropriate time. Therefore, it can provide a novel approach for improving the human-machine interaction of exoskeletons.

Accurate Recognition of Motion Patterns Based on Artificial Visual Neural Network

In order to improve the accuracy of motion pattern recognition, this paper combines the artificial visual neural network to construct a motion pattern recognition system. Moreover, this paper discusses the psychological perception properties of human eyes to color stimuli and gives a description of the observation field of view where the color stimuli are located. At the same time, this paper analyzes the phenomenon of color adaptation and provides a method of color appearance matching through modeling to achieve color appearance matching under variable observation conditions. Based on the chromatic adaptation transformation, a chromatic appearance model is given, which can predict the corresponding color and also predict the chromatic appearance properties of color stimuli under given observation conditions. In addition, this paper constructs an intelligent motion pattern recognition system combined with artificial visual neural network. The experimental results show that the motion pattern recognition system based on artificial visual neural network can accurately identify the motion pattern category.

Sports Analysis and Action Optimization in Physical Education Teaching Practice Based on Internet of Things Sensor Perception.

With the progress of the Internet of things technology in recent years, all aspects of people's lives have also been affected. More and more people are immersed in the virtual world and ignore the real activities. According to the survey, nearly 50% of people in China are in subhealth, mainly modern diseases caused by long-term inactivity. Therefore, to form a good habit of physical exercise, we must start from an early age. Starting from the physical education teaching in primary and secondary schools and from the perspective of modern scientific and technological facilities, this paper discusses the practical sports analysis and action optimization of physical education teaching based on the perception of the Internet of things. Starting from the practice of Internet of things sensor sensing in physical education teaching, we have successively determined the multisensor motion acquisition system algorithm, motion pattern recognition algorithm, and motion energy consumption algorithm, which provides modern equipment for motion analysis and motion optimization in physical education teaching practice, which breaks the current situation that traditional teachers spend a lot of time and energy for students. Combining sports mode with sports energy consumption can not only analyze sports data accurately and in real time but also optimize and predict students' sports behavior in time. We hope to supervise and urge primary and secondary school students to exercise through technical means to improve the quality of primary and secondary school students' exercise and improve people's health through physical exercise.

Application of Accelerometer to Monitor Students' Exercise Load in 50 m Round Trip.

With the further advancement of microelectronics innovation and sensors, sensors can be broadly implanted in cell phone gadgets, compact gadgets, and so forth. The utilization of speed increase sensors for human running checking has expansive application possibilities. From one perspective, the everyday development of the human body is firmly connected with the physical and emotional wellness of the person. Observing the day-to-day developments of the human body is of incredible importance in planning a logical running activity plan and working on actual wellbeing. On the other hand, it is also of practical value to monitor human abnormal movements. This kind of abnormal movement caused by accidental falls can bring certain harm to the human body. Real-time monitoring of the fall can provide timely assistance to the person and reduce the risk brought by the fall. This article analyzes and summarizes the research theories and common research methods in the field of 50 m round-trip movement monitoring based on the acceleration sensor. According to the process of 50 m round-trip movement pattern recognition, the data collection, preprocessing, feature extraction, and selection of 50 m round-trip movement are evaluated. The classification and recognition of each module were analyzed. This article proposes a human body motion recognition mechanism based on acceleration sensors by looking at the three trademark upsides, the wavefront edge, wavefront limit, and time stretch between the pinnacle and valley of the speed increase sensor vertical information waveform, and joining the rule of choice tree order to accomplish the activities of hunching down, taking off, and running. To get an accurate recognizable proof and recognize ways of behaving, a human fall identification calculation is proposed. This calculation removes human movement attributes throughout the fall and focuses on four sorts of falls: forward fall, reverse fall, left fall, and right fall by utilizing the connection of the three tomahawks of the speed increase sensor. The trial results show that the normal right acknowledgment pace of the human body's 50 m full-circle running way of behaviour is more than 90%, which has specific useful application esteem.

Human (but not animal) motion can be recognized at first sight – After treatment for congenital blindness

The long-standing nativist vs. empiricist debate asks a foundational question in epistemology – does our knowledge arise through experience or is it available innately? Studies that probe the sensitivity of newborns and patients recovering from congenital blindness are central in informing this dialogue.One of the most robust sensitivities our visual system possesses is to ‘biological motion’ – the movement patterns of humans and other vertebrates. Various biological motion perception skills (such as distinguishing between movement of human and non-human animals, or between upright and inverted human movement) become evident within the first months of life. The mechanisms of acquiring these capabilities, and specifically the contribution of visual experience to their development, are still under debate.We had the opportunity to directly examine the role of visual experience in biological motion perception, by testing what level of sensitivity is present immediately upon onset of sight following years of congenital visual deprivation. Two congenitally blind patients who underwent sight-restorative cataract-removal surgery late in life (at the ages of 7 and 20 years) were tested before and after sight restoration. The patients were shown displays of walking humans, pigeons, and cats, and asked to describe what they saw.Visual recognition of movement patterns emerged immediately upon eye-opening following surgery, when the patients spontaneously began to identify human, but not animal, biological motion. This recognition ability was evident contemporaneously for upright and inverted human displays.These findings suggest that visual recognition of human motion patterns may not critically depend on visual experience, as it was evident upon first exposure to un-obstructed sight in patients with very limited prior visual exposure, and furthermore, was not limited to the typical (upright) orientation of humans in real-life settings.