Motion Branches Research Articles

Innovative approaches in imaging photoplethysmography for remote blood oxygen monitoring

Peripheral Capillary Oxygen Saturation (SpO2) has received increasing attention during the COVID-19 pandemic. Clinical investigations have demonstrated that individuals afflicted with COVID-19 exhibit notably reduced levels of SpO2 before the deterioration of their health status. To cost-effectively enable individuals to monitor their SpO2, this paper proposes a novel neural network model named “ITSCAN” based on Temporal Shift Module. Benefiting from the widespread use of smartphones, this model can assess an individual’s SpO2 in real time, utilizing standard facial video footage, with a temporal granularity of seconds. The model is interweaved by two distinct branches: the motion branch, responsible for extracting spatiotemporal data features and the appearance branch, focusing on the correlation between feature channels and the location information of feature map using coordinate attention mechanisms. Accordingly, the SpO2 estimator generates the corresponding SpO2 value. This paper summarizes for the first time 5 loss functions commonly used in the SpO2 estimation model. Subsequently, a novel loss function has been contributed through the examination of various combinations and careful selection of hyperparameters. Comprehensive ablation experiments analyze the independent impact of each module on the overall model performance. Finally, the experimental results based on the public dataset (VIPL-HR) show that our model has obvious advantages in MAE (1.10%) and RMSE (1.19%) compared with related work, which implies more accuracy of the proposed method to contribute to public health.

EM-Gait: Gait recognition using motion excitation and feature embedding self-attention

Gait recognition, which can realize long-distance and contactless identification, is an important biometric technology. Recent gait recognition methods focus on learning the pattern of human movement or appearance during walking, and construct the corresponding spatio-temporal representations. However, different individuals have their own laws of movement patterns, simple spatial–temporal features are difficult to describe changes in motion of human parts, especially when confounding variables such as clothing and carrying are included, thus distinguishability of features is reduced. To this end, we propose the Embedding and Motion (EM) block and Fine Feature Extractor (FFE) to capture the motion mode of walking and enhance the difference of local motion rules. The EM block consists of a Motion Excitation (ME) module to capture the changes of temporal motion and an Embedding Self-attention (ES) module to enhance the expression of motion rules. Specifically, without introducing additional parameters, ME module learns the difference information between frames and intervals to obtain the dynamic change representation of walking for frame sequences with uncertain length. By contrast, ES module divides the feature map hierarchically based on element values, blurring the difference of elements to highlight the motion track. Furthermore, we present the FFE, which independently learns the spatio-temporal representations of human body according to different horizontal parts of individuals. Benefiting from EM block and our proposed motion branch, our method innovatively combines motion change information, significantly improving the performance of the model under cross appearance conditions. On the popular dataset CASIA-B, our proposed EM-Gait is better than the existing single-modal gait recognition methods.

AEMNet: Unsupervised Video Anomaly Detection Method Based on Attention-Enhanced Memory Networks

Video anomaly detection has always been a challenging task in computer vision due to data imbalance and susceptibility to scene variations such as lighting and occlusions. In response to this challenge, this paper proposes an unsupervised video anomaly detection method based on an attention-enhanced memory network. The method utilizes a dual-stream network structure of autoencoders, enhancing the model’s learning ability for important features in appearance and motion by introducing coordinate attention mechanisms and variance attention mechanisms, emphasizing significant characteristics of static objects and rapidly moving regions. By adding memory modules to both the appearance and motion branches, the network structure’s memory information is reinforced, enabling it to capture long-term spatiotemporal dependencies in videos and thereby improving the accuracy of anomaly detection. Furthermore, by optimizing the network structure’s activation functions to handle negative inputs, it enhances its nonlinear modeling capabilities, enabling better adaptation to complex environments, including variations in lighting and occlusions, further improving the effectiveness of anomaly detection. The paper conducts comparative experiments and ablation studies using three public available datasets and various models. The results demonstrate that compared to baseline models, the AUC performance is improved by 3.9%, 4.7%, and 1.7% on UCSD Ped2, CHUK Avenue, and ShanghaiTech datasets, respectively. When compared with the other models, the average AUC performance is improved by 4.3%, 5.4%, and 6.2%, with an average improvement of 8.75% in the ERR metric, validating the effectiveness and adaptability of the proposed method. The code can be obtained at the following URL: https://github.com/AcademicWhite/AEMNet .

Motion Branch Transformation of 3(Rc)PU Parallel Mechanism with Reconfigurable Joint and Kinematic Performance Index

A novel 3(Rc)PU parallel mechanism based on the reconfigurable circular groove cross joint (Rc joint) is proposed. The motion branch transformation relationship and constraint performance evaluation method for the mechanism are studied. According to constrained screw analysis, the (Rc)PU limb provides a constraint couple to the moving platform in one motion mode and no constraint to the platform in another motion mode. According to the two configurations of the limb, the 3(Rc)PU mechanism has four different motion branches, namely 3T, 1R3T, 2R3T, and 3R3T. Based on the input selection principle, the input selection scheme of the 3(Rc)PU mechanism is determined, and it is concluded that, to achieve stable motion, at least two input drive pairs need to be applied on each limb. A unified kinematic model with four motion branches is established, and the geometric constraint equations of the 3(Rc)PU parallel mechanism are derived. After selecting the input actuated joints, the local minimization transmission index (LMTI) of the reconfigurable mechanism under different motion branches is established to evaluate the motion/force transmission performance of the reconfigurable mechanism under different motion branches. The conclusion shows that the LMTI values satisfy the corresponding constraint conditions, and the 3(Rc)PU parallel mechanism has good motion/force transmission performance.

Morphable thick-panel origami

Beyond being a traditional art form, origami can be applied to create complex deployable systems. Origami structures with one degree of freedom (DOF) are easy to actuate and have high stiffness but lack the ability to achieve different tasks. Previous multiple DOF origami structures are the opposite, having good task generalizability but low stiffness. In this paper, we design morphable origami structures with multiple one-DOF motion branches by using reconfigurable linkages. However, the kinematic models of previous thick-panel techniques are spherical linkages and overconstrained linkages, which limit the ability of structure morphing. This paper first proposes an approach to create reconfigurable linkages by connecting two one-DOF linkages and removing common parts. The new linkages have two motion branches corresponding to the two original linkages, and each motion branch is one-DOF. Then, we use the relationships between these reconfigurable linkages and thick-panel origami structures to construct morphable thick-panel origami structures. Furthermore, we also analyze their kinematics and motion branches. The concept of this paper, using reconfigurable linkages to design thick-panel origami structures, will improve the potential of origami techniques for different tasks.

SKT-MOT and DyTracker: A Multiobject Tracking Dataset and a Dynamic Tracker for Speed Skating Video

Speed skating serves as a significant application domain for multiobject tracking (MOT), presenting unique challenges such as frequent occlusion, highly similar appearances, and motion blur. To address these challenges, this paper constructs an MOT dataset called SKT-MOT for speed skating and analyzes the shortcomings of existing datasets and methods. Accordingly, we propose a dynamic MOT method called DyTracker. The method builds upon the DeepSORT baseline and enhances three key modules. At the global level, we design the track dynamic management (TDM) algorithm. In the motion branch, a novel metric is proposed to evaluate occlusion and Kalman filter dynamic update (KFDU) is implemented. In the appearance branch, we account for the difference in human posture and propose the feature dynamic selection and updating (FDSU) strategy. This makes our DyTracker flexible and efficient to achieve a multiobject tracking accuracy (MOTA) of 93.70% and identification F1 (IDF1) score of 92.39% on SKT-MOT, which is a significant advantage over existing SOTA methods. To validate the generalization of our proposed module, two dynamic update modules are inserted into other methods and validated on the public dataset MOT17, and the accuracy is generally improved by 0.2%–0.6%.

On the kinematics of general plane-symmetric Bricard mechanisms

The spatial 6R single-loop overconstrained Bricard mechanisms have been attracting continuous attention owing to their remarkable mobility. However, kinematic analysis and characterization of diversified Bricard mechanisms with different structures remain challenging due to their geometric versatility. This paper presents a comprehensive method with an intuitive geometric interpretation for the kinematic modeling and analysis of general plane-symmetric Bricard mechanisms. The concept of ‘half Bricard linkage’ is first introduced to simplify the kinematics of the studied plane symmetric originals with the idea of kinematic equivalence. From a geometric perspective, the forward kinematics of half Bricard linkage can be transformed into the intersection of two coplanar ellipses and resolved straightforwardly. Using the proposed method, the kinematic properties of a general plane-symmetric Bricard mechanism, such as dead center configurations, motion branches, and their connectivity, can be characterized intuitively according to the intersection situations of two spatial surfaces generated from the planar ellipses. Several well-studied examples are elaborated as case studies to demonstrate the effectiveness and advantages of the novel geometric method for the kinematic analysis of general plane-symmetric Bricard mechanisms.

Action recognition for sports combined training based on wearable sensor technology and SVM prediction

In the field of sports, coaches have mainly relied on observing the performance of athletes on the spot to formulate suitable training plans for athletes, which has extremely high requirements for the professionalism of coaches. Based on the above requirements, this paper designs a sports action recognition system for sports enthusiasts based on the SVM algorithm optimization model, and for the purpose of verifying the applicability of the system to different sports fields, experiments are carried out on basketball actions and race walking actions. The system uses wearable sensors to capture the motion data of the user, and then analyzes and identifies the user's actions through the SVM algorithm optimization model. By standardizing the user's sports combination training under the system algorithm, the user can improve their training efficiency and reduce the risk of injury. To establish the human body motion model, this paper divides the human skeleton model into five motion branches. The rotation freedom constraints and joint rotation angle range limits are added to the model to ensure the accuracy of the motion analysis. Combining the forward kinematics of the robot and the homogeneous coordinate transformation, the human body joint rotation motion model and the human bone position and posture model are established. In the end, the user can standardize the sports combination training under the system algorithm. In this paper, through the research of wearable sensor technology and sports combined training action recognition, and apply it to practical life, it aims to promote its development and application.

Design, analysis, and experiment of a piezoelectric stick–slip rotary actuator with asynchronous-driven dual stators

The piezoelectric stick–slip actuator with a compact structure shows great merits for space-critically limited applications, such as high-precision positioning and micro-nano operations. It can be used as a rotating part in micro-nano manipulators. However, piezoelectric stick–slip actuators operate under the friction-inertia principle. The backward motion is almost inevitable at low driving frequencies, hugely detrimental to manipulation accuracy, thus posing challenges to the configuration design and driving strategy of the actuator. This paper proposes a stick–slip rotary actuator with asynchronous-driven dual stators. Based on the principle of minimum motion branch chains and piezoelectric stacks, a dual-stator stick–slip rotary actuator with 2-R force-couple flexible stators is designed to enhance structure compactness. Then, an asynchronous driving method is proposed that enables the actuator to increase stickiness and reduce slippage without changing the driving waveform to minimize the backward motion. After that, the kinetic model with electromechanical-frictional coupling properties is developed, and the motion and load characteristics of the actuator are analyzed through MATLAB/Simulink simulations. The causes of the motion and load characteristics are discussed, and measures to improve the motion speed and load capacity are proposed. Finally, a prototype is fabricated, and several experiments are carried out. The results show that the backward motion of the actuator is significantly suppressed in the asynchronous driving mode, and its output displacement exhibits good stepping characteristics. The actuator has the advantages of a simple structure, little volume, small backward motion, and high resolution, demonstrating structural and performance superiorities for micro-nano manipulators in space-constrained applications.

A Kirigami-Inspired Metamorphic Double-Loop Linkage With Multiple Single-Degree-of-Freedom Reconfiguration Branches

Abstract Reconfigurable mechanisms attract increasing attention for the characteristic to change structure configurations and motion behaviors in various multifunctional systems. By drawing inspiration from a kirigami fold, this paper presents a metamorphic double-loop linkage with multiple single-degree-of-freedom (DOF) reconfiguration branches, which is generated through combining two reconfigurable 6R plane-symmetric linkages. Mobility analysis of the metamorphic linkage is carried out in terms of the screw-loop theory equation, as well as the induced reconfiguration position identification. Then, this paper presents the specific reconfiguration conditions for the two single-loop components in this linkage, and a physical prototype fabricated by 3D printing verifies the ability of configuration reconstruction. Sixteen types of evolved motion branches are constructed by implementing different reconfiguration constraints, respectively, and corresponding kinematic paths are described by employing the specific angle between axes of joints. Furthermore, these evolved motion branches of this metamorphic linkage provide novel examples of mobile assemblies among several fundamental single-loop linkages.

Anomaly detection with dual-stream memory network

Anomaly detection is an essential but challenging task. Existing DNN-based approaches tend to ignore the impact of network history state on extracting spatio-temporal correlations between video events. To address this problem, a Dual-Stream Memory Network (DSM-Net) has been proposed. It leverages historical information from the network to create a dual-stream memory module serving as complementary knowledge for the anomaly detection network. The memory module performs writing and reading in the form of a queue of data features. The writing records the historic information of video events through a moving average encoder, and the reading uses optical flow to uncover behavioral patterns in RGB images. Using a memory sharing strategy, the semantic information of the appearance branch and the motion branch can be integrated to reinforce the network. Results demonstrate that the proposed method on various standard datasets performs favorably when compared to existing methods.

Configuration switch and path selection between Schönflies motion and non-Schönflies motion based on quotient manifold of novel reconfigurable parallel mechanisms

This paper investigates configuration switch and path selection of novel reconfigurable parallel mechanisms with two double-four-bar modular linkages in terms of subgroups SE(2), SO(3) and X(z). The motion representation of reconfigurable parallel mechanisms with three motion branches is proposed based on intersection operation of manifold generated, and the switching principle of motion generators is presented to realize the switching of the three motion branches of the reconfigurable parallel mechanism through the transformation of the submanifold generated by the suitable common motion generators. Two double-four-bar modular linkages are designed to constitute a subchain in terms of the switching principle, and the configuration transformation of the subchain is proposed with respects to the configuration switch of each modular linkage following different forms of the quotient submanifold generated. A reconfigurable parallel mechanism is synthesized by means of rearranging the configurations of two modular linkages of three identical subchains. Especially, the paths of three configurations of a reconfigurable parallel mechanism are selected to satisfy switching.

Multiple bifurcated reconfiguration of double-loop metamorphic mechanisms with prismatic joints

The design and analysis of multi-loop metamorphic mechanisms are challenging in the research of metamorphic mechanisms. Inspired by the parallelogram mechanism, this paper proposes a metamorphic mechanism by adding a link to the parallelogram mechanism and changing the properties of some kinematic joints. The metamorphic mechanism has two loops and is composed of six links connected by six revolute joints and one prismatic joint. The topological structure of the metamorphic mechanism is represented by a directed graph, and the multiple bifurcation of the metamorphic mechanism is analyzed by using screw theory and high-order kinematics. The metamorphic mechanism has six bifurcation points and four different motion branches, including one parallelogram branch, one isosceles trapezoid branch and two oscillating block branches. Meanwhile, by combining these metamorphic mechanisms, some novel metamorphic mechanisms with variable sizes and different configurations can be obtained. The metamorphic mechanism proposed in this paper provides a fresh perspective on how to design new metamorphic mechanisms.

Design and analysis of a novel metamorphic remote-centre-of-motion mechanism with parallelogram joints

Owing to their movements through a fixed point, remote-centre-of-motion (RCM) mechanisms can be widely used in applications such as surgery robotics, exoskeleton robots, haptic devices, pointing mechanisms. Most existing RCM mechanisms only have a constant mobility and RCM point. To adapt to more situations with variable motion requirements, a novel metamorphic RCM mechanism is proposed in this paper based on an analysis of the fundamental methods for metamorphic and RCM mechanisms. The mechanism is composed of two limbs with parallelogram joints and has three motion branches with different RCMs and two furcation points. The screw constraint multisets of the end-effector are utilized to analyse the mobilities and motions of reconfigurations by spanning different screw constraint systems. For the two furcation points and three motion branches, kinematic models and branch transformations are established. In addition, one actuation arrangement and its combined Jacobian are proposed and analysed. The workspace with no singularities except for the furcation points is shown at the end. The mechanism can move with three types of RCMs, which expands the design method of both RCM and metamorphic mechanisms.

Hybrid-attention and frame difference enhanced network for micro-video venue recognition

Many micro-video related applications, such as personalized location recommendation and micro-video verification, can be benefited greatly from the venue information. Most existing works focus on integrating the information from multi-modal for exact venue category recognition. It is important to make full use of the information from different modalities. However, the performance may be limited by the lacked acoustic modality or textual descriptions in uploaded micro-videos. Therefore, in this paper visual modality is explored as the only modality according to its rich and indispensable semantic information. To this end, a hybrid-attention and frame difference enhanced network (HAFDN) is proposed to generate the comprehensive venue representation. Such network mainly contains two parallel branches: content and motion branches. Specifically, in the content branch, a domain-adaptive CNN model combined with temporal shift module (TSM) is employed to extract discriminative visual features. Then, a novel hybrid attention module (HAM) is introduced to enhance extracted features via three attention mechanisms. In HAM, channel attention, local and global spatial attention mechanisms are used to capture salient visual information from different views. In addition, convolutional Long Short-Term Memory (convLSTM) is enforced after HAM to better encode the long spatial-temporal dependency. A difference-enhanced module parallel with HAM is devised to learn the content variations among adjacent frames, which is usually ignored in prior works. Moreover, in the motion branch, 3D-CNNs and LSTM are used to capture movement variation as a supplement of content branch in a different form. Finally, the features from two branches are fused to generate robust video-level representations for predicting venue categories. Extensive experimental results on public datasets verify the effectiveness of the proposed micro-video venue recognition scheme. The source code is available at https://github.com/hs8945/HAFDN.

See the wind: Wind scale estimation with optical flow and VisualWind dataset

Wind sensing by learning from video clips could empower cameras to sense the wind scale and significantly improve the spatiotemporal resolution of existing professional weather records that are often at the city scale. Humans can interpret the wind scale from the motion of surrounding environment objects, especially the moving dynamics of trees in the wind. The goal of this paper is to train cameras to sense the wind by capturing such motion information using optical flow and machine learning models. To this end, we introduce a novel video dataset of over 6000 labeled video clips, covering eleven wind classes of the Beaufort scale. The videos are collected from social media, pubic cameras, and self-recording with varying numbers of clips in each class. Every video clip has a length of 10 s with varied frame rates, and contains scenes of various trees swaying in different scales of wind from an approximately fixed viewpoint. The variation in scene over the course of a single video is minimal. We propose a dual-branch model to estimate the wind scale including a motion branch, which uses optical flow to extract the tree movement dynamics, and a visual branch, to provide visual complementary clues for wind scale estimation. The two branches are fused adaptively in the end of the network, achieving 86.69 % accuracy which confirms the effectiveness of the proposed method. We have conducted experiments compared with a two-stage baseline model and a model only consisting of the motion branch, achieving the best accuracy with the potential to significantly improve the efficiency in time and storage. The dataset and the code are publicly accessible online.

Exploring Motion and Appearance Information for Temporal Sentence Grounding

This paper addresses temporal sentence grounding. Previous works typically solve this task by learning frame-level video features and align them with the textual information. A major limitation of these works is that they fail to distinguish ambiguous video frames with subtle appearance differences due to frame-level feature extraction. Recently, a few methods adopt Faster R-CNN to extract detailed object features in each frame to differentiate the fine-grained appearance similarities. However, the object-level features extracted by Faster R-CNN suffer from missing motion analysis since the object detection model lacks temporal modeling. To solve this issue, we propose a novel Motion-Appearance Reasoning Network (MARN), which incorporates both motion-aware and appearance-aware object features to better reason object relations for modeling the activity among successive frames. Specifically, we first introduce two individual video encoders to embed the video into corresponding motion-oriented and appearance-aspect object representations. Then, we develop separate motion and appearance branches to learn motion-guided and appearance-guided object relations, respectively. At last, both motion and appearance information from two branches are associated to generate more representative features for final grounding. Extensive experiments on two challenging datasets (Charades-STA and TACoS) show that our proposed MARN significantly outperforms previous state-of-the-art methods by a large margin.

Enhanced Attention Tracking With Multi-Branch Network for Egocentric Activity Recognition

The emergence of wearable devices has opened up new potentials for egocentric activity recognition. Although some methods integrate attention mechanisms into deep neural networks to capture fine-grained human-object interactions in a weak-supervision manner, they either ignore exploiting the temporal consistency or generate attention based on considering appearance cues only. To address these limitations, in this paper, we propose an enhanced attention-tracking method, combined with multi-branch network (EAT-MBNet), for egocentric activity recognition. Specifically, we propose class-aware attention maps (CAAMs) by employing a self-attention-based module to refine the class activation maps (CAMs). Our proposed method can enhance the semantic dependency between the activity categories and the feature maps. To highlight the discriminative features from the regions of interest across frames, we propose a flow-guided attention-tracking (F-AT) module, by simultaneously leveraging historical attention and motion patterns. Furthermore, we propose a cross-modality modeling branch based on an interactive GRU module, which captures the time-synchronized long-term relationships between the appearance and motion branches. Experimental results on four egocentric activity benchmarks demonstrate that the proposed method achieves state-of-the-art performance.

Mechanics of incompressible test bodies moving on λ‐spheres

The considerations of the mechanics of incompressible test bodies moving on spheres are generalized to the case of λ‐spheres. Both surfaces are examples of Riemannian manifolds realized as two‐dimensional manifolds with strictly positive Gaussian curvature under the condition λ < 1/3 for the λ‐spheres. A convenient parametrization of λ‐spheres embedded as two‐dimensional surfaces of revolution into the three‐dimensional Euclidean space is derived. The so‐obtained parametrization is expressed in a concise form via the elliptic integrals of the first, second, and third kind. Next, the geodetic motion is considered. The explicit solutions for two branches of the incompressible motion are obtained in the parametric form. For the special case of geodetics corresponding to meridians taken as geodesics their analysis is performed in detail. In the latter case, the so‐obtained geodetic solutions are reduced to the incomplete elliptic integrals of the first, second, and third kind.

Dynamic Modeling and Solution of 6-DOF Parallel Mechanism

When a parallel robot used in a scene that requires force control, rapid attitude adjustment, or precise positioning, we need to know the dynamic characteristics of the moving platform and the motion branch, so it is necessary to do the dynamic analysis of the parallel robot. In this study, for the six-degree-of-freedom parallel mechanism, the Newton-Eulerian method is used to model the dynamics, and then the inverse dynamics simulation is performed through the ADAMS simulation software to verify the correctness of the established dynamic equations. Finally, the Euler integration method is used to solve the dynamic equations numerically. When establishing dynamic equations, it is more convenient to use spiral coordinates to express the angular motion of the motion platform of the parallel mechanism. However, it is difficult to solve the equation numerically. When solving the equations in this study, Euler angles are used to express angular motion. The Euler angle is used as an iterative variable representing the angular motion in the solving process. Then the Euler angle is converted into a rotation matrix, and the parameters of the spiral coordinates are obtained through the rotation matrix, and the dynamic equation is finally solved. The simulation and calculation results show that the established dynamic equation is correct, and the solution to the dynamic equation is also correct.