Force Estimation Method Research Articles

Toward Vision-Based High Sampling Interaction Force Estimation With Master Position and Orientation for Teleoperation

In this study, a vision-based high sampling rate interaction force estimation method is proposed for teleoperation systems that uses master position and orientation information without using physical sensors such as force/torque (F/T) or tactile sensors. The proposed method uses red-green-blue (RGB) images, six-axis robot pose and motor current data, gripper position and current data, as well as master position and orientation information as inputs without requiring force sensors. To estimate the interaction forces, a deep neural network composed of densely connected convolutional network (DenseNet) and long short-term memory (LSTM) is proposed. The database was created by operators using grip and picking motions to interact with 10 objects over a teleoperation system. In addition, we compared the proposed method with different deep learning networks that used different sets of inputs. The results show that the proposed model can estimate 1 kHz interaction force based on 60 Hz images and 1 kHz master inputs. Moreover, the results indicate that the master position and orientation information are useful in estimating the interaction force at a high sampling rate through the result of the change in the network input.

A learning-based tip contact force estimation method for tendon-driven continuum manipulator

Although tendon-driven continuum manipulators have been extensively researched, how to realize tip contact force sensing in a more general and efficient way without increasing the diameter is still a challenge. Rather than use a complex modeling approach, this paper proposes a general tip contact force-sensing method based on a recurrent neural network that takes the tendons’ position and tension as the input of a recurrent neural network and the tip contact force of the continuum manipulator as the output and fits this static model by means of machine learning so that it may be used as a real-time contact force estimator. We also designed and built a corresponding three-degree-of-freedom contact force data acquisition platform based on the structure of a continuum manipulator designed in our previous studies. After obtaining training data, we built and compared the performances of a multi-layer perceptron-based contact force estimator as a baseline and three typical recurrent neural network-based contact force estimators through TensorFlow framework to verify the feasibility of this method. We also proposed a manually decoupled sub-estimators algorithm and evaluated the advantages and disadvantages of those two methods.

Contact force estimation of hydraulic soft bending actuators for gripping

Force measurement of soft actuators has drawn much research interest to realize closed-loop feedback. This work develops an estimation method of contact force of hydraulic soft bending actuators for gripping instead of installing force sensors at interaction position. A simplified analytical model is developed to reveal the relationships among the hydraulic pressure, the bending angle, and the contact force. The method includes the following three steps. Firstly, the bending angle varying with the hydraulic pressure is calibrated when the soft actuators are operated in free space. Secondly, the modulus of elasticity is identified by blocking the soft actuators at initial state. Thirdly, the output force of the soft actuators touching an object is estimated according to the model by measuring the bending angle and the hydraulic pressure through visual and pressure sensors. Experiments on soft actuators with two different structures are implemented to verify the effectiveness of the estimation method. The results show that the estimation method has expected accuracy and robustness. Finally, the estimation method is applied to the closed-loop control of contact force. It is found that good dynamical and steady control performance can be achieved by using the estimated feedback.

Robust Estimation of Contact Force and Location for Magnetic-Field-Based Soft Tactile Sensor Considering Magnetic Source Inconsistency.

Flexible magnetic-field-based tactile sensors (FMFTS) have numerous advantages including low cost, ease of manufacture, simple wiring, high sensitivity, and so on. Flexible magnetic-field-based tactile sensors need to be calibrated before use to build accurate mapping between contact force and magnetic field intensity measured by magnetic sensors; however, when considering remanence inconsistency of magnetic source, each FMFTS needs to be calibrated independently to enhance accuracy, and the complex preparation prevents FMFTS from being used conveniently. A robust estimation method of contact force and location that can tolerate remanence inconsistency of magnetic source in FMFTS is proposed. Firstly, the position and orientation of magnetic source were tracked using the Levenberg–Marquart algorithm, and the tracking results were insensitive to the remanence of magnetic source with appropriate cost function. Secondly, the mapping between magnitude and location of contact force and position and orientation of magnetic source was built with calibration of one sensor; the mapping only depends on the structural response of flexible substrate, and thus can be extended to estimate external force and location for other sensors with the same structure. The proposed method was evaluated in both simulations and experiments, and the results confirm that the estimation of magnitude and location of external force for FMFTS with the same structure and different remanence could reach acceptable accuracy, depending on single calibration. The proposed method can be used to simplify the calibration procedure and remove the barrier for large-scale application of FMFTS and replacement of damaged FMFTS.

Uncertainty quantification for impact location and force estimation in composite structures

Structural health monitoring of impact location and severity using Lamb waves has been proven to be a reliable method under laboratory conditions. However, real-life operational and environmental conditions (vibration noise, temperature changes, different impact scenarios, etc.) and measurement errors are known to generate variation in Lamb wave features which may significantly affect the accuracy of these estimates. Therefore, these uncertainties should be considered, as a deterministic approach may lead to erroneous decisions. In this article, a novel data-driven stochastic Kriging-based method for impact location and maximum force estimation, that is able to reliably quantify the output uncertainty is presented. The method utilises a novel modification of the kriging technique (normally used for spatial interpolation of geostatistical data) for statistical pattern matching and uncertainty quantification using Lamb wave features to estimate the location and maximum force of impacts. The data was experimentally obtained from a composite panel equipped with piezoelectric sensors. Comparison with a deterministic benchmark method developed in prior studies shows that the proposed method gives a more reliable estimate for experimental impacts under various simulated environmental and operational conditions by estimating the uncertainty. The developed method highlights the suitability of data-driven methods for uncertainty quantification, by taking advantage of the relationship between data points in the reference database that is a mandatory component of these methods (and is often seen as a disadvantage). By quantifying the uncertainty, there is more information for operators to reliably locate impacts and estimate the severity, leading to robust maintenance decisions.

Cable force estimation of cables with small sag considering inclination angle effect

Cable force estimation is essential for security assessment of cable-stayed bridges. Cable force estimation methods based on the relationship between cable force and frequency have been extensively studied and used during both construction phase and service phase. However, the effect induced by inclination angle of the cable is not included in the establishment of frequency-cable force relationship as horizontal cable model is normally employed. This study aims to investigate the influence of the inclination angle on vibration based cable force estimation and provide practical formulas accordingly. Firstly numerical examples of fixed-fixed and hinged-hinged cables are simulated to illustrate the necessity of considering the inclination angle effect on the modal parameters and cable force estimation for inclined cables with small sag. Then practical formulas considering the inclination angle effect to estimate the cable force of fixed-fixed and hinged-hinged cables via the fundamental frequency are established accordingly. For the inclined cables with unknown boundary conditions, the coefficients reflecting boundary condition are predicted via the practical formulas for fixed-fixed and hinged-hinged cables. And the cable force considering the influence of inclination angle and unknown boundary conditions is obtained by iteration method. Finally, numerical examples are presented to demonstrate the effectiveness of the proposed method.

A novel sEMG-based force estimation method using deep-learning algorithm

This paper discusses the problem of force estimation represented by surface electromyography (sEMG) signals collected from an armband-like collection device. The scheme is proposed for the sake of two dimensions of sEMG signals: spatial and temporal information. From the point of space, first, appropriate channel number across all subjects is investigated. During this progress, an electrode channel selection method based on Spearman’s rank order correlation coefficient is utilized to detect signals from active muscle. Then, to reduce the computation and highlight the channel information, linear regression (LR) algorithm is conducted to weight each channel. Besides, the recurrent neural network (RNN) is used to capture the temporal information and model the relation between sEMG and output force. Experiments conducted on four subjects demonstrate that six channels are enough to characterize the muscle activity. By combining the selected channels with different weight coefficients, LR algorithm can fit the output force better than simply averaging them. Furthermore, RNN with long short-term memory cell shows the superiority in time series modeling, which can improve our results to a greater degree. Experimental results prove the feasibility of the proposed method.

Estimation of Extreme Cable Forces of Cable-Stayed Bridges Based on Monitoring Data and Random Vehicle Models

For long-span cable-stayed bridges, cables serve as one of the most important components to guarantee structural integrity. Forces of stay cables indicate not only the performance of cables themselves but also the overall condition of bridges. In order to help stakeholders to make maintenance decisions, an extreme cable force estimation method was proposed based on cable force measurements and traffic data from the weighing system. First, raw monitoring data were preprocessed based on a median filtering to obtain usable cable force signals. The multiresolution wavelet method was used to extract traffic-induced force component from mixed signals. Then, a Monte Carlo-based random vehicle model was developed using traffic data from the weighing system. Based on field temperature measurements and simulation of traffic-induced effects, extreme cable forces with respect to vehicle loads and temperature effects were predicted by extreme value theory. The Generalized Pareto Distribution (GPD) was adopted to establish the probability distribution models of the daily maximum cable force. Then, the extreme value within a return period of 100 years was determined and compared with the design loading demand. Finally, the effectiveness of the proposed method was validated through a cable-stayed bridge in China. As a result, the low-frequency varying component of cable force response had positive correlation with environmental temperatures, and the extreme value of the predicted cable force under prospective traffic volumes was within limit interval value according to the design code. The conclusions can be utilized by bridge owners to make maintenance decisions.

Image-Based Force Estimation in Medical Applications: A Review

Minimally invasive robotic interventions have highlighted the need to develop efficient techniques to measure forces applied to the soft tissues. Since the last decade, many scholars have focused on micro-scale and macro-scale robotic manipulations. Early articles used the model of soft tissue mathematically and tracked the displacement of the contour of the object in the vision system to provide the corresponding force to the user. Lack of knowledge of different materials and the computational complexity led to a transition from model-based to learning-based approaches to interpret the relation between object deformations, extracted from the vision system, and the real forces applied to the object. The dramatic growth of machine learning techniques and its integration with computer vision has brought novel learning-based visual data processing methods to the area. The application of the image-based force estimation methods in a controlled medical intervention has also received significant attention in the last five years. A decent number of surveys have been published on micromanipulation in recent years, especially for cell microinjection. However, the state of the art in meso- and macro-scale medical robotic interventions has not been reviewed. The aim and contribution of this paper are to fill the stated gap by reviewing the recent advances in image-based force estimation in robotic interventions. The survey shows that learning-based force estimation methods are growing significantly by using deep learning-based methods. The survey will encourage researchers and surgeons to apply learning-based algorithms to real-time medical and health-related operations.

Noninvasive Method of Force Estimation of Lower Limb Muscles in Cricket Fast Bowlers: Can This be an Application to Minimize Potential Injury?

Noninvasive Method of Force Estimation of Lower Limb Muscles in Cricket Fast Bowlers: Can This be an Application to Minimize Potential Injury?

Development of an Intention-Based Adaptive Neural Cooperative Control Strategy for Upper-Limb Robotic Rehabilitation

Robotic rehabilitation therapy has become an important technology to recover the motor ability of disabled individuals. Clinical studies indicate that involving the active intention of patient into rehabilitation training contributes to promoting the performance of therapies. An adaptive neural cooperative control strategy is developed in this letter to realize intention-based human-cooperative rehabilitation training. The human motion intention is estimated by fusing the human-robot interaction forces and the muscular forces into a Gaussian radial basis function network. A biological force estimation method is proposed to obtain the muscular forces of biceps and triceps based on surface electromyography signals and Kalman filter. A robust adaptive sliding mode controller is integrated into the cooperative control scheme to ensure the accuracy and stability of inner position control loop with uncertainties. The minimum jerk cost principle is used to improve the smoothness and continuity of trajectory. To evaluate the effectiveness of the proposed control scheme, further experimental investigations are conducted on a planar upper-limb rehabilitation robot with ten volunteers. The results indicate that the proposed control strategy has significant potential to modulate the interaction compliance and cooperation process during training.

Muscle Force Estimation Based on Neural Drive Information From Individual Motor Units.

Estimation of muscle contraction force based on the macroscopic feature of surface electromyography (SEMG) has been widely reported, but the use of microscopic neural drive information has not been thoroughly investigated. In this study, a novel method is proposed to process individual motor unit (MU) activities (firing sequences and action potential waveforms) derived from the decomposition of high density SEMG (HD-SEMG), and it is applied to muscle force estimation. In the proposed method, a supervised machine learning approach was conducted to determine the twitch force of each MU according to its action potential waveforms, which enables separate calculation of every MU's contribution to force. Thus, the muscle force was predicted through a physiologically meaningful muscle force model. In the experiment, HD-SEMG data were recorded from the abductor pollicis brevis muscles of eight healthy subjects during their performance of thumb abduction with the force increasing gradually from zero to four force levels (10%, 20%, 30%, 40% of the maximal voluntary contraction), while the true muscle force was measured simultaneously. When the proposed method was used, the root mean square difference (RMSD) of the error of the estimated force with respect to the measured force was reported to be 8.3% ± 2.8%. The proposed method also significantly outperformed the other four common methods for force estimation (RMSD: from 11.7% to 20%, ), demonstrating its effectiveness. This study offers a useful tool for exploiting the neural drive information towards muscle force estimation with improved precision. The proposed method has wide applications in precise motor control, sport and rehabilitation medicine.

A novel method for impact force estimation in composite plates under simulated environmental and operational conditions

During its lifetime, an aircraft structure is subjected to various impacts from various sources such as tool drops, hail, ground service equipment, etc. In modern composite structures, these impacts have a significant chance of generating barely visible damage (BVID) which may lead to catastrophic failure of a structure if left undetected to grow. However, BVID is difficult to detect during routine visual inspection without specialised non-destructive inspection and thus there is large interest in developing monitoring systems for estimating the location and severity of impact events. Currently, most systems and methods have been developed for controlled lab conditions and do not consider the wide range of impact parameters in real life operation (environmental conditions, vibration, impactor stiffness, angle, etc) which may severely compromise the accuracy of these methods. In this study we have explored two methods for maximum impact force estimation, deconvolution and a novel gradient method, for the purpose of reliable severity assessment in composite aircraft structures under simulated environmental and operational conditions. It is shown that both methods allow accurate and robust estimation of the maximum impact force from various cases of impacts (variation of impact energy, mass, stiffness, angle, temperature, source) using minimum initial data from a single impact case. From further testing it is demonstrated that the gradient method is robust towards the effects of impact localisation errors and noise. The gradient method also has much less computational and storage requirements and is thus more feasible to integrate with current data acquisition systems being developed for structural health monitoring. Thus, we conclude that the proposed gradient method is suitable for impact force monitoring and severity assessment in composite aircraft structures in the simulated environmental and operational conditions.

Generalization of spatio-temporal deep learning for vision-based force estimation

Abstract Robot-assisted minimally-invasive surgery is increasingly used in clinical practice. Force feedback offers potential to develop haptic feedback for surgery systems. Forces can be estimated in a vision-based way by capturing deformation observed in 2D-image sequences with deep learning models. Variations in tissue appearance and mechanical properties likely influence force estimation methods’ generalization. In this work, we study the generalization capabilities of different spatial and spatio-temporal deep learning methods across different tissue samples. We acquire several data-sets using a clinical laparoscope and use both purely spatial and also spatio-temporal deep learning models. The results of this work show that generalization across different tissues is challenging. Nevertheless, we demonstrate that using spatio-temporal data instead of individual frames is valuable for force estimation. In particular, processing spatial and temporal data separately by a combination of a ResNet and GRU architecture shows promising results with a mean absolute error of 15.450 compared to 19.744 mN of a purely spatial CNN.

External Force Estimation in Linear Series Elastic Actuator Without Load-Side Encoder

This article presents a novel estimation method of the external force in linear series elastic actuators (SEAs). Conventionally, such force is estimated through the deflection of an elastic element, placed between motor and end-effector in SEAs (measured using two encoders). However, in some applications (e.g., wearable devices), it is difficult to mount a load-side encoder, due to cost or manufacturing issue. Moreover, methods based on encoders have limitations in terms of bandwidth, and, in turn, in estimating fast-varying impact forces. Additionally, SEAs may have a nonnegligible end-effector mass, therefore gravity (or accelerations of the actuator support) may add a contribution to the external force, not accounted by deflection-based estimation methods. To address these issues, we propose a method based on the Kalman filter in combination with the use of two MEMS accelerometers, mounted on the load-side and on the support of the SEA, for external force estimation and gravity (or support acceleration) compensation, respectively. We tested the proposed method even in a reduced version, with a single load-side accelerometer. Finally, it is shown that using the acceleration signal also improves the robustness against spring stiffness variations, compared to conventional methods. The experimental results show the effectiveness of the proposed methods.

Propulsive forces on water polo players’ feet from eggbeater kicking estimated by pressure distribution analysis

ABSTRACT The purpose of this study was to characterise the unsteady propulsive force during eggbeater kicking by a fluid force estimation method based on pressure distribution analysis. The eggbeater kick was performed by six male water polo players. The participants’ eggbeater kicking motions were recorded by three cameras, and the kinematic foot variables were analysed. The pressure distributions around the foot were measured by four pairs of pressure sensors attached to the dorsal and plantar surfaces of the participants’ right foot. The resultant fluid force acting on the foot was estimated from the measured pressure and area of the foot. The calculated propulsive force increased with the pressure difference between the plantar and dorsal sides of the foot, which was mainly related to the decrease in pressure on the dorsal side, and peaked when the foot passed its maximum velocity and began to decelerate. These results cannot be elucidated only by conventional biomechanical theories of swimming propulsion (Newton’s laws of motion and the quasi-steady approach) but instead indicate a high possibility that the exerted propulsive force is induced by the effects of unsteady water flow.

Force Control of In-Wheel-Motored Electric Vehicles

Electric vehicles (EVs) are gaining attention, and novel use of cars such as vehicle-to-home is becoming widespread. In particular, in-wheel-motor (IWM) type EVs, which have four motors inside each wheel, are expected to be an ideal power train system because it can achieve precise driving force control and estimation. To exploit these features of IWM-EVs, researchers have proposed numerous control methods, most of which are velocity or position-based control for running or trajectory tracking. Force control technologies are quite popular in robotics divisions, and these methods are quite suitable for the new applications of EVs. By taking advantage of the precise estimation performance of IWM-EVs, the authors have proposed to apply force control to EVs and built the concept of human-friendly EVs as done in the robotics division. We show the external force estimation method using wheel resolvers and propose hand-assisted position adjustment method based on impedance control. The appropriate control structure is discussed for IWM-EVs, and its effectiveness is demonstrated with numerical simulations and experiments. We anticipate this research to generate many control applications suitable for the new application of cars.

Noncontact cable force estimation with unmanned aerial vehicle and computer vision

AbstractCables and hangers are critical components of long‐span bridges, tension forces of them are needed to be accurately measured for ensuring the safety of bridges. Traditionally, cable tension forces are measured by attached accelerometers or elastomagnetic (EM) sensors, however, applying these sensors into engineering practice are time‐consuming, labor‐intensive, and highly dangerous. To address these problems, an unmanned aerial vehicle (UAV)‐based noncontact cable force estimation method with computer vision technologies was proposed in this article. Basic concept of the proposed method is to use the UAV‐installed camera for capturing vibration images of cables from a certain distance and cable dynamic properties are extracted by analyzing captured images. It includes two aspects: (a) a line segments detector (LSD) was employed for detecting cable edges from captured video and a line matching algorithm was further proposed for extracting dynamic displacements; (b) the frequency difference of adjacent higher modal frequencies identified from relative displacements of the cable was employed for cable force calculation to avoid the difficulty of extracting fundamental frequency from UAV‐captured video. It should be noted that relative displacement herein refers to the difference between displacements of two points on the same cable. Advantages of the proposed method lie in that the proposed LSD and matching algorithm are more robust than traditional correlation‐based algorithm for calculating dynamic displacements of bridge cables and it does not need to adjust predefined parameters (i.e., subset size in correlation‐based algorithms). In addition, the combination of relative displacement and frequency difference‐based cable force estimation has the capability of enhancing the Fourier spectrum magnitude of bridge cables and reducing the effect of UAV motion on extraction of cable vibration frequencies. The effectiveness and robustness of the proposed method was verified by using an experimental inclined cable and field‐testing data of a long‐span suspension bridge. Results show that calculated cable forces with UAV technology have a good agreement with reference values measured by attached accelerometers and fixed camera, demonstrating correctness and robustness of the proposed method for cable force estimation.

Adaptive wave neural network nonsingular terminal sliding mode control for an underwater manipulator with force estimation

This paper proposes an adaptive wave neural network nonsingular terminal sliding mode control (AWNN-NTSMC) strategy with force estimation, which is exploited to address the path tracking control problem of the underwater manipulator under lumped disturbances. The proposed control scheme contains three parts: a nonsingular terminal sliding mode surface (NTSMS) part, an AWNN part, and a force estimation part. The NTSMS is designed to make the system states achieve fast convergence in the sliding mode phase. The AWNN theory is utilized to approximate the lumped disturbances via online adjustment of the network parameters. The force estimation method is applied in compensating the effect of external force on the control system. Besides, a saturated function instead of the signum function is used aiming to the chattering suppression. Asymptotic stability of the closed-loop system is guaranteed by the Lyapunov stability. Finally, by using a six degree of freedom (DOF) underwater manipulator, comparative simulation results validate the better tracking performance and stronger robustness against disturbances of our proposed scheme.

Grasping Force Control of Multi-Fingered Robotic Hands through Tactile Sensing for Object Stabilization.

Grasping force control is important for multi-fingered robotic hands to stabilize the grasped object. Humans are able to adjust their grasping force and react quickly to instabilities through tactile sensing. However, grasping force control through tactile sensing with robotic hands is still relatively unexplored. In this paper, we make use of tactile sensing for multi-fingered robot hands to adjust the grasping force to stabilize unknown objects without prior knowledge of their shape or physical properties. In particular, an online detection module based on Deep Neural Network (DNN) is designed to detect contact events and object material simultaneously from tactile data. In addition, a force estimation method based on Gaussian Mixture Model (GMM) is proposed to compute the contact information (i.e., contact force and contact location) from tactile data. According to the results of tactile sensing, an object stabilization controller is then employed for a robotic hand to adjust the contact configuration for object stabilization. The spatio-temporal property of tactile data is exploited during tactile sensing. Finally, the effectiveness of the proposed framework is evaluated in a real-world experiment with a five-fingered Shadow Dexterous Hand equipped with BioTac sensors.