Smart Insole Research Articles

Identity Recognition by Walking Outdoors Using Multimodal Sensor Insoles

Recently, gait attracts attention as a practical biometric for devices that naturally possess walking pattern sensing. In the present study, we explored the feasibility of using a multimodal smart insole for identity recognition. We used sensor insoles designed and implemented by us to collect kinetic and kinematic data from 59 participants that walked outdoors. Then, we evaluated the performance of four neural network architectures, which are a baseline convolutional neural network (CNN), a CNN with a multi-stage feature extractor, a CNN with an extreme learning machine classifier using sensor-level fusion and CNN with extreme learning machine classifier using feature-level fusion. The networks were trained with segmented insole data using 0%, 50%, and 70% segmentation overlap, respectively. For 70% segmentation overlap and both-side data, we obtained mean accuracies of 72.8% ±0.038, 80.9% ±0.036, 80.1% ±0.021 and 93.3% ±0.009, for the four networks, respectively. The results suggest that multimodal sensor-enabled footwear could serve biometric purposes in the next generation of body sensor networks.

121 A Newly-Developed Smart Insole System with Instant Reminder: Paves the Way towards Integrating Artificial Intelligence (AI) Technology to Improve Balance and Prevent Falls

Abstract Background Falls in senior people have high incidence& lead to severe injuries [1]. Application of smart wearable systems (with sensors to monitor user’s balance and corresponding instant reminder to let tusers adjust posture/motion) can effectively improve static standing balance [2], reduce reaction time and body sway in response to balance perturbation [3], improve walking pattern [4], and reduce the risk of falls [5, 6]. However, previous systems have not considered the daily monitor of user’s balance and falling risks, and the personalized reminder. Artificial intelligence (AI) and big data analytics have been widely used to monitor the daily physical activity [7], while few studies have utilized them to improve balance/gait and prevent falls. Methods This study has optimized previous devices by integrating AI technology and developed a new smart insole system. The system consisted of insoles with embedded sensors that can capture the foot motion and plantar pressure, smart watch that connected with insoles wirelessly and then transmitted the foot motion and force data to Cloud server via Wi-Fi, central Cloud server for big data transmission and storage, workstation for big data analytics and machine learning, and user interface for data visualization (e.g. smartphone, tablet, and/or laptop). Results & Discussion The system transmission rate was up to 30 Hz. The collected big data contained all sensor signals captured before and after delivering reminder, and from day-to-day monitoring of users. The customized reminder varied in the type, frequency, magnitude, and amount/dosage. This AI smart insole system enabled the monitor of daily balance and falling risks and the provision of timely-updated and customized reminder to users, which could potentially reduce the risk of falls and slips. It can also act as a balance-training device.

A High-Fidelity Wearable System for Measuring Lower-Limb Kinetics and Kinematics

There are many important challenges in gait analysis, which has many applications in healthcare, rehabilitation, therapy, and exercise training. However, gait analysis is typically performed in a gait laboratory, which is inaccessible to the general population and is not available in natural gait environments (e.g., outdoors). In this paper, we discuss the development of a high-fidelity, cost-effective, wireless sensor network to address the challenge of efficient gait monitoring in real-world walking scenarios. The sensor network is designed in a modular way to capture plantar forces and knee angle, angular velocity, and angular acceleration. A force module called a smart insole is designed to measure the plantar forces. The module is comprised of force sensitive resistors (FSRs) and a signal conditioning circuit. Various signal conditioning techniques, including a novel technique called transfrequency, are investigated to provide a linear mapping for FSR measurements and to provide data acquisition fidelity. The motion module includes a low-cost inertial measurement unit (IMU) augmented with a Kalman filter to provide filtered knee kinematics. A qualitative evaluation of the sensor network communication module is achieved by considering the internal communication protocols between the modules and the external wireless transmission protocol used to deliver data to an end-point terminal PC. Experiments are conducted to validate the motion and force modules. Then, the overall, integrated system is compared to gold standard laboratory results, demonstrating a successful application for gait identification. The results show that the sensor network accurately captures important gait parameters and features.

Bring Gait Lab to Everyday Life: Gait Analysis in Terms of Activities of Daily Living

With the development of the Internet of Things (IoT), wearable technologies have been proposed to measure gait parameters in everyday life. However, since both diseases and activities could influence gait patterns, clinicians cannot use the measured gait parameters for clinical applications without knowing the corresponding activities. To address this problem, a novel gait analysis method—“gait analysis in terms of activities of daily living (ADLs)”—was proposed based on a wearable Smart Insole system. Twenty six gait parameters were extracted to realize a systematic gait analysis. Novel activity recognition algorithms based on characteristics of human gait were proposed to recognize ADLs, including “sitting,” “standing,” “walking,” “running,” “ascend stairs,” and “descend stairs” with high accuracy and low computation load. To evaluate the performance of “gait analysis in terms of ADLs,” an experiment consisting of a sequence of different ADLs was designed to simulate the scenario of everyday life. In the result, gait parameters measured during different activities were automatically highlighted with different colors, which made it easy to see whether the gait pattern change was caused by activities or diseases. Besides, a refined gait analysis could be realized by individually extracting and analyzing the gait parameters of a specific activity. The results indicate that “gait analysis in terms of ADLs” is a feasible method to reach the aim of bringing gait lab to everyday life.

Self-Powered Smart Insole for Monitoring Human Gait Signals.

With the rapid development of low-power consumption wireless sensors and wearable electronics, harvesting energy from human motion to enable self-powered sensing is becoming desirable. Herein, a pair of smart insoles integrated with piezoelectric poly(vinylidene fluoride) (PVDF) nanogenerators (NGs) are fabricated to simultaneously harvest energy from human motion and monitor human gait signals. Multi-target magnetron sputtering technology is applied to form the aluminum electrode layers on the surface of the PVDF film and the self-powered insoles are fabricated through advanced 3D seamless flat-bed knitting technology. Output responses of the NGs are measured at different motion speeds and a maximum value of 41 V is obtained, corresponding to an output power of 168.1 μW. By connecting one NG with an external circuit, the influence of external resistance, capacitor, and motion speed on the charging characteristics of the system is systematically investigated. To demonstrate the potential of the smart insoles for monitoring human gait signals, two subjects were asked to walk on a treadmill at different speeds or with a limp. The results show that one can clearly distinguish walking with a limp from regular slow, normal, and fast walking states by using multiscale entropy analysis of the stride intervals.

Additive manufacturing of smart insole by direct printing of pressure sensitive material

3D printing technology is gaining attention as one of the key technologies that will lead the fourth industrial revolution. In addition, the field of application is from trial production to mass production of products. However, in many cases, the application area is limited to the geometrical shape building of products. This paper shows an example of application technology for manufacturing the smart insole by applying 3D printing technology for the purpose of giving various functions as well as making geometric shape. In the production of smart insole, 3D printing technology is applied to direct printing of pressure sensitive sensor and highly conductive wire as well as flexible insole object itself. The materials used in the production of flexible products, conductors and sensor are different, and in order to produce the sensor by 3D printing, suitable materials have been directly manufactured and used. This paper shows that the 3D printing technology can be directly applied to the manufacturing of multi functional products through the above manufacturing process as well as the experimental results after the actual production.

Reel-to-reel fabrication of strain sensing threads and realization of smart insole

Integrating sensors into our daily wear such as shoes or clothing will revolutionize healthcare and fitness. In this paper, we demonstrate scalable reel-to-reel fabrication of strain sensing threads, which are then used to realize smart shoe insole for gait monitoring. Fabrication includes coating of bare threads with conductive ink using a 3D printed cartridge that enables simultaneous stretching and dip coating, followed by drying in a heated chamber. Notable pre-stretching of threads before dip coating is proposed as an essential step to ensure conductivity and strain measurement under extreme stretching conditions. In this paper, the strain sensitive threads are made from coating of carbon ink on Gütermann elastic threads. They are shown to have gauge factor of 12. To realize a smart shoe insole, threads were arranged in a two dimensional grid of 16 × 8 threads cast inside a highly elastomeric EcoFlex sole. Custom circuitry enables wireless readout from sensor array to any Bluetooth enabled device. We propose advanced calibration and normalization algorithm to correct for mismatch in sensitivity between threads in an array. Further signal processing of the real-time strain information from the 2D grid enables generation of precise 16 × 8 pixel 2D pressure foot map. These foot maps can be used for gait monitoring and other physical activity. Detailed characterization and validation on individual threads and of the smart insole have been provided.

Gait event detection algorithm based on smart insoles

Gait event detection algorithm based on smart insoles

Highly sensitive and stable printed pressure sensor with microstructured grid arrays

Flexible pressure sensors with high sensitivity, broad pressure range and a large array are highly desired for wearable applications. In this work, a flexible pressure sensor with a polyvinyl chloride/carbon black microgrid films was obtained by screen printing. Compared to other recent pressure sensors, these printed pressure sensors with microstructured grids have a broad pressure range up to 666.7 kPa, an excellent sensitivity of typically 4.71 kPa−1 in the dynamic range between 0 and 15 kPa, a fast response of typically 25 ms, a high stability, a good durability in a broad pressure range for more than 5300 cycles and long effectiveness. We showed that the printed pressure sensors can be customized to fit in a smart insole. Moreover, a wrist pulse could be detected accurately by a single sensor. The proposed printed pressure sensors will have a great potential a smart devices in health monitoring, for preventive health care, and in rehabilitation care.

User Identification from Gait Analysis Using Multi-Modal Sensors in Smart Insole.

Recent studies indicate that individuals can be identified by their gait pattern. A number of sensors including vision, acceleration, and pressure have been used to capture humans’ gait patterns, and a number of methods have been developed to recognize individuals from their gait pattern data. This study proposes a novel method of identifying individuals using null-space linear discriminant analysis on humans’ gait pattern data. The gait pattern data consists of time series pressure and acceleration data measured from multi-modal sensors in a smart insole used while walking. We compare the identification accuracies from three sensing modalities, which are acceleration, pressure, and both in combination. Experimental results show that the proposed multi-modal features identify 14 participants with high accuracy over 95% from their gait pattern data of walking.

Smart Insole-Based Indoor Localization System for Internet of Things Applications

With the development of Internet of Things (IoT), indoor localization has been a research focus in recent years. For inertial measurement unit (IMU)-based indoor localization method, zero velocity update (ZUPT) uses the known velocity at stationary epoch as a benchmark to calibrate the velocity drift. However, stationary epoch only takes up 24% of a whole gait cycle time, and the velocity drift at the remaining 76% time is usually estimated according to an assumption that velocity has a linear drift over time, which would introduce errors. In this paper, a two-step velocity calibration method was proposed based on human gait characteristics with Smart Insole: known velocity update (KUPT) and double-foot position calibration (DFPC). KUPT could measure the velocity from heel-strike to toe-off based on the recorded real-time foot angle and the shoe dimensions, which increases the time period when the velocity could be measured from 24% to 62% of a whole gait cycle time. DFPC method could fuse the position information of both feet based on the symmetrical characteristic of human gait to further increase the reliability of the localization results. The statistical result of a 20 times 20-m walking experiment showed that KUPT method was more accurate and reliable than ZUPT method for both feet, and DFPC method could further improve the result of KUPT method. Another experiment about walking in an indoor environment for 91 m showed that the proposed KUPT+DFPC method had an error of about 0.78 m which is acceptable for most IoT applications.

DEVELOPMENT OF GAIT DIAGNOSTIC TECHNOLOGY FOR LONG-DISTANCE RUNNERS VIA SMART INSOLES

Aim. The article deals with the development of methods and software for the analysis of the running pattern in long-distance runners using smart insoles to prevent the development of athlete’s injuries. Materials and Methods. Insoles with embedded pressure sensors, an accelerometer, a gyroscope, and a specially designed software for analyzing the pattern of running and pressure distribution in the foot. Results. The technique of diagnostics and analysis of the running pattern, which allows preventing the development of tendon inflammation, Morton neuroma, and plantar fasciitis, is presented. Conclusion. The technique and the software deve­loped allow to analyze the patterns of running in stayers, diagnose, and prevent the development of injuries

Highly stretchable and sensitive liquid-type strain sensor based on a porous elastic rope/elastomer matrix composite structure

In this study, we present a new liquid-type strain sensor based on porous elastic rope/elastomer matrix (PER/EM) composite structure. The liquid-impregnated porous elastic rope (LIPER) was immersed into a flexible silicone matrix to fabricate the sensor, making the manufacture process easy. The effect of liquid viscosity on the hysteresis of the sensor has been understood through selecting water, ethylene glycol and hydrogel as the sensing liquids. The most attractive performance of the PER/EM strain sensor is its ability to detect both stretching and compression deformations as well as to realize temperature sensing. Simultaneously, high stretch strain over 100% and pressure up to 612 kPa can be subjected. A minimum tensile strain of 0.5% and a minimum applied pressure of 5 kPa can be detected, indicating the good sensitivity of the as-fabricated strain sensor to the subtle deformation. In particular, our devices exhibited the excellent long-term durability after completing >6000 stretching-releasing cycles (100% strain) and >10000 compression-expansion cycles (50 kPa). On this basis, the PER/EM strain sensor was successfully employed to detect the full-scale human motion ranging from swallowing to joint bending. We further made a smart insole and achieved the real-time monitoring of walking, running and jumping. Our results show that the as-fabricated PER/EM strain sensor has the great potential for the applications in healthcare, human motion monitoring and electronic skin.

Lower-body posture estimation with a wireless smart insole.

Typical optical marker and camera based systems for motion capture suffer from several limitations. They are restricted to indoor environments, have difficulties tracking multiple people simultaneously and require expensive camera setups. In this work, we present a new method for lower-body posture estimation with a wireless smart insole using end-to-end training of a deep neural network. Our model is able to predict the movement of the entire lower body (including the hip, knee, ankle and toe) accurately in a wide range of activities. Inference only takes 1.62ms and hence can be used in real-time. The proposed method can potentially provide a very efficient and portable solution for applications like sports analysis, rehabilitation and virtual reality.

Benchmarking study of the forces and centre of pressure derived from a novel smart-insole against an existing pressure measuring insole and force plate

This study identified a novel method of validating the force and Centre of Pressure (COP) obtained from pressure-measuring insoles, utilizing commercially available equipment. The focus of this research was to compare force and COP data between a newly developed Smart Insole (SI) and the Pedar (PE) insole using an independent point of reference device – the Kistler force plate (FP). Results showed good correlation between forces measured from both systems and the FP (R2 > 0.98) with RMSE of 11.2% and 5.6% respectively. For the COP, the Systematic Error (SE), k, and R2, are presented. For the SI, the COPx and COPy data achieved k = 0.763 and 1.009 (R2 = 0.855 and 0.990). For the PE, k = 0.155 and 1.0143 (R2 = 0.014 and 0.995). The SI is comparable to PE in terms of accuracy when measuring force and COP, with the advantages of being accurate, portable and cheaper.

Classification of Gait Type Based on Deep Learning Using Various Sensors with Smart Insole.

In this paper, we proposed a gait type classification method based on deep learning using a smart insole with various sensor arrays. We measured gait data using a pressure sensor array, an acceleration sensor array, and a gyro sensor array built into a smart insole. Features of gait pattern were then extracted using a deep convolution neural network (DCNN). In order to accomplish this, measurement data of continuous gait cycle were divided into unit steps. Pre-processing of data were then performed to remove noise followed by data normalization. A feature map was then extracted by constructing an independent DCNN for data obtained from each sensor array. Each of the feature maps was then combined to form a fully connected network for gait type classification. Experimental results for seven types of gait (walking, fast walking, running, stair climbing, stair descending, hill climbing, and hill descending) showed that the proposed method provided a high classification rate of more than 90%.

A Smart Environment-Adapting Timed-Up-and-Go System Powered by Sensor-Embedded Insoles

With the growth of the elder population, fall risk evaluation is crucial to prevent elders from serious injuries, as well as reduce related financial burdens. A balance assessment, timed up and go (TUG), has been widely used to estimate fall risk. The standardized TUG focuses on flat ground walking with no environmental variance. Therefore, it falls short of assessing an individual’s gait adaptability. Being able to adjust steps in response to environmental changes, for example, needing to navigate around or over a child’s toy left on the sidewalk, is essential to avoid fall risk and fundamental to community ambulation. To this end, we propose four environment-adapting TUGs designed to assess one’s ability to adapt gait in complex environments and a compatible system named Smart Insole TUG (SITUG), which provides real-time, feature-rich, and ease-of-operation TUG analysis. Based on experimental results, SITUG is capable of extracting gait related spatial–temporal features with all mean accuracies over 92%. Besides, the system achieves a mean accuracy of 92.23% in segmenting five TUG phases.

Identification of Motor Symptoms Related to Parkinson Disease Using Motion-Tracking Sensors at Home (KÄVELI): Protocol for an Observational Case-Control Study.

BackgroundClinical characterization of motion in patients with Parkinson disease (PD) is challenging: symptom progression, suitability of medication, and level of independence in the home environment can vary across time and patients. Appointments at the neurological outpatient clinic provide a limited understanding of the overall situation. In order to follow up these variations, longer-term measurements performed outside of the clinic setting could help optimize and personalize therapies. Several wearable sensors have been used to estimate the severity of symptoms in PD; however, longitudinal recordings, even for a short duration of a few days, are rare. Home recordings have the potential benefit of providing a more thorough and objective follow-up of the disease while providing more information about the possible need to change medications or consider invasive treatments.ObjectiveThe primary objective of this study is to collect a dataset for developing methods to detect PD-related symptoms that are visible in walking patterns at home. The movement data are collected continuously and remotely at home during the normal lives of patients with PD as well as controls. The secondary objective is to use the dataset to study whether the registered medication intakes can be identified from the collected movement data by looking for and analyzing short-term changes in walking patterns.MethodsThis paper described the protocol for an observational case-control study that measures activity using three different devices: (1) a smartphone with a built-in accelerometer, gyroscope, and phone orientation sensor, (2) a Movesense smart sensor to measure movement data from the wrist, and (3) a Forciot smart insole to measure the forces applied on the feet. The measurements are first collected during the appointment at the clinic conducted by a trained clinical physiotherapist. Subsequently, the subjects wear the smartphone at home for 3 consecutive days. Wrist and insole sensors are not used in the home recordings.ResultsData collection began in March 2018. Subject recruitment and data collection will continue in spring 2019. The intended sample size was 150 subjects. In 2018, we collected a sample of 103 subjects, 66 of whom were diagnosed with PD.ConclusionsThis study aims to produce an extensive movement-sensor dataset recorded from patients with PD in various phases of the disease as well as from a group of control subjects for effective and impactful comparison studies. The study also aims to develop data analysis methods to monitor PD symptoms and the effects of medication intake during normal life and outside of the clinic setting. Further applications of these methods may include using them as tools for health care professionals to monitor PD remotely and applying them to other movement disorders.Trial RegistrationClinicalTrials.gov NCT03366558; https://clinicaltrials.gov/ct2/show/NCT03366558 International Registered Report Identifier (IRRID)DERR1-10.2196/12808

A smart insole for monitoring plantar pressure based on the fiber Bragg grating sensing technique

The analysis of plantar pressure distribution is essential in the field of biomedical and sports-related applications. In this study, a smart insole was developed for the measurement of plantar pressure distribution and the evaluation of body postures using optical fiber Bragg grating (FBG) sensing technology. Four FBG sensors characterized by four different center Bragg wavelengths, 1528 ± 0.3, 1532 ± 0.3, 1535 ± 0.3 and 1539 ± 0.3 nm, were located at the first metatarsus, third metatarsus, fifth metatarsus and heel position, respectively. The measurement sensitivity of all the FBG sensors was 0.000412 nm/kPa, approximately. Silica gel material of modulus = 10 MPa was selected to incorporate the FBG sensors. All FBG sensors were multiplexed together with one optical fiber cable. The performance and functional properties of all FBG-based pressure sensors were calibrated in the laboratory to evaluate plantar pressure distribution. A male subject was selected for performing four tasks, namely standing in an upright position, leaning forward, squat position and forward fold. During standing tests, plantar pressure observed at the heel position was around 57% higher than that at the first and third metatarsus, while the pressure of the fifth metatarsus position presents minimal pressure, which is only 37% that of the pressure of the heel position. When the subject performs leaning forward, the squat position and forward fold posture, the first and third metatarsi show maximum pressure, while the pressure decreases at the fifth metatarsus position. However, almost zero pressure is observed at the heel position when the subject changes the body postures of leaning forward, squat and forward fold posture. The extreme pressure of the forward fold posture was 1750 kPa acquired at the first metatarsus, which is 52% and 62% higher than those at the fifth and third metatarsi, respectively. Therefore, the smart insole successfully recorded both plantar pressure distribution and body posture changes regarding the wavelength values collected by the FBG sensors.

Automatic Biomechanical Workload Estimation for Construction Workers by Computer Vision and Smart Insoles

AbstractConstruction workers are commonly subject to ergonomic risks due to awkward working postures or lifting/carrying heavy objects. Accordingly, accurate ergonomic assessment is needed to help ...