Analysis of multimodal data obtained from users of smart textiles designed for mental wellbeing

Abstract

Smart textile and wearable technologies will form an integral part of Internet of Things ecologies, including those implemented in mental health service environments. Smart textiles were developed and provided to users who were asked to discuss their feelings whilst holding/interacting with a smart textile during a walk. In the study discussed, respondents spoke aloud while using prototype handheld smart textiles with wireless capability, soft switches and accelerometers. Voice recordings were collected and transformed into verbal transcripts which were comparatively analysed to assess their validity for understanding individuals' emotional states in Internet of Things-enabled mental health service design. This paper reports on the talk aloud protocol, and the creation of a data analytics model for semantically analysing the transcripts in order to identify the ‘anxious’ from the ‘not so anxious’ participants. The model comprises of two main parts: Latent Semantic Analysis, for semantically analysing the transcripts; and the Fuzzy C-Means clustering algorithm to naturally place the data into the two groups. The analysis revealed significant differences between the vocabulary used by the ‘anxious’ and ‘not so anxious’ participants. Finally, we demonstrate how the voice recorded data can help understand the patterns detected in accelerometer data collected from the smart textiles, using the data of one participant. This approach further provides an understanding of how smart textile objects can be utilised to communicate participant reactions to environments and situations as part of a Person-Centred approach to smart textile and service design development with mental health service providers.