Abstract
This article describes a stochastic framework for integrating human whole-body motions with natural language. Human whole-body motions in daily life are measured by inertial measurement units (IMU) and subsequently encoded into motion primitives. Sentences are manually attached to the human motion primitives for their descriptions. Two aspects of semantics and syntactics are represented by stochastic modules. One stochastic module trains the linking of motion primitives to words, and the other module represents word order in the sentence structure. These two modules are helpful toward converting human whole-body motions into descriptions, where multiple words are generated from the human motions by the first module, and the second module searches for syntactically consistent sentences consisting of the generated words. The proposed framework is tested on a large dataset of human whole-body motions and their descriptive sentences. The linking of human motions to natural language enables robots to understand observations of human behavior as sentences.
Highlights
H UMANS are animals that create structure
We have developed a framework for integrating human or robotic whole-body motions with natural language by the statistical method [22]–[24]
We describe a statistical framework for conversion from human whole-body motions to descriptive sentences
Summary
H UMANS are animals that create structure. We use languages of words and symbols to do so, and it would be no exaggeration to say that these words and symbols support humanity’s highly advanced society and culture. The function of language is to put the world in order, to maintain that order, and, from that, to create a new order. But we can view it in structural terms by understanding it as a combination of symbols. When the world is organized as low-information symbols, it can be conveyed through conversational language, or recorded for future generations as characters. The results of our history, where we understand social order, maintain it, and recreate a new order, become the advanced knowledge systems of humanity
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Free) 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
