Converting Human Movements into a Conceptual Painting based on Music Using a Motion Capture System: Action Research

Abstract

Introduction: Motion capturing is a concept which has been taken into consideration in recent decades and it has been used for various purposes like improving the performance of patients with motor problems and increasing motivation and creativity of users. It can also be used in film industry both in two- and three-dimensional animations. Research has shown that this system may be used to enhance creativity and intellect. The aim of this study is to use the Vicon motion capturing device to convert creative human movements into a conceptual painting with the help of music. Materials and Methods: For cultural adaption of the system, the first chapter of “HAFTVAD” music album called “The Introduction” composed by Masoud Nekoyi and the story of Rostam and Sohrab from Shahnameh, written by the Persian poet Ferdowsi, were chosen as the musical groundwork and the scenario for this survey. A human model was trained to perform movements in accordance with the musical and fictional workarounds of the study; the model was also allowed to use self-created styles in movements in the final performance. The segmental movements of the model body were identified in the Vicon Vero system by wearing clothes equipped with reflective markers. This information was transmitted into the Unity engine, and without modeling the body's segments, different colors and brushes were defined to illustrate the path of the movements of the hands, feet, head, and heart. Results: By defining various brushes in different colors, human movements were converted into a conceptual painting in accordance with the music. Conclusion: This study discussed a creative process, entitled Motion Paint, which was implemented in Iran for the first time. This method can be used to make the process of exercise therapy more attractive, especially in chronic motor problems.