Wobble Strings: Spatially divided stroboscopic effect for augmenting wobbly motion of string instruments

Abstract

Visual stimuli in the form of special lighting effects are often used to provide additional entertainment when musicians perform. Many technologies have been developed to synchronize visual and audio effects; for example, a CMOS camera can be used to capture the motions of the strings of string instruments as a visual medium. However, because a CMOS sensor scans video line-by-line in sequence, fast moving objects are distorted during the scanning sequence. This morphing and distortion are known as the rolling-shutter effect, which is considered an artistic photographic technique such as strip photography and slit-scan photography. This effect can only be seen in a camera viewfinder or on a PC screen and is usually not perceived by the naked eye. We aimed to overcome this problem by developing a system to allow the rolling-shutter effect to be observed in real time using spatially divided stroboscopic projection. The system produces a wobbly slow-motion effect by animating the sweep lines. Our system also alters the color and texture of strings using a projection of the color and texture sweep lines. Furthermore, it can also change the shape of strings to arbitrary two-dimensional shapes such as geometric patterns or patterns consisting of characters.