Introducing song form intelligence into streaming audio

Abstract

When receiving streaming media over a low bandwidth wireless connection, users can experience not only packet losses but also extended service interruptions. During this time no packets are received and, if not addressed, these dropped packets cause unacceptable interruptions in the audio stream. A long dropout of this kind may be overcome by ensuring that the buffer at the client is large enough. However, when using fixed bit rate technologies this may only be done by buffering packets for an extended period before starting to play the track. During this period, many users are likely to lose interest or become frustrated. This paper presents a novel semantic audio error concealment algorithm which enables the audio to start playing within two or three seconds, while at the same time using a small proportion of the available bandwidth to fill the client device buffer with received packets but categorised into structures of the song. A pattern matching runtime algorithm works to identify portions of the audio stream and when a dropout does occur the relevant sections of the buffered audio are inserted so as to create a perfect match for the lost audio. Our algorithm can be shown to increase reliability on bursty wireless networks.