Haralick feature extraction from timefrequency images for automatic detection and classification of audio anomalies for road surveillance

Abstract

In this paper, we propose a novel method for the detection of road accidents by analyzing audio streams for road surveillance application. In the last decade, due to the increase in a number of people and transportation vehicles, traffic accidents have become one of a major public issue worldwide. The vast number of injuries and death due to road traffic accident reveals the story of a global crisis of road safety. A total of 52,160 road traffic accidents (RTA), 1130 injuries and 85 fatalities were registered during the year 2000, in the state of Qatar. An increase in the number of transportation vehicles around cities has risen the need for more security and safety in public environments. The most obvious reason for a person's death during accidents is the absence or prolong response time of the first aid facility, which is due to the delay in the information of the accident being reached to the police, hospital or ambulance team. In the last couple of years, several surveillance systems have been proposed based on image and video processing for automatically detecting road accidents and car crashes to ensure a quick response by emergency teams. However, in some situations such as adverse weather conditions or cluttered environments, the visual information is not sufficient enough, whereas analyzing the audio tracks can significantly improve the overall reliability of surveillance systems. In this paper we propose a novel method that automatically identifies hazardous situations such as tire skidding and car crashes in presence of background noises, by analyzing the audio streams. Previous studies show that methods for the detection, estimation, and classification of nonstationary signals can be enhanced by utilizing the time-frequency (TF) characteristics of such signals. The TF-based techniques have been proved to outperform classical techniques based on either time or frequency domains, in analyzing real-life nonstationary signals. Time-frequency distributions (TFDs) give additional information about signals that cannot be extracted from either the time domain or frequency domain representations, i.e. the instantaneous frequency of the components of a signal. In order to utilize this extra information provided by TF domain representation, the proposed approach extracts TF image features from quadratic time-frequency distributions (QTFDs), for the detection of audio anomalies. The extended modified-B distribution (EMBD) is utilized to transform a 1-dimensional audio signal into 2-dimensional TF representation which is interpreted as an image. The image descriptors based features are then extracted from the TF representation to classify visually the audio signals into background or abnormal activity patterns in the TF domain. The proposed features are based on Haralick's texture features extracted from the TF representation of audio signals considered and processed as a textured image. These features are used to characterize and hence classify audio signals into M classes. This research study demonstrates that a TF image pattern recognition approach offers significant advantages over standard signal classification methods that utilize either t-domain only or f-domain only features. The proposed method has been experimentally validated on a large open source database of sounds, including several kinds of background noise. The events to be recognized are superimposed on different background sounds of roads and traffic jam. The obtained results are compared with a recent study, utilizing the same large and complex data set of audio signals, and the same experimental setup. The overall classification results confirm the superior performance of the proposed approach with accuracy improvement up to 6%.