Audio Fingerprinting Research Articles

Earthquake Fingerprints: Extracting Waveform Features for Similarity-Based Earthquake Detection

Seismologists are increasingly adopting data mining and machine learning techniques to detect weak earthquake signals in large seismic data sets. The detection performance of these new methods, especially their sensitivity and false detection rate, depends on the choice of feature representation for waveform data. We have previously introduced Fingerprint and Similarity Thresholding (FAST), a new method for waveform-similarity-based earthquake detection that uses a pattern mining approach to detect earthquake signals without template waveforms. FAST has two key steps: fingerprint extraction and efficient indexing for similarity search. In this work, we focus on FAST fingerprint extraction: the method used to map short-duration waveforms to a set of features, called waveform fingerprints, used for detection. We describe the FAST fingerprint extraction method, a data-adaptive variation on the Waveprint audio fingerprinting method tailored for use in continuous seismic data. We compare the performance of the FAST fingerprint extraction method with existing fingerprinting techniques designed for audio identification. To overcome the challenges associated with using limited or incomplete event catalogs to evaluate detection algorithms, we propose a framework for quantifying the performance of different fingerprint extraction methods in the context of blind similarity-based detection. Our framework uses computational experiments on benchmark data sets, constructed with known event waveforms, to compute a measure of fingerprint effectiveness. We use this framework to show that, among the audio fingerprinting schemes considered in this work, our proposed FAST fingerprint extraction method achieves the most consistent performance in distinguishing similar, low signal-to-noise earthquake waveforms from noise in waveform data sets from eight stations in the Northern California Seismic Network.

A Case for Reproducibility in MIR: Replication of ‘A Highly Robust Audio Fingerprinting System’

Claims made in many Music Information Retrieval (MIR) publications are hard to verify due to the fact that (i) often only a textual description is made available and code remains unpublished – leaving many implementation issues uncovered; (ii) copyrights on music limit the sharing of datasets; and (iii) incentives to put effort into reproducible research – publishing and documenting code and specifics on data – is lacking. In this article the problems around reproducibility are illustrated by replicating an MIR work. The system and evaluation described in ‘A Highly Robust Audio Fingerprinting System’ is replicated as closely as possible. The replication is done with several goals in mind: to describe difficulties in replicating the work and subsequently reflect on guidelines around reproducible research. Added contributions are the verification of the reported work, a publicly available implementation and an evaluation method that is reproducible.

A Framework for Infrastructure-Free Indoor Localization Based on Pervasive Sound Analysis

Even as modern indoor positioning systems become more precise and computationally lightweight, most rely on specific infrastructure to be installed, leading to increased setup and maintenance costs. As such, multiple infrastructure-free solutions were devised relying on signals such as magnetic field, ambient light, and movement. In this paper, we propose a framework for determining the user’s location through the sound recorded by the user’s device. With this goal, we present two algorithms: SoundSignature and SoundSimilarity. With SoundSignature, we extract acoustic fingerprints from the recorded audio and employ them in a support vector machine classifier. With SoundSimilarity, where we employ a novel audio similarity measure to detect if users are in the same location as other users or microphone equipped devices. Both of these algorithms require no infrastructure and are computationally lightweight, thus allowing their use either in conjunction with other infrastructure-free technologies or standalone. The training of these algorithms requires nothing more than a smartphone or a similar device under normal usage conditions, eliminating the need of any dedicated equipment.

Key‐dependent audio fingerprinting technique based on a quantisation minimum‐distance hash extractor in the DWT domain

A novel key-dependent audio fingerprinting technique is proposed by introducing the quantisation minimum distance (QMD) as a hash extractor in the discrete wavelet transform (DWT) domain. The quantiser dithers of the QMD are generated using a chaotic map whose initial value is used as a secret key for fingerprint extraction. Experimental results show that the proposed audio fingerprinting technique achieves with a small fingerprint size an excellent discrimination between audio signals of different contents and outperforms the existing Philips robust hash based techniques in terms of the robustness.

Monitoring System Based on Acoustic Fingerprint for the Crimping Process of the Composite Insulators

The crimping process is mainly determined by the initial tolerance field of the fitting-rod assembly and the hardness of the fittings. The initial tolerance field of the fitting-rod assembly results in a higher crimping pressure as the tolerance field increases. The greater the hardness of the fittings, the higher the crimping pressure. In practice, due to the above mentioned factors, it is impossible to set an optimum crimping pressure to ensure a correct fitting-rod assembly. The article presents a system for monitoring the acoustic emissions during the crimping process of composite insulators. The system achieves an automatic classification of the type of crimping, a database of the acoustic emission graph recordings, the storage of the values as TDMS files, and generates automatic reports of the crimping process. This method was successfully tested.

Real-Time Detection of Important Sounds with a Wearable Vibration Based Device for Hearing-Impaired People

Hearing-impaired people do not hear indoor and outdoor environment sounds, which are important for them both at home and outside. By means of a wearable device that we have developed, a hearing-impaired person will be informed of important sounds through vibrations, thereby understanding what kind of sound it is. Our system, which operates in real time, can achieve a success rate of 98% when estimating a door bell ringing sound, 99% success identifying an alarm sound, 99% success identifying a phone ringing, 91% success identifying honking, 93% success identifying brake sounds, 96% success identifying dog sounds, 97% success identifying human voice, and 96% success identifying other sounds using the audio fingerprint method. Audio fingerprint is a brief summary of an audio file, perceptively summarizing a piece of audio content. In this study, our wearable device is tested 100 times a day for 100 days on five deaf persons and 50 persons with normal hearing whose ears were covered by earphones that provided wind sounds. This study aims to improve the quality of life of deaf persons, and provide them a more prosperous life. In the questionnaire performed, deaf people rate the clarity of the system at 90%, usefulness at 97%, and the likelihood of using this device again at 100%.

A hybrid technique for speech segregation and classification using a sophisticated deep neural network.

Recent research on speech segregation and music fingerprinting has led to improvements in speech segregation and music identification algorithms. Speech and music segregation generally involves the identification of music followed by speech segregation. However, music segregation becomes a challenging task in the presence of noise. This paper proposes a novel method of speech segregation for unlabelled stationary noisy audio signals using the deep belief network (DBN) model. The proposed method successfully segregates a music signal from noisy audio streams. A recurrent neural network (RNN)-based hidden layer segregation model is applied to remove stationary noise. Dictionary-based fisher algorithms are employed for speech classification. The proposed method is tested on three datasets (TIMIT, MIR-1K, and MusicBrainz), and the results indicate the robustness of proposed method for speech segregation. The qualitative and quantitative analysis carried out on three datasets demonstrate the efficiency of the proposed method compared to the state-of-the-art speech segregation and classification-based methods.

On Smartphone Power Consumption in Acoustic Environment Monitoring Applications

In this paper, we provide a systematic analysis of smartphone power consumption in background audio recording scenarios. We report comprehensive measurement results for several Android smartphone models with diverse hardware specifications, running different types of Android operating systems (OS). Our experimental results show that with respect to background audio sensing applications, the major impact on smartphone power consumption is associated with the core audio recording functionality. On the other hand, moderately complex digital signal processing algorithms (e.g., sound pressure level estimation, acoustic feature extraction, and audio fingerprinting) that operate simultaneously with background audio recording do not significantly affect overall smartphone power consumption or battery life. Power consumption varies significantly from model to model, and we found no obvious correlation between the power consumption of background audio recording scenarios and factors such as the year of release, the OS version, or the model of smartphone processor or system-on-chip (SoC) hardware.

Investigation of deturbing methods for sonic boom signatures

There is a desire to certify aircraft for supersonic flight using microphone measurements of the aircraft’s sonic boom signature on the ground. If the signature propagates through the planetary boundary layer before it reaches the ground, the signature becomes distorted because of atmospheric turbulence, corrupting the measurement. Methods to remove the turbulence effect, or deturbing methods, currently exist but are restricted to typical N-wave shapes or knowledge of the turbulence structure beforehand. This work investigates methods of deturbing sonic boom signatures independent of the N-wave shape and not requiring prior knowledge of turbulence. Improvements are made to the averaging method and subtraction method of deturbing, and a new method is investigated which uses audio fingerprinting, an audio identification algorithm used by music identification apps such as Shazam. The effects of each deturbing method are described using a wide set of physical and subjective metrics. [Work supported by the FAA. The opinions, findings, conclusions, and recommendations expressed in this material are those of the authors and do not necessarily reflect the views of ASCENT FAA Center of Excellence sponsor organizations.]

Audio fingerprint hierarchy searching strategies on GPGPU massively parallel computer

ABSTRACTAudio fingerprint was developed for representing the audio based on the content of waveform. With the audio fingerprint database, we can easily manage the song/music with high reliability and flexibility. However, with the well-developed Internet of today, the audio data have become bigger and bigger which make the management of audio/music data more difficult. There are two problems that we need to solve when the audio fingerprint database turn into bigdata: the size of the database needs to be sufficient for storing 10 millions of audio fingerprint and the strategies for searching the nearest song in acceptable time for thousands of queries at once [Nguyen Mau, T., & Inoguchi, Y. (2016). Audio fingerprint hierarchy searching on massively parallel with multi-gpgpus using K-modes and lsh. Eighth international conference on knowledge and systems engineering (KSE) (pp. 49–54). IEEE]. In this research, we propose the methods for storing the audio fingerprint using multiple GPGPU and nearest song searching strategies based on these databases. We also showed that our methods have the significant result for deploying the real system in the future.

Recognition of Activities of Daily Living Based on Environmental Analyses Using Audio Fingerprinting Techniques: A Systematic Review.

An increase in the accuracy of identification of Activities of Daily Living (ADL) is very important for different goals of Enhanced Living Environments and for Ambient Assisted Living (AAL) tasks. This increase may be achieved through identification of the surrounding environment. Although this is usually used to identify the location, ADL recognition can be improved with the identification of the sound in that particular environment. This paper reviews audio fingerprinting techniques that can be used with the acoustic data acquired from mobile devices. A comprehensive literature search was conducted in order to identify relevant English language works aimed at the identification of the environment of ADLs using data acquired with mobile devices, published between 2002 and 2017. In total, 40 studies were analyzed and selected from 115 citations. The results highlight several audio fingerprinting techniques, including Modified discrete cosine transform (MDCT), Mel-frequency cepstrum coefficients (MFCC), Principal Component Analysis (PCA), Fast Fourier Transform (FFT), Gaussian mixture models (GMM), likelihood estimation, logarithmic moduled complex lapped transform (LMCLT), support vector machine (SVM), constant Q transform (CQT), symmetric pairwise boosting (SPB), Philips robust hash (PRH), linear discriminant analysis (LDA) and discrete cosine transform (DCT).

Genre Classification for Brazilian Music using Independent and Discriminant Features

Digital music files are largely available both online and in private local collections. These databases may comprise hundreds or thousands of files, which in some cases may not carry tagged information about their content, making the search for the desired audio files very time consuming. An important task in this context is to organize the available database according to the prevailing musical genre. The purpose of this work is to develop an automatic music genre classification system able to identify international music genres (i.e. pop, rock, classic, soul, funk) and also typical Brazilian rhythms such as Samba, Forr\'o and Brazilian Popular Music. The proposed signal processing chain comprises two stages. Initially, audio signal features are computed and their relevance for music genre identification estimated. Independent component analysis is applied to reduce mutual redundancy among the audio attributes. In the following, different classifiers based on neural networks and support vector machines are applied for music genre identification. The proposed system efficiency is evaluated using an experimental dataset.

Monitoring Tool Wear in Drilling Process Using Spindle Noise Features

The use of worn cutting tools has a detrimental effect on the surface finish of a workpiece, tool precision and internal machine stress. Worn tools also decrease productivity through unplanned stops, tool changes and increase the production of scrap material. This study investigates an inexpensive and non-intrusive method of inferring tool wear by measuring the audible sounds emitted during a drilling process. A microphone was used to record the machining operation sound of S50C steel, which was drilled using a computer numerical control (CNC) milling machine in wet conditions. The audio signature was examined using a spectrogram, and the extracted sound features of the rotating spindle motor in the frequency domain were used to correlate with tool wear. The results indicated that the frequency of spindle noise was unrelated to tool wear, but although the magnitude of spindle noise significantly increased in accordance with tool wear progression.

Automatic Recovery and Verification of Subtitles for Large Collections of Video Clips

This paper describes an experimental system that can create good quality subtitle files for video clips derived from broadcast content. The system is designed to run automatically without the need for human verification. The approach utilises existing metadata sources, an off-air broadcast archive and an archive of original subtitle files along with audio fingerprinting and speech-to-text technology to identify the source programme. It then locates the position of the video clip, verifies the match between the video clip and the subtitles and create a new subtitle file. This paper also reports on the results of the work using a large corpus of over 7,000 video clips and further, smaller sets of clips from different television genres, and explores where improvements might be made. It also looks at the limitations of the current approach discussing alternative methods for providing subtitles for video clips.

Audio Identification by Sampling Sub-fingerprints and Counting Matches

It is challenging to retrieve audio clips from large audio datasets not only due to the high dimensionality of audio but also due to the large number of audios. Fingerprinting methods primarily focus on the use of semantic-level techniques to speed up retrieval and neglect low-level support. This paper shows that the performance of audio retrieval can be exploited by properly organizing and manipulating audio fingerprint data. The proposed sampling and counting method markedly improves the retrieval speed while maintaining a high recall rate and high precision for short audio clips. The proposed inverted index structure for fingerprints quickly shrinks the scope of the candidate set while requiring considerably less memory. The experiments show that the proposed method is faster and yields more consistent performance in terms of recall rate and precision than do the state-of-the-art methods. The proposed method is scalable for big audio data and is robust to background noise, resampling, MP3 conversion, white noise addition, bandpass filtering, chorus, echo, flanger, gsm compression, and tremolo. The average precision and recall rates are 99.995% and 99.78%, respectively, for dataset 1 and average 98.97% and 95.15%, respectively, for dataset 2. Specifically, the precision and recall rates for query audio clips of 3 s in length in the targeted dataset 1 are 99.94% and 99.64%, respectively.

SNORAP: A Device for the Correction of Impaired Sleep Health by Using Tactile Stimulation for Individuals with Mild and Moderate Sleep Disordered Breathing.

Sleep physiology and sleep hygiene play significant roles in maintaining the daily lives of individuals given that sleep is an important physiological need to protect the functions of the human brain. Sleep disordered breathing (SDB) is an important disease that disturbs this need. Snoring and Obstructive Sleep Apnea Syndrome (OSAS) are clinical conditions that affect all body organs and systems that intermittently, repeatedly, with at least 10 s or more breathing stops that decrease throughout the night and disturb sleep integrity. The aim of this study was to produce a new device for the treatment of patients especially with position and rapid eye movement (REM)-dependent mild and moderate OSAS. For this purpose, the main components of the device (the microphone (snore sensor), the heart rate sensor, and the vibration motor, which we named SNORAP) were applied to five volunteer patients (male, mean age: 33.2, body mass index mean: 29.3). After receiving the sound in real time with the microphone, the snoring sound was detected by using the Audio Fingerprint method with a success rate of 98.9%. According to the results obtained, the severity and the number of the snoring of the patients using SNORAP were found to be significantly lower than in the experimental conditions in the apnea hypopnea index (AHI), apnea index, hypopnea index, in supine position’s AHI, and REM position’s AHI before using SNORAP (Paired Sample Test, p < 0.05). REM sleep duration and nocturnal oxygen saturation were significantly higher when compared to the group not using the SNORAP (Paired Sample Test, p < 0.05).

A Low Cost Wireless Acoustic Sensor for Ambient Assisted Living Systems

Ambient Assisted Living (AAL) has become an attractive research topic due to growing interest in remote monitoring of older people. Development in sensor technologies and advances in wireless communications allows to remotely offer smart assistance and monitor those people at their own home, increasing their quality of life. In this context, Wireless Acoustic Sensor Networks (WASN) provide a suitable way for implementing AAL systems which can be used to infer hazardous situations via environmental sounds identification. Nevertheless, satisfying sensor solutions have not been found with the considerations of both low cost and high performance. In this paper, we report the design and implementation of a wireless acoustic sensor to be located at the edge of a WASN for recording and processing environmental sounds which can be applied to AAL systems for personal healthcare because it has the following significant advantages: low cost, small size, audio sampling and computation capabilities for audio processing. The proposed wireless acoustic sensor is able to record audio samples at least to 10 kHz sampling frequency and 12-bit resolution. Also, it is capable of doing audio signal processing without compromising the sample rate and the energy consumption by using a new microcontroller released at the last quarter of 2016. The proposed low cost wireless acoustic sensor has been verified using four randomness tests for doing statistical analysis and a classification system of the recorded sounds based on audio fingerprints.

Connecting chord progressions with specific pieces of music

Musicians can conceptualize harmony in terms of its connection to specific pieces of music. However, research appears to indicate that harmony plays a relatively unimportant role in music identification tasks. The present study examines the ability of listeners of varying levels of musical expertise to identify music from chord progressions. Participants were asked to identify well-known classical and pop/rock pieces from their chord progressions, which were recorded using either piano tones or Shepard tones and were played at six transpositional levels. Although musical training and invariance of surface melodic and rhythmic features were found to have an advantageous effect on the identification task, even some non-musicians were able to identify music from chord progressions in conditions of low invariance of surface features. Implications of these results for our understanding of how listeners mentally represent and remember harmony are discussed.

Efficient music identification using ORB descriptors of the spectrogram image

Audio fingerprinting has been an active research field typically used for music identification. Robust audio fingerprinting technology is used to successfully perform content-based audio identification regardless of the audio signal being subjected to various types of distortion. These distortions affect the time-frequency correlation relating to pitch and speed changes. In this paper, experiments are done using the computer vision technique ORB (Oriented FAST and Rotated BRIEF) for robust audio identification. Investigations are conducted for ORB, relating to its advantage of robustness against distortions including speed and pitch changes. The ORB prototype compares the features of the spectrogram image query to a database of spectrogram images of the songs. For the initial experiment, a Brute-Force matcher is used to compare the ORB descriptors. Results show that the ORB prototype performs robustly to real-world distortions with fast, reliable performance against distortions such as speed and pitch which justifies the research done.

State Identification of Smart Devices Using Acoustic Fingerprinting

The widespread use of smart phones has brought the security and privacy problems. In the past research, fingerprinting of physical device can be used in authenticating system. However, using the fingerprinting to identify and track the user is useful in many legitimate scenes. In this paper, we propose a novel acoustic fingerprinting approach that uses the microphones and speakers of devices to assess the state of target device. While a person is walking or running with a smart phone, a certain amount of vibration is generated, which can be investigated as the basis of state identification. For mining the subtle vibration variations, firstly, a special high-frequency control simulation to speakers is introduced. Then an unsupervised learning method called Incremental Slow Feature Analysis (IncSFA) is applied to construct the state fingerprinting to represent the slow variation triggered by external driving. At last, two alternate approaches as support vector machine (SVM) and Bayes classifier are used to classify the different states from different smart phones.