Audio Fingerprinting Techniques Research Articles

Auditory Image Perception and Verbal Brain Subsystems

This paper describes a novel system that uses music emotion and human face as features for automatic highlights extraction for drama video. These high-level audiovisual features are used because music evokes emotion response from the viewer and characters express emotion on their faces. In addition, a novel scheme is developed to improve the accuracy of music emotion recognition in drama video. Specifically, emotion recognition is performed not on the input audio signal but on the noise-free music available from the album of the incidental music, with the presence of incidental music detected by an audio fingerprint technique. Besides the conventional subjective evaluation, we propose a new metric for quantitative performance evaluation of highlights extraction. Experiments conducted over four different types of drama videos demonstrate that the proposed system significantly outperforms baseline ones in terms of both subjective and objective measures.

SAMAF

Audio fingerprinting techniques were developed to index and retrieve audio samples by comparing a content-based compact signature of the audio instead of the entire audio sample, thereby reducing memory and computational expense. Different techniques have been applied to create audio fingerprints; however, with the introduction of deep learning, new data-driven unsupervised approaches are available. This article presents Sequence-to-Sequence Autoencoder Model for Audio Fingerprinting (SAMAF), which improved hash generation through a novel loss function composed of terms: Mean Square Error, minimizing the reconstruction error; Hash Loss, minimizing the distance between similar hashes and encouraging clustering; and Bitwise Entropy Loss, minimizing the variation inside the clusters. The performance of the model was assessed with a subset of VoxCeleb1 dataset, a“speech in-the-wild” dataset. Furthermore, the model was compared against three baselines: Dejavu, a Shazam-like algorithm; Robust Audio Fingerprinting System (RAFS), a Bit Error Rate (BER) methodology robust to time-frequency distortions and coding/decoding transformations; and Panako, a constellation-based algorithm adding time-frequency distortion resilience. Extensive empirical evidence showed that our approach outperformed all the baselines in the audio identification task and other classification tasks related to the attributes of the audio signal with an economical hash size of either 128 or 256 bits for one second of audio.

Key‐dependent audio fingerprinting technique based on a quantisation minimum‐distance hash extractor in the DWT domain

A novel key-dependent audio fingerprinting technique is proposed by introducing the quantisation minimum distance (QMD) as a hash extractor in the discrete wavelet transform (DWT) domain. The quantiser dithers of the QMD are generated using a chaotic map whose initial value is used as a secret key for fingerprint extraction. Experimental results show that the proposed audio fingerprinting technique achieves with a small fingerprint size an excellent discrimination between audio signals of different contents and outperforms the existing Philips robust hash based techniques in terms of the robustness.

Recognition of Activities of Daily Living Based on Environmental Analyses Using Audio Fingerprinting Techniques: A Systematic Review.

An increase in the accuracy of identification of Activities of Daily Living (ADL) is very important for different goals of Enhanced Living Environments and for Ambient Assisted Living (AAL) tasks. This increase may be achieved through identification of the surrounding environment. Although this is usually used to identify the location, ADL recognition can be improved with the identification of the sound in that particular environment. This paper reviews audio fingerprinting techniques that can be used with the acoustic data acquired from mobile devices. A comprehensive literature search was conducted in order to identify relevant English language works aimed at the identification of the environment of ADLs using data acquired with mobile devices, published between 2002 and 2017. In total, 40 studies were analyzed and selected from 115 citations. The results highlight several audio fingerprinting techniques, including Modified discrete cosine transform (MDCT), Mel-frequency cepstrum coefficients (MFCC), Principal Component Analysis (PCA), Fast Fourier Transform (FFT), Gaussian mixture models (GMM), likelihood estimation, logarithmic moduled complex lapped transform (LMCLT), support vector machine (SVM), constant Q transform (CQT), symmetric pairwise boosting (SPB), Philips robust hash (PRH), linear discriminant analysis (LDA) and discrete cosine transform (DCT).

A low-complexity audio fingerprinting technique for embedded applications

This paper proposes a new optimized audio-based fingerprinting technology for embedded applications. The target use case is related to TV content synchronization and its numerous applications for Social TV and second screen applications. The proposed technology can be used for automatically identifying the program being watched by capturing the sound of the TV set. It can also be used to know which program is being watched and to precisely estimate the timestamp of the currently broadcast moment with respect to the beginning of the program. This is very useful for second screen applications where notifications (e.g. quizzes, additional information, commercials) have to be sent to viewers with a perfect synchronization relatively to the broadcast TV program. The robustness of the proposed technique is first evaluated on a large music database and then by considering a realistic use case where this technology is embedded in a smartphone.

Synchronizing multimodal recordings using audio-to-audio alignment

Research on the interaction between movement and music often involves analysis of multi-track audio, video streams and sensor data. To facilitate such research a framework is presented here that allows synchronization of multimodal data. A low cost approach is proposed to synchronize streams by embedding ambient audio into each data-stream. This effectively reduces the synchronization problem to audio-to-audio alignment. As a part of the framework a robust, computationally efficient audio-to-audio alignment algorithm is presented for reliable synchronization of embedded audio streams of varying quality. The algorithm uses audio fingerprinting techniques to measure offsets. It also identifies drift and dropped samples, which makes it possible to find a synchronization solution under such circumstances as well. The framework is evaluated with synthetic signals and a case study, showing millisecond accurate synchronization.

Audio Fingerprinting Based on Constant Q Transform for TV Commercial Advertisement Identification

오디오 핑거프린팅 기술은 잡음과 에코 등으로 인한 왜곡에도 성공적으로 음원을 식별해야한다. 이러한 오디오 핑거프린팅 기술을 TV광고식별에 적용하고자 한다. 본 논문은 TV 광고 식별을 위한 강인한 오디오 핑거프린팅 방식을 제안한다. 제안된 방법에서 사용되는 Constant Q 변환 기반에서 추출된 현저한 오디오 피크 쌍 핑거프린트는 실제 다양한 잡음환경에서 오디오 핑거프린팅 시스템의 정확도를 향상시키고, 낮은 복잡도를 가진다. 실험결과는 제안된 방식이 기존의 오디오 핑거프린팅 방식에 비해 다양한 잡음환경에서도 안정적이며 신뢰할 수 있는 검색 정확도를 제공함을 보여준다. In spite of distortion caused by noise and echo, the audio fingerprinting technique must identify successfully an audio source. This audio fingerprinting technique is applying for TV commercial advertisement identification. In this paper, we propose a robust audio fingerprinting method for TV commercial advertisement identification. In the proposed method, a prominent audio peak pair fingerprint based on constant Q transform improves the accuracy of the audio fingerprinting system in real noisy environments. Experimental results confirm that the proposed method is quite robust than previous audio fingerprinting method in different noise conditions and achieves promising accurate results.

Automatic highlights extraction for drama video using music emotion and human face features

This paper describes a novel system that uses music emotion and human face as features for automatic highlights extraction for drama video. These high-level audiovisual features are used because music evokes emotion response from the viewer and characters express emotion on their faces. In addition, a novel scheme is developed to improve the accuracy of music emotion recognition in drama video. Specifically, emotion recognition is performed not on the input audio signal but on the noise-free music available from the album of the incidental music, with the presence of incidental music detected by an audio fingerprint technique. Besides the conventional subjective evaluation, we propose a new metric for quantitative performance evaluation of highlights extraction. Experiments conducted over four different types of drama videos demonstrate that the proposed system significantly outperforms baseline ones in terms of both subjective and objective measures.

An effective and efficient indexing scheme for audio fingerprinting

With the proliferation of multimedia data, there is an increasing need to support the ability to automatically analyse and interpret their contents. In this paper, we propose a new effective and efficient indexing scheme for a fingerprint database that stores songs. A song is represented by a fingerprint extracted from the audio fingerprinting technique and it can be recognised by matching an extracted fingerprint to a database of known audio fingerprints. We are given a fingerprint database of songs and focus our attention on the problem of effective and efficient database search. However, the nature of high dimensionality of fingerprints suffers from the curse of dimensionality, i.e., as the dimension increases, the search performance decreases exponentially. In order to tackle this problem, we propose a new indexing scheme based on the inverted file and duplicate fingerprint indexes with toggled bits. In addition, we also present a new search algorithm based on the offset match principle to avoid expen...

A PUBLIC AUDIO IDENTIFICATION EVALUATION FRAMEWORK FOR BROADCAST MONITORING

This paper presents the first public framework for the evaluation of audio fingerprinting techniques. Although the domain of audio identification is very active, both in the industry and the academic world, there is at present no common basis to compare the proposed techniques. This is because corpuses and evaluation protocols differ among the authors. The framework we present here corresponds to a use-case in which audio excerpts have to be detected in a radio broadcast stream. This scenario, indeed, naturally provides a large variety of audio distortions that makes this task a real challenge for fingerprinting systems. Scoring metrics are discussed with regard to this particular scenario. We then describe a whole evaluation framework including an audio corpus, together with the related groundtruth annotation, and a toolkit for the computation of the score metrics. An example of an application of this framework is finally detailed, that took place during the evaluation campaign of the Quaero project. This evaluation framework is publicly available for download and constitutes a simple, yet thorough, platform that can be used by the community in the field of audio identification to encourage reproducible results.

Audio Fingerprinting Based on Multiple Hashing in DCT Domain

Audio fingerprinting techniques aim at successfully performing content-based audio identification even when the audio signals are slightly or seriously distorted. In this letter, we propose a novel audio fingerprinting technique based on multiple hashing. In order to improve the robustness of hashing, multiple hash strings are generated through the discrete cosine transform (DCT) which is applied to the temporal energy sequence in each subband. Experimental results show that the proposed algorithm outperforms the Philips Robust Hash (PRH) algorithm under various distortions.