Audio Fingerprinting Research Articles

Piano Sheet Music Identification Using Dynamic N-gram Fingerprinting

This article introduces a method for large-scale retrieval of piano sheet music images. We study this problem in two different scenarios: camera-based sheet music identification and MIDI-sheet image retrieval. Our proposed method combines bootleg score features with a novel hashing scheme called dynamic N-gram fingerprinting. This hashing scheme ensures that every fingerprint is discriminative enough to warrant a table lookup, which improves both retrieval accuracy and runtime. On experiments using all piano sheet music images in the IMSLP database, the proposed method achieves >0.8 mean reciprocal rank with sub-second runtimes. As a practical application, we use our system to find matches between the Lakh MIDI dataset and IMSLP, which augments the IMSLP sheet music data with symbolic music information for a subset of pieces. We release our code and Lakh-IMSLP matches to facilitate future study.

A Comprehensive Survey of Personalized Music Identifier System

Music occupies a very important space in the heart and life of common people and it is rather subjective and universal nature indeed. Music Identifier System is obviously concerned with providing a very meaningful and personalized recommendation of items i.e. songs, music, playlist according to the mood, emotion, interest and preference of the users or listeners. With the advancement of technologies, rapid development of internet, it has become very common to use the streaming services to listen and enjoy music or songs in more convenient ways. In this paper, an attempt has been made to perform a comparative analysis, systematic research, empirical thorough review on various approaches or strategies proposed and applied by different researchers in the task of designing an effective system for music identification or recommendation. The basic theme of the paper includes music identifier system, its components, and different features along with emphasize on the methods, metrics, general framework and state-of-art strategies proposed during the last two decades or so, have been empirically reviewed. The existing studies were found lacking with systematic research work on the behaviour, requirements and preferences of the users plus poor level of extraction of features and limitations in the area of evaluation of performance of the music identifier systems. Although, the study reveals that systems based on effective, social information, emotional-traits, content, context and knowledge have been widely applied and improved the quality of identification or recommendation of music to a large extend but still it is not enough. In future, more in-depth studies or research work need to be conducted based on enlarging the scope of further development of personalized contextual awareness based music identifier system and generating a continuous and automatic top playlist of music and songs with added tracks matching with profile, mood, emotional traits, and behaviour of the user in a mobile environment.

Adaptability of Ultrasonic Lamb Wave Touchscreen to the Variations in Touch Force and Touch Area.

Previous studies on Lamb wave touchscreen (LWT) were carried out based on the assumption that the unknown touch had the consistent parameters with acoustic fingerprints in the reference database. The adaptability of LWT to the variations in touch force and touch area was investigated in this study for the first time. The automatic collection of the databases of acoustic fingerprints was realized with an experimental prototype of LWT employing three pairs of transmitter–receivers. The self-adaptive updated weight coefficient of the used transmitter–receiver pairs was employed to successfully improve the accuracy of the localization model established based on a learning method. The performance of the improved method in locating single- and two-touch actions with the reference database of different parameters was carefully evaluated. The robustness of the LWT to the variation of the touch force varied with the touch area. Moreover, it was feasible to locate touch actions of large area with reference databases of small touch areas as long as the unknown touch and the reference databases met the condition of equivalent averaged stress.

Audio Fingerprint Retrieval Method Based on Feature Dimension Reduction and Feature Combination

Audio Fingerprint Retrieval Method Based on Feature Dimension Reduction and Feature Combination

Dropping malware through sound injection: A comparative analysis on Android operating systems

Trojan droppers are regularly ranked among the top 5 worst malware threats; especially for Android. Software for malware detection opts to either identify the payload when executed in memory or detect the connection from which droppers attempt to download malicious payloads. Much effort has been put into securing network channels and stopping droppers from delivering their payloads. This paves the way for alternative routes of infiltration. Our paper extends work on inaudible sound covert channel attacks and audio fingerprinting to use them for close vicinity attacks as potential mass dropper techniques. We demonstrate that malware modules can be dropped over the air to multiple devices using the same sound medium from approximately four to five meters away. Instead of using network connections, we conceal the malware within musics inaudible frequencies. Then we use a seemingly innocent app that implements an algorithm similar to Shazams audio fingerprinting to transmit the payload over the devices microphone and execute it. To accomplish this, we combine Short-Time Discrete Fourier Transform, peak frequency mapping and dilation techniques to a Meterpreter payload as fluctuations in magnitude within inaudible frequencies. To our knowledge, there is currently no technique capable of transmitting payloads to multiple neighboring systems without discrete network connections between the payload remote location and the device. We, therefore, present realistic attack cases and provide solutions for these attack vectors, including modulation and high-peak compression of sound dependent channel frequencies.

(In)secure Acoustic Mobile Authentication

Acoustic fingerprinting aims to identify a mobile device based on its internal microphone(s) and speaker(s) which are unique due to manufacturing imperfection. This paper seeks a thorough understanding of the (in)security of exploring acoustic fingerprints for achieving distributed mobile authentication. Our contributions are threefold. First, we present a new acoustic fingerprint-emulation attack and demonstrate that it is a common vulnerability of acoustic mobile authentication systems. Second, we propose a dynamic challenge-response defense to secure acoustic mobile authentication systems against the acoustic fingerprint-emulation attack. Finally, we thoroughly investigate existing acoustic fingerprinting schemes and identify the best option for accurate, secure, and deployable acoustic mobile authentication systems.

Folded space of machine listening

The paper investigates new machine listening technologies through a comparison of phenomenological and empirical/media-archeological approaches. While phenomenology associates listening with subjectivity, empiricism takes into account the technical operations involved with listening processes in both human and non-human apparatuses. Based on this theoretical framework, the paper undertakes a media-archeological investigation of two algorithms employed in copyright detection: “acoustic fi ngerprinting” and “audio watermarking”. In the technical operations of sound recognition algorithms, empirical analysis suggests the coexistence of a multiplicity of spatialities: from the “sound event”, which occurs in three-dimensional physical space, to its mathematical representation in vector space, and to the one-dimensional informational space of data processing and machine-to-machine communication. Recalling Deleuze’s defi nition of “the fold”, we defi ne these coexistent spatial dimensions in techno-culturally mediated sound as “the folded space” of machine listening. We go on to argue that the issue of space in machine listening consists of the virtually infi nite variability of the sound event being subjected to automatic recognition. The diffi culty lies in conciliating the theoretically enduring information transmitted by sound with the contingent manifestation of sound affected by space. To make machines able to deal with the site-specifi city of sound, recognition algorithms need to reconstruct the three-dimensional space on a signal processing level, in a sort of reverse-engineering of the sound phenomenon that recalls the concept of “implicit sonicity” defi ned by Wolfgang Ernst. While the metaphors and social representations adopted to describe machine listening are often anthropomorphic – and the very term “listening”, when referring to numerical operations, can be seen as a metaphor in itself – we argue that both human listening and machine listening are co-defi ned in a socio-technical network, in which the listening space no longer coincides with the position of the listening subject, but is negotiated between human and nonhuman agencies.

Integrating Dilated Convolution into DenseLSTM for Audio Source Separation

Herein, we proposed a multi-scale multi-band dilated time-frequency densely connected convolutional network (DenseNet) with long short-term memory (LSTM) for audio source separation. Because the spectrogram of the acoustic signal can be thought of as images as well as time series data, it is suitable for convolutional recurrent neural network (CRNN) architecture. We improved the audio source separation performance by applying the dilated block with a dilated convolution to CRNN architecture. The dilated block has the role of effectively increasing the receptive field in the spectrogram. In addition, it was designed in consideration of the acoustic characteristics that the frequency axis and the time axis in the spectrogram are changed by independent influences such as speech rate and pitch. In speech enhancement experiments, we estimated the speech signal using various deep learning architectures from a signal in which the music, noise, and speech were mixed. We conducted the subjective evaluation on the estimated speech signal. In addition, speech quality, intelligibility, separation, and speech recognition performance were also measured. In music signal separation, we estimated the music signal using several deep learning architectures from the mixture of the music and speech signal. After that, the separation performance and music identification accuracy were measured using the estimated music signal. Overall, the proposed architecture shows the best performance compared to other deep learning architectures not only in speech experiments but also in music experiments.

Segmental DCT Coefficient Reversal Based Anti-Collusion Audio Fingerprinting Mechanism

Collusion attacks are challenging to tackle in audio fingerprinting. A new direction to resist collusion attacks is to degrade the perceptual quality of the colluded files so that these files cannot be reused. The existing method in this direction has low embedding capacity and limited anti-collusion performance when the number of colluders is odd. In this letter, we present an anti-collusion mechanism that has a higher embedding capacity and can significantly degrade the perceptual quality of the colluded files regardless of the number of colluders. In the proposed mechanism, we first segment the host audio file into frames and perform the discrete cosine transform (DCT) on each frame. Then multiple fingerprint bits are embedded into each frame by reversing the DCT coefficients of the corresponding frequency band. As a result, when a collusion attack occurs, our proposed embedding mechanism can introduce perceptibly annoying differences between frames in the colluded file, which leads to severe perceptual quality degradation. Theoretical analysis and experimental results validate the superiority of the proposed anti-collusion mechanism.

Audio fingerprint for automatic Balinese rindik music identification using gaussian mixture model

Rindik is a traditional gamelan instrument that has been known for a long time in which is made from bamboo. Rindik music can be identified by just hearing a little bit snippets from the music that can only be done by professional rindik players. This research assumes doing some training in the system can make the system recognize the music by just using some rindik music snippets. This research focuses on comparing some features extraction method for automatic rindik music identification by using some rindik music snippet to find out which of these features perform the best to represent rindik music. This research will use an audio fingerprint method for song identification. MFCC (Mel Frequency Cepstral Coefficient) and Spectral Subband Centroid will be used and compared as the music’s feature, and Gaussian Mixture Model will be used to model the music’s fingerprint. The result of this research show both methods give excellent results. Both features only need 10-second duration data to get over 90% overall accuracies. Both Feature gives excellent performance with GMM for automatic rindik song identification task. However, Spectral subband centroid show better result with the highest accuracies is 99.3%.

A Study on the Criteria for Determining the Infringement of Music Copyright

저작권 침해 판단 기준은 피침해 저작물과의 ‘의거 관계’, 창작적 표현의 ‘실질적 유사성’ 여부 등의 기준에 의해 판단한다. 본 논문에서는 음악 저작권 침해 판단 기준을 창작적 표현 범위에 대한 ‘실질적 유사성’을 중심으로 분석하였다. 저작권 침해 판단 기준을 제안하기 위해서 다른 저작물별로 정량적 기준과 정성적 기준에 의해서 비교 분석할 수 있었다. 이러한 기준을 비추어 보았을 때, 음악 저작물의 침해 판단 기준에도 정량적 기준과 정성적 기준을 도출하여 기존의 국내･외 판례들을 비교하여 적용하였다 그러나 음악 저작물에 대한 정량적, 정성적 침해 판단 기준이 명확하지 않다는 것을 확인하였다. 따라서 해당 음악 저작권 침해 판단의 한계 부분을 명확하게 하기 위해서 개선방안으로 기술적 요소를 도입할 필요가 있다. 현재까지 음악 저작권 침해 판단 기술은 AI 기반의 데이터 마이닝 분석 기법들을 활용한 음악 식별 시스템을 통해서 추출한 음원 간 유사한 패턴인 수치화된 표현으로 가능하다. 그러나 관용적 표현과 창작적 표현에 대한 기준 수립이 선행되어야 하지만 아직까지 해당 한계점에 대해서는 다루어지지 않고 있다. 또한 침해 판단 기술 활용의 예로 논문을 제출할 때 유사도 검증을 위해서 카피킬러 서비스를 이용하여 제출한다. 음악의 경우도 해당 기준이 필요하다. 즉, 음악을 만듦에 있어서 침해 가능성을 예측 가능한 서비스가 필요하다. 따라서 카피킬러의 논문 유사도 검색 서비스처럼 음악 데이터를 데이터베이스 및 기술적 수단을 활용하여 저작자가 본인의 음악을 침해 분석하였을 때 기존 음악과 유사율이 몇 % 정도이고 창작적 표현이 몇 % 정도인지 검증이 가능한 기술적 도입이 필요하다. 이로 인해 음악 저작권 침해 판단 기준의 한계를 극복하자는 게 이 글의 주장이다.In this paper, the criteria for judging infringement of music copyrights were analyzed centering on the ‘substantial similarity’ of the scope of creative expression. In order to propose criteria for judging copyright infringement, they could be compared and analyzed by quantitative and qualitative standards for different works. Quantitative and qualitative standards were derived from judging infringement of music works by comparing existing domestic and foreign precedents. However, it was confirmed that the criteria for determining quantitative and qualitative infringement on music works were not clear. Therefore, it is necessary to introduce technical elements as a measure of improvement to clarify the limitations of the judgment of infringement of the music copyright. So far, music copyright infringement judging technology is possible with digitized expressions, a similar patterns between sound sources extracted through a music identification system using AI-based data mining analysis techniques. However, although the establishment of standards for idiomatic and creative expressions should precede, the limitations have not yet been addressed. For example, when submitting a paper as an example of the use of infringement judgment technology, a copy-killer service is used to verify similarity. In the case of music, such standards are also needed. Therefore, it is necessary to introduce a technology that can verify what percentage is similar to existing music and is creative expression each. As a result, the article argues that the government should overcome the limitations of judging standards for infringement of music copyrights.

Comparative Study of Dimensionality Reduction Techniques for Spectral–Temporal Data

This paper studies the use of three different approaches to reduce the dimensionality of a type of spectral–temporal features, called motion picture expert group (MPEG)-7 audio signature descriptors (ASD). The studied approaches include principal component analysis (PCA), independent component analysis (ICA), and factor analysis (FA). These approaches are applied to ASD features obtained from audio items with or without distortion. These low-dimensional features are used as queries to a dataset containing low-dimensional features extracted from undistorted items. Doing so, we may investigate the distortion-resistant capability of each approach. The experimental results show that features obtained by the ICA or FA reduction approaches have higher identification accuracy than the PCA approach for moderately distorted items. Therefore, to extract features from distorted items, ICA or FA approaches should also be considered in addition to the PCA approach.

Propagation of ultrasonic guided waves in plant leaves

In this work, dispersive wave propagation in plant leaves is studied using a long distance laser-based ultrasonic method. Guided waves traveling through and along the midrib, as well as in the lamina, are measured and analyzed via wave attenuation, characteristic diagram and dispersion curves. Additionally, an estimation of the leaf elastic properties using the Rayleigh–Lamb model, supported by thickness-resonance frequency constraints, is proposed. This study paves the way for the development of acoustic fingerprints for the identification of leaves and their non-invasive and fairly non-destructive mechanical characterization, which is highly related to the plant hydraulics and photosynthetic activity.

Design and development of biometric voting system using fingerprint and facial recognition

The election process adopted in Nigeria is mostly paper based or manual. This manually handled process can be time-consuming, and prone to security breaches and electoral fraud. Consequently, this work proposes a biometrically authenticated voting system to provide inexpensive solution to the electoral process in Nigeria. In particular, the biometric database is constructed by filling it with the audio fingerprints and the associated metadata of many audio clips, and then the fingerprint of an unknown clip (or the distorted version of the clip brought by compression or standard audio processing) is extracted and compared to that of the clip in the database. If the fingerprint of the unknown clip is in the database, it will be correctly identified by the matching procedures. The project is implemented with biometric system i.e. fingerprint scanning and facial recognition. Accordingly, this is used to ensure enhanced security, and to avoid fake, repeated voting. Results obtained showed that the proposed system is fast, accurate, reliable, and cost effective, thus a viable option to replace the manual voting process.
 Keywords: Biometrics, electoral process, facial recognition, finger print scanning

Audio Fingerprint Extraction Based on Locally Linear Embedding for Audio Retrieval System

With the appearance of a large amount of audio data, people have a higher demand for audio retrieval, which can quickly and accurately find the required information. Audio fingerprint retrieval is a popular choice because of its excellent performance. However, there is a problem about the large amount of audio fingerprint data in the existing audio fingerprint retrieval method which takes up more storage space and affects the retrieval speed. Aiming at the problem, this paper presents a novel audio fingerprinting method based on locally linear embedding (LLE) that has smaller fingerprints and the retrieval is more efficient. The proposed audio fingerprint extraction divides the bands around each peak in the frequency domain into four groups of sub-regions and the energy of every sub-region is computed. Then the LLE is performed for each group, respectively, and the audio fingerprint is encoded by comparing adjacent energies. To solve the distortion of linear speed changes, a matching strategy based on dynamic time warping (DTW) is adopted in the retrieval part which can compare two audio segments with different lengths. To evaluate the retrieval performance of the proposed method, the experiments are carried out under different conditions of single and multiple groups’ dimensionality reduction. Both of them can achieve a high recall and precision rate and has a better retrieval efficiency with less data compared with some state-of-the-art methods.

Acoustic analysis of starting jets in an anechoic chamber: implications for volcano monitoring

Explosive volcanic eruptions are associated with a plethora of geophysical signals. Among them, acoustic signals provide ample information about eruptive dynamics and are widely used for monitoring purposes. However, a mechanistic correlation of monitoring signals, underlying source processes and reasons for short-term variations is incomplete. Scaled laboratory experiments can mimic a wide range of explosive volcanic eruption conditions. Here, starting (non-steady) compressible gas jets are created using a shock tube in an anechoic chamber and their acoustic signature is recorded with a microphone array. Noise sources are mapped in time and frequency using wavelet analysis and their dependence from pressure ratio, non-dimensional mass supply and exit-to-throat area ratio is deciphered. We observed that the pressure ratio controls the establishment of supersonic conditions and their duration, and influences the interaction between shock, shear layer, and vortex ring. The non-dimensional mass supply affects the duration of the discharge, the maximum velocity of the flow, and the existence of a trailing jet. Lower values of exit-to-throat area ratio induce a faster decay of the acoustic fingerprint of the jet flow. The simplistic experiments presented here, and their acoustic analysis will serve as an essential starting point to infer source conditions prior to and during impulsive volcanic eruptions.

Detection of speech playback attacks using robust harmonic trajectories

In this paper, a new feature set is proposed for use in a playback attack detector (PAD) aimed at safeguarding a passphrase and speaker-verified protected system that can be remotely accessed from an arbitrary location using an arbitrary telecommunication channel. The new feature set, termed VoicedTracks, is a time-frequency map of the most robust harmonic trajectories in an utterance and serves as an audio fingerprint that can uniquely identify an utterance despite a moderate amount of noise and channel distortion. Experimental results are obtained using a specially designed in-house database; the impact of various noise types and SNR levels is further investigated using a publicly available database. An analysis of playback scores across several combinations of telecommunication channel types, playback devices and additive noise demonstrates robustness of the feature set to channel distortion and additive noise, thus making it suitable for use in a copy-detection based PAD (cd-PAD) designed for applications such as telephone banking. Relative to other cd-PADs the proposed approach was better able to defend against playback attacks when telephone channels were involved. An analysis of its performance across the replay configurations used in the ASVspoof 2017 V2 evaluation set suggests that the proposed cd-PAD is highly effective in detecting those playback attacks that are most likely to spoof the speaker verification system.

컨볼루션 신경망 기반 모노 채널 음악-대사 음원 분리 기술을 이용한 방송물 배경 음악 식별

컨볼루션 신경망 기반 모노 채널 음악-대사 음원 분리 기술을 이용한 방송물 배경 음악 식별

SAMAF

Audio fingerprinting techniques were developed to index and retrieve audio samples by comparing a content-based compact signature of the audio instead of the entire audio sample, thereby reducing memory and computational expense. Different techniques have been applied to create audio fingerprints; however, with the introduction of deep learning, new data-driven unsupervised approaches are available. This article presents Sequence-to-Sequence Autoencoder Model for Audio Fingerprinting (SAMAF), which improved hash generation through a novel loss function composed of terms: Mean Square Error, minimizing the reconstruction error; Hash Loss, minimizing the distance between similar hashes and encouraging clustering; and Bitwise Entropy Loss, minimizing the variation inside the clusters. The performance of the model was assessed with a subset of VoxCeleb1 dataset, a“speech in-the-wild” dataset. Furthermore, the model was compared against three baselines: Dejavu, a Shazam-like algorithm; Robust Audio Fingerprinting System (RAFS), a Bit Error Rate (BER) methodology robust to time-frequency distortions and coding/decoding transformations; and Panako, a constellation-based algorithm adding time-frequency distortion resilience. Extensive empirical evidence showed that our approach outperformed all the baselines in the audio identification task and other classification tasks related to the attributes of the audio signal with an economical hash size of either 128 or 256 bits for one second of audio.

Audio Fingerprint based on Power Spectral Density and Hamming Distance Measure

Audio Fingerprint based on Power Spectral Density and Hamming Distance Measure