Audio Forensics Research Articles

Exploring the Effectiveness of the Phase Features on Double Compressed AMR Speech Detection

Determining whether an audio signal is single compressed (SC) or double compressed (DC) is a crucial task in audio forensics, as it is closely linked to the integrity of the recording. In this paper, we propose the utilization of phase spectrum-based features for detecting DC narrowband and wideband adaptive multi-rate (AMR-NB and AMR-WB) speech. To the best of our knowledge, phase spectrum features have not been previously explored for DC audio detection. In addition to introducing phase spectrum features, we propose a novel parallel LSTM system that simultaneously learns the most representative features from both the magnitude and phase spectrum of the speech signal and integrates both sets of information to further enhance its performance. Analyses demonstrate significant differences between the phase spectra of SC and DC speech signals, suggesting their potential as representative features for DC AMR speech detection. The proposed phase spectrum features are found to perform as well as magnitude spectrum features for the AMR-NB codec, while outperforming the magnitude spectrum in detecting AMR-WB speech. The proposed phase spectrum features yield 8% performance improvement in terms of true positive rate over the magnitude spectrogram features. The proposed parallel LSTM system further improves DC AMR-WB speech detection.

1D-CNN-based audio tampering detection using ENF signals

The extensive adoption of digital audio recording has revolutionized its application in digital forensics, particularly in civil litigation and criminal prosecution. Electric network frequency (ENF) has emerged as a reliable technique in the field of audio forensics. However, the absence of comprehensive ENF reference datasets limits current ENF-based methods. To address this, this study introduces ATD, a blind audio forensics framework based on a one-dimensional convolutional neural network (1D-CNN) model. ATD can identify phase mutations and waveform discontinuities within the tampered ENF signal, without relying on an ENF reference database. To enhance feature extraction, the framework incorporates characteristics of the fundamental harmonics of ENF signals. In addition, a denoising method termed ENF noise reduction (ENR) based on the variational mode decomposition (VMD) and robust filtering algorithm (RFA) is proposed to reduce the impact of external noise on embedded electric network frequency signals. This study investigates three distinct types of audio tampering—deletion, insertion, and replacement—culminating in the design of binary-class tampering detection scenarios and four-class tampering detection scenarios tailored to these tampering types. ATD achieves a tampering detection accuracy of over 93% in the four-class scenario and exceeds 96% in the binary-class scenario. The effectiveness, efficiency, adaptability, and robustness of ATD in the two and four classification scenarios have been confirmed by extensive experiments.

Detecting Forged Audio Files Using "Mixed Paste" Command: A Deep Learning Approach Based on Korean Phonemic Features.

The ubiquity of smartphones today enables the widespread utilization of voice recording for diverse purposes. Consequently, the submission of voice recordings as digital evidence in legal proceedings has notably increased, alongside a rise in allegations of recording file forgery. This trend highlights the growing significance of audio file authentication. This study aims to develop a deep learning methodology capable of identifying forged files, particularly those altered using "Mixed Paste" commands, a technique not previously addressed. The proposed deep learning framework is a composite model, integrating a convolutional neural network and a long short-term memory model. It is designed based on the extraction of features from spectrograms and sequences of Korean consonant types. The training of this model utilizes an authentic dataset of forged audio recordings created on an iPhone, modified via "Mixed Paste", and encoded. This hybrid model demonstrates a high accuracy rate of 97.5%. To validate the model's efficacy, tests were conducted using various manipulated audio files. The findings reveal that the model's effectiveness is not contingent on the smartphone model or the audio editing software employed. We anticipate that this research will advance the field of audio forensics through a novel hybrid model approach.

Automatic speech recognition and the transcription of indistinct forensic audio: how do the new generation of systems fare?

This study provides an update on an earlier study in the “Capturing Talk” research topic, which aimed to demonstrate how automatic speech recognition (ASR) systems work with indistinct forensic-like audio, in comparison with good-quality audio. Since that time, there has been rapid technological advancement, with newer systems having access to extremely large language models and having their performance proclaimed as being human-like in accuracy. This study compares various ASR systems, including OpenAI’s Whisper, to continue to test how well automatic speaker recognition works with forensic-like audio. The results show that the transcription of a good-quality audio file is at ceiling for some systems, with no errors. For the poor-quality (forensic-like) audio, Whisper was the best performing system but had only 50% of the entire speech material correct. The results for the poor-quality audio were also generally variable across the systems, with differences depending on whether a .wav or .mp3 file was used and differences between earlier and later versions of the same system. Additionally, and against expectations, Whisper showed a drop in performance over a 2-month period. While more material was transcribed in the later attempt, more was also incorrect. This study concludes that forensic-like audio is not suitable for automatic analysis.

A novel Approach for Audio-based Video Analysis via MFCC Features

The video analysis is a momentous task of computer vision where larger videos are analyzed and converted to comparatively shorter summaries with key-frames satisfying a typical criterion. However, in the recent times audio-based video analysis is an important field where, audio features are used to categorize video frames into various classes. In this work, we present an audio-based video analysis model named as ABVS. The proposed approach employs 40 MFCC audio features from input audios to learn a multiclass classifier. The ABVS model is trained on ANN, SVM, KNN, DT and a majority voting ensemble to categorize a given video based on audio at high precision and low error rate. Our proposed approach is trained and validated on a benchmark urbansound8k dataset with an accuracy of 90% and loss of 0.10 on ANN and SVM model whereas, KNN achieves an accuracy of 89%. The ABVS model can be deployed in real-time applications to extract key-frames from the video scenes to troubleshoot the malfunctioning of critical infrastructure in industrial sectors. Our proposed model may be deployed in audio forensics for criminal voice investigation.

Anti Forensik Voice Note Menggunakan Whatsapp Mod

Currently there is a social media application that provides additional features in the form of voice changers by utilizing the voice note feature of Whatsapp Mod which can blur the identity of the voice owner. In several articles, Whatsapp Mod claims that this application is an anti-banned application, so it is feared that Whatsapp Mod is an application that uses anti-forensic techniques in it. This study aims to prove whether voice modifiers using voice notes on Whatsapp Mod can be a form of anti-forensic activity which can later make it difficult for investigators to conduct investigations. This will be proven by the use of a special method in handling audio forensics in which there are four stages, namely Acquisition, Enhancement, Decoding, and Voice Recognition. Audio forensics analysis that was carried out between voice recordings modified using Whatsapp Mod voice notes and original voice recordings showed results that were not identical. This proves that the voice modifier using the Whatsapp Mod voice note is considered an effective anti-forensic tool.

The Eastman transcripts: A case study calling Australian linguists to action against legal misconceptions about language in forensic evidence

ABSTRACT This paper presents a new case study examining the use of police transcripts to assist the court in understanding poor-quality forensic audio admitted as evidence in criminal trials. The 1995 trial it studies was the first major Australian case to include extensive expert opinions about police transcripts provided by the prosecution. Despite the fact that experts on both sides noted serious problems with the police transcripts, the judge allowed them to assist the jury, with the expert opinions offered as (ineffectual) subsequent commentary. The legal procedures the judge used in doing this were upheld on appeal, and have been followed ever since as a model for judges admitting opinions of both police and experts. The paper demonstrates how these procedures (unintentionally) privileged the opinions of police “ad hoc” experts over those of genuine experts, enabling the erroneous transcripts to influence not only the 1995 verdict, but a 1997 appeal and a 2014 inquiry. Analysis reveals the reason for these anomalies as the fact that the procedures incorporate misconceptions about spoken language and its representation in a transcript, which, though they have been thoroughly refuted by linguistic science over many decades, remain deeply embedded in the “common knowledge” accepted by wider society – including powerful institutions such as the law. The paper ends by calling on Australian linguists to find effective ways to address the misconceptions that affect the legal handling of forensic audio, by building further on the success of other branches of forensic linguistics in seeking direct engagement with the judiciary outside the trial process. The first step in achieving this is for linguists to gain a thorough understanding of how the legal procedures for handling poor-quality forensic audio operate, both in principle and in practice. The aim of the present paper is to contribute to that understanding.

Robust Audio Copy-Move Forgery Detection Using Constant Q Spectral Sketches and GA-SVM

Audio recordings used as evidence have become increasingly important to litigation. Before their admissibility as evidence, an audio forensic expert is often required to help determine whether the submitted audio recordings are altered or authentic. Within this field, the copy-move forgery detection (CMFD), which focuses on finding possible forgeries that are derived from the same audio recording, has been an urgent problem in blind audio forensics. However, most of the existing methods require idealistic pre-segmentation and artificial threshold selection to calculate the similarity between segments, which may result in serious misleading and misjudgment especially on high frequency words. In this work, we present a robust method for detecting and locating an audio copy-move forgery on the basis of constant Q spectral sketches (CQSS) and the integration of a customised genetic algorithm (GA) and support vector machine (SVM). Specifically, the CQSS features are first extracted by averaging the logarithm of the squared-magnitude constant Q transform. Then, the CQSS feature set is automatically optimised by a customised GA combined with SVM to obtain the best feature subset and classification model at the same time. Finally, the integrated method, named CQSS-GA-SVM, is evaluated against the state-of-the-art approaches to blind detection of copy-move forgeries on real-world copy-move datasets with read English and Chinese corpus, respectively. The experimental results demonstrate that the proposed CQSS-GA-SVM exhibits significantly high robustness against post-processing based anti-forensics attacks and adaptability to the changes of the duplicated segment duration, the training set size, the recording length, and the forgery type, which may be beneficial to improving the work efficiency of audio forensic experts.

Audio forensics behind the Iron Curtain: from raw sounds to expert testimony

ABSTRACT This essay investigates the construction of forensic audio expertise in the legal and security system of Communist Czechoslovakia and shows that the contested nature of speaker identification and sound-based objectivity contributed to the formulation of probabilistic claims in forensics. It explores the practices of the Department of Fonoscopy: a unique research laboratory of audio forensics that systematically examined the spectrographic, linguistic, and auditory means of sound analysis for the purpose of identifying unknown voices and environments in audio recordings. Bringing together the notions of “forensic cultures” and “sonic skills”, this article addresses the scientific, cultural, and political underpinning of the nascent field of audio expertise as well as the changing status of sound-based knowledge and forms of representation in forensics. In establishing fonoscopic expertise before the court and in the broader praxis of police investigation, the idea of vocal fingerprints and the use of sound visualisation technologies became instrumental. This essay pays special attention to the dynamics of the intricate process in which acoustic “raw material” (from anonymous calls, wiretapped phone lines, recorded conversations, or police interrogation rooms) was transformed into different kinds of legal and criminalistic evidence in the service of the totalitarian surveillance state.

Source Microphone Identification Using Swin Transformer

Microphone identification is a crucial challenge in the field of digital audio forensics. The ability to accurately identify the type of microphone used to record a piece of audio can provide important information for forensic analysis and crime investigations. In recent years, transformer-based deep-learning models have been shown to be effective in many different tasks. This paper proposes a system based on a transformer for microphone identification based on recorded audio. Two types of experiments were conducted: one to identify the model of the microphones and another in which identical microphones were identified within the same model. Furthermore, extensive experiments were performed to study the effects of different input types and sub-band frequencies on system accuracy. The proposed system is evaluated on the Audio Forensic Dataset for Digital Multimedia Forensics (AF-DB). The experimental results demonstrate that our model achieves state-of-the-art accuracy for inter-model and intra-model microphone classification with 5-fold cross-validation.

Robust audio copy-move forgery detection on short forged slices using sliding window

Audio copy-move forgery detection (CMFD), which aims at finding possible forgeries that are derived from the same audio recording, has been one of the most difficult challenges in blind audio forensics. However, the existing works rarely pay attention to the forgery of short slices within voiced segments, which may result in missed detection. In this work, we present a robust audio CMFD method on the basis of sliding window (SW) and constant Q cepstral coefficients (CQCC). Specifically, we first propose two SW strategies to illustrate how to reveal the possible copy-move forgeries within and between voiced segments. Then, we combine the CQCC feature with the Pearson correlation coefficients to measure the similarity between different short slices. Finally, we evaluate the proposed method, named SW-CQCC, against the state-of-the-art approaches to CMFD on real-world read English and Chinese corpora. Experimental results demonstrate that our SW-CQCC exhibits significantly high effectiveness and robustness in detecting short forged slices within voiced segments, which may be helpful to audio forensic examinations.

Source Acquisition Device Identification from Recorded Audio Based on Spatiotemporal Representation Learning with Multi-Attention Mechanisms

Source acquisition device identification from recorded audio aims to identify the source recording device by analyzing the intrinsic characteristics of audio, which is a challenging problem in audio forensics. In this paper, we propose a spatiotemporal representation learning framework with multi-attention mechanisms to tackle this problem. In the deep feature extraction stage of recording devices, a two-branch network based on residual dense temporal convolution networks (RD-TCNs) and convolutional neural networks (CNNs) is constructed. The spatial probability distribution features of audio signals are employed as inputs to the branch of the CNN for spatial representation learning, and the temporal spectral features of audio signals are fed into the branch of the RD-TCN network for temporal representation learning. This achieves simultaneous learning of long-term and short-term features to obtain an accurate representation of device-related information. In the spatiotemporal feature fusion stage, three attention mechanisms-temporal, spatial, and branch attention mechanisms-are designed to capture spatiotemporal weights and achieve effective deep feature fusion. The proposed framework achieves state-of-the-art performance on the benchmark CCNU_Mobile dataset, reaching an accuracy of 97.6% for the identification of 45 recording devices, with a significant reduction in training time compared to other models.

Transformer for authenticating the source microphone in digital audio forensics

In recent years, digital audio has played a significant role in providing information used as evidence in criminal investigations. The authenticity of an audio recording must be proven for it to be admitted into evidence in a legal proceeding. In this paper, we propose a robust system based on transformer deep learning architecture for identifying the source microphone with high accuracy, which is applicable in digital audio forensics. To evaluate the proposed model, we use the Audio Forensic Dataset for Digital Multimedia Forensics (AF-DB) and the King Saud University speech database (KSU-DB) datasets. Experiments show that the proposed model achieves state-of-the-art classification accuracy on the two databases. The model achieves an overall accuracy of 94.18% and 84.19% for Inter-model and Intra-model microphone classification, respectively, for the AF-DB and 99.38% for the KSU-DB speech dataset.

ANALYSIS OF THE IMPACT OF DISTORTION ON SOUND RECORDINGS AS ANTI FORENSIC ACTIVITIES

Anti-forensics on audio is aimed at complicating investigations on audio forensics, on sound recordings. Sound recordings can be altered or manipulated in various ways as well as the provision of distortion effects on sound recordings. Effect such distortions will make it difficult for investigators to find out the owner of the original voice. Analysis of distortion effects on sound recordings for anti-forensic activities, has not been widely carried out. Distortion can be an effective anti-forensic technique because the sound produced will be noisy, making it difficult for investigators to conduct investigations. In this study, testing was carried out using 3 types of distortion, namely Hard Clipping, Hard Overdrive and Odd Harmonics. To find out the extent to which the three types of distortions make it difficult to identify the owner of the original sound, the variables that affect each type of distortion are set at low, medium, and high levels. Formant values from the original and distorted sound samples were compared for later analysis using the Anova One-Way approach to show whether the original sound was identical and the other three voices were distorted. The test was carried out using 10 sound samples. From the results of the anova analysis, it is known that the types of Distortion of Hard Clipping and Odd Harmonics with variables at high levels can manipulate sound recordings, making it difficult to recognize the authenticity of a sound recording. Unlike the case with the type of Distortion of Hard Overdrive with variable level high – low and Hard Clipping and Odd Harmonics with variable level low – medium, it proves that sound recordings can still be identified.

A Universal Audio Steganalysis Scheme Based on Multiscale Spectrograms and DeepResNet

Given the popularity of audio and video applications, compressed audio has become an important carrier of covert communication on the Internet. Many novel compressed audio steganography schemes have emerged that offer good hiding capability and aural concealment. In this paper, a universal steganalysis scheme called MultiSpecNet is proposed to detect steganography based on multiple embedding domains (advanced audio coding (AAC) and MPEG-1 Audio Layer III (MP3)), which are currently the two most popular compressed audio standards. The basic idea is that modification of either domain by a steganography scheme will change the time-frequency relationship of the audio signal after decoding. The proposed approach adopts the spectrogram as the input feature to extract richer information. DeepResNet is used to learn the distinguishing feature representations, and multiscale spectrograms are used to enrich the feature diversity. The experimental results show that the proposed scheme is effective at detecting different steganography schemes based on the AAC and MP3 embedding domains. The detection accuracy of the proposed scheme is higher than that achieved by other state-of-the-art schemes. Using spectrograms as the input, DeepResNet achieves better performance than schemes using quantized modified discrete cosine transform (MDCT) coefficients and mel-spectrogram, although the quantized MDCT coefficient is the parameter modified by the steganography schemes directly and mel-spectrogram is very popular and effective for general audio signal analysis. To the best of our knowledge, this work is the first audio steganalysis scheme that can detect multiple steganography schemes in both the MP3 and AAC embedding domains. The method proposed in this paper can be extended to audio steganalysis for other codecs or for audio forensics purposes.

Analisis Rekaman Suara Magic Call pada Provider Seluler dengan Rekaman Suara Asli.

Abstract
 Magic call is a voice changer modification service from the provider that allows you to change the original voice when calling. This study was conducted to analyze the characteristics of the similarity level of magic call voice recordings on Telkomsel and XL providers with original voice recordings for forensic audio purposes. The voice recordings of magic call Telkomsel and XL use 4 voice recording samples consisting of 2 women and 2 men using their original voices. The data were analyzed using pitch, formant, and spectrogram analysis techniques. The data were then extracted using praat and gnumeric tools. The findings in this study showed that the four-voice recordings of 2 women and 2 men from magic call were compared with the original voice recordings showing non-identical results. This can be seen from the analysis results table which shows that 13 words from the sound recordings have very high non-identical values compared to identical values. It can be concluded that the magic call on Telkomsel and XL providers are not identical.
 
 Keywords— Forensic Audio, Sound Recording, Provider, Magic Call. 

Detection of audio copy-move-forgery with novel feature matching on Mel spectrogram

Audio copy-move-forgery created by copying one or more segments of an audio file and pasting it in a different position within the same audio is one of the most widely used methods in the field of audio forensics. This type of forgery is easy to apply but difficult to detect in the case of post-processing operations applied to forged speech to hide traces of forgeries. This paper proposes a robust method for the detection and localization of the audio copy-move forgery using a keypoint-based approach to the Mel spectrogram representation of audio. In the proposed method, first, the Mel spectrogram image is created from the input audio. Then, SIFT keypoints are obtained from each RGB color channel of this image. The obtained keypoints from each channel are matched via feature vectors to reveal the clues of the forgery regions, and the image sub-blocks whose keypoints are determined to be the center are labeled as forged blocks. Then the blocks in the neighborhood of the forged blocks are investigated whether forged or not. The proposed post-processing stage completes the determination of the forged regions. This stage eliminates the possible false positives and marks the forged areas in the spectrogram image. The forged segments are marked in the audio file by utilizing the positions of the forged regions in the spectrogram image. Experimental studies are carried out on two pitch-based datasets, using TIMIT and Arabic Speech Corpus. The paper presents the detailed performance results of popular referenced studies on these datasets. The performance results prove that the proposed method is more robust against common post-processing operations such as noise addition, filtering operation, and especially compression operation.

Other than Ethical: STS-Oriented Approaches to Communist Audio Forensics

This article considers the benefits of constructivist approaches to the history of audio forensics. It is argued that science and technology studies (STS) open up a new avenue of research on historical uses of sound recordings in the communist security apparatus and offer a perspective that is considerably different from the mainstream historiographical treatment of the state audio surveillance. This claim serves as a basis for discussing the Czechoslovak programme of audio forensics (1975–1989).

A Framework for Deciding How to Create and Evaluate Transcripts for Forensic and Other Purposes

Transcripts are used successfully in many areas of contemporary society. However, some uses of transcripts show systemic problems, with significant negative consequences. The key to finding effective solutions in these areas is to determine which factors contribute most strongly to the problems – which may be different from those to which they are commonly ascribed. This systematic review offers a conceptual framework for understanding the nature of transcripts in general, and the factors that contribute to a transcript's reliability and suitability for purpose. It then demonstrates how the framework can explain the (mostly) successful use of transcripts in two domains: court proceedings and linguistics research. Next, it uses the framework to examine two problematic cases: transcripts of forensic audio used as evidence in criminal trials, and transcripts of police interviews with suspects. A crucial observation is that, while it is common, and understandable, to focus on the transcriber as the source of problems with transcripts, transcription is actually a complex process involving practitioners in multiple roles, of which the transcriber role is not always the most important. Solving problems thus requires coordination of a range of factors. The analysis ends with practical suggestions for how to seek solutions for both the problematic areas reviewed, with attention to the role that linguistic science needs to play. The conclusion amplifies recent calls to consolidate transcription as a dedicated field of study within linguistics.

Does Automatic Speech Recognition (ASR) Have a Role in the Transcription of Indistinct Covert Recordings for Forensic Purposes?

The transcription of covert recordings used as evidence in court is a huge issue for forensic linguistics. Covert recordings are typically made under conditions in which the device needs to be hidden, and so the resulting speech is generally indistinct, with overlapping voices and background noise, and in many cases the acoustic record cannot be analyzed via conventional phonetic techniques (i.e. phonetic segments are unclear, or there are no cues at all present acoustically). In the case of indistinct audio, the resulting transcripts that are produced, often by police working on the case, are often questionable and despite their unreliable nature can be provided as evidence in court. Injustices can, and have, occurred. Given the growing performance of automatic speech recognition (ASR) technologies, and growing reliance on such technologies in everyday life, a common question asked, especially by lawyers and other legal professionals, is whether ASR can solve the problem of what was said in indistinct forensic audio, and this is the main focus of the current paper. The paper also looks at forced alignment, a way of automatically aligning an existing transcriptions to audio. This is an area that needs to be explored in the context of forensic linguistics because transcripts can technically be “aligned” with any audio, making it seem as if it is “correct” even if it is not. The aim of this research is to demonstrate how automatic transcription systems fare using forensic-like audio, and with more than one system. Forensic-like audio is most appropriate for research, because there is greater certainty with what the speech material consists of (unlike in forensic situations where it cannot be verified). Examples of how various ASR systems cope with indistinct audio are shown, highlighting that when a good-quality recording is used ASR systems cope well, with the resulting transcript being usable and, for the most part, accurate. When a poor-quality, forensic-like recording is used, on the other hand, the resulting transcript is effectively unusable, with numerous errors and very few words recognized (and in some cases, no words recognized). The paper also demonstrates some of the problems that arise when forced-alignment is used with indistinct forensic-like audio—the transcript is simply “forced” onto an audio signal giving completely wrong alignment. This research shows that the way things currently stand, computational methods are not suitable for solving the issue of transcription of indistinct forensic audio for a range of reasons. Such systems cannot transcribe what was said in indistinct covert recordings, nor can they determine who uttered the words and phrases in such recordings, nor prove that a transcript is “right” (or wrong). These systems can indeed be used advantageously in research, and for various other purposes, and the reasons they do not work for forensic transcription stems from the nature of the recording conditions, as well as the nature of the forensic context.