Audio Forensics Research Articles

Identification of AMR decompressed audio

More and more conversation recordings from phone calls are used as forensic evidence. To decide whether an unknown speech recording comes from mobile phone or not becomes an important issue in digital audio forensics. The communicating conversation recorded by mobile phones is encoded by Adaptive Multi-Rate (AMR) audio codec, which was adopted as the standard speech codec by 3GPP and widely used in GSM and UMTS. Therefore, AMR decompressed audio detection can be used to identify the source of the digital audio recording. Furthermore, it is helpful to locate the forgery position of the splicing AMR decompressed audio for forensic purposes. In this article, we focus on the identification of AMR decompressed audio, namely, given the waveform of an audio, we wish to identify whether it has been previously compressed by AMR codec or not. The artifacts introduced by the AMR codec will help to detect the source of the recordings. Based on our analysis, we find that the sample repetition rate of the AMR decompressed waveform is significantly greater than the regular waveform. Therefore, we employ the sample repetition rate as a feature to identify the AMR decompressed audio. The experimental results show that this feature is robust and effective.

Identifying Compression History of Wave Audio and Its Applications

Audio signal is sometimes stored and/or processed in WAV (waveform) format without any knowledge of its previous compression operations. To perform some subsequent processing, such as digital audio forensics, audio enhancement and blind audio quality assessment, it is necessary to identify its compression history. In this article, we will investigate how to identify a decompressed wave audio that went through one of three popular compression schemes, including MP3, WMA (windows media audio) and AAC (advanced audio coding). By analyzing the corresponding frequency coefficients, including modified discrete cosine transform (MDCT) and Mel-frequency cepstral coefficients (MFCCs), of those original audio clips and their decompressed versions with different compression schemes and bit rates, we propose several statistics to identify the compression scheme as well as the corresponding bit rate previously used for a given WAV signal. The experimental results evaluated on 8,800 audio clips with various contents have shown the effectiveness of the proposed method. In addition, some potential applications of the proposed method are discussed.

Identification of Electronic Disguised Voices

Since voice disguise has been an increasing tendency in illegal applications, and it has a great negative impact on establishing the authenticity of audio evidence for audio forensics, it is important to be able to identify whether a suspected voice has been disguised or not. However, few studies on such identification have been reported. In this paper, we propose an algorithm to identify electronic disguised voices. Since voice disguise, in essence, the modification of the frequency spectrum of speech signals, and mel-frequency cepstrum coefficients (MFCCs) can be used to well describe frequency spectral properties, MFCC-based features are supposed to be effective for the identification of disguised voices. In this paper, MFCC statistical moments including mean values and correlation coefficients are extracted as acoustic features. Then, an algorithm based on the extracted features and support vector machine classifiers is proposed to separate disguised voices from original voices. Extensive experiments show that the detection rates higher than 90% of the voices from various speech databases and disguised by various methods can be achieved, indicating that the identification performance of this algorithm is remarkable.

Audio Recording Location Identification Using Acoustic Environment Signature

An audio recording is subject to a number of possible distortions and artifacts. Consider, for example, artifacts due to acoustic reverberation and background noise. The acoustic reverberation depends on the shape and the composition of the room, and it causes temporal and spectral smearing of the recorded sound. The background noise, on the other hand, depends on the secondary audio source activities present in the evidentiary recording. Extraction of acoustic cues from an audio recording is an important but challenging task. Temporal changes in the estimated reverberation and background noise can be used for dynamic acoustic environment identification (AEI), audio forensics, and ballistic settings. We describe a statistical technique based on spectral subtraction to estimate the amount of reverberation and nonlinear filtering based on particle filtering to estimate the background noise. The effectiveness of the proposed method is tested using a data set consisting of speech recordings of two human speakers (one male and one female) made in eight acoustic environments using four commercial grade microphones. Performance of the proposed method is evaluated for various experimental settings such as microphone independent, semi- and full-blind AEI, and robustness to MP3 compression. Performance of the proposed framework is also evaluated using Temporal Derivative-based Spectrum and Mel-Cepstrum (TDSM)-based features. Experimental results show that the proposed method improves AEI performance compared with the direct method (i.e., feature vector is extracted from the audio recording directly). In addition, experimental results also show that the proposed scheme is robust to MP3 compression attack.

Acoustic Environment Identification and Its Applications to Audio Forensics

An audio recording is subject to a number of possible distortions and artifacts. Consider, for example, artifacts due to acoustic reverberation and background noise. The acoustic reverberation depends on the shape and the composition of a room, and it causes temporal and spectral smearing of the recorded sound. The background noise, on the other hand, depends on the secondary audio source activities present in the evidentiary recording. Extraction of acoustic cues from an audio recording is an important but challenging task. Temporal changes in the estimated reverberation and background noise can be used for dynamic acoustic environment identification (AEI), audio forensics, and ballistic settings. We describe a statistical technique to model and estimate the amount of reverberation and background noise variance in an audio recording. An energy-based voice activity detection method is proposed for automatic decaying-tail-selection from an audio recording. Effectiveness of the proposed method is tested using a data set consisting of speech recordings. The performance of the proposed method is also evaluated for both speaker-dependent and speaker-independent scenarios.

Acoustic echoes reveal room shape

Imagine that you are blindfolded inside an unknown room. You snap your fingers and listen to the room's response. Can you hear the shape of the room? Some people can do it naturally, but can we design computer algorithms that hear rooms? We show how to compute the shape of a convex polyhedral room from its response to a known sound, recorded by a few microphones. Geometric relationships between the arrival times of echoes enable us to "blindfoldedly" estimate the room geometry. This is achieved by exploiting the properties of Euclidean distance matrices. Furthermore, we show that under mild conditions, first-order echoes provide a unique description of convex polyhedral rooms. Our algorithm starts from the recorded impulse responses and proceeds by learning the correct assignment of echoes to walls. In contrast to earlier methods, the proposed algorithm reconstructs the full 3D geometry of the room from a single sound emission, and with an arbitrary geometry of the microphone array. As long as the microphones can hear the echoes, we can position them as we want. Besides answering a basic question about the inverse problem of room acoustics, our results find applications in areas such as architectural acoustics, indoor localization, virtual reality, and audio forensics.

Smart grid forensic science: applications, challenges, and open issues

Smart grid forensic science is a newly flourishing research area that is tightly coupled with cyber and physical security of the smart grid. Post-mortem analysis of a power system after a cyber attack or natural disaster generally provides the most accurate comprehension of the real-world vulnerabilities of the system and helps to protect the grid against similar attacks in the future as well as avoid failures during disasters. Besides increasing the security level of the smart grid, smart grid forensics aids evidence collection for the service of criminal justice. For instance, data extracted from smart meters and data collectors can provide evidence to legal proceedings in electricity theft matters. Furthermore, authentication and timestamping audio recordings using power grid frequency have been employed in several recent academic studies, as well as by the Metropolitan Police Forensic Audio Laboratory in London. Briefly, smart grid forensic science is emerging as a powerful security component of the power system. On the other hand, storage and processing of the enormous amount of data introduce significant challenges together with the privacy issue. In this article, we introduce the emerging application areas of smart grid forensic science, discuss the challenges, and outline the open issues in the topic. This article aims to serve as a roadmap for future smart grid forensic studies.

Digital signal processing in forensic acoustics cases

Forensic acoustics typically considers speaker identification, authenticity analysis, audio enhancement and transcript verification by analyzing and processing audio or speech signals. Other applications in forensic acoustics include different analyses of non-speech audio signals. This presentation will discuss three contentious litigation cases in which various digital processing techniques formed the foundation of scientifically-based conclusions. The first is a products liability case involving the death of a firefighter and his Personal Alert Safety System. When movement of a firefighter ceases, a high-pitched audible alert is emitted. When numbers of these devices were recorded, the task was to determine if the deceased firefighter’s device was included within the multiple devices examined. In the second case the order of the discharges of various caliber weapons based on a 9-1-1 call was determined via digital signal processing. The third study utilized processing techniques to differentiate and identify the type of sounds produced in a number of murder trials. These cases represent the varied challenges in forensic audio and acoustics investigations, and show the difficulty of applying broad theory to practice, the necessity for innovation in the field and the basis for establishing scientific reliability in order to secure admissibility into the courtroom.

Selection of speech/voice vectors in forensic voice identification

The problem of identifying speakers from voice analysis is a serious one. Many procedures have been proposed, some based on signal processing techniques common to automatic speech recognition. Yet it is clear that humans very often can make highly accurate identifications, even under challenging listening conditions that are common in forensic audio. A number of procedures have been developed which mimic human perception for this purpose: a semi-automatic forensic speaker recognition system using four sets of parameters, or vectors, based on a substantial number of related speech parameters. Identifications of 28 males in a field of 10 foil voices provided these data; the technique involved three (complete) replications of the approach. It was found that identification scores for the three of these vectors (voice quality, vowel formants, fundamental frequency) were very high and that for the temporal vector, positive but modest. Moreover, it also was found that every one of the vector-summation scores identified the target speaker. These results were based on high quality simulated field recordings, and demonstrate the efficacy of modeling biological systems (human perception) to solve challenging processing problems.

Further Considerations for the Analysis of ENF Data for Forensic Audio and Video Applications

The Electric Network Frequency (ENF) Criterion is becoming an established forensic analysis tool for determining the originating date and time of a recording and helping to authenticate digital audio and video evidence. To date, proposed comparison metrics for an automated approach have been based on variants of the mean squared error or the correlation coefficient. These algorithms are discussed and empirically compared in the context of ENF pattern matching. The operation of an electrical network under normal and adverse operating conditions is considered in relation to ENF behaviour, and the possible relationships between a network’s ENF dynamics and a networks identity are discussed. This article compliments one previous by the author relating to the theory and practice in the use of an automated approach to the ENF Criterion.

Environment Recognition for Digital Audio Forensics Using MPEG-7 and MEL Cepstral Features

Environment Recognition for Digital Audio Forensics Using MPEG-7 and MEL Cepstral FeaturesEnvironment recognition from digital audio for forensics application is a growing area of interest. However, compared to other branches of audio forensics, it is a less researched one. Especially less attention has been given to detect environment from files where foreground speech is present, which is a forensics scenario. In this paper, we perform several experiments focusing on the problems of environment recognition from audio particularly for forensics application. Experimental results show that the task is easier when audio files contain only environmental sound than when they contain both foreground speech and background environment. We propose a full set of MPEG-7 audio features combined with mel frequency cepstral coefficients (MFCCs) to improve the accuracy. In the experiments, the proposed approach significantly increases the recognition accuracy of environment sound even in the presence of high amount of foreground human speech.

Audio Environment Recognition using Zero Crossing Features and MPEG-7 Descriptors

Problem statement: This study investigated zero crossing features and selected MPEG-7 audio descriptors for environment sound recognition applications such as audio forensics. Approach: The study implemented several experiments focusing on the problems of environment recognition from audio particularly for forensic applications. Results: It was investigated the effect of the temporal zero crossing feature as well as selected MPEG-7 audio low level descriptors on environment sound recognition. The performance was evaluated against a varying number of training sounds and samples per training file. Conclusion/Recommendations: Experimental results showed that higher recognition accuracy is achieved by increasing the number of training files and by decreasing the number of samples per training file. This study presented an audio environment recognition using zero crossing features and MPEG-7 Descriptors.

Factors affecting the intelligibility of recorded speech: Considerations for forensic audio “best evidence”.

Derived from a traditional common law rule of evidence, the “best evidence” standard as applied to recorded audio prescribes that an original recording, and not a duplicated or altered copy, will be presented in legal proceedings. The intent of this standard is to ensure that the integrity of the original evidence is preserved, such that a court is reasonably assured that it is being presented with the most complete and accurate record of the evidence. However, when considering forensic audio recordings of speech, which are frequently made in adverse acoustic environments, presentation of such recordings in their original form may not afford a court with the opportunity for a complete and accurate assessment of the evidence in question—namely, what words are being spoken on the recording? The current paper summarizes the technological and listener-based factors that should be considered when speech intelligibility is of prime importance in meeting the best evidence standard for presentation of forensic audio in court proceedings. Illustrative examples from recent court cases will be provided.

Speech Watermarking: An Approach for the Forensic Analysis of Digital Telephonic Recordings*

In this article, the authors discuss the problem of forensic authentication of digital audio recordings. Although forensic audio has been addressed in several articles, the existing approaches are focused on analog magnetic recordings, which are less prevalent because of the large amount of digital recorders available on the market (optical, solid state, hard disks, etc.). An approach based on digital signal processing that consists of spread spectrum techniques for speech watermarking is presented. This approach presents the advantage that the authentication is based on the signal itself rather than the recording format. Thus, it is valid for usual recording devices in police-controlled telephone intercepts. In addition, our proposal allows for the introduction of relevant information such as the recording date and time and all the relevant data (this is not always possible with classical systems). Our experimental results reveal that the speech watermarking procedure does not interfere in a significant way with the posterior forensic speaker identification.

An automated approach to the Electric Network Frequency (ENF) criterion - Theory and practice

A relatively recent forensic technique developed to help establish the authenticity of audio and video recordings is the Electric Network Frequency (ENF) criterion. This paper confirms the applicability of the ENF criterion for use in mainland UK and describes an automated approach to matching ENF estimates taken from a questioned recording to a database of ENF values. It will be shown that the automated approach has the added advantage of producing statistical data capable of indicating the strength of the match. Consideration has been given to the robust extraction of ENF signals from evidential recordings and the resulting system and signal processing procedures outlined have been used successfully by the Metropolitan Police Forensic Audio Laboratory in London to extract and match ENF data from evidential recordings
 to an ENF database.

Study on the Selection of Specific Filters for Enhancement of Recorded Speech for Speaker Identification

Understanding the noise characteristics for finding appropriate filtering technique/s so as to obtain sufficiently clear speech samples for Speaker Identification, is one of the challenging tasks in Forensic Acoustics. Speaker's idiosyncratic speech should not be affected when the noise reduction is carried out; otherwise, Speaker Identification becomes highly erroneous. We have collected fifty noisy speech samples reported to be recorded in different modes from actual crime cases received in the laboratory. The samples are analyzed after subjecting to various filtering techniques and compared with the clear speech mixed with the noise collected from non-speech portion. Distortion levels on the speech are studied at various stages of application of filters in terms of SNR and Speaker Specific Information. Retaining the Speaker Specific Information as primary concern of our study, the limitation of filtering techniques depending on the characteristic and intensity level of noise is worked out for noisy speech samples. Subsequently a statistical study is also conducted. Listening tests were conducted to ensure that the perceptual features of the original noisy speech are preserved while applying filters. This work demonstrates the efficiency of Noise reduction filters in improving SNR and their controlled applications for preserving Speaker dependent features depending on the various noise characteristics embedded on speech samples. Audio Forensics has a challenging history of enhancement problems of speech samples received for examination. It is observed that out of the total speech samples received for Speaker Identification in the Laboratory, a large number of recordings requires enhancement. Speech is a non-linear time series represented in terms of complex number. Hence separating noise from noisy speech in spectral domain results into countless solutions. The main objective of a Noise Cancellation system is to obtain a clear signal with higher quality of speech signal. The presence of noise in speech signals can create higher degree of mismatch in performance of speech processing systems used for Speaker Identification as well as Speech Recognition. Inappropriate filtering of noise corresponds to extracting features of noise together with the actual speech signal during the feature extraction process. However, the desired parametric representation carries a high amount of error rate. The presence of broadband noise and a very low SNR deteriorate the intelligibility of most of the recorded speech samples. Speaker's idiosyncratic speech is affected when the noise reduction is carried out. Thus the Speaker Identification

Audio forensic examination

The field of audio forensics involves many topics familiar to the general audio digital signal processing (DSP) community, such as speech recognition, talker identification, and signal quality enhancement. There is potentially much to be gained by applying modern DSP theory to problems of interest to the forensics community, and this article is written to give the DSP audience some insight into the types of problems and challenges that face practitioners in audio forensic laboratories. However, this article must also present several of the frustrations and pitfalls encountered by signal processing experts when dealing with typical forensic material due to the standards and practices of the legal system.

GSM interference cancellation for forensic audio: a report on work in progress

GSM interference cancellation for forensic audio: a report on work in progress

Applications of ENF criterion in forensic audio, video, computer and telecommunication analysis

This article reports on the electric network frequency criterion as a means of assessing the integrity of digital audio/video evidence and forensic IT and telecommunication analysis. A brief description is given to different ENF types and phenomena that determine ENF variations. In most situations, to reach a non-authenticity opinion, the visual inspection of spectrograms and comparison with an ENF database are enough. A more detailed investigation, in the time domain, requires short time windows measurements and analyses. The stability of the ENF over geographical distances has been established by comparison of synchronized recordings made at different locations on the same network. Real cases are presented, in which the ENF criterion was used to investigate audio and video files created with secret surveillance systems, a digitized audio/video recording and a TV broadcasted reportage. By applying the ENF Criterion in forensic audio/video analysis, one can determine whether and where a digital recording has been edited, establish whether it was made at the time claimed, and identify the time and date of the registering operation.

GSM interference cancellation for forensic audio: a report on work in progress

A central aspect of forensic phonetic casework concerns the transcription of noisy recordings. An increasing problem in this area of work is the contamination of recordings with interference caused by radio transmissions from GSM mobile phones. Transmitting phones emit short duration radio-frequency pulses at a rate of 217 Hz. The induced interference signal contains the 217 Hz fundamental and a large number of harmonics that overlap the frequency range of speech, and therefore severely degrade speech intelligibility. Listener fatigue is increased due to the harsh sound of theinterference, and overall the transcription of such audio samples is problematic. This paper describes the progressing development of a filter to assist the forensic phonetician in carrying out the transcription of such contaminated recordings