Temporal Masking Research Articles

Circuit optimization techniques to mitigate the effects of soft errors in combinational logic

Soft errors in combinational logic circuits are emerging as a significant reliability problem for VLSI designs. Technology scaling trends indicate that the soft error rates (SER) of logic circuits will be dominant factor for future technology generations. SER mitigation in logic can be accomplished by optimizing either the gates inside a logic block or the flipflops present on the block boundaries. We present novel circuit optimization techniques that target these elements separately as well as in unison to reduce the SER of combinational logic circuits. First, we describe the construction of a new class of flip-flop variants that leverage the effect of temporal masking by selectively increasing the length of the latching window thereby preventing faulty transients from being registered. In contrast to previous flip-flop designs that rely on logic duplication and complicated circuit design styles, the new variants are redesigned from the library flip-flop using efficient transistor sizing. We then propose a flip-flop selection method that uses slack information at each primary output node to determine the flip-flop configuration that produces maximum SER savings. Next, we propose a gate sizing algorithm that trades off SER reduction and area overhead. This approach first computes bounds on the maximum achievable SER reduction by resizing a gate. This bound is then used to prune the circuit graph, arriving at a smaller set of candidate gates on which we perform incremental sensitivity computations to determine the gates that are the largest contributors to circuit SER. Third, we propose a unified, co-optimization approach combining flip-flop selection with the gate sizing algorithm. The joint optimization algorithm produces larger SER reductions while incurring smaller circuit overhead than either technique taken in isolation. Experimental results on a variety of benchmarks show average SER reductions of 10.7X with gate sizing, 5.7X with flip-flop assignment, and 30.1X for the combined optimization approach, with no delay penalties and area overheads within 5-6%. The runtimes for the optimization algorithms are on the order of 1-3 minutes.

Measures of hearing threshold and temporal processing across the adult lifespan

Psychophysical data on hearing sensitivity and various measures of supra-threshold auditory temporal processing are presented for large groups of young (18–35 y), middle-aged (40–55 y) and older (60–89 y) adults. Hearing thresholds were measured at 500, 1414 and 4000 Hz. Measures of temporal processing included gap-detection thresholds for bands of noise centered at 1000 and 3500 Hz, stimulus onset asynchronies for monaural and dichotic temporal-order identification for brief vowels, and stimulus onset/offset asynchronies for the monaural temporal masking of vowel identification. For all temporal-processing measures, the impact of high-frequency hearing loss in older adults was minimized by a combination of low-pass filtering the stimuli and use of high presentation levels. The performance of the older adults was worse than that of the young adults on all measures except gap-detection threshold at 1000 Hz. Middle-aged adults performed significantly worse than the young adults on measures of threshold sensitivity and three of the four measures of temporal-order identification, but not for any of the measures of temporal masking. Individual differences are also examined among a group of 124 older adults. Cognition and age were found to be significant predictors, although only 10–27% of the variance could be accounted for by these predictors.

Temporal whitening: Transient noise perceptually equalizes the 1/f temporal amplitude spectrum

Naturally occurring luminance distributions are approximately 1/f in their spatial and temporal amplitude spectra. By systematically varying the spatio-temporal profile of broadband noise stimuli, we demonstrate that humans invariably overestimate the proportion of high spatial and temporal frequency energy. Critically, we find that that the strength of this bias is of a magnitude that predicts a perceptually equalized response to the spatio-temporal fall off in the natural amplitude spectrum. This interpretation is supported by our finding that the magnitude of this transient response bias, while evident across a broad range of narrowband spatial frequencies (0.25-8 cycles/deg), decreases above 2 cycles/deg, which itself compensates for the increase in temporal frequency energy previously observed at high spatial frequencies as a consequence of small fixational eye movements (M. Rucci, R. Iovin, M. Poletti, & F. Santini, 2007). Additional temporal masking and adaptation experiments reveal a transiently biased asymmetry. Whereas temporal frequencies >4 Hz mask and adapt 1- and 15-Hz targets, lower masking and adaptation frequencies have much less effect on sensitivity to 15-Hz compared with 1-Hz targets. These results imply that the visual system over-represents its transient input to an extent that predicts an equalized temporal channel response to the low-frequency-biased structure of natural scenes.

Combining simultaneous with temporal masking.

Simultaneous and temporal masking are two frequently used techniques in psychology and vision science. Although there are many studies and theories related to each masking technique, there are no systematic investigations of their mutual relationship, even though both techniques are often applied together. Here, the authors show that temporal masking can both undo and enhance the deteriorating effects of simultaneous masking depending on the stimulus onset asynchrony between the simultaneous and temporal masks. For the task and stimuli used in this study, temporal masking was largely unaffected by the properties of the simultaneous mask. In contrast, simultaneous masking seems to depend strongly on spatial grouping and was strongly affected by the properties of the temporal mask. These findings help to identify the nature of both temporal and simultaneous masking and promote understanding of the role of spatial and temporal grouping in visual perception.

Otoacoustic Emission Theories and Behavioral Estimates of Human Basilar Membrane Motion Are Mutually Consistent

When two pure tones (or primaries) of slightly different frequencies (f (1) and f (2)) are presented to the ear, new frequency components are generated by nonlinear interaction of the primaries within the cochlea. These new components can be recorded in the ear canal as otoacoustic emissions (OAE). The level of the 2f (1)-f (2) OAE component is known as the distortion product otoacoustic emission (DPOAE) and is regarded as an indicator of the physiological state of the cochlea. The current view is that maximal level DPOAEs occur for primaries that produce equal excitation at the f (2) cochlear region, but this notion cannot be directly tested in living humans because it is impossible to record their cochlear responses while monitoring their ear canal DPOAE levels. On the other hand, it has been claimed that the temporal masking curve (TMC) method of inferring human basilar membrane responses allows measurement of the levels of equally effective pure tones at any given cochlear site. The assumptions of this behavioral method, however, lack firm physiological support in humans. Here, the TMC method was applied to test the current notion on the conditions that maximize DPOAE levels in humans. DPOAE and TMC results were mutually consistent for frequencies of 1 and 4 kHz and for levels below around 65 dB sound pressure level. This match supports the current view on the generation of maximal level DPOAEs as well as the assumptions of the behavioral TMC method.

Effects of envelope discontinuities on perceptual restoration of amplitude-compressed speech

An interrupted signal may be perceptually restored and, as a result, perceived as continuous, when the interruptions are filled with loud noise bursts. Additionally, when the signal is speech, an improvement in intelligibility may be observed. The perceived continuity of interrupted tones is reduced when the signal level is ramped down and up before and after the noise burst, respectively--an effect that has been attributed to envelope discontinuities at the tone-noise interface [Bregman, A. S., and Dannenbring, G. L. (1977). Can. J. Psychiatry 31, 151-159]. The hypothesis of the present study was that the perceptual restoration of speech would also be reduced with similar envelope discontinuities that may occur in real life due to the release time constants of hearing-aid compression. In an effort to make the conditions more relevant to hearing aids, speech was amplitude-compressed and normal-hearing listeners of varying ages were recruited. Envelope amplitude ramps were placed at the onsets/offsets of speech segments of interrupted sentences and the restoration effect was measured in two ways: objectively as the improvement in intelligibility when noise was added in the gaps and subjectively through the perceived continuity measured by subjects' own reporting. Both measures showed a reduction as the ramp duration increased--a trend observed for subjects of all ages and for all ramp configurations. These findings can be attributed to envelope discontinuities, with an additional contribution from reduced speech information due to ramping and temporal masking from loud noise bursts.

Visual Perception: Saccadic Omission — Suppression or Temporal Masking?

Although we don't perceive visual stimuli during saccadic eye movements, new evidence shows that our brains do process these stimuli and they can influence our subsequent visual perception.

Temporal structure of chromatic channels revealed through masking

The human color and luminance-driven systems appear to be mediated by mechanisms with distinct spatio-temporal tuning properties, with the iso-luminant color-driven system comparatively less sensitive to high rates of temporal modulation. While color and luminance defined speed and rate discrimination studies indicate perceptual access to the outputs of multiple, overlapping temporal frequency-selective mechanisms (channels), a detailed functional characterization of their response is currently restricted to the luminance-driven domain. Threshold elevations for 1- and 10-Hz targets were measured as a function of the temporal frequency of a spatio-temporally overlaid masking stimulus (1-30 Hz). Target and masking stimuli were iso-oriented gratings spatio-temporally modulated along each axis of human color space: L - M (red-green), S + S- (violet-yellow; S-cone isolating), and L + M + S (achromatic). Qualitatively equivalent patterns of temporal frequency-dependent masking were observed when measured within and, with one exception (red-green target, achromatic mask), between cardinal axes of human color space: fitted by one low-pass and one bandpass Gaussian function. That both low-pass and higher bandpass masking functions were evident between iso-luminant cardinal color axes (L - M and S + S-) suggests that both low and high temporal frequency masking may be cortically mediated.

Relationships between timing in reading and language ability and auditory temporal masking.

Broad timing difficulties may play a role in reading problems, but its role and a potential relationship with auditory temporal processing are unclear. The present study explored relationships between variables that might identify timing as a possible link between reading and auditory temporal masking. Tone identification thresholds were measured for 75 adults for backward masking and comodulation masking release (CMR) stimuli. Adults also completed a battery of nonverbal IQ, naming speed, motor speed, language, and reading measures. Backward masking stimuli consisted of a 20‐ms, 1‐kHz tone immediately preceding a 300‐ms, 600–1400 Hz bandpass noise masker. A notched‐noise condition included a spectral notch between 800 and 1200 Hz. CMR stimuli consisted of a reference condition in which a 1‐kHz tone was temporally centered in a 20‐Hz wide narrow‐band modulated noise masker alone and one with eight flanking bands of modulated noise in adjacent frequency regions present. CMR was measured as the improvement in threshold with flanking bands present relative to the single‐masker reference. Auditory stimuli were presented via headphones. Results indicated that nonverbal intelligence predicted performance on masking tasks, supporting previous research. Likewise, motor speed predicted performance for some of the masking tasks, suggesting that the underlying timing elements are not independent.

Pitfalls in behavioral estimates of basilar-membrane compression in humans.

Psychoacoustic estimates of basilar-membrane compression often compare on- and off-frequency forward masking. Such estimates involve assuming that the recovery from forward masking for a given signal frequency is independent of masker frequency. To test this assumption, thresholds for a brief 4-kHz signal were measured as a function of masker-signal delay. Comparisons were made between on-frequency (4 kHz) and off-frequency (either 2.4 or 4.4 kHz) maskers, adjusted in level to produce the same amount of masking at a 0-ms delay between masker offset and signal onset. Consistent with the assumption, forward-masking recovery from a moderate-level (83 dB SPL) 2.4-kHz masker and a high-level (92 dB SPL) 4.4-kHz masker was the same as from the equivalent on-frequency maskers. In contrast, recovery from a high-level (92 dB SPL) 2.4-kHz forward masker was slower than from the equivalent on-frequency masker. The results were used to simulate temporal masking curves, taking into account the differences in on- and off-frequency masking recoveries at high levels. The predictions suggest that compression estimates assuming frequency-independent masking recovery may overestimate compression by as much as a factor of 2. The results suggest caution in interpreting forward-masking data in terms of basilar-membrane compression, particularly when high-level maskers are involved.

Digital Video Watermarking in P-Frames With Controlled Video Bit-Rate Increase

Most video watermarking algorithms embed the watermark in I-frames, but refrain from embedding in P- and B-frames, which are highly compressed by motion compensation. However, P-frames appear more frequently in the compressed video and their watermarking capacity should be exploited, despite the fact that embedding the watermark in P-frames can increase the video bit rate significantly. This paper gives a detailed overview of a common approach for embedding the watermark in I-frames. This common approach is adopted to use P-frames for video watermarking. We show that by limiting the watermark to nonzero-quantized AC residuals in P-frames, the video bit-rate increase can be held to reasonable values. Since the nonzero-quantized AC residuals in P-frames correspond to nonflat areas that are in motion, temporal and texture masking are exploited at the same time. We also propose embedding the watermark in nonzero quantized AC residuals with spatial masking capacity in I-frames. Since the locations of the nonzero-quantized AC residuals is lost after decoding, we develop a watermark detection algorithm that does not depend on this knowledge. Our video watermark detection algorithm has controllable performance. We demonstrate the robustness of our proposed algorithm to several different attacks.

Temporal order discrimination of tonal sequences by younger and older adults: The role of duration and rate

The effect of tone duration and presentation rate on the discrimination of the temporal order of the middle two tones of a four-tone sequence was investigated in young normal-hearing (YNH) and older hearing-impaired (OHI) listeners. The frequencies and presentation level of the tone sequences were selected to minimize the effect of hearing loss on the performance of the OHI listeners. Tone durations varied from 20 to 400 ms and presentation rates from 2.5 to 25 toness. Two experiments were conducted with anisochronous (nonuniform duration and rate across entire sequence) and isochronous (uniform rate and duration) sequences, respectively. For the YNH listeners, performance for both isochronous and anisochronous sequences was determined primarily by presentation rate such that performance decreased at rates faster than 5 toness. For anisochronous tone sequences alone, the effects of rate were more pronounced at short tone durations. For the OHI listeners, both presentation rate and tone duration had an impact on performance for both isochronous and anisochronous sequences such that performance decreased as rate increased above 5 toness or duration decreased below 40 ms. Temporal masking was offered as an explanation for the interaction of short durations and fast rates on temporal order discrimination for the anisochronous sequences.

Estimates of compression at low and high frequencies using masking additivity in normal and impaired ears.

Auditory compression was estimated at 250 and 4000 Hz by using the additivity of forward masking technique, which measures the effects on signal threshold of combining two temporally nonoverlapping forward maskers. The increase in threshold in the combined-masker condition compared to the individual-masker conditions can be used to estimate compression. The signal was a 250 or 4000 Hz tone burst and the maskers (M1 and M2) were bands of noise. Signal thresholds were measured in the presence of M1 and M2 alone and combined for a range of masker levels. The results were used to derive response functions at each frequency. The procedure was conducted with normal-hearing and hearing-impaired listeners. The results suggest that the response function in normal ears is similar at 250 and 4000 Hz with a mid level compression exponent of about 0.2. However, compression extends over a smaller range of levels at 250 Hz. The results confirm previous estimates of compression using temporal masking curves (TMCs) without assuming a linear off-frequency reference as in the TMC procedure. The impaired ears generally showed less compression. Importantly, some impaired ears showed a linear response at 250 Hz, providing a further indication that low-frequency compression originates in the cochlea.

A reverberation-robust automatic speech recognition system based on temporal masking

Previously, we have proposed a reverberation-robust system for automatic speech recognition (ASR) based on a temporal masking principle. In the first pathway of this system, speech is analysed by a bank of auditory filters, in order to provide acoustic features for the recogniser. In the second pathway, a bandpass modulation filter (1.5 Hz - 8.2 Hz) detects regions of the envelope in each filter channel that contain strong speech energy. Regions of the modulation filter output that exceed a threshold are labelled as reliable evidence for the speech in a time-frequency mask; regions that fall below the threshold are dominated by reverberation and labelled as unreliable. The acoustic features and time-frequency mask are then decoded by a “missing data” ASR system. Here we describe modifications of this system that bring it into closer agreement with purported mechanisms of human perceptual compensation for reverberation, as determined by psychophysical studies [Watkins & Makin, JASA 121, 257-266]. Specifically, we use a metric for identifying regions of reliable speech that gives greater emphasis to reverberation “tails,” and is computed independently for each frequency band. The performance of the system is evaluated on a corpus of spoken digits convolved with recorded room impulse responses.

Additivity of auditory masking using Gaussian-shaped tones

Both temporal and spectral masking have been studied extensively in the literature. Mostly, they have been regarded as separate phenomena. Very little is known about the interaction between these two effects, i.e. masking in the time-frequency domain. Data on the time-frequency spread of masking evoked by a single Gaussian-shaped tone pulse are presented in an accompanying study at the same conference (Necciari et al.). The current study gathers data on the additivity of masking by up to four, approximately equally effective Gaussian maskers (ERB=600 Hz), separated either along the time or the frequency axis. For temporal separation, the amount of masking increases with the number of maskers, with excess masking (exceeding linear additivity) of up to 25 dB. For frequency separation (preliminary data) excess masking amounts up to 15 dB, and the higher-frequency masker (relative to the target) contributes more to the additivity than the lower-frequency maskers. Experiments with multiple maskers combining both temporal and frequency separation are underway. Combined with the single maker data, these data may serve as a basis for modeling time-frequency masking effects in complex signals. Work partly supported by OEAD (WTZ project AMADEUS) and ANR

Cochlear nonlinearity in normal-hearing subjects as inferred psychophysically and from distortion-product otoacoustic emissions

The aim was to investigate the correlation between estimates of compression ratio, compression threshold, and cochlear gain for normal-hearing subjects inferred from temporal masking curves (TMC) and distortion-product otoacoustic emission (DPOAEs) input-output (I/O) curves. Special care was given to reduce the influence of DPOAE fine structure on the I/O curves. Data were collected for 10 subjects and for cochlear places with characteristic frequencies (CFs) of 0.5, 1, 2, and 4 kHz. Except at 4 kHz, little correlation was found between ratio estimates obtained with the two methods. One reason is that the DPOAE I/O curves show plateaus or dips which result in unexpectedly high compression estimates. Moderately high correlation was found between the compression-threshold estimates obtained with the two methods, although DPOAE-based values were systematically lower than those based on the TMCs. Both methods confirmed that compression ratio and threshold are approximately constant across the CF range considered. Cochlear gain could be estimated from TMCs only and it was found to increase slightly with increasing CF. It is discussed that DPOAE I/O curves may be inappropriate to infer precise individual nonlinear cochlear characteristics in normal-hearing listeners, particularly at low CFs [Work supported by IMSERSO 131/06, PROFIT CIT-390000-2005-4, MEC BFU-2006-07536, and The Oticon Foundation.]

A psychophysical test of the Kummer primary-level rule for measuring distortion-product otoacoustic emission input/output functions

The standard procedure for measuring distortion-product otoacoustic emission (DPOAEs) input/output (I/O) curves involves using primaries whose levels conform to the rule of Kummer [Kummer et al., J. Acoust. Soc. Am. 103, 3431-44 (1998)]. The assumption is that the DP originates at the F2 cochlear site and primaries that conform to this rule produce equal excitation at that site, which maximizes the DP amplitude. These ideas were tested psychophysically by measuring temporal masking curves (TMCs) for masker frequencies equal to the primary frequencies. A TMC-based level rule was obtained by plotting the levels of the F1 masker against those for the F2 one for corresponding masker-probe intervals. DPOAE I/O curves were then measured using the Kummer rule, the TMC-based rule, and primary levels optimized individually to maximize DP amplitude. DPOAEs for the TMC-based and the Kummer rule had statistically similar amplitudes, but they were both lower than those measured with individually-optimized levels. This undermines the assumption that maximum DP responses occur for primaries that produce equal excitation at the F2 cochlear site. Individually-optimized levels showed that L1 should be higher than prescribed by the Kummer rule. [Work supported by IMSERSO-131/06, PROFIT-CIT-390000-2005-4, and MEC-BFU-2006-07536, and The Oticon Foundation.]

An efficient masking model for audio coding exploiting spectro-temporal masking

Perceptual audio coding achieves part of its coding efficiency by spectrally shaping the quantization noise such that it is masked by the audio signal to be encoded. In order to determine how much quantization noise is allowed within each frequency band and time interval a masking model is used to predict a masking curve specifying the maximally allowed quantization noise level within each frequency band. In most audio coders only spectral masking properties of the audio signal are used. The model by Dau et al. [J. Acoust. Soc. Am. 99, Vol. 3615, 1996] provides an interesting approach to also model temporal masking. Since this model operates as an artificial observer it only predicts whether the quantization noise is audible or not in the presence of the audio signal. In order to determine the most efficient quantization noise shape, the encoder needs to iteratively adapt the noise shape and evaluate each option with the model. This implies a highly computational complex encoding algorithm. In this contribution we will present an efficient masking model based on the Dau et al. model that only requires a single evaluation of the input signal to determine the maximally allowed noise level within each frequency band.

Cochlear nonlinearity in normal-hearing subjects as inferred psychophysically and from distortion-product otoacoustic emissions

The aim was to investigate the correlation between compression exponent, compression threshold, and cochlear gain for normal-hearing subjects as inferred from temporal masking curves (TMCs) and distortion-product otoacoustic emission (DPOAEs) input-output (I/O) curves. Care was given to reduce the influence of DPOAE fine structure on the DPOAE I/O curves. A high correlation between compression exponent estimates obtained with the two methods was found at 4 kHz but not at 0.5 and 1 kHz. One reason is that the DPOAE I/O curves show plateaus or notches that result in unexpectedly high compression estimates. Moderately high correlation was found between compression threshold estimates obtained with the two methods, although DPOAE-based values were around 7 dB lower than those based on TMCs. Both methods show that compression exponent and threshold are approximately constant across the frequency range from 0.5 to 4 kHz. Cochlear gain as estimated from TMCs was found to be approximately 16 dB greater at 4 than at 0.5 kHz. In conclusion, DPOAEs and TMCs may be used interchangeably to infer precise individual nonlinear cochlear characteristics at 4 kHz, but it remains unclear that the same applies to lower frequencies.

Wavelet filter bank based wide-band audio coder

New system for wide-band audio signals compression is presented. The system is based on a two stage wavelet filter bank principle. Special psychoacoustic model was designed for the system. Data compression is nowadays very topical issue. Due to the development of information technologies more sophisticated compression algorithms may be designed. Most of the present compression systems, e.g. MPEG, ATRAC or Vorbis, use a simple filter bank or DCT transformation. Both methods suffer from the origin of undesirable artefacts which are disturbing for the sound perception. Wavelet filter bank is used to suppress these artefacts and to increase data compression. Frequency dependant windowing function is used for improve of frequency resolution at low frequencies along with faster response at high frequencies. Wavelet transformation based coding algorithm can improve temporal masking effects over conventional perceptual coding algorithms for data compression.