Speech Recognition Task Research Articles

Neuromorphic Time-Multiplexed Reservoir Computing With On-the-Fly Weight Generation for Edge Devices.

The human brain has evolved to perform complex and computationally expensive cognitive tasks, such as audio-visual perception and object detection, with ease. For instance, the brain can recognize speech in different dialects and perform other cognitive tasks, such as attention, memory, and motor control, with just 20 W of power consumption. Taking inspiration from neural systems, we propose a low-power neuromorphic hardware architecture to perform classification on temporal data at the edge. The proposed architecture uses a neuromorphic cochlea model for feature extraction and reservoir computing (RC) framework as a classifier. In the proposed hardware architecture, the RC framework is modified for on-the-fly generation of reservoir connectivity, along with binary feedforward and reservoir weights. Also, a large reservoir is split into multiple small reservoirs for efficient use of hardware resources. These modifications reduce the computational and memory resources required, thereby resulting in a lower power budget. The proposed classifier is validated for speech and human activity recognition (HAR) tasks. We have prototyped our hardware architecture using Intel's cyclone-10 low-power series field-programmable gate array (FPGA), consuming only 4790 logic elements (LEs) and 34.9-kB memory, making it a perfect candidate for edge computing applications. Moreover, we have implemented a complete system for speech recognition with the feature extraction block (cochlea model) and the proposed classifier, utilizing 15 532 LEs and 38.4-kB memory. By using the proposed idea of multiple small reservoirs along with on-the-fly generation of reservoir binary weights, our architecture can reduce the power consumption and memory requirement by order of magnitude compared to existing FPGA models for speech recognition tasks with similar complexity.

Listening Effort in Hearing Aid Users: Is It Related to Hearing Aid Use and Satisfaction?

Listening effort is primarily reflective of real-world performance. Therefore, it is crucial to evaluate the listening effort to predict the performance of hearing aid (HA) users in their daily lives. This study aimed to investigate the relationship between listening effort, daily HA use time, and HA satisfaction. This is a cross-sectional study. Thirty-three bilateral behind-the-ear HA users (17 females and 16 males) between 19 and 37 years were participated. All participants had bilateral, symmetric, moderate sensorineural hearing loss and at least 6 months of experience using HAs. The pure-tone average thresholds (PTA) of the participants' left and right ears were 55.34 ± 4.38 and 54.85 ± 5.05, respectively. First, daily HA use times of the last 30 days were derived from data logging. Second, participants were asked to fill in the Satisfaction with Amplification in Daily Life Scale questionnaire (SADL). Lastly, participants performed the dual-task paradigm to evaluate listening effort. The dual-task paradigm consisted of a primary speech recognition task that included three different individualized signal-to-noise ratio (SNR) conditions, that is, SNR100, SNR80, and SNR50, which the participant could understand 100, 80, and 50% of the speech, respectively. The secondary task was a visual reaction time task that required participants to press the key in response to a visual probe (an image of a white or red rectangle). Multiple linear regression analyses were used to model the effect of factors (daily HA use time and HA satisfaction) on reaction times (RT) of each three individualized SNR sessions. Mean daily HA use time of the participants was 5.72 ± 4.14 hours. Mean RTs of SNR50, SNR80, and SNR100 conditions were 1,050.61 ± 286.49, 893.33 ± 274.79, and 815.45 ± 233.22 ms, respectively. Multiple linear regression analyses showed that daily HA use time and HA satisfaction are significantly related to listening effort in all SNR conditions. For SNR80 condition; F (2,30) = 47.699, p < 0.001, with an adjusted R 2 of 0.745. As far as we know, this study is the first to demonstrate a strong link between listening effort, daily HA use time, and HA satisfaction. Evaluating listening effort following the HA fitting session may provide preliminary information about the treatment success of HA.

Analysis of Different Keying Modes and Brain Sound Image Establishment Mode in Piano Playing

The piano is known as the king of musical instruments for its rich expressiveness. Pianists get rich emotion with tone control. Touch key mode is the primary method of tone control. The purpose of this paper is to conduct research and analysis on different keying modes and brain-sound image establishment modes in piano performance. In this paper, we first propose a fuzzy mathematical proximity comparison method and an audio feature extraction method. Most of the audio features are derived from speech recognition tasks. They can reduce the original waveform sampling signal, thereby accelerating the machine’s understanding of the semantic meaning of audio. Using comparative analysis method, fuzzy mathematical proximity comparison method, and audio feature extraction as research methods, a model analysis research model based on different key touch methods in piano playing was established, and the relationship between piano key touch and brain sound image was studied. The experimental results of this paper show that after using the research method in this paper, the error rate of the data is controlled within 5%. Compared with previous research methods, the error rate is lower and has certain practical value.

Application of Speech Recognition Technology in Chinese English Simultaneous Interpretation of Law

Speech recognition is an important research field in natural language processing. In Chinese and English, which have rich data resources, the performance of end-to-end speech recognition model is close to that of Hidden Markov Model—Deep Neural Network (HMM-DNN) model. However, for the low resource speech recognition task of Chinese English hybrid, the end-to-end speech recognition system does not achieve good performance. In the case of limited mixed data between Chinese and English, the modeling method of end-to-end speech recognition is studied. This paper focuses on two end-to-end speech recognition models: connection timing distribution and attention based codec network. In order to improve the performance of Chinese English hybrid speech recognition, this paper studies how to improve the performance of the coder based on connection timing distribution model and attention mechanism, and tries to combine the two models to improve the performance of Chinese English hybrid speech recognition. In low resource Chinese English mixed data, the advantages of different models are used to improve the performance of end-to-end models, so as to improve the recognition accuracy of speech recognition technology in legal Chinese English simultaneous interpretation.

The Impact of Acute Tinnitus on Listening Effort: A Study Based on Clinical Observations of Sudden Sensorineural Hearing Loss Patients.

This study investigates the relationship between listening effort and acute tinnitus over the clinical course of sudden sensorineural hearing loss (SSNHL) before and after treatment. Thirty SSNHL patients with acute tinnitus were enrolled in this prospective study. Each patient was evaluated before treatment and after 1 and 3 months of follow-up. Listening effort was evaluated in the unaffected ears in two conditions (with and without background noise) using a dual-task paradigm, which included a primary (speech recognition) task and a secondary (visual reaction time) task. Tinnitus severity was assessed with the Tinnitus Handicap Inventory (THI). It was observed that background noise significantly increased listening effort in SSNHL patients with acute tinnitus before and after treatment. THI scores and listening effort in quiet conditions (** p = 0.009) were significantly decreased three months after treatment. In an analysis of the relation between tinnitus severity and listening effort, it was found that the THI total score was significantly correlated with listening effort in quiet (* p = 0.0388) and noisy conditions (* p = 0.044) before treatment. We concluded that SSNHL patients with acute tinnitus exerted greater listening effort in the presence of background noise than in quiet conditions. Furthermore, listening effort was reduced as tinnitus improved in SSNHL patients during the three months after treatment. Both before and after 3 months of treatment, patients who were more affected and emotionally distressed by tinnitus tended to exert more listening effort in both quiet and noisy environments.

Robust Multi-Scenario Speech-Based Emotion Recognition System.

Every human being experiences emotions daily, e.g., joy, sadness, fear, anger. These might be revealed through speech—words are often accompanied by our emotional states when we talk. Different acoustic emotional databases are freely available for solving the Emotional Speech Recognition (ESR) task. Unfortunately, many of them were generated under non-real-world conditions, i.e., actors played emotions, and recorded emotions were under fictitious circumstances where noise is non-existent. Another weakness in the design of emotion recognition systems is the scarcity of enough patterns in the available databases, causing generalization problems and leading to overfitting. This paper examines how different recording environmental elements impact system performance using a simple logistic regression algorithm. Specifically, we conducted experiments simulating different scenarios, using different levels of Gaussian white noise, real-world noise, and reverberation. The results from this research show a performance deterioration in all scenarios, increasing the error probability from 25.57% to 79.13% in the worst case. Additionally, a virtual enlargement method and a robust multi-scenario speech-based emotion recognition system are proposed. Our system’s average error probability of 34.57% is comparable to the best-case scenario with 31.55%. The findings support the prediction that simulated emotional speech databases do not offer sufficient closeness to real scenarios.

Evidence for cortical adjustments to perceptual decision criteria during word recognition in noise

Extensive increases in cingulo-opercular frontal activity are typically observed during speech recognition in noise tasks. This elevated activity has been linked to a word recognition benefit on the next trial, termed “adaptive control,” but how this effect might be implemented has been unclear. The established link between perceptual decision making and cingulo-opercular function may provide an explanation for how those regions benefit subsequent word recognition. In this case, processes that support recognition such as raising or lowering the decision criteria for more accurate or faster recognition may be adjusted to optimize performance on the next trial. The current neuroimaging study tested the hypothesis that pre-stimulus cingulo-opercular activity reflects criterion adjustments that determine how much information to collect for word recognition on subsequent trials. Participants included middle-age and older adults (N = 30; age = 58.3 ± 8.8 years; m ± sd) with normal hearing or mild sensorineural hearing loss. During a sparse fMRI experiment, words were presented in multitalker babble at +3 dB or +10 dB signal-to-noise ratio (SNR), which participants were instructed to repeat aloud. Word recognition was significantly poorer with increasing participant age and lower SNR compared to higher SNR conditions. A perceptual decision-making model was used to characterize processing differences based on task response latency distributions. The model showed that significantly less sensory evidence was collected (i.e., lower criteria) for lower compared to higher SNR trials. Replicating earlier observations, pre-stimulus cingulo-opercular activity was significantly predictive of correct recognition on a subsequent trial. Individual differences showed that participants with higher criteria also benefitted the most from pre-stimulus activity. Moreover, trial-level criteria changes were significantly linked to higher versus lower pre-stimulus activity. These results suggest cingulo-opercular cortex contributes to criteria adjustments to optimize speech recognition task performance.

The XMUSPEECH System for Accented English Automatic Speech Recognition

In this paper, we present the XMUSPEECH systems for Track 2 of the Interspeech 2020 Accented English Speech Recognition Challenge (AESRC2020). Track 2 is an Automatic Speech Recognition (ASR) task where the non-native English speakers have various accents, which reduces the accuracy of the ASR system. To solve this problem, we experimented with acoustic models and input features. Furthermore, we trained a TDNN-LSTM language model for lattice rescoring to obtain better results. Compared with our baseline system, we achieved relative word error rate (WER) improvements of 40.7% and 35.7% on the development set and evaluation set, respectively.

Detection and Recognition of Asynchronous Auditory/Visual Speech: Effects of Age, Hearing Loss, and Talker Accent.

This investigation examined age-related differences in auditory-visual (AV) integration as reflected on perceptual judgments of temporally misaligned AV English sentences spoken by native English and native Spanish talkers. In the detection task, it was expected that slowed auditory temporal processing of older participants, relative to younger participants, would be manifest as a shift in the range over which participants would judge asynchronous stimuli as synchronous (referred to as the “AV simultaneity window”). The older participants were also expected to exhibit greater declines in speech recognition for asynchronous AV stimuli than younger participants. Talker accent was hypothesized to influence listener performance, with older listeners exhibiting a greater narrowing of the AV simultaneity window and much poorer recognition of asynchronous AV foreign-accented speech compared to younger listeners. Participant groups included younger and older participants with normal hearing and older participants with hearing loss. Stimuli were video recordings of sentences produced by native English and native Spanish talkers. The video recordings were altered in 50 ms steps by delaying either the audio or video onset. Participants performed a detection task in which they judged whether the sentences were synchronous or asynchronous, and performed a recognition task for multiple synchronous and asynchronous conditions. Both the detection and recognition tasks were conducted at the individualized signal-to-noise ratio (SNR) corresponding to approximately 70% correct speech recognition performance for synchronous AV sentences. Older listeners with and without hearing loss generally showed wider AV simultaneity windows than younger listeners, possibly reflecting slowed auditory temporal processing in auditory lead conditions and reduced sensitivity to asynchrony in auditory lag conditions. However, older and younger listeners were affected similarly by misalignment of auditory and visual signal onsets on the speech recognition task. This suggests that older listeners are negatively impacted by temporal misalignments for speech recognition, even when they do not notice that the stimuli are asynchronous. Overall, the findings show that when listener performance is equated for simultaneous AV speech signals, age effects are apparent in detection judgments but not in recognition of asynchronous speech.

Taris: An online speech recognition framework with sequence to sequence neural networks for both audio-only and audio-visual speech

It is widely accepted that the visual modality of speech provides complementary information to the speech recognition task, and many models have been introduced in order to make good use of the visual channel. This article develops Taris, a fully differentiable neural network model capable of decoding both audio-only and audio-visual speech in real time. We achieve this by connecting our previously proposed models AV Align and Taris, which are both end-to-end differentiable approaches to audio-visual speech integration and online speech recognition respectively. We evaluate AV Taris under the same conditions as AV Align and Taris on one of the largest publicly available audio-visual speech datasets, LRS2. Our results show that AV Taris is superior to the audio-only variant of Taris, demonstrating the utility of the visual modality to speech recognition within the real time decoding framework defined by Taris. Compared to an equivalent Transformer-based AV Align model that takes advantage of full sentences without meeting the real-time requirement, we report an absolute degradation of approximately 3% with AV Taris. As opposed to the more popular alternative for online speech recognition, namely the RNN Transducer, Taris offers a greatly simplified fully differentiable training pipeline. We speculate that AV Taris has the potential to popularise the adoption of Audio-Visual Speech Recognition (AVSR) technology and overcome the inherent limitations of the audio modality in less optimal listening conditions.11Our code is publicly available at https://github.com/georgesterpu/Taris.

Mlphon: A Multifunctional Grapheme-Phoneme Conversion Tool Using Finite State Transducers

In this article we present the design and the development of a knowledge based computational linguistic tool, Mlphon [em.el.foːɳ] for Malayalam language. Mlphon computationally models linguistic rules using finite state transducers and performs multiple functions including grapheme to phoneme (g2p) and phoneme to grapheme (p2g) conversions, syllabification, phonetic feature analysis and script grammar check. This open source software tool, released under MIT license, is developed as a one-stop solution to handle different speech related text processing tasks for automatic speech recognition, text to speech synthesis and non-speech natural language processing tasks including syllable subword based language modeling, phoneme diversity analysis and text sanity check. The tool is evaluated on a manually crafted gold standard lexicon. Mlphon performs orthographic syllabification with 99% accuracy with a syllable error rate of 0.62% on the gold standard lexicon. For grapheme to phoneme conversion task, overall phoneme recognition accuracy of 99% with a phoneme error rate of 0.55% is obtained on gold standard lexicon. Additionally an extrinsic evaluation of Mlphon is performed by employing the pronunciation lexicon created using Mlphon, in Malayalam automatic speech recognition (ASR) task. Performance analysis in terms of the computation time of lexicon creation process and the word error rate (WER) on ASR task are presented along with a comparison over other automated tools for lexicon creation. Pronunciation lexicons with more than 100k commonly used Malayalam words in phonemised and syllabified forms is created and they are published as open language resources along with this work. We also demonstrate the usage of Mlphon on different natural language processing applications - syllable subword ASR, assisted pronunciation learning, phoneme diversity analysis and text sanity check. Being a knowledge based solution with open source code, Mlphon can be adapted to other languages of similar script nature.

Self-Supervised Pre-Training for Attention-Based Encoder-Decoder ASR Model

End-to-end (E2E) models, including the attention-based encoder-decoder (AED) models, have achieved promising performance on the automatic speech recognition (ASR) task. However, the supervised training process of the E2E model needs a large amount of speech-text paired data. In contrast, self-supervised pre-training can pre-train the model on the unlabeled data and then fine-tune it on the limited labeled data to realize better performance. Most of the previous self-supervised pre-training methods focus on learning hidden representations from speech but ignore how to utilize the unpaired text. As a result, previous works often pre-train an acoustic encoder and then fine-tune it as a classification based ASR model, such as Connectionist Temporal Classification (CTC) based model, rather than an AED model. In this paper, we propose a self-supervised pre-training method for the AED model (SP-AED). The SP-AED method contains acoustic pre-training for the encoder, linguistic pre-training for the decoder, and an adaptive combination fine-tuning for the whole system. We first design a linguistic pre-training method for decoder by utilizing the text-only data. The decoder will be pre-trained as a noise-condition language model to learn the prior distribution of the text. Then, we pre-train the AED encoder with the wav2vec2.0 method with some modifications. Finally, we combine the pre-trained encoder and decoder and fine-tune them on the limited labeled data. We design an adaptive combination method during fine-tuning by modifying the decoder's input and output to prevent catastrophic forgetting. Experiments prove that compared with the random initialized models, the SP-AED pre-trained models can realize up to 17% relative improvement. And with similar model size or computational cost, we can get comparable results to other classification-based models on both English and Chinese corpus.

Multitaper-Mel Spectrograms for Keyword Spotting

Keyword spotting (KWS) is one of the speech recognition tasks most sensitive to the quality of the feature representation. However, the research on KWS has traditionally focused on new model topologies, putting little emphasis on other aspects like feature extraction. This paper investigates the use of the multitaper technique to create improved features for KWS. The experimental study is carried out for different test scenarios, windows and parameters, datasets, and neural networks commonly used in embedded KWS applications. Experiment results confirm the advantages of using the proposed improved features.

An Innovative Approach Utilizing Binary-View Transformer for Speech Recognition Task

The deep learning advancements have greatly improved the performance of speech recognition systems, and most recent systems are based on the Recurrent Neural Network (RNN). Overall, the RNN works fine with the small sequence data, but suffers from the gradient vanishing problem in case of large sequence. The transformer networks have neutralized this issue and have shown state-of-the-art results on sequential or speech-related data. Generally, in speech recognition, the input audio is converted into an image using Mel-spectrogram to illustrate frequencies and intensities. The image is classified by the machine learning mechanism to generate a classification transcript. However, the audio frequency in the image has low resolution and causing inaccurate predictions. This paper presents a novel end-to-end binary view transformer-based architecture for speech recognition to cope with the frequency resolution problem. Firstly, the input audio signal is transformed into a 2D image using Mel-spectrogram. Secondly, the modified universal transformers utilize the multi-head attention to derive contextual information and derive different speech-related features. Moreover, a feed-forward neural network is also deployed for classification. The proposed system has generated robust results on Google's speech command dataset with an accuracy of 95.16% and with minimal loss. The binary-view transformer eradicates the eventuality of the over-fitting problem by deploying a multi-view mechanism to diversify the input data, and multi-head attention captures multiple contexts from the data's feature map.

Neural Architecture Search for LF-MMI Trained Time Delay Neural Networks

State-of-the-art automatic speech recognition (ASR) system development is data and computation intensive. The optimal design of deep neural networks (DNNs) for these systems often require expert knowledge and empirical evaluation. In this paper, a range of neural architecture search (NAS) techniques are used to automatically learn two types of hyper-parameters of factored time delay neural networks (TDNN-Fs): i) the left and right splicing context offsets; and ii) the dimensionality of the bottleneck linear projection at each hidden layer. These techniques include the differentiable neural architecture search (DARTS) method integrating architecture learning with lattice-free MMI training; Gumbel-Softmax and pipelined DARTS methods reducing the confusion over candidate architectures and improving the generalization of architecture selection; and Penalized DARTS incorporating resource constraints to balance the trade-off between performance and system complexity. Parameter sharing among TDNN-F architectures allows an efficient search over up to <inline-formula><tex-math notation="LaTeX">$7^{28}$</tex-math></inline-formula> different systems. Statistically significant word error rate (WER) reductions of up to 1.2&#x0025; absolute and relative model size reduction of 31&#x0025; were obtained over a state-of-the-art 300-hour Switchboard corpus trained baseline LF-MMI TDNN-F system featuring speed perturbation, i-Vector and learning hidden unit contribution (LHUC) based speaker adaptation as well as RNNLM rescoring. Performance contrasts on the same task against recent end-to-end systems reported in the literature suggest the best NAS auto-configured system achieves state-of-the-art WERs of 9.9&#x0025; and 11.1&#x0025; on the NIST Hub5&#x2019; 00 and Rt03 s test sets respectively with up to 96&#x0025; model size reduction. Further analysis using Bayesian learning shows that the proposed NAS approaches can effectively minimize the structural redundancy in the TDNN-F systems and reduce their model parameter uncertainty. Consistent performance improvements were also obtained on a UASpeech dysarthric speech recognition task.

Non-diacritized Arabic speech recognition based on CNN-LSTM and attention-based models

Arabic language has a set of sound letters called diacritics, these diacritics play an essential role in the meaning of words and their articulations. The change in some diacritics leads to a change in the context of the sentence. However, the existence of these letters in the corpus transcription affects the accuracy of speech recognition. In this paper, we investigate the effect of diactrics on the Arabic speech recognition based end-to-end deep learning. The applied end-to-end approach includes CNN-LSTM and attention-based technique presented in the state-of-the-art framework namely, Espresso using Pytorch. In addition, and to the best of our knowledge, the approach of CNN-LSTM with attention-based has not been used in the task of Arabic Automatic speech recognition (ASR). To fill this gap, this paper proposes a new approach based on CNN-LSTM with attention based method for Arabic ASR. The language model in this approach is trained using RNN-LM and LSTM-LM and based on nondiacritized transcription of the speech corpus. The Standard Arabic Single Speaker Corpus (SASSC), after omitting the diacritics, is used to train and test the deep learning model. Experimental results show that the removal of diacritics decreased out-of-vocabulary and perplexity of the language model. In addition, the word error rate (WER) is significantly improved when compared to diacritized data. The achieved average reduction in WER is 13.52%.

Listening Effort in Cochlear Implant Users: The Effect of Speech Intelligibility, Noise Reduction Processing, and Working Memory Capacity on the Pupil Dilation Response.

This study aimed to evaluate the effect of speech recognition performance, working memory capacity (WMC), and a noise reduction algorithm (NRA) on listening effort as measured with pupillometry in cochlear implant (CI) users while listening to speech in noise. Speech recognition and pupil responses (peak dilation, peak latency, and release of dilation) were measured during a speech recognition task at three speech-to-noise ratios (SNRs) with an NRA in both on and off conditions. WMC was measured with a reading span task. Twenty experienced CI users participated in this study. With increasing SNR and speech recognition performance, (a) the peak pupil dilation decreased by only a small amount, (b) the peak latency decreased, and (c) the release of dilation after the sentences increased. The NRA had no effect on speech recognition in noise or on the peak or latency values of the pupil response but caused less release of dilation after the end of the sentences. A lower reading span score was associated with higher peak pupil dilation but was not associated with peak latency, release of dilation, or speech recognition in noise. In CI users, speech perception is effortful, even at higher speech recognition scores and high SNRs, indicating that CI users are in a chronic state of increased effort in communication situations. The application of a clinically used NRA did not improve speech perception, nor did it reduce listening effort. Participants with a relatively low WMC exerted relatively more listening effort but did not have better speech reception thresholds in noise.

In‐Materio Reservoir Computing in a Sulfonated Polyaniline Network (Adv. Mater. 48/2021)

Reservoir Computing In materio reservoir computing is a direct way for mimicking spatiotemporal dynamics as found in animal brains. In article number 2102688, Hirofumi Tanaka, Wilfred G. van der Wiel, Takuya Matsumoto, and co-workers report a sulfonated polyaniline organic electrochemical random network device fabricated by simple dropcasting. They demonstrate in materio reservoir computing for a speech-recognition task, exploiting the rich time dynamics of the electrochemical system.

Bimodal variational autoencoder for audiovisual speech recognition

Multimodal fusion is the idea of combining information in a joint representation of multiple modalities. The goal of multimodal fusion is to improve the accuracy of results from classification or regression tasks. This paper proposes a Bimodal Variational Autoencoder (BiVAE) model for audiovisual features fusion. Reliance on audiovisual signals in a speech recognition task increases the recognition accuracy, especially when an audio signal is corrupted. The BiVAE model is trained and validated on the CUAVE dataset. Three classifiers have evaluated the fused audiovisual features: Long-short Term Memory, Deep Neural Network, and Support Vector Machine. The experiment involves the evaluation of the fused features in the case of whether two modalities are available or there is only one modality available (i.e., cross-modality). The experimental results display the superiority of the proposed model (BiVAE) of audiovisual features fusion over the state-of-the-art models by an average accuracy difference \(\simeq \) 3.28% and 13.28% for clean and noisy, respectively. Additionally, BiVAE outperforms the state-of-the-art models in the case of cross-modality by an accuracy difference \(\simeq \) 2.79% when the only audio signal is available and 1.88% when the only video signal is available. Furthermore, SVM satisfies the best recognition accuracy compared with other classifiers.

The effects of stimulus modality, task complexity, and cuing on working memory and the relationship with speech recognition in older cochlear implant users

IntroductionThe role of working memory (WM) in speech recognition of older cochlear implant (CI) users remains unclear. This study 1) examined the effects of aging and CI on WM performance across different modalities (auditory vs. visual) and cuing conditions, and 2) assessed how specific WM measures relate to sentence and word recognition in noise. MethodFourteen Older CI users, 12 Older acoustic-hearing (AH) listeners with age-appropriate hearing loss, and 15 Young normal-hearing (NH) listeners were tested. Participants completed two simple span tasks (auditory digit and visual letter span), two complex WM tasks (reading span and cued-modality WM with simultaneously presented auditory digits and visual letters), and two speech recognition tasks (sentence and word recognition in speech-babble noise). ResultsThe groups showed similar simple span performance, except that Older CI users had lower auditory digit span than Young NH listeners. Both older groups had similar reading span performance, but scored significantly lower than Young NH listeners, indicating age-related declines in attentional and phonological processing. A similar group effect was observed in the cued-modality WM task. All groups showed higher recall for auditory digits than for visual letters and the advantage was most evident without modality cuing. All groups displayed greater cuing benefits for visual recall than for auditory recall, suggesting that participants consistently allocated more attention to auditory stimuli regardless of cuing. For Older CI users, after controlling for the previously reported spectral resolution, auditory-uncued WM performance was significantly correlated with word recognition but not sentence recognition. ConclusionsComplex WM was significantly affected by aging but not by CI. Neither aging nor CI significantly affected modality cuing benefits in the WM task. For Older CI users, complex auditory WM with attentional control may better reflect the cognitive load of speech recognition in noise than simple span or complex visual WM.