Auditory Modality Research Articles

Discourse Analysis of Chinese Provincial and Regional Foreign Promotional Videos in the Framework of Fairclough’s Three-Dimensional Model

Chinese culture is a convergence of local cultures, and an in-depth understanding of local culture is the same as an understanding of Chinese culture. By constructing analysis framework, this paper not only analyzes text mode from the perspective of Halliday’s three meta-language functions, but also studies collaborative discourse propaganda effect of visual mode, auditory mode, and text mode from the perspective of semantic coherence and intertextuality at the level of discourse practice. This paper also expounds the performance of ideology and discourse power in propaganda film of Shanxi Province. The study has drawn the conclusion that regional propaganda films are mostly characterized by material and relational processes, third-person, and unmarked theme structure. Communication effect of multimodal synergy maximizes propaganda effect during text production and consumption. And regional propaganda films reflect the historical, cultural and political will and China’s determination to open up to the outside world at the same time.

Circadian Fluctuations of Cognitive and Psychomotor Performance in Medical Students: The Role of Daytime and Chronotype Patterns

Objective: Our research aimed to explore the impact of chronotype and time of day on the cognitive and psychomotor performance of medical students around 25 years old. The study provides evidence that an individual’s circadian rhythm affects their daily performance and cognitive processes.Methods: To measure selective attention and inhibition, we utilized the Stroop test, while three subtests from the fourth edition of Wechsler Adult Intelligence Scale (WAIS-IV) were employed to measure simple attention and working memory. In addition, the Leed psychomotor tester was utilized to measure perception and motor reaction time, and the morningness-eveningness questionnaire was administered to investigate students’ chronotype. Statistical analysis were based on many tests depending on the correlation types. We mainly used ANOVA repeated measures, Bonferroni post-hoc test, ANOVA I, chi-square, and Pearson correlation tests.Results: Our findings suggest that students exhibit decreased selective attention (F=128.842b, p<0.01, η<sup>2</sup>=0.868, 95% CI=2.744–16.998) and weakened inhibition during the late afternoon (6 p.m.) when the largest margin of error was committed (1.56±2.90) compared to 10 a.m. (0.82±1.72), leading to increased difficulty resisting interference; except that the size effect was small (η<sup>2</sup>>0.13). Simple attention reaches its peak in the morning around 10 a.m., while memory performance and auditory modality of working memory reach their peaks at 10 a.m. and the end of the afternoon (6 p.m.), respectively.Conclusion: Our study underscores the importance of taking wake time and body temperature into account when examining task performance. Overall, the study highlights the importance of considering circadian rhythm and chronotype when designing academic and athletic schedules.

Successful alpha neurofeedback training enhances working memory updating and event-related potential activity

Neurofeedback (NF) is a promising method to self-regulate human brain activity for cognition enhancement. Due to the unclear results of alpha NF training on working memory updating as well as the impact of feedback modality on NF learning, this study aimed to understand further the underlying neural mechanism of alpha NF training effects on working memory updating, where the NF learning was also compared between visual and auditory feedback modalities. A total of 30 participants were assigned to Visual NF, Auditory NF, and Control groups. Working memory updating was evaluated by n-back (n =2,3) tasks before and after five alpha upregulation NF sessions. The result showed no significant difference in NF learning performance between the Visual and Auditory groups, indicating that the difference in feedback modality did not affect NF learning. In addition, compared to the control group, the participants who achieved successful NF learning showed a significant increase in n-back behavioral performance and P3a amplitude in 2-back and a significant decrease in P3a latency in 3-back. Our results in n-back further suggested that successful alpha NF training might improve updating performance in terms of the behavioral and related event-related potential (ERP) measures. These findings contribute to the understanding of the effect of alpha training on memory updating and the design of NF experimental protocol in terms of feedback modality selection.

Rapid plasticity in the ventral visual stream elicited by a newly learnt auditory script in congenitally blind adults

Accumulating evidence in the last decades has given rise to a new theory of brain organization, positing that cortical regions are recruited for specific tasks irrespective of the sensory modality via which information is channeled. For instance, the visual reading network has been shown to be recruited for reading via the tactile Braille code in congenitally blind adults. Yet, how rapidly non-typical sensory input modulates activity in typically visual regions is yet to be explored. To this aim, we developed a novel reading orthography, termed OVAL, enabling congenitally blind adults to quickly acquire reading via the auditory modality. OVAL uses the EyeMusic, a visual-to-auditory sensory-substitution-device (SSD) to transform visually presented letters optimized for auditory transformation into sound. Using fMRI, we show modulation in the right ventral visual stream following 2-h of same-day training. Crucially, following more extensive training (i.e., ∼12 h) we show that OVAL reading recruits the left ventral visual stream including the location of the Visual Word Form Area, a key graphene-responsive region within the visual reading network. Our results show that while after 2 h of SSD training we can already observe the recruitment of the deprived ventral visual stream by auditory stimuli, computation-selective cross-modal recruitment requires longer training to establish.

A multimodal cortical network of sensory expectation violation revealed by fMRI.

The brain is subjected to multi-modal sensory information in an environment governed by statistical dependencies. Mismatch responses (MMRs), classically recorded with EEG, have provided valuable insights into the brain's processing of regularities and the generation of corresponding sensory predictions. Only few studies allow for comparisons of MMRs across multiple modalities in a simultaneous sensory stream and their corresponding cross-modal context sensitivity remains unknown. Here, we used a tri-modal version of the roving stimulus paradigm in fMRI to elicit MMRs in the auditory, somatosensory and visual modality. Participants (N = 29) were simultaneously presented with sequences of low and high intensity stimuli in each of the three senses while actively observing the tri-modal input stream and occasionally reporting the intensity of the previous stimulus in a prompted modality. The sequences were based on a probabilistic model, defining transition probabilities such that, for each modality, stimuli were more likely to repeat (p = .825) than change (p = .175) and stimulus intensities were equiprobable (p = .5). Moreover, each transition was conditional on the configuration of the other two modalities comprising global (cross-modal) predictive properties of the sequences. We identified a shared mismatch network of modality general inferior frontal and temporo-parietal areas as well as sensory areas, where the connectivity (psychophysiological interaction) between these regions was modulated during mismatch processing. Further, we found deviant responses within the network to be modulated by local stimulus repetition, which suggests highly comparable processing of expectation violation across modalities. Moreover, hierarchically higher regions of the mismatch network in the temporo-parietal area around the intraparietal sulcus were identified to signal cross-modal expectation violation. With the consistency of MMRs across audition, somatosensation and vision, our study provides insights into a shared cortical network of uni- and multi-modal expectation violation in response to sequence regularities.

AUDITORY TRAINING IN CHILDREN WITH HEARING LOSS: A CASE STUDY

verbal communication is studied primarily through the auditory modality. Adverse impact on language development, cognition, academics, including verbal communication when there is a decrease in hearing sensitivity. Hearing loss will result in a reduced ability to understand and produce intelligible speech. Objective: this study aims to provide initial evidence of changes in ability to identify speech sounds after intervention with auditory training. Method: The research method used is a single- subject experiment. Data were collected through interviews with the client’s parents, direct, observations of the client, test and a document study of the client’s medical record. Data analysis was carried out by comparing the ability to identify speech sounds pre- and post- therapy. Result: After 10 sessions of speech therapy using auditory training, the client's ability to detect sounds increased by 83.3%. Conclusion: providing auditory training to identify sounds is important for children with hearing loss. Suggestion: Further research is needed to obtain data and information about the effectiveness of using the Auditory Training method in children with hearing loss.

Young adults and multisensory time perception: Visual and auditory pathways in comparison.

The brain continuously encodes information about time, but how sensorial channels interact to achieve a stable representation of such ubiquitous information still needs to be determined. According to recent research, children show a potential interference in multisensory conditions, leading to a trade-off between two senses (sight and audition) when considering time-perception tasks. This study aimed to examine how healthy young adults behave when performing a time-perception task. InExperiment 1, we tested the effects of temporary sensory deprivation on both visual and auditory senses in a group of young adults. InExperiment2, we compared the temporal performances of young adults in the auditory modality with those of two samples of children (sighted and sighted but blindfolded) selected from a previous study. Statistically significant results emerged when comparing the two pathways: young adults overestimated and showed a higher sensitivity to time in the auditory modality compared to the visual modality. Restricting visual and auditory input did not affect their time sensitivity. Moreover, children were more accurate at estimating time than young adults after a transientvisual deprivation. This implies thatas we mature, sensory deprivation does not constitute a benefittotime perception, and supports the hypothesis of a calibration process between senses with age.However, more research is needed to determine how this calibration process affectsthe developmental trajectories of time perception.

Attentional focusing and filtering in multisensory categorization.

Selective attention refers to the ability to focus on goal-relevant information while filtering out irrelevant information. In a multisensory context, how do people selectively attend to multiple inputs when making categorical decisions? Here, we examined the role of selective attention in cross-modal categorization in two experiments. In a speed categorization task, participants were asked to attend to visual or auditory targets and categorize them while ignoring other irrelevant stimuli. A response-time extended multinomial processing tree (RT-MPT) model was implemented to estimate the contribution of attentional focusing on task-relevant information and attentional filtering of distractors. The results indicated that the role of selective attention was modality-specific, with differences found in attentional focusing and filtering between visual and auditory modalities. Visual information could be focused on or filtered out more effectively, whereas auditory information was more difficult to filter out, causing greater interference with task-relevant performance. The findings suggest that selective attention plays a critical and differential role across modalities, which provides a novel and promising approach to understanding multisensory processing and attentional focusing and filtering mechanisms of categorical decision-making.

Effects of Spatial Uncertainty on Tactile Vigilance

We examined effects of task demand on tactile vigilance by manipulating spatial uncertainty, which has been shown to affect visual vigilance. While holding a tactor in each hand, participants received a long or short (critical) vibration, and pressed a foot pedal when the critical stimulus occurred. In the “certain” condition, the spatial location of the stimulus alternated predictably across hands. In the “uncertain” condition, the stimulus was randomly assigned to either hand. We measured response time; proportion hits and false alarms; sensitivity and response bias; and stress and workload. Vigilance decrement was observed; sensitivity decreased and response time increased. Spatial uncertainty did not affect the occurrence or magnitude of the decrement. Results suggest that effects of cognitive demands are more similar between tactile and auditory modalities compared to the visual modality. Interface designers should be aware that spatial uncertainty likely has more detrimental effects on visual vigilance, compared to tactile vigilance.

Multimodal meaning-making of aggression in English song narrative: A cognitive-pragmatic perspective

This article addresses cognitive and pragmatic aspects of the multimodal meaning-making of AGGRESSION in the English song narrative. The cognitive-pragmatic foundations integrate cognitive and pragmatic aspects with multimodal ones, which allows to explain how signs of different modes and semiotic resources instantiate the concept and which cognitive features are decisive for (re)construction of meaning by performers and viewers. Cognitive features have verbal and non-verbal implementation through visual and auditory modes. The verbal semiotic resource, implemented through auditory and visual modes, contains lexical and pragmatic means of direct/indirect realization of aggression. Lexical means include words that name/describe/express aggression, as well as lexemes that acquire the aggressive meaning in a certain context. Pragmatic means are represented by expressive statements realizing threat, order, reproach, and accusation. The non-verbal semiotic resource is also represented through both modes and is characterized by semiotic heterogeneity, including prosodic/mimic/gestural features, various aspects of performers’ appearance, music clip design and lighting. They include hoarse voice/high exalted vocals/screaming; aggressive look/bulging eyes/wide open mouth/tense distorted face; active movements/fight; dark clothes/ dark make-up/tattoos. The musical component is characterized by the lower register, alternating consonance and dissonance, and a fast tempo. The lighting includes the play of light and shadow, and flashes. The construction of aggression appears as a result of the integration of semiotic elements into a multimodal blend, which is an emergent entity. Semantic relations between the blend components are based on the complementarity or contradiction of their elements, which makes it possible to single out convergent and divergent constructive patterns.

Modality-Specificity of the Neural Correlates of Linguistic and Non-Linguistic Demand.

Imaging studies of language processing in clinical populations can be complicated to interpret for several reasons, one being the difficulty of matching the effortfulness of processing across individuals or tasks. To better understand how effortful linguistic processing is reflected in functional activity, we investigated the neural correlates of task difficulty in linguistic and non-linguistic contexts in the auditory modality and then compared our findings to a recent analogous experiment in the visual modality in a different cohort. Nineteen neurologically normal individuals were scanned with fMRI as they performed a linguistic task (semantic matching) and a non-linguistic task (melodic matching), each with two levels of difficulty. We found that left hemisphere frontal and temporal language regions, as well as the right inferior frontal gyrus, were modulated by linguistic demand and not by non-linguistic demand. This was broadly similar to what was previously observed in the visual modality. In contrast, the multiple demand (MD) network, a set of brain regions thought to support cognitive flexibility in many contexts, was modulated neither by linguistic demand nor by non-linguistic demand in the auditory modality. This finding was in striking contradistinction to what was previously observed in the visual modality, where the MD network was robustly modulated by both linguistic and non-linguistic demand. Our findings suggest that while the language network is modulated by linguistic demand irrespective of modality, modulation of the MD network by linguistic demand is not inherent to linguistic processing, but rather depends on specific task factors.

Survey on multimodal approaches to emotion recognition

Emotion is an instinctive state of mind created by the neurophysiological changes occurring in the human body as reactions to various internal or external stimuli. Emotions play a vital role in decision-making. The choices one makes in day-to-day life determine their behaviour and thus their character. Emotion and behaviour recognition are the key processes in ascertaining Emotional Intelligence (EQ) which is the inherent human potential to understand and manage one’s own emotions in positive ways. But the process requires high expertise in the field of psychology and is exhaustive and time-consuming. This has opened a new horizon for exploring the computational recognition of EQ. Emotion Recognition (ER) is one of its sub-processes that identifies various human emotional states. Emotions are detected from physiological signals and also through non-invasive, vision-based algorithms by exploiting video and audio modalities. With the emergence of big data and state-of-art deep learning architectures combined with the vast availability of emotion-rich video content from various streaming platforms, Multimodal Emotion Recognition (MER) which detects emotions through multiple and complementary input modalities from video has gathered momentum in recent years. This survey paper elaborately discusses the unimodal ER through visual, auditory, and linguistic modalities and reviews MER with combined features from these modalities. It also discusses the joint representations and fusion mechanisms used to acquire the intermodal correlations. Finally, we put forward the limitations and gaps identified in the literature along with a few suggestions for future work.

Advantages and Difficulties of Psychological Consultations Online

This article presents a report by Tatiana Stepanova at the student scientific online conference Transactional Analysis in the 21st century on the topic Advantages and difficulties of psychological consultations online. The report is devoted to the analysis of the article by Julie Hay Working Online (2020). The author of the report considers the main advantages of a psychologists work online: cheapness for the consultant and for a client, convenience, more security of the psychologist, and others. The gains that the psychologist receives in the auditory and visual modalities are shown, as well as the difficulties that arise in the kinesthetic modality. Also, the paper pays attention to the difficulties that may arise during psychological counseling online. This concerns the difficulty of working with certain types of clients, possible problems in organizing the space from which the access to the Internet is carried out.

Preliminary evidence comparing three auditory attention network tests in people with aphasia and matched controls

ABSTRACT Background The Attention Network Test (ANT) was initially developed to measure three types of attention, alerting attention, orienting attention, and executive control attention, using a single cued-flanker task. Its use has since been extended into the auditory modality where an auditory-spatial ANT revealed that auditory alerting and orienting attention were impaired in people with aphasia (PWA). However, there are a few limitations that may preclude the auditory-spatial ANT from future use with PWA. Aims The purpose of this study was to compare the ability of three auditory ANTs to measure alerting attention, orienting attention, and executive control attention in PWA and matched controls, with the final goal being to identify one task to refine in future studies. We hypothesized that all three tasks would measure alerting attention, orienting attention, and executive control attention in the control group. We further expected group differences such that the aphasia group would benefit less from the alerting and orienting cues, and experience greater executive control costs than the control group. Methods & Procedures Seventeen PWA and 20 matched controls completed three auditory ANTs. The auditory ANTs used the same cues to measure alerting attention and orienting attention, however they differed on the tasks used to measure executive control attention. Two executive control tasks were pitch discrimination tasks: participants decided if the target word (Auditory Stroop ANT) or tone (Auditory Pitch ANT) was high or low in pitch. The third executive control task was a temporal discrimination task: participants decided if the target tone was short or long in duration (Auditory Duration ANT). Outcomes & Results All three auditory ANTs measured executive control attention. However, only the Auditory Pitch and Duration ANTs measured alerting attention and no task measured orienting attention. Group and task also interacted within executive control attention; the Auditory Duration ANT was the only task to meaningfully distinguish the aphasia group’s executive control attention from the control groups’. Conclusions Our results suggest that the Auditory Duration ANT may have the greatest clinical utility of the three auditory ANTs trialed. However, further work is needed to optimize the Auditory Duration ANT for clinical use.

Exploring behavioral adjustments of proportion congruency manipulations in an Eriksen flanker task with visual and auditory distractor modalities

The present study investigated global behavioral adaptation effects to conflict arising from different distractor modalities. Three experiments were conducted using an Eriksen flanker paradigm with constant visual targets, but randomly varying auditory or visual distractors. In Experiment 1, the proportion of congruent to incongruent trials was varied for both distractor modalities, whereas in Experiments 2A and 2B, this proportion congruency (PC) manipulation was applied to trials with one distractor modality (inducer) to test potential behavioral transfer effects to trials with the other distractor modality (diagnostic). In all experiments, mean proportion congruency effects (PCEs) were present in trials with a PC manipulation, but there was no evidence of transfer to diagnostic trials in Experiments 2A and 2B. Distributional analyses (delta plots) provided further evidence for distractor modality-specific global behavioral adaptations by showing differences in the slope of delta plots with visual but not auditory distractors when increasing the ratio of congruent trials. Thus, it is suggested that distractor modalities constrain global behavioral adaptation effects due to the learning of modality-specific memory traces (e.g., distractor–target associations) and/or the modality-specific cognitive control processes (e.g., suppression of modality-specific distractor-based activation). Moreover, additional analyses revealed partial transfer of the congruency sequence effect across trials with different distractor modalities suggesting that distractor modality may differentially affect local and global behavioral adaptations.

Sensorimotor foundations of speech perception in infancy

The perceptual system for speech is highly organized from early infancy. This organization bootstraps young human learners' ability to acquire their native speech and language from speech input. Here, we review behavioral and neuroimaging evidence that perceptual systems beyond the auditory modality are also specialized for speech in infancy, and that motor and sensorimotor systems can influence speech perception even in infants too young to produce speech-like vocalizations. These investigations complement existing literature on infant vocal development and on the interplay between speech perception and production systems in adults. We conclude that a multimodal speech and language network is present before speech-like vocalizations emerge.

To Inform or to Instruct? An Evaluation of Meaningful Vibrotactile Patterns to Support Automated Vehicle Takeover Performance

Automated vehicles may occasionally require drivers to take over. The complexity of the takeover process warrants the design of effective human–machine interfaces that assist drivers in regaining control, especially when the visual and auditory sensory modalities are occupied. Vibrotactile displays, which can represent information about the status, direction, and position of driving environment elements, have been suggested as one promising approach, but their effectiveness to aid in takeover transitions has not been fully evaluated. This study investigated the effects of meaningful tactile signal patterns, used as takeover requests, on automated vehicle takeover performance. Forty participants rode in a simulated SAE Level 3 automated vehicle and completed a series of takeover tasks with two tactile pattern formats, i.e., informative (which displayed status information of surrounding vehicles) and instructional (that displayed the appropriate takeover maneuver), and three in-vehicle locations (seat back, seat pan, and a seat back and seat pan combination). Takeover response options included lane changes only or brake applications followed by changing lanes, depending on the locations of surrounding vehicles. Results indicate that only meaningful instructional tactile signals, in either the seat back or seat pan, were associated with worse takeover response time and maximum resulting acceleration compared to signals without any patterns. Additionally, tactile information presented on the seat back was perceived as the most useful and satisfying. Findings from this study can inform the development of next-generation human–machine interfaces that utilize tactile stimulation in a wide range of environments with automation.

Audio-Corsi: a novel system to evaluate audio-spatial memory skills.

Spatial memory (SM) is a multimodal representation of the external world, which different sensory inputs can mediate. It is essential in accomplishing everyday activities and strongly correlates with sleep processes. However, despite valuable knowledge of the spatial mechanisms in the visual modality, the multi-sensory aspects of SM have yet to be thoroughly investigated due to a lack of proper technologies.This work presents a novel acoustic system built around 3D audio spatial technology. Our goal was to examine if an afternoon nap can improve memory performance, measured through the acoustic version of the Corsi Block Tapping Task (CBTT), named Audio-Corsi. We tested five adults over two days. During one of the two days (Wake), participants performed the Audio-Corsi before (Pre) and after (Post) a wake resting period; while the other day (Sleep), participants performed the Audio-Corsi before (Pre) and after (Post) a nap. Day orders were randomized. We calculated the memory span for the Pre and Post session in both the Wake and Sleep days. Preliminary results show a significant difference in the memory span between the Wake and Sleep days. Specifically, memory span decreased between the pre-and post-test during the wake day. The opposite trend was found for the sleep day. Results indicate that SM can be improved by sleeping also in the acoustic modality other than the visual one.Clinical Relevance- The technology and procedure we designed and developed could be suitable in clinical and experimental settings to study high-level cognitive skills in the auditory sensory modality and their relationship with sleep, especially when vision is absent or distorted (i.e. blindness).

STUDENT LEARNING STYLE OF MATHEMATICS EDUCATION BASED ON GENDER

This study aimed to identify and describe the learning modalities of students in mathematics education based on gender. This research is a type of quantitative research that uses a descriptive design. The research was conducted at the Mathematics Education Study Program, USN Kolaka. The population in this study were all students of the mathematics education class 2018, totalling 40 students. The sampling technique in this study used non-probability sampling in the form of saturated sampling. The instrument used in this study was a questionnaire containing statements used to obtain research information about visual (V), auditory (A), read/write (R), and kinesthetic (K) learning modalities, which are called VARK. The data analysis technique in this study used descriptive analysis based on the trend analysis of learning modalities. This trend data is then described in the form of tables and graphs. The results showed that the 2018 USN Kolaka Mathematics Education Study Program students' learning styles had a multimodal preference. Male students have a preference for multimodal learning style in bimodal (AR and AK) as 5 (41.66%) of 12 students. On the other hand, female students tend towards a multimodal learning style in bimodal (AR), as much as 6 (21.4%) of 28 students.

Phonological and morphological literacy skills in English and Chinese: A cross-linguistic neuroimaging comparison of Chinese-English bilingual and monolingual English children.

Over the course of literacy development, children learn to recognize word sounds and meanings in print. Yet, they do so differently across alphabetic and character-based orthographies such as English and Chinese. To uncover cross-linguistic influences on children's literacy, we asked young Chinese-English simultaneous bilinguals and English monolinguals (N = 119, ages 5-10) to complete phonological and morphological awareness (MA) literacy tasks. Children completed the tasks in the auditory modality in each of their languages during functional near-infrared spectroscopy neuroimaging. Cross-linguistically, comparisons between bilinguals' two languages revealed that the task that was more central to reading in a given orthography, such as phonological awareness (PA) in English and MA in Chinese, elicited less activation in the left inferior frontal and parietal regions. Group comparisons between bilinguals and monolinguals in English, their shared language of academic instruction, revealed that the left inferior frontal was less active during phonology but more active during morphology in bilinguals relative to monolinguals. MA skills are generally considered to have greater language specificity than PA skills. Bilingual literacy training in a skill that is maximally similar across languages, such as PA, may therefore yield greater automaticity for this skill, as reflected in the lower activation in bilinguals relative to monolinguals. This interpretation is supported by negative correlations between proficiency and brain activation. Together, these findings suggest that both the structural characteristics and literacy experiences with a given language can exert specific influences on bilingual and monolingual children's emerging brain networks for learning to read.