Nonverbal Recognition Research Articles

Multiparty conversation via multirobot system: incorporation of nonverbal user responses for continued conversation

Recent years have seen the advent of conversational humanoid robots. Implementing multiple robots is a promising approach for continuing human–robot conversations when a robot encounters speech-recognition errors. This strategy is effective for the user's verbal responses; however, people generally use nonverbal responses, such as nods or smiles, to respond to the interlocutor. In this study, we proposed a conversational strategy for twin robots in which the second robot recognized a human's nonverbal responses and interrupted the conversation by mentioning these responses. Moreover, we developed an interrogative dialogue system using simple nonverbal recognition modules. To verify the effectiveness of this conversational strategy, a subject-based experiment was conducted in which humans conversed with the two robots, and their impressions were evaluated through a questionnaire. We compared the outcomes under two conditions: whether the second robot's interruption was inserted or otherwise. Our results indicate that language-only ineffectiveness in responding to ambiguous responses regarding satisfaction and comprehension was mitigated. Thus, such a dialogue system is advantageous because it can facilitate robust conversation without relying solely on speech recognition.

Verbal and non-verbal recognition memory assessment: validation of a computerized version of the Recognition Memory Test.

The use of computerized devices for neuropsychological assessment (CNADs) as an effective alternative to the traditional pencil-and-paper modality has recently increased exponentially, both in clinical practice and research, especially due to the pandemic. However, several authors underline that the computerized modality requires the same psychometric validity as "in-presence" tests. The current study aimed at building and validating a computerized version of the verbal and non-verbal recognition memory test (RMT) for words, unknown faces and buildings. Seventy-two healthy Italian participants, with medium-high education and ability to proficiently use computerized systems, were enrolled. The sample was subdivided into six groups, one for each age decade. Twelve neurological patients with mixed aetiology, age and educational level were also recruited. Both the computerized and the paper-and-pencil versions of the RMT were administered in two separate sessions. In healthy participants, the computerized and the paper-and-pencil versions of the RMT showed statistical equivalence for words, unknown faces and buildings. In the neurological patients, no statistical difference was found between the performance at the two versions of the RMT. A moderate-to-good inter-rater reliability between the two versions was also found in both samples. Finally, the computerized version of the RMT was perceived as acceptable by both healthy participants and neurological patients at System Usability Scale (SUS). The computerized version of the RMT can be used as a reliable alternative to the traditional version.

Looking differently at locative events: the cognitive impact of linguistic preferences

Abstract While the Talmian dichotomy between satellite-framed and verb-framed languages has been amply studied for motion events, it has been less discussed for locative events, even if Talmy considers these to be included in motion events. This paper discusses such locative events, starting from the significant cross-linguistic variation among Dutch, French, and English. Dutch habitually encodes location via cardinal posture verbs (CPVs; ‘SIT’, ‘LIE’, ‘STAND’) expressing the orientation of the Figure, French prefers orientation-neutral existence verbs like être ‘be’ and English – unlike for motion events – straddles the middle with a marked preference for be but the possibility to occasionally rely on CPVs. Through the analysis of recognition performances and gazing behaviours in a non-verbal recognition task, this study confirms a (subtle) cognitive impact of different linguistic preferences on the mental representation of locative events. More specifically, they confirm the continuum suggested by Lemmens (2005, Parcours linguistiques. Domaine anglais (pp. 223–244). Publications de l’Université St Etienne.) for the domain of location with French on the one extreme and Dutch on the other with English in-between, behaving like French in some contexts but like Dutch in others.

Предикторы невербального опознания эмоциональных экспрессий лица

<p>The work was carried out in the paradigm of analyzing the structure of erroneous responses when performing the identification of emotional facial expressions. The subjects performed the task of non-verbal recognition of facial expressions at tachistoscopic exposure time (16 ms, 8 ms, 4 ms). The search for predictors that determine the partial proportions of responses for different combinations (response, expression shown) was performed among the subscales of the Boyko empathy questionnaire, the EmIn questionnaire, the analytical-holistic scale, and the Alekseev thinking styles questionnaire. The study involved students from Moscow universities. Sample size: 49 people, 9 men, 40 women, average age m=22.2 years, sd=4.45.The identified predictors are specific (not universal) in nature, different for different combinations of the presented image and the assessment performed. We believe that this result should be interpreted as indicating the presence of competing parallel processes associated with testing for the presence of different emotional expressions. The selected system of predictors is not complete. A promising direction for further research is expanding the scope of the search for predictors through methods that measure individual personality characteristics and the level of anxiety.</p>

Exploring The Influence of Virtual Reality on Interpersonal Communication: A Novel Study In Communication Science

This study aims to investigate the influence of virtual reality on interpersonal communication, focusing on aspects that have never been studied before in communication science. In an increasingly advanced digital age, virtual reality has become a phenomenon that attracts the attention of many people. However, there is still a lack of research that comprehensively reveals its impact on interpersonal communication. The research method used in this study is a combination of qualitative and quantitative approaches. Through the use of surveys and in-depth interviews, we obtain data that includes individual perceptions and experiences related to interpersonal communication in a virtual reality environment. Our respondents consisted of participants who were experienced in the use of virtual reality and those who had never used the technology before. Our results suggest that the use of virtual reality has the potential to influence various aspects of interpersonal communication, including emotional expression, nonverbal recognition, and levels of interpersonal closeness. These discoveries provide new insights into how virtual reality can enrich our communication experience in different contexts. This research makes an important contribution to the development of communication science, especially in understanding the role of new technologies such as virtual reality in shaping interpersonal interactions. The implications of this research could be used to develop practical guidelines for the use of virtual reality in interpersonal communication, as well as inform the design and development of better technologies in the future.

Which components of famous people recognition are lateralized? A study of face, voice and name recognition disorders in patients with neoplastic or degenerative damage of the right or left anterior temporal lobes

We administered to large groups of patients with neoplastic or degenerative damage affecting the right or left ATL, the ‘Famous People Recognition Battery’ (FPRB), in which subjects are required to recognize the same 40 famous people through their faces, voices and names, to clarify which components of famous people recognition are lateralized. At the familiarity level, we found, as expected, a dissociation between a greater impairment of patients with right ATL lesions on the non-verbal (face and voice) recognition modalities and of those with left ATL lesions on name familiarity. Equally expected were results obtained at the naming level, because the worse naming scores for faces and voices were observed in left-sided patients. Less foregone were, for two reasons, results obtained at the semantic level. First, no difference was found between the two hemispheric groups when scores obtained on the verbal (name) and non-verbal (face and voice) recognition modalities were account for. Second, the face and voice recognition modalities showed a different degree of right lateralization. All groups of patients showed, indeed, both at the familiarity and at the semantic level, a greater difficulty in the recognition of voices regarding faces, but this difference reached significance only in patients with right ATL lesions, suggesting a greater right lateralization of the more complex task of voice recognition. A model aiming to explain the greater right lateralization of the more perceptually demanding voice modality of person recognition is proposed.

A Study on the Application of Computer-Aided Dual-Coding Theory in English Vocabulary Teaching

Learning English needs a huge amount of vocabulary recalls as well as multiple grammatical structures and practices. With the development and improvement of the humanization and quality of English teaching, the multidimensional teaching of English has formed a powerful learning model and teaching method. The teaching of dual-coding theory assisted by computer teaching software mainly focuses on the dual activation representation of language combined with nonverbal graphic recognition to complete the recognition of information. According to the dual-coding hypothesis, in Gilford’s intelligence theory, the capacity of spatial and information recognition and reading is highly significant. With the large amount of visual recognition and information acquisition, images involve spatial domain information reading, which is a learning method based on the principle of information feature recognition. Based on the principle of dual-coding theory, it is a very effective way to present information and recall specific information in the form of vision and language. In this study, the application research for the purpose of English vocabulary teaching is adopted that has strong innovative and practical characteristics. According to the steps and contents of English vocabulary learning, the study is carried out step by step from the context in detail, so it continuously improves the learning quality and efficiency of English vocabulary teaching. Vocabulary teaching can help the students better improve their cognitive level of vocabulary and meet the vocabulary test requirements of the new institution entrance examination. Further, how to improve the effect of senior high school English vocabulary teaching is a major topic in the field of senior high school English teaching. In order to sort out this issue, I proposed a computer-aided dual-coding theory to help quickly understand vocabulary usage in order to increase learning efficiency. The Paivio–Desrochers model is used in this study. In addition, this study also attempts to explore the application of dual-coding theory in senior high school English vocabulary teaching.

Superior verbal but not nonverbal memory in congenital blindness.

Previous studies suggest that people who are congenitally blind outperform sighted people on some memory tasks. Whether blindness-associated memory advantages are specific to verbal materials or are also observed with nonverbal sounds has not been determined. Congenitally blind individuals (n = 20) and age and education matched blindfolded sighted controls (n = 22) performed a series of auditory memory tasks. These included: verbal forward and backward letter spans, a complex letter span with intervening equations, as well as two matched recognition tasks: one with verbal stimuli (i.e., letters) and one with nonverbal complex meaningless sounds. Replicating previously observed findings, blind participants outperformed sighted people on forward and backward letter span tasks. Blind participants also recalled more letters on the complex letter span task despite the interference of intervening equations. Critically, the same blind participants showed larger advantages on the verbal as compared to the nonverbal recognition task. These results suggest that blindness selectively enhances memory for verbal material. Possible explanations for blindness-related verbal memory advantages include blindness-induced memory practice and 'visual' cortex recruitment for verbal processing.

Transcranial Magnetic Stimulation Trains at 1 Hz Frequency of the Right Posterior Parietal Cortex Facilitate Recognition Memory

Neuroimaging, neuropsychological, and brain stimulation studies have led to contrasting findings regarding the potential roles of the lateral parietal lobe in episodic memory. Studies using brain stimulation methods reported in the literature do not offer unequivocal findings on the interactions with stimulation location (left vs. right hemisphere) or timing of the stimulation (encoding vs. retrieval). To address these issues, active and sham 1 Hz repetitive transcranial magnetic stimulation (rTMS) trains of 600 stimuli were applied over the right or left posterior parietal cortex (PPC) before the encoding or before the retrieval phase of a recognition memory task of unknown faces in a group of 40 healthy subjects. Active rTMS over the right but not the left PPC significantly improved non-verbal recognition memory performance without any significant modulation of speed of response when applied before the retrieval phase. In contrast, rTMS over the right or the left PPC before the encoding phase did not modulate memory performance. Our results support the hypothesis that the PPC plays a role in episodic memory retrieval that appears to be dependent on both the hemispheric lateralization and the timing of the stimulation (encoding vs. retrieval).

Prediction of Communication Effectiveness During Media Skills Training Using Commercial Automatic Non-verbal Recognition Systems.

It is well recognised that social signals play an important role in communication effectiveness. Observation of videos to understand non-verbal behaviour is time-consuming and limits the potential to incorporate detailed and accurate feedback of this behaviour in practical applications such as communication skills training or performance evaluation. The aim of the current research is twofold: (1) to investigate whether off-the-shelf emotion recognition technology can detect social signals in media interviews and (2) to identify which combinations of social signals are most promising for evaluating trainees’ performance in a media interview. To investigate this, non-verbal signals were automatically recognised from practice on-camera media interviews conducted within a media training setting with a sample size of 34. Automated non-verbal signal detection consists of multimodal features including facial expression, hand gestures, vocal behaviour and ‘honest’ signals. The on-camera interviews were categorised into effective and poor communication exemplars based on communication skills ratings provided by trainers and neutral observers which served as a ground truth. A correlation-based feature selection method was used to select signals associated with performance. To assess the accuracy of the selected features, a number of machine learning classification techniques were used. Naive Bayes analysis produced the best results with an F-measure of 0.76 and prediction accuracy of 78%. Results revealed that a combination of body movements, hand movements and facial expression are relevant for establishing communication effectiveness in the context of media interviews. The results of the current study have implications for the automatic evaluation of media interviews with a number of potential application areas including enhancing communication training including current media skills training.

Cognitive Functions in Adult-Onset Phenotypes of X-Linked Adrenoleukodystrophy.

X-linked adrenoleukodystrophy (X-ALD) is a rare genetic disorder characterized by progressive demyelination ranging from mild myelopathic forms (adrenomyeloneuropathy [AMN]) to severe cerebral variants (adult cerebral adrenoleukodystrophy [ACALD]). The aim of this study was to compare cognitive function in adult-onset X-ALD phenotypes. Cognitive function in various domains (intelligence, attention, memory, executive function, and processing speed) was assessed in 172 adults (117 with AMN, 30 with arrested ACALD, and 25 with acute ACALD) using comprehensive neuropsychological batteries. Phenotype differences were examined by analyses of variance. X-ALD phenotypes significantly differed in nonverbal intelligence, sustained attention, verbal encoding, nonverbal recognition, and processing speed (ps < 0.050). No group differences emerged regarding verbal intelligence, verbal retrieval and recognition, and executive function (ps > 0.050). Specifically, patients with acute ACALD showed severe cognitive deficits compared to AMN and normal data, with largest effects on processing speed. Contrary, cognition was overall intact in patients with AMN, independent of sex and corticospinal tract involvement, and those with arrested ACALD showed mild cognitive dysfunction, particularly in verbal encoding and processing speed. Cerebral demyelination in patients with X-ALD causes white matter dementia, mainly characterized by an extreme slowdown in processing speed associated with deficits in attention and learning. Most patients with AMN show intact cognitive function. Future prospective, longitudinal studies with more sensitive imaging techniques are required to clarify whether early mild cognitive dysfunction found in some patients with AMN may be associated with subtle myelin abnormalities that do not yet appear as white matter lesions on cerebral MRI (cMRI) but have the potential to serve as early predictors of later cerebral progression. ANN NEUROL 2021;90:266-273.

Association of visual motor processing and social cognition in schizophrenia

Patients with schizophrenia have difficulties in social cognitive domains including emotion recognition and mentalization, and in sensorimotor processing and learning. The relationship between social cognitive deficits and sensorimotor function in patients with schizophrenia remains largely unexplored. With the hypothesis that impaired visual motor processing may decelerate information processing and subsequently affects various domains of social cognition, we examined the association of nonverbal emotion recognition, mentalization, and visual motor processing in schizophrenia. The study examined mentalization using the verbal subset of the Chinese version of Theory of Mind (CToM) Task, an equivalent task of the Faux Pas Test; emotion recognition using the Diagnostic Analysis of Nonverbal Accuracy 2-Taiwan version (DANVA-2-TW), and visual motor processing using a joystick tracking task controlled for basic motor function in 34 individuals with chronic schizophrenia in the community and 42 healthy controls. Patients with schizophrenia had significantly worse performance than healthy controls in social cognition, including facial, prosodic emotion recognition, and mentalization. Visual motor processing was also significantly worse in patients with schizophrenia. Only in patients with schizophrenia, both emotion recognition (mainly in prosodic modality, happy, and sad emotions) and mentalization were positively associated with their learning capacity of visual motor processing. These findings suggest a prospective role of sensorimotor function in their social cognitive deficits. Despite that the underlying neural mechanism needs further research, our findings may provide a new direction for restoration of social cognitive function in schizophrenia by enhancing visual motor processing ability.

Dyslexia &amp; the Pons

Introduction Dyslexia is a learning disorder. It is centered on the Pons. It affects between 5 to 17% of the population. The Pons is where the balance takes place. People with Dyslexia have trouble with balance. The Pons is also central to the sub consciousness, which is tied in with REM sleep. The sleep equation therefore applies. The Pons is involved in communication by facial expression. People with Dyslexia is a communication disorder since those with it have trouble linking visual and hearing senses. The Cranial Nerve VIII, Vestibular Nerve, responsible for hearing comes out of the Pons. The tracts for sight and hearing pass through the Lateral Geniculate Body (LGB) and the Medical Geniculate Body. (MGB) which are located on the anterior of the Thalamus. Reading and wringing, which is a problem with Dyslexia, involves both seeing and hearing. We translate visual signal to vocal words and writing. Dyslexia also have trouble with writing letters. They do letter backwards. The is an orientation problem. It may have to do with the left hemisphere which usually is responsible for language (Broca’s and Wernicke’s Areas). The Pons too is responsible for communication, especially the nonverbal facial recognition.

Errors in identifying emotion in body postures and facial expressions among pediatric patients with bipolar disorder

ABSTRACT Introduction Facial emotion processing impairments have been consistently demonstrated among patients with pediatric bipolar disorder (PBD). However, less is known about other domains of nonverbal emotion recognition in this group. Therefore, the aim of the current study was to investigate emotion identification for body postures in addition to facial expressions among PBD patients and matched healthy comparison (HC) subjects. Method Youth with PBD (n = 25) and HC subjects (n = 25), equal on intellectual and demographic variables, completed the body postures and faces subtests of the Diagnostic Analysis of Nonverbal Accuracy-2 scale (DANVA-2) along with diagnostic and clinical assessments. Results Compared to HC, PBD patients made significantly more errors identifying emotion in both body postures and faces. Poorer performance on the faces subtest was associated with more problematic peer functioning. Greater manic symptoms were associated with poorer performance on negative emotional stimuli but not positive. Conclusions In addition to impairments in facial emotion processing, youth with PBD also show impairments in the ability to correctly identify emotion from body postures. In this group, greater manic symptomatology appears to impair processing for negative facial expressions and body postures, but not positive ones. Interestingly, facial emotion processing impairments appear to be a better indicator of real-world psychosocial difficulties among PBD patients compared to body postures. Psychosocial treatments for youth with PBD should focus on improving the ability to accurately identify and respond to various facial and body postural cues, as well as symptom management and emotion regulation strategies.

"Motion or Emotion? Recognition of Emotional Bodily Expressions in Children With Autism Spectrum Disorder With and Without Intellectual Disability".

The recognition of emotional body movement (BM) is impaired in individuals with Autistic Spectrum Disorder ASD, yet it is not clear whether the difficulty is related to the encoding of body motion, emotions, or both. Besides, BM recognition has been traditionally studied using point-light displays stimuli (PLDs) and is still underexplored in individuals with ASD and intellectual disability (ID). In the present study, we investigated the recognition of happy, fearful, and neutral BM in children with ASD with and without ID. In a non-verbal recognition task, participants were asked to recognize pure-body-motion and visible-body-form stimuli (by means of point-light displays-PLDs and full-light displays-FLDs, respectively). We found that the children with ASD were less accurate than TD children in recognizing both the emotional and neutral BM, either when presented as FLDs or PLDs. These results suggest that the difficulty in understanding the observed BM may rely on atypical processing of BM information rather than emotion. Moreover, we found that the accuracy improved with age and IQ only in children with ASD without ID, suggesting that high level of cognitive resources can mediate the acquisition of compensatory mechanisms which develop with age.

Low-Frequency Repetitive Transcranial Magnetic Stimulation of the Right Dorsolateral Prefrontal Cortex Enhances Recognition Memory in Alzheimer's Disease.

The lack of effective pharmacological or behavioral interventions for memory impairments associated with Alzheimer's disease (AD) emphasizes the need for the investigation of approaches based on neuromodulation. This study examined the effects of inhibitory repetitive transcranial magnetic stimulation (rTMS) of prefrontal cortex on recognition memory in AD patients. In a first experiment, 24 mild AD patients received sham and real 1Hz rTMS over the left and right dorsolateral prefrontal cortex (DLPFC), in different sessions, between encoding and retrieval phases of a non-verbal recognition memory task. In a second experiment, another group of 14 AD patients underwent sham controlled repeated sessions of 1Hz rTMS of the right DLPFC across a two week treatment. Non-verbal recognition memory task was performed at baseline, at the end of the two weeks period and at a follow up of 1 month. Right real rTMS significantly improved memory performance compared to right sham rTMS (p = 0.001). Left real rTMS left the memory performance unchanged as compared with left sham rTMS (p = 0.46). The two sham conditions did not differ between each other (p = 0.24). In the second experiment, AD patients treated with real rTMS showed an improvement of memory performance at the end of the two weeks treatment (p = 0.0009), that persisted at 1-month follow-up (p = 0.002). These findings provide evidence that inhibitory rTMS over the right DLPFC can improve recognition memory function in AD patients. They also suggest the importance of a new approach of non-invasive brain stimulation as a promising treatment in AD.

Early detection of memory impairments in older adults: standardization of a short version of the verbal and nonverbal Recognition Memory Test.

In several neurological conditions, in elderly and cognitively impaired subjects, memory functioning must be evaluated to early detect the cognitive deterioration processes. In particular, recognition memory assessment is an essential step in the clinical and neuropsychological evaluation of early memory impairments. The Recognition Memory Test (RMT) developed by Smirni et al. (G Ital Psicol XXXVII(1):325-343, 2010) is an effective instrument to assess verbal and nonverbal recognition memory in the Italian population. The current study provides a new, brief, and reliable RMT format to evaluate recognition memory on elderly subjects and it reports normative data in an older adult Italian population sample (including 100 participants well distributed across sex, education, and age categories). The shortened version of RMT keeps the administration procedures and materials of the original Italian RMT constant, i.e., words, faces, and buildings. Multiple regression analysis revealed significant effects of age and educational level on performance but no effect of sex. Inferential cutoffs have been determined and equivalent scores computed. The availability of equivalent scores for the Recognition Memory Test will prove useful in the clinical evaluation of patients' memory profiles.

One is all you need: intrahemispheric processing benefits nonverbal visual recognition

ABSTRACTSeveral attempts have been made to understand when and how the two hemispheres of the brain work together to encode and retrieve information during memory tasks, but it remains unclear whether they are equally capable of encoding and retrieval, particularly when the stimuli do not evoke a leftward processing asymmetry. Using a divided visual field paradigm, we presented nonverbal visual stimuli to one visual field/hemisphere at encoding, and at retrieval presented the stimuli either to the same or opposite visual field/hemisphere. Recognition responses were faster and more accurate when the stimuli were initially presented at encoding and retrieval to the same hemisphere (Experiment 1), even when delay intervals between study and test were short (Experiment 2). Taken together, these findings suggest that recognition decisions for stimuli initially presented to a single hemisphere occur more quickly at shorter lags, perhaps due to a stronger memory representation in the original hemisphere of input compared to the indirectly activated hemisphere. Our results are significant because they demonstrate that each hemisphere of the brain can function to encode and retrieve memory representations equally well, as long as the stimuli contain no linguistic information.

Standardization and validation of a parallel form of the verbal and non-verbal recognition memory test in an Italian population sample.

In the neuropsychological assessment of several neurological conditions, recognition memory evaluation is requested. Recognition seems to be more appropriate than recall to study verbal and non-verbal memory, because interferences of psychological and emotional disorders are less relevant in the recognition than they are in recall memory paradigms. In many neurological disorders, longitudinal repeated assessments are needed to monitor the effectiveness of rehabilitation programs or pharmacological treatments on the recovery of memory. In order to contain the practice effect in repeated neuropsychological evaluations, it is necessary the use of parallel forms of the tests. Having two parallel forms of the same test, that kept administration procedures and scoring constant, is a great advantage in both clinical practice, for the monitoring of memory disorder, and in experimental practice, to allow the repeated evaluation of memory on healthy and neurological subjects. First aim of the present study was to provide normative values in an Italian sample (n = 160) for a parallel form of a verbal and non-verbal recognition memory battery. Multiple regression analysis revealed significant effects of age and education on recognition memory performance, whereas sex did not reach a significant probability level. Inferential cutoffs have been determined and equivalent scores computed. Secondly, the study aimed to validate the equivalence of the two parallel forms of the Recognition Memory Test. The correlations analyses between the total scores of the two versions of the test and correlation between the three subtasks revealed that the two forms are parallel and the subtasks are equivalent for difficulty.

Facial and prosodic emotion recognition in social anxiety disorder

ABSTRACTIntroduction: Patients with social anxiety disorder (SAD) have a cognitive preference to negatively evaluate emotional information. In particular, the preferential biases in prosodic emotion recognition in SAD have been much less explored. The present study aims to investigate whether SAD patients retain negative evaluation biases across visual and auditory modalities when given sufficient response time to recognise emotions.Methods: Thirty-one SAD patients and 31 age- and gender-matched healthy participants completed a culturally suitable non-verbal emotion recognition task and received clinical assessments for social anxiety and depressive symptoms. A repeated measures analysis of variance was conducted to examine group differences in emotion recognition.Results: Compared to healthy participants, SAD patients were significantly less accurate at recognising facial and prosodic emotions, and spent more time on emotion recognition. The differences were mainly driven by the lower accuracy and longer reaction times for recognising fearful emotions in SAD patients. Within the SAD patients, lower accuracy of sad face recognition was associated with higher severity of depressive and social anxiety symptoms, particularly with avoidance symptoms.Conclusion: These findings may represent a cross-modality pattern of avoidance in the later stage of identifying negative emotions in SAD. This pattern may be linked to clinical symptom severity.