Recognition Failures Research Articles

User Interaction Patterns and Breakdowns in Conversing with LLM-Powered Voice Assistants

Conventional Voice Assistants (VAs) rely on traditional language models to discern user intent and respond to their queries, leading to interactions that often lack a broader contextual understanding, an area in which Large Language Models (LLMs) excel. However, current LLMs are largely designed for text-based interactions, thus making it unclear how user interactions will evolve if their modality is changed to voice. In this work, we investigate whether LLMs can enrich VA interactions via an exploratory study with participants (N=20) using a ChatGPT-powered VA for three scenarios (medical self-diagnosis, creative planning, and discussion) with varied constraints, stakes, and objectivity. We observe that LLM-powered VA elicits richer interaction patterns that vary across tasks, showing its versatility. Notably, LLMs absorb the majority of VA intent recognition failures. We additionally discuss the potential of harnessing LLMs for more resilient and fluid user-VA interactions and provide design guidelines for tailoring LLMs for voice assistance.

The Request of Xiangling: A Controversial Case on Silencing

Dream of the Red Chamber has been one of the well-known novels in China, which has contributed to an area named “redology.” In the realm of redology, there are a lot of controversies about the silencing of Xiangling’s request for Baochai (one main character of the Red Chamber) to teach her poetry. However, few researchers have tried to analyze this conversation through a linguistic angle. This paper will introduce the Austinian approach to silencing to this controversial case in Dream of the Red Chamber, which could make essential distinctions between important concepts, and provide a coherent explanation for the reason why Xiangling insisted on her specific request to learn poetry and why Baochai had such systematic recognition failures. Through the case study, this paper proves that Xiangling suffers illocutionary silencing in her conversation with Baochai by refuting possible counterarguments. It is significant not only to the explanation of this novel, but also to expansion of application of Austin’s speech act theory.

Episodic memory assessment: effects of sex and age on performance and response time during a continuous recognition task.

Continuous recognition tasks (CRTs) assess episodic memory (EM), the central functional disturbance in Alzheimer's disease and several related disorders. The online MemTrax computerized CRT provides a platform for screening and assessment that is engaging and can be repeated frequently. MemTrax presents complex visual stimuli, which require complex involvement of the lateral and medial temporal lobes and can be completed in less than 2 min. Results include number of correct recognitions (HITs), recognition failures (MISSes = 1-HITs), correct rejections (CRs), false alarms (FAs = 1-CRs), total correct (TC = HITs + CRs), and response times (RTs) for each HIT and FA. Prior analyses of MemTrax CRT data show no effects of sex but an effect of age on performance. The number of HITs corresponds to faster RT-HITs more closely than TC, and CRs do not relate to RT-HITs. RT-HITs show a typical skewed distribution, and cumulative RT-HITs fit a negative survival curve (RevEx). Thus, this study aimed to define precisely the effects of sex and age on HITS, CRs, RT-HITs, and the dynamics of RTs in an engaged population. MemTrax CRT online data on 18,255 individuals was analyzed for sex, age, and distributions of HITs, CRs, MISSes, FAs, TC, and relationships to both RT-HITs and RT-FAs. HITs corresponded more closely to RT-HITs than did TC because CRs did not relate to RT-HITs. RT-FAs had a broader distribution than RT-HITs and were faster than RT-HITs in about half of the sample, slower in the other half. Performance metrics for men and women were the same. HITs declined with age as RT-HITs increased. CRs also decreased with age and RT-FAs increased, but with no correlation. The group over aged 50 years had RT-HITs distributions slower than under 50 years. For both age ranges, the RevEx model explained more than 99% of the variance in RT-HITs. The dichotomy of HITs and CRs suggests opposing cognitive strategies: (1) less certainty about recognitions, in association with slower RT-HITs and lower HIT percentages suggests recognition difficulty, leading to more MISSes, and (2) decreased CRs (more FAs) but faster RTs to HITs and FAs, suggesting overly quick decisions leading to errors. MemTrax CRT performance provides an indication of EM (HITs and RT-HITs may relate to function of the temporal lobe), executive function (FAs may relate to function of the frontal lobe), processing speed (RTs), cognitive ability, and age-related changes. This CRT provides potential clinical screening utility for early Alzheimer's disease and other conditions affecting EM, other cognitive functions, and more accurate impairment assessment to track changes over time.

Differential contributions of body form, motion, and temporal information to subjective action understanding in naturalistic stimuli.

We investigated the factors underlying naturalistic action recognition and understanding, as well as the errors occurring during recognition failures. Participants saw full-light stimuli of ten different whole-body actions presented in three different conditions: as normal videos, as videos with the temporal order of the frames scrambled, and as single static representative frames. After each stimulus presentation participants completed one of two tasks-a forced choice task where they were given the ten potential action labels as options, or a free description task, where they could describe the action performed in each stimulus in their own words. While generally, a combination of form, motion, and temporal information led to the highest action understanding, for some actions form information was sufficient and adding motion and temporal information did not increase recognition accuracy. We also analyzed errors in action recognition and found primarily two different types. One type of error was on the semantic level, while the other consisted of reverting to the kinematic level of body part processing without any attribution of semantics. We elaborate on these results in the context of naturalistic action perception.

Deciphering pixel insights: A deep dive into deep learning strategies for enhanced indoor depth estimation

Depth estimation is one of the crucial tasks for autonomous systems, which provides important information about the distance between the system and its surroundings. Traditionally, Light Detection and Ranging and stereo cameras have been used for distance measurement, despite the significant cost. In contrast, monocular cameras offer a more cost-effective solution, but lack inherent depth information. The synergy of big data and deep learning has led to various advanced architectures for monocular depth estimation. However, due to the characteristics of the monocular depth estimation case that is ill posed problem, we incorporate Attention Gates (AG) within an encoder-decoder based architecture. This helps prevent pattern recognition failures caused by variations in object sizes that share identical depth values. Our research involves evaluating popular pretrained architectures, assessing the impact of using AG, and creating effective head blocks to tackle depth estimation challenges. Notably, our approach demonstrates improved evaluation metrics on the DIODE dataset, positioning Attention U-Net as a promising solution. Therefore, utilizing the superior performance obtained by Attention U-Net in performing monocular depth estimation on low-cost autonomous systems could relatively reduce the cost of using lidar or stereo cameras in measuring distance.11https://github.com/KrisnaPinasthika/Deciphering-Pixel-Insights

Willful testimonial injustice as a form of epistemic injustice

AbstractIn the debate on epistemic injustice, it is generally assumed that testimonial injustice as one form of epistemic injustice cannot be committed (fully) deliberately or intentionally because it involves unconscious identity prejudices. Drawing on the case of sexual violence against refugees in European refugee camps, this paper argues that there is a form of testimonial injustice—willful testimonial injustice—that is deliberate. To do so, the paper argues (a) that the hearer intentionally utilizes negative identity prejudices for a particular purpose and (b) that the hearer is aware of the fact that the intentionally used prejudices are in fact prejudices. Furthermore, the paper shows how testimonial injustice relates to recognition failures both in terms of a causal as well as a constitutive claim. In fact, introducing willful testimonial injustice can support the constitutive claim of such a relation that has so far received little attention. Besides arguing for a novel form of testimonial injustice and contributing to the recent debate on the relation between epistemic injustice and recognition failures, this paper is also motivated by the attempt to draw attention to the inhumane conditions for refugees at the border of Europe as well as elsewhere.

Distracted driving detection based on the fusion of deep learning and causal reasoning

Distracted driving is one of the key factors that cause drivers to ignore potential road hazards and then lead to accidents. Existing efforts in distracted behavior recognition are mainly based on deep learning (DL) methods, which identifies distracted behaviors by analyzing static characteristics of images. However, the convolutional neural network (CNN) — based DL methods lack the causal reasoning ability for behavior patterns. The uncertainty of driving behaviors, noise of the collected data, and occlusion between body agents, bring additional challenges to existing DL methods to recognize distracted behaviors continuously and accurately. Therefore, in this paper, we propose a distracted behavior recognition method based on the Temporal–Spatial double-line DL network (TSD-DLN) and causal And-or graph (C-AOG). TSD-DLN fuses the attention feature extracted from the dynamic optical flow information and the spatial feature of the single video frame to recognize the distracted driving posture. Furthermore, a causal knowledge fence based on C-AOG is fused with TSD-DLN to improve the recognition robustness. The C-AOG represents the causality of behavior state fluent change and adopts counterfactual reasoning to suppress behavior recognition failures caused by frame features distortion or occlusion between body agents. We compared the performance of the proposed method with other state-of-the-art (SOTA) DL methods on two public datasets and self-collected dataset. Experimental results demonstrate that proposed method significantly outperforms other SOTA methods when acquiring distracted driving behavior by processing consecutive frames. In addition, the proposed method exhibits accurate continuous recognition and robustness under incomplete observation scenarios.

Personalised socially assistive robot for cardiac rehabilitation: Critical reflections on long-term interactions in the real world.

Lack of motivation and low adherence rates are critical concerns of long-term rehabilitation programmes, such as cardiac rehabilitation. Socially assistive robots are known to be effective in improving motivation in therapy. However, over longer durations, generic and repetitive behaviours by the robot often result in a decrease in motivation and engagement, which can be overcome by personalising the interaction, such as recognising users, addressing them with their name, and providing feedback on their progress and adherence. We carried out a real-world clinical study, lasting 2.5 years with 43 patients to evaluate the effects of using a robot and personalisation in cardiac rehabilitation. Due to dropouts and other factors, 26 patients completed the programme. The results derived from these patients suggest that robots facilitate motivation and adherence, enable prompt detection of critical conditions by clinicians, and improve the cardiovascular functioning of the patients. Personalisation is further beneficial when providing high-intensity training, eliciting and maintaining engagement (as measured through gaze and social interactions) and motivation throughout the programme. However, relying on full autonomy for personalisation in a real-world environment resulted in sensor and user recognition failures, which caused negative user perceptions and lowered the perceived utility of the robot. Nonetheless, personalisation was positively perceived, suggesting that potential drawbacks need to be weighed against various benefits of the personalised interaction.

Control framework for collaborative robot using imitation learning-based teleoperation from human digital twin to robot digital twin

Despite the deployment of collaborative robots for various industrial processes, their teaching and control remain comparatively difficult tasks compared with general industrial robots. Various imitation learning methods involving the transfer of human poses to a collaborative robot have been proposed. However, most of these methods depend heavily on deep learning-based human recognition algorithms that fail to recognize complicated human poses. To address this issue, we propose an automated/semi-automated vision-based teleoperation framework using human digital twin and a collaborative robot digital twin models. First, a human pose is recognized and reasoned to a human skeleton model using a convolution encoder-decoder architecture. Next, the developed human digital twin model is taught using the skeletons. As human and collaborative robots have different joints and rotation architectures, pose mapping is achieved using the proposed Bezier curve-based smooth approximation. Then, a real collaborative robot is controlled using the developed robot digital twin. Furthermore, the proposed framework works successfully using a human digital twin in the case of recognition failures of human poses. To verify the effectiveness of the proposed framework, transfers of several human poses to a real collaborative robot are tested and analyzed.

Metamorphic testing of named entity recognition systems: A case study

Named entity recognition (NER) is a widely used natural language processing technique; it plays a key role in information extraction from sentences. To be able to test the correctness of NER systems is important, but it is expensive because an automated test oracle is normally unavailable. To address the oracle problem, this study proposes to apply metamorphic testing (MT). The authors conduct a case study with Litigant, an industrial NER system of the Ant Group, and show that MT can effectively detect real-life bugs in the absence of an ideal oracle. The authors further investigate the causes for a series of entity recognition failures detected. Outcomes of this research further justify the application of MT to the natural language processing domain as well as provide hints for practitioners to improve the quality process of their NER systems.

Forensic implications of fingerprint verification failure among people with skin diseases.

Fingerprint analysis is one of the earliest and most commonly used methods of personal identification by police and forensic scientists at crime scenes. It has two key advantages: permanence and uniqueness. However, some skin diseases like eczema, psoriasis and viral exanthems can hinder the process of analysis and thwart identification; so it is important to recognise the effects a skin condition may have on fingerprint identification. The aim of this narrative review is to provide an overview of common skin diseases that are associated with fingerprint recognition failures as well as to recapitulate the forensic implications.

Double-meaning agreements by two robots to conceal incoherent agreements to user's opinions

ABSTRACT In conversations between people and social robots, it is important that robots show agreements to user's opinions from the aspect of building relationships. However, robot's agreements are often incoherent to user's opinions due to speech recognition failures. This paper proposes a new approach called double-meaning agreement to conceal the incoherence. In this approach, we exploit an interaction protocol by two robots. The interaction protocol makes an agreement that has a double meaning and enables a user to interpret robot's incoherent agreement as coherent. To evaluate effects of double-meaning agreement, we conducted an experiment. The results showed that participants who talked with two robots using double-meaning agreement had better feelings of being understood by the robots than those who talked with one robot without double-meaning agreement. These findings will contribute to developing social robots to keep a conversation coherent and to build social relationships.

Understanding Speech in Interactive Narratives with Crowdsourced Data

Speech recognition failures and limited vocabulary coverage pose challenges for speech interaction with characters in games. We describe an end-to-end system for automating characters from a large corpus of recorded human game logs, and demonstrate that inferring utterance meaning through a combination of plan recognition and surface text similarity compensates for recognition and understanding failures significantly better than relying on surface similarity alone.

Recallable but not recognizable: The influence of semantic priming in recall paradigms

When people can successfully recall a studied word, they should be able to recognize it as having been studied. In cued-recall paradigms, however, participants sometimes correctly recall words in the presence of strong semantic cues but then fail to recognize those words as actually having been studied. Although the conditions necessary to produce this unusual effect are known, the underlying neural correlates have not been investigated. Across five experiments, involving both behavioral and electrophysiological methods (EEG), we investigated the cognitive and neural processes that underlie recognition failures. Experiments 1 and 2 showed behaviorally that assuming that recalled items can be recognized in cued-recall paradigms is a flawed assumption, because recognition failures occur in the presence of cues, regardless of whether those failures are measured. With event-related potentials (ERPs), Experiments 3 and 4 revealed that successfully recalled words that are recognized are driven by recollection at recall and then by a combination of recollection and familiarity at ensuing recognition. In contrast, recognition failures did not show that memory signature and may instead be driven by semantic priming at recall and followed at recognition stages by negative-going ERP effects consistent with implicit processes, such as repetition fluency. These results demonstrate that recall - long-characterized as predominantly reflecting recollection-based processing in episodic memory - may at times also be served by a confluence of implicit cognitive processes.

Post-Mortem Iris Recognition—A Survey and Assessment of the State of the Art

Post-mortem biometrics entails utilizing the biometric data of a deceased individual for determining or verifying human identity. Due to fundamental biological changes that occur in a person’s biometric traits after death, post-mortem data can be significantly different from ante-mortem data, introducing new challenges for biometric sensors, feature extractors and matchers. This paper surveys research to date on the problem of using iris images acquired after death for automated human recognition. A comprehensive review of existing literature is complemented by a summary of the most recent results and observations offered in these publications. This survey is unique in several elements. Firstly, it is the first publication to consider iris recognition where gallery images are acquired before death (peri-mortem images) and the probe images are acquired after death from the same subjects. Secondly, results are presented from the largest database of peri-mortem and post-mortem iris images, collected from 213 subjects by two independent institutions located in the U.S. and Poland. Thirdly, post-mortem recognition viability is assessed using more than 20 iris recognition algorithms, ranging from the classic (e.g., Gabor filtering-based) to the modern (e.g., deep learning-based). Finally, we provide a medically informed commentary on post-mortem iris, analyze the reasons for recognition failures, and identify key directions for future research.

Theodicies as Failures of Recognition

This paper examines the ethical failure of theodicies by integrating the perspectives of philosophical argumentation and literary reading and analysis. The paper consists of two main parts. In the first part, we propose an ethical critique of metaphysical realism by analyzing its inability to recognize the perspectival plurality and diversity of suffering. As theodicies seek to explain how an omnipotent, omniscient, and absolutely benevolent God could allow the world to contain evil and suffering, it can be argued that metaphysical realism—i.e., the thesis that the world possesses its own fundamental structure independently of human perspectives of conceptualization and inquiry—is a problematic starting point of theodicism. We examine the failure of recognition of others’ suffering inherent in theodicies as a failure based on the search for an overall reductive and objectifying picture (a “God’s-Eye View”) that is constitutive of metaphysical realism. The second part of the paper shows why we should include insights from imaginative literature in our attempts to understand the recognition failures of theodicies. Emphasizing the literary, philosophical, and theological relevance of various modern rewritings of the Book of Job, which has been a crucially important sub-text for many later literary works in which the protagonists render a particular kind of human experience—unmerited suffering—we turn more closely to some literary examples, such as Joseph Roth’s novels Hiob and Die Rebellion. The tensions that are created around the moral controversy of the experiences of injustice and suffering and the human and religious reasoning and justification of violence are examined. The ambiguous ending of Hiob that adds an apparently hopeful and almost fairytale-like redemption to the story plays a crucial role in the interpretation provided in the paper. By analyzing some literary examples and their relation to the literary Job tradition, the recognition-failures of theodicist attempts to provide meaning into suffering—attempts based on metaphysical realism, as argued in the first part of the paper—are highlighted. Finally, we also critically consider the charge that theodicism could only be theoretically formulated and argue that a sharp distinction between theory and practice in this area is itself an act of non-recognition, or a failure to recognize suffering.

Recognition of Banknote Fitness Based on a Fuzzy System Using Visible Light Reflection and Near-infrared Light Transmission Images.

Fitness classification is a technique to assess the quality of banknotes in order to determine whether they are usable. Banknote classification techniques are useful in preventing problems that arise from the circulation of substandard banknotes (such as recognition failures, or bill jams in automated teller machines (ATMs) or bank counting machines). By and large, fitness classification continues to be carried out by humans, and this can cause the problem of varying fitness classifications for the same bill by different evaluators, and requires a lot of time. To address these problems, this study proposes a fuzzy system-based method that can reduce the processing time needed for fitness classification, and can determine the fitness of banknotes through an objective, systematic method rather than subjective judgment. Our algorithm was an implementation to actual banknote counting machine. Based on the results of tests on 3856 banknotes in United States currency (USD), 3956 in Korean currency (KRW), and 2300 banknotes in Indian currency (INR) using visible light reflection (VR) and near-infrared light transmission (NIRT) imaging, the proposed method was found to yield higher accuracy than prevalent banknote fitness classification methods. Moreover, it was confirmed that the proposed algorithm can operate in real time, not only in a normal PC environment, but also in an embedded system environment of a banknote counting machine.

A Unifying Model of Orientation Crowding in Peripheral Vision

Peripheral vision is fundamentally limited not by the visibility of features, but by the spacing between them [1]. When too close together, visual features can become "crowded" and perceptually indistinguishable. Crowding interferes with basic tasks such as letter and face identification and thus informs our understanding of object recognition breakdown in peripheral vision [2]. Multiple proposals have attempted to explain crowding [3], and each is supported by compelling psychophysical and neuroimaging data [4-6] that are incompatible with competing proposals. In general, perceptual failures have variously been attributed to the averaging of nearby visual signals [7-10], confusion between target and distractor elements [11, 12], and a limited resolution of visual spatial attention [13]. Here we introduce a psychophysical paradigm that allows systematic study of crowded perception within the orientation domain, and we present a unifying computational model of crowding phenomena that reconciles conflicting explanations. Our results show that our single measure produces a variety of perceptual errors that are reported across the crowding literature. Critically, a simple model of the responses of populations of orientation-selective visual neurons accurately predicts all perceptual errors. We thus provide a unifying mechanistic explanation for orientation crowding in peripheral vision. Our simple model accounts for several perceptual phenomena produced by crowding of orientation and raises the possibility that multiple classes of object recognition failures in peripheral vision can be accounted for by a single mechanism.

Impaired processing of self-face recognition in anorexia nervosa.

Body image disturbances and massive weight loss are major clinical symptoms of anorexia nervosa (AN). The aim of the present study was to examine the influence of body changes and eating attitudes on self-face recognition ability in AN. Twenty-seven subjects suffering from AN and 27 control participants performed a self-face recognition task (SFRT). During the task, digital morphs between their own face and a gender-matched unfamiliar face were presented in a random sequence. Participants' self-face recognition failures, cognitive flexibility, body concern and eating habits were assessed with the Self-Face Recognition Questionnaire (SFRQ), Trail Making Test (TMT), Body Shape Questionnaire (BSQ) and Eating Disorder Inventory-2 (EDI-2), respectively. Subjects suffering from AN exhibited significantly greater difficulties than control participants in identifying their own face (p=0.028). No significant difference was observed between the two groups for TMT (all p>0.1, non-significant). Regarding predictors of self-face recognition skills, there was a negative correlation between SFRT and body mass index (p=0.01) and a positive correlation between SFRQ and EDI-2 (p<0.001) or BSQ (p<0.001). Among factors involved, nutritional status and intensity of eating disorders could play a part in impaired self-face recognition.

Failure of Location Recognition in the Rorschach

Abstract. Eye movements during the Rorschach Inkblot Method (RIM) were monitored to investigate the existence of location recognition failures that were accompanied by clients’ explanations in the inquiry phase. We hypothesized that perceived locations may have differed from actual explained locations if fixations did not occur – and visual attention was diverted to other locations – just before the response in the free association phase. The eye movement data of 29 participants under a Rorschach administration were collected, and 688 responses were obtained. Of these, 195 responses that involved perception of small, specific locations were used to investigate the association between pre-response eye movements and queried locations. Six responses of three participants showed fixations within 3 s before the time of response at locations different from the locations provided in the explanation. Responses made to similar but different locations were indicative of potential failures of location recognition.