Semantic Anomaly Detection Research Articles

Semantic-aware normalizing flow with feature fusion for image anomaly detection

In the field of computer vision, anomaly detection is a binary classification task used to identify exceptional instances within image datasets. Typically, it can be divided into two aspects: texture defect detection and semantic anomaly detection. Existing methods often use pre-trained feature extractors to singly capture semantic or spatial features of images, and then employ different classifiers to handle these two types of anomaly detection tasks. However, these methods fail to fully utilize the synergistic relationship between these two types of features, resulting in algorithms that excel in one type of anomaly detection task but perform poorly in the other type. Therefore, we propose a novel approach that successfully combines these two types of features into a normalizing flow learning module to address both types of anomaly detection tasks. Specifically, we first adopt a pre-trained Vision Transformer (ViT) model to capture both texture and semantic features of input images. Subsequently, using the semantic features as input, we design a novel normalizing flow model to fit the semantic distribution of normal data. In addition, we introduce a feature fusion module based on attention mechanisms to integrate the most relevant texture and semantic information between these two types of features, significantly enhancing the model’s ability to simultaneously represent the spatial texture and semantic features of the input image. Finally, We conduct comprehensive experiments on well-known semantic and texture anomaly detection datasets, namely Cifar10 and MVTec, to evaluate the performance of our proposed method. The results demonstrate that our model achieves outstanding performance in both semantic and texture anomaly detection tasks, particularly achieving state-of-the-art results in semantic anomaly detection.

Semantic anomaly detection with large language models

Semantic anomaly detection with large language models

Anomaly detection based on multi-teacher knowledge distillation

Anomaly detection on high-dimensional data is crucial for real-world industrial applications. Recent works adopt the Knowledge Distillation (KD) technique to improve the accuracy of anomaly detection Neural Networks (NN). Most KD-based solutions only adopt a single teacher NN and have not yet fully incorporated the distinct advantages of different NN structures. To fill this gap, this paper proposes a novel Multi-teacher Knowledge Distillation approach, which effectively integrates multiple teachers with importance weights to provide guidance for the accurate anomaly detection of students. However, the importance weights are hard to get when training only with normal data. To overcome this challenge, we use an autoencoder-based reconstruction process to update teacher importance weights. In the meantime, the student model parameters are optimized by giving a set of teacher importance weights. Anomalies are then detected based on the deviations between the outputs of teacher and student, as well as the reconstruction errors through the student network. Our proposed approach is evaluated on both CIFAR10 and MVTec datasets. The results show good performance on both high-level semantic anomaly detection and low-level pixel anomaly detection.

Scene Semantic Anomaly Detection of Multi-Sensor in Autonomous Driving

Autonomous driving is an outstanding representative of artificial intelligence, and while it has unlimited opportunities, it also faces unprecedented challenges. The potential risks of autonomous vehicles cause a significant threat to our current traffic system. Multi-sensor data in autonomous vehicles will form scene semantics. When sensors are intruded, however, discordant elements of scene semantics will appear. Anomaly detection of scene semantics can improve the robustness of intrusion detection systems for autonomous driving. In this paper, we proposed a method for scene semantic anomaly detection. We obtain the scene semantic representation by performing a semantic-level fusion of sensor data. Then, to achieve a scene semantic anomaly detection, convert abnormal semantic detection to out-of-distribution detection. To test the effectiveness of our method, we performed experiments on a publicly available dataset of autonomous driving.

Moses or Noah? A case of ‘potato-potahto’ when using a foreign language

AbstractResearch among bilinguals suggests a foreign language effect for various tasks requiring a more systematic processing style. For instance, bilinguals seem less prone to heuristic reasoning when solving problem statements in their foreign (FL) as opposed to their native (NL) language. The present study aimed to determine whether such an effect might also be observed in the detection of semantic anomalies. Participants were presented NL and FL questions with and without anomalies while their eye movements were recorded. Overall, they failed to detect the anomaly in more than half of the trials. Furthermore, more illusions occurred for questions presented in the FL, indicating an FL disadvantage. Additionally, eye movement analyses suggested that reading patterns for anomalies are predominantly similar across languages. Our results therefore substantiate theories suggesting that FL use induces cognitive load, causing increased susceptibility to illusions due to partial semantic processing.

Semantic Anomaly Detection in Medical Time Series.

The main goal of this project was to define and evaluate a new unsupervised deep learning approach that can differentiate between normal and anomalous intervals of signals like the electrical activity of the heart (ECG). Denoising autoencoders based on recurrent neural networks with gated recurrent units were used for the semantic encoding of such time frames. A subsequent cluster analysis conducted in the code space served as the decision mechanism labelling samples as anomalies or normal intervals, respectively. The cluster ensemble method called cluster-based similarity partitioning proved itself well suited for this task when used in combination with density-based spatial clustering of applications with noise. The best performing system reached an adjusted Rand index of 0.11 on real-world ECG signals labelled by medical experts. This corresponds to a precision and recall regarding the detection task of around 0.72. The new general approach outperformed several state-of-the-art outlier recognition methods and can be applied to all kinds of (medical) time series data. It can serve as a basis for more specific detectors that work in an unsupervised fashion or that are partially guided by medical experts.

Semantic anomaly detection in school-aged children during natural sentence reading - A study of fixation-related brain potentials.

In this study, we investigated the effects of context-related semantic anomalies on the fixation-related brain potentials of 12–13-year-old Finnish children in grade 6 during sentence reading. The detection of such anomalies is typically reflected in the N400 event-related potential. We also examined whether the representation invoked by the sentence context extends to the orthographic representation level by replacing the final words of the sentence with an anomalous word neighbour of a plausible word. The eye-movement results show that the anomalous word neighbours of plausible words cause similar first-fixation and gaze duration reactions, as do other anomalous words. Similarly, we observed frontal negativity in the fixation-related potential of the unrelated anomalous words and in the anomalous word neighbours. This frontal negativity was larger in both anomalous conditions than in the response elicited by the plausible condition. We thus show that the brain successfully uses context to separate anomalous words from plausible words on a single letter level during free reading. From the P600 response of the scalp waveform, we observed that the P600 was delayed in the anomalous word neighbour condition. We performed group-level decomposition on the data with ICA (independent component analysis) and analysed the time course and source structure of the decomposed data. This analysis of decomposed brain signals not only confirmed the delay of the P600 response but also revealed that the frontal negativity concealed s more typical and separate N400 response, which was similarly delayed in the anomalous word neighbour condition, as was the P600 response. Source analysis of these independent components implicated the right frontal eye field as the cortical source for the frontal negativity and the middle temporal and parietal regions as cortical sources for the components resembling the N400 and P600 responses. We interpret the delays present in N400 and P600 responses to anomalous word neighbours to reflect competition with the representation of the plausible word just one letter different.

Single-Trial Detection of Semantic Anomalies From EEG During Listening to Spoken Sentences.

We propose a method for the automatic detection of mismatched feelings that occur in communication. As our first step, we examined the semantically anomalous feelings from EEGs when participants listened to spoken sentences. Previous studies have shown that the event-related potentials (ERP) of an electroencephalogram (EEG) are evoked in the auditory and visual modalities where a semantic anomaly occurs. We expand this knowledge and detect it from a single-trial ERP using machine learning techniques. We recorded the brain activity of eight participants as they listened to sentences that contained semantic anomalies and found that a combination of feature selection using linear discriminant analysis and linear kernel support vector machines achieved the highest accuracy that exceeded 60%. By applying this technique, we plan to detect other types of anomalies in practical situations.

Conscious and unconscious detection of semantic anomalies.

When asked What superhero is associated with bats, Robin, the Penguin, Metropolis, Catwoman, the Riddler, the Joker, and Mr. Freeze? people frequently fail to notice the anomalous word Metropolis. The goals of this study were to determine whether detection of semantic anomalies, like Metropolis, is conscious or unconscious and whether this detection is immediate or delayed. To achieve these goals, participants answered anomalous and nonanomalous questions as their reading times for words were recorded. Comparisons between detected versus undetected anomalies revealed slower reading times for detected anomalies-a finding that suggests that people immediately and consciously detected anomalies. Further, comparisons between first and second words following undetected anomalies versus nonanomalous controls revealed some slower reading times for first and second words-a finding that suggests that people may have unconsciously detected anomalies but this detection was delayed. Taken together, these findings support the idea that when we are immediately aware of a semantic anomaly (i.e., immediate conscious detection) our language processes make immediate adjustments in order to reconcile contradictory information of anomalies with surrounding text; however, even when we are not consciously aware of semantic anomalies, our language processes still make these adjustments, although these adjustments are delayed (i.e., delayed unconscious detection).

Revisiting the scrambling complexity hypothesis in sentence processing: a self-paced reading study on anomaly detection and scrambling in Hindi

The scrambling complexity hypothesis based on working memory or locality accounts as well as syntactic accounts have proposed that processing a scrambled structure is difficult. However, the locus of this difficulty in sentence processing remains debatable. Several studies on multiple languages have explored the effect of scrambling on sentence processing and not all languages have shown an advantage for the canonical word order. Using a self-paced reading paradigm, we studied the effect of scrambling on semantic anomaly detection in Hindi sentence comprehension employing three word order types. Reading times on critical verbs, judgment latency, and error rates showed significant effect of word order type. The results further revealed significant interactions between word order and anomaly type. The patterns of results suggest that the canonical word order does not necessarily have a processing advantage in terms of speed and accuracy over non-canonical orders and do not provide support to sentence processing accounts that assume an advantage for canonical structures. The results indicate that processing speed depends on the distance between the subject and the verb, thus supporting a locality dependent working memory based model of sentence processing. The results provide evidence for the role of specific cognitive processes in Hindi sentence processing with further implications for language and literacy acquisition in Hindi.

Morphed facial expressions elicited a N400 ERP effect: A domain‐specific semantic module?

Previous studies have revealed that decoding of facial expressions is a specific component of face comprehension and that semantic information might be processed separately from the basic stage of face perception. In order to explore event-related potentials (ERPs) related to recognition of facial expressions and the effect of the semantic content of the stimulus, we analyzed 20 normal subjects. Faces with three prototypical emotional expressions (fear, happiness, and sadness) and with three morphed expressions were presented in random order. The neutral stimuli represented the control condition. Whereas ERP profiles were similar with respect to an early negative ERP (N170), differences in peak amplitude were observed later between incongruous (morphed) expressions and congruous (prototypical) ones. In fact, the results demonstrated that the emotional morphed faces elicited a negative peak at about 360 ms, mainly distributed over the posterior site. The electrophysiological activity observed may represent a specific cognitive process underlying decoding of facial expressions in case of semantic anomaly detection. The evidence is in favor of the similarity of this negative deflection with the N400 ERP effect elicited in linguistic tasks. A domain-specific semantic module is proposed to explain these results.

Childhood Head Injury and Metacognitive Processes in Language and Memory

We studied the metacognitive functioning of children with severe and mild traumatic brain injury (TBI) and typically developing children. To test metacognition for memory, children were tested on a modified Judgment of Learning task. We found that children with severe TBI were impaired in their ability to predict recall of specific items prior to study-recall trials, but were unimpaired in predicting recall on a delayed test when the judgment was made after study-recall trials. Metacognitive knowledge impairment for memorial abilities was also demonstrated in children with severe TBI by poor estimation of memory span and exaggerated overconfidence in performance. To test metacognition within the language domain, we gave children a sentence anomaly detection and repair task in which spoken sentences were monitored for semantic anomalies. Children with severe TBI were impaired on the detection of semantic anomalies, especially under conditions of high memory load. However, metalinguistic knowledge in the form of adequate repairs of anomalous sentences, was preserved. Results are discussed in terms of effects of age at test and injury severity.