Teacher Network Research Articles

AFF-DSTnet: fabric anomaly detection based on adaptive feature fusion dual-student–teacher network

AFF-DSTnet: fabric anomaly detection based on adaptive feature fusion dual-student–teacher network

Modality-Aware Distillation Network for Microvascular Invasion Prediction of Hepatocellar Carcinoma From MRI Images.

Microvascular invasion (MVI) of hepatocellular carcinoma (HCC) is a crucial histopathologic prognostic factor associated with cancer recurrence after liver transplantation or hepatectomy. Recently, clinicoradiologic characteristics are combined with medical images to enhance the HCC prediction. However, compared to medical imaging data, the clinicoradiologic characteristics (e.g., APOe4 genotyping) is not easy to collect or even unavailable, as it requires more efforts of clinicians and more medical instruments for collecting diverse measurements. This work explores how to transfer the knowledge of a teacher network learned from non-image clinical data and image data to a student network with only image data such that the student network can leverage the transferred clinical information to boost HCC classification with only imaging data as input. Specifically, we present a modality-aware distillation network (MD-Net) to transform non-image clinicoradiologic from the teacher network to the student network. The teacher network integrates non-image clinicoradiologic characteristics with two 3D MRI modality images via two MRI-clinical-fusion modules and a symmetric attention (SA) module, while the student network extracts features from two modality MRI data via two MRI-only modules and then refine these two MRI features via a SA module. A classification-level distillation and a feature-level distillation are jointly utilized to transfer the clinical information between teacher and student networks. Furthermore, we design a novel self-supervised task to predict clinicoradiologic characteristics from the imaging data to further enhance the downstream HCC classification. The experimental results from our collected dataset and a multi-modal sarcasm detection dataset have demonstrated the effectiveness of our approach. Specifically, we achieved an AUC score of 71.86% and 75.51% respectively, surpassing the performance of the state-of-the-art classification methods.

An Unsupervised Anomaly Detection Method for Railway Fasteners Based on Knowledge-Distilled Generative Adversarial Networks

The integrity and stability of railway fasteners are of vital importance to railway safety. To address the challenges of limited anomaly samples, irregular defect geometries, and complex operational conditions in rail fastener anomaly detection, this paper proposes an unsupervised anomaly detection method using a knowledge-distilled generative adversarial network. First, the proposed method employs collaborative teacher–student learning to model normal sample distributions, where the student network reconstructs input images as normal outputs while a discriminator identifies anomalies by comparing input and reconstructed images. Second, a multi-scale attention-coupling feature-enhancement mechanism is proposed, effectively integrating hierarchical semantic information with spatial-channel attention to achieve both precise target localization and robust background suppression in the teacher network. Third, an enhanced anomaly discriminator is designed to incorporate an enhanced pyramid upsampling module, through which fine-grained details are preserved via multi-level feature map aggregation, resulting in significantly improved sensitivity for small-sized anomaly detection. Finally, the proposed method achieved an AUC of 94.0%, an ACC of 92.5%, and an F1 score of 91.6% on the MNIST dataset, and an AUC of 94.7%, an ACC of 90.1%, and an F1 score of 87.8% on the railway fastener dataset, which proves the superior anomaly detection ability of this method.

RESPONSIVENESS OF TEACHERS ON CHILD PROTECTION POLICY IMPLEMENTATION: CREATING BRIGHT FUTURE FOR LEARNERS

The primary purpose of this research is to delve into the intricate realities of teachers' lived experiences in implementing child protection policies. By examining their day-to-day interactions and practices, the study seeks to uncover how policies are interpreted and applied in educational settings. The qualitative phenomenological design is well-suited for exploring the lived experiences and perspectives of teachers involved in implementing child protection policies within educational settings. In this study, purposeful sampling will be utilized to select a diverse group of teachers with varied backgrounds, experiences, and roles relevant to the implementation of Child Protection Policies (CPP) within educational settings. Participants will be chosen deliberately based on their potential to provide rich and meaningful data aligned with the research objectives. The selection process will begin with identifying potential participants through coordination with school administrators and teacher networks in Laak, Davao de Oro, ensuring that insights reflect the local educational landscape and challenges unique to the area. The insights gained from the study underscore the critical role of teachers, school administrators, and other stakeholders in safeguarding the well-being of children. Moreover, the collaborative practices mentioned in Themes 2 and 3, such as teamwork among teachers, clear communication channels, and coordinated interventions, reflect a learning environment where both teachers and students develop behaviors through reciprocal interactions. As teachers learn from one another and adapt best practices in child protection, they also indirectly teach students to seek help, build trust, and form healthy relationships with adults. As the landscape of education continues to evolve, it is crucial to reflect on the future directions for child protection practices within schools. Ensuring the safety and well-being of students requires continuous adaptation, collaboration, and innovation. Moving forward, there is a need to strengthen the integration of child protection policies with broader educational goals, emphasizing not only the prevention of abuse but also the holistic development of students. Keywords: Responsiveness of Teachers, Child Protection Policy Implementation, Creating Bright Future for Learners

Knowledge distillation-enhanced marine optical remote sensing object detection with transformer and dual-path architecture

With the growing demand for marine surveillance and resource management, accurate marine object detection has become crucial for both military operations and civilian applications. However, this task faces inherent challenges including complex environmental interference, diverse object scales and morphologies, and dynamic imaging conditions. To address these issues, this paper proposes a marine optical remote sensing object detection architecture based on transformer and dual path architecture (MOD-TD), aiming to improve the accuracy and robustness of maritime target detection. The encoder integrates a Holistic Focal Feature Interwined (HFFI) module that employs parallel pathways to progressively refine local textures and global semantic representations, enabling adaptive feature fusion across spatial hierarchies. The decoder introduces task-specific query decoupling for classification and localization, combined with an Enhanced Multi-scale Attention (EMSA) mechanism that dynamically aggregates contextual information from multiple receptive fields. Furthermore, the framework incorporates a Multivariate Matching strategy with Gaussian spatial constraints to improve anchor-object correspondence in complex marine scenarios. To balance detection accuracy with computational efficiency, a knowledge distillation framework is implemented where a compact student model learns distilled representations through multi-granularity alignment with a teacher network, encompassing intermediate feature guidance and output-level probability calibration. Comprehensive evaluations on the SeaDronesSee and DOTA-Marine datasets validate the architecture’s superior detection performance and environmental adaptability compared to existing methods, demonstrating significant advancements in handling multi-scale objects under variable marine conditions. This work establishes a new paradigm integrating architectural innovation and model compression strategies for practical marine observation systems.

Co-Development and Collaboration for Responsive Teacher Education: A Teacher Leader Network to Support Reform-Based Science Instruction

ABSTRACT This paper highlights research conducted within a collaborative science teacher network, which provides professional development (PD) to elementary teachers to support engaging children in practices of science through open-ended inquiry opportunities. This network is anchored in a teacher resource center, the SciTeach Center (hereafter the Center), located at our university. The PD offered through the Center emerges from teachers’ specific needs and contexts and is developed through a collective process, in which teachers work together to learn in a professional community. The overarching question that guides this paper is related to their positions as Teacher Leaders within the context of education curricular reform and change: What does the process of responsive PD (co-planning, co-development, and co-teaching) reveal about supporting teachers in enacting reform-based science teaching/instruction? We present two vignettes from case studies of participating teachers to analyze central events in our collaborations and to highlight patterns and contradictions that have emerged through ongoing collaborative analysis and PD. As we examine the structures that are in place to guide our collaboration, we consider how they mediate Teacher Leaders’ agency, with a particular lens on our relational work relative to learning as science teacher educators. We explore the ways in which a collaborative structure based on a continual process of reflection, dialogue, and action positions teacher leaders in relation to the science education curriculum and elaborate how this work provides insights into teachers’ views on children, on science, and on self, and how these views intertwine and shift over time.

Unsupervised domain adaptation with multi-level distillation boost and adaptive mask for medical image segmentation.

Unsupervised domain adaptation with multi-level distillation boost and adaptive mask for medical image segmentation.

Recruiting Teacher IF Modality for Nephropathy Diagnosis: A Customized Distillation Method With Attention-Based Diffusion Network.

The joint use of multiple modalities for medical image processing has been widely studied in recent years. The fusion of information from different modalities has demonstrated the performance improvement for a lot of medical tasks. For nephropathy diagnosis, immunofluorescence (IF) is one of the most widely-used multi-modality medical images due to its ease of acquisition and the effectiveness for certain nephropathy. However, the existing methods mainly assume different modalities have the equal effect on the diagnosis task, failing to exploit multi-modality knowledge in details. To avoid this disadvantage, this paper proposes a novel customized multi-teacher knowledge distillation framework to transfer knowledge from the trained single-modality teacher networks to a multi-modality student network. Specifically, a new attention-based diffusion network is developed for IF based diagnosis, considering global, local, and modality attention. Besides, a teacher recruitment module and diffusion-aware distillation loss are developed to learn to select the effective teacher networks based on the medical priors of the input IF sequence. The experimental results in the test and external datasets show that the proposed method has a better nephropathy diagnosis performance and generalizability, in comparison with the state-of-the-art methods.

SMART DEFENSE: A STUDENT-TEACHER FRAMEWORK FOR ADAPTIVE ADVERSARIAL ROBUSTNESS

Deep neural networks' (DNNs') dependability and security depend on defence against hostile assaults. The most advanced defence techniques available today are very resilient to hostile assaults. Nevertheless, these defence strategies are unable to differentiate between normal instances (NEs) and adversary examples (AEs). As a consequence, they execute categorisation using the same defence procedure for both instances, which degrades NE performance. In this research, we offer a new defence approach based on the student-teacher framework that can identify AEs and apply the defence process exclusively to AEs, hence minimising the deterioration of classification performance for NEs. We train the student network to predict the undistorted hidden layer features of the teacher network (target DNN), focussing on the fact that adversarial assaults would always succeed if the hidden layer features are distorted. As a consequence, our approach can identify AEs by comparing the hidden layer features of the teacher and student networks, and then use the penultimate layer characteristics that the student network predicted to retrieve the classification result of AEs. By conducting comprehensive tests on relevant image classification benchmark datasets, including as CIFAR-10, CIFAR-100, and TinyImagenet, we show that our approach outperforms state-of-the-art techniques in both detection and defence. Additionally, we demonstrate that our approach produces strong detection and defence results for a completely white-box assault, which presumes an attacker is aware of every detail of our detection and defence system.

MultiDistiller: Efficient Multimodal 3D Detection via Knowledge Distillation for Drones and Autonomous Vehicles

Real-time 3D object detection is a cornerstone for the safe operation of drones and autonomous vehicles (AVs)—drones must avoid millimeter-scale power lines in cluttered airspace, while AVs require instantaneous recognition of pedestrians and vehicles in dynamic urban environments. Although significant progress has been made in detection methods based on point clouds, cameras, and multimodal fusion, the computational complexity of existing high-precision models struggles to meet the real-time requirements of vehicular edge devices. Additionally, during the model lightweighting process, issues such as multimodal feature coupling failure and the imbalance between classification and localization performance often arise. To address these challenges, this paper proposes a knowledge distillation framework for multimodal 3D object detection, incorporating attention guidance, rank-aware learning, and interactive feature supervision to achieve efficient model compression and performance optimization. Specifically: To enhance the student model’s ability to focus on key channel and spatial features, we introduce attention-guided feature distillation, leveraging a bird’s-eye view foreground mask and a dual-attention mechanism. To mitigate the degradation of classification performance when transitioning from two-stage to single-stage detectors, we propose ranking-aware category distillation by modeling anchor-level distribution. To address the insufficient cross-modal feature extraction capability, we enhance the student network’s image features using the teacher network’s point cloud spatial priors, thereby constructing a LiDAR-image cross-modal feature alignment mechanism. Experimental results demonstrate the effectiveness of the proposed approach in multimodal 3D object detection. On the KITTI dataset, our method improves network performance by 4.89% even after reducing the number of channels by half.

Feature distance-weighted adaptive decoupled knowledge distillation for medical image segmentation.

This paper aims to apply decoupled knowledge distillation (DKD) to medical image segmentation, focusing on transferring knowledge from a high-performance teacher network to a lightweight student network, thereby facilitating model deployment on embedded devices. We initially decouple the distillation loss into pixel-wise target class knowledge distillation (PTCKD) and pixel-wise non-target class knowledge distillation (PNCKD). Subsequently, to address the limitations of the fixed weight paradigm in PTCKD, we propose a novel feature distance-weighted adaptive decoupled knowledge distillation (FDWA-DKD) method. FDWA-DKD quantifies the feature disparity between student and teacher, generating instance-level adaptive weights for PTCKD. We design a feature distance weighting (FDW) module that dynamically calculates feature distance to obtain adaptive weights, integrating feature space distance information into logit distillation. Lastly, we introduce a class-wise feature probability distribution loss to encourage the student to mimic the teacher's spatial distribution. Extensive experiments conducted on the Synapse and FLARE22 datasets demonstrate that our proposed FDWA-DKD achieves satisfactory performance, yielding optimal Dice scores and, in some instances, surpassing the performance of the teacher network. Ablation studies further validate the effectiveness of each module within our proposed method. Our method overcomes the constraints of traditional distillation methods by offering instance-level adaptive learning weights tailored to PTCKD. By quantifying student-teacher feature disparity and minimizing class-wise feature probability distribution loss, our method outperforms other distillation methods.

A semi-supervised prototypical network for prostate lesion segmentation from multimodality MRI

Objective.Prostate lesion segmentation from multiparametric magnetic resonance images is particularly challenging due to the limited availability of labeled data. This scarcity of annotated images makes it difficult for supervised models to learn the complex features necessary for accurate lesion detection and segmentation.Approach.We proposed a novel semi-supervised algorithm that embeds prototype learning into mean-teacher (MT) training to improve the feature representation for unlabeled data. In this method, pseudo-labels generated by the teacher network simultaneously serve as supervision for unlabeled prototype-based segmentation. By enabling prototype segmentation to operate across labeled and unlabeled data, the network enriches the pool of "lesion representative prototypes", and allows prototypes to flow bidirectionally-from support-to-query and query-to-support paths. This intersected, bidirectional information flow strengthens the model's generalization ability. This approach is distinct from the MT algorithm as it involves few-shot training and differs from prototypical learning for adopting unlabeled data for training.Main results.This study evaluated multiple datasets with 767 patients from three different institutions, including the publicly available PROSTATEx/PROSTATEx2 datasets as the holdout institute for reproducibility. The experimental results showed that the proposed algorithm outperformed state-of-the-art semi-supervised methods with limited labeled data, observing an improvement in Dice similarity coefficient with increasing labeled data, ranging from 0.04 to 0.09.Significance.Our method shows promise in improving segmentation outcomes with limited labeled data and potentially aiding clinicians in making informed patient treatment and management decisions88The algorithm implementation has been made available on GitHub viagit@github.com:yanwenCi/semi-proto-seg.git...

FPGA-Accelerated Lightweight CNN in Forest Fire Recognition

Using convolutional neural networks (CNNs) to recognize forest fires in complex outdoor environments is a hot research direction in the field of intelligent forest fire recognition. Due to the storage-intensive and computing-intensive characteristics of CNN algorithms, it is difficult to implement them at edge terminals with limited memory and computing resources. This paper uses a FPGA (Field-Programmable Gate Array) to accelerate CNNs to realize forest fire recognition in the field environment and solves the problem of the difficulty in giving consideration to the accuracy and speed of a forest fire recognition network in the implementation of edge terminal equipment. First, a simple seven-layer lightweight network, LightFireNet, is designed. The network is compressed using a knowledge distillation method and the classical network ResNet50 is used as the teacher network to supervise the learning of LightFireNet so that its accuracy rate reaches 97.60%. Compared with ResNet50, the scale of LightFireNet is significantly reduced. Its model parameter amount is 24 K and its calculation amount is 9.11 M, which are 0.1% and 1.2% of ResNet50, respectively. Secondly, the hardware acceleration circuit of LightFireNet is designed and implemented based on the FPGA development board ZYNQ Z7-Lite 7020. In order to further compress the network and speed up the forest fire recognition circuit, the following three methods are used to optimize the circuit: (1) the network convolution layer adopts a depthwise separable convolution structure; (2) the BN (batch normalization) layer is fused with the upper layer (or full connection layer); (3) half float or ap_fixed<16,6>-type data is used to express feature data and model parameters. After the circuit function is realized, the LightFireNet terminal circuit is obtained through the circuit parallel optimization method of loop tiling, ping-pong operation, and multi-channel data transmission. Finally, it is verified on the test dataset that the accuracy of the forest fire recognition of the FPGA edge terminal of the LightFireNet model is 96.70%, the recognition speed is 64 ms per frame, and the power consumption is 2.23 W. The results show that this paper has realized a low-power-consumption, high-accuracy, and fast forest fire recognition terminal, which can thus be better applied to forest fire monitoring.

Semi-supervised medical image segmentation based on dual swap data mixing and cross EMA strategies.

Semi-supervised medical image segmentation methods based on mean teacher (MT) framework provide a promising means for addressing the dense prediction problems with limited annotated images and numerous unlabeled images. However, the confirmation bias caused by the distribution difference between labeled and unlabeled data and the parameters-coupling problem of MT prevent the model from further improving the segmentationperformance. To reduce confirmation bias and alleviate the parameter coupling problem in MT framework, a novel data augmentation strategy and a cross exponential moving averaging (crossEMA) architecture are proposed in thiswork. Specifically, a dual swap mixing data augmentation method was first proposed, which exchanges the patches between labeled and unlabeled images twice to decrease the confirmation bias caused by distribution divergency. Subsequently, a novel architecture for both student and teacher networks was designed with structurally identical dual decoders, one of which adopted a dropout operation. Labeled, unlabeled, and mixed images are fed into this MT architecture. For unlabeled data, the pseudo-labels generated by the dual decoders of the teacher network were used to supervise the predictions of the corresponding decoders of the student network. For mixed data, the real labels of the labeled data are mixed with the pseudo-labels of the unlabeled data predicted by the teacher network to form the supervisory information, which is used to constrain the prediction consistency for mixed data between student and teacher networks. To overcome the parameter coupling problem between the student and teacher networks, the encoder parameters of the teacher network were updated using an exponential moving average (EMA) strategy, while its dual decoder parameters were updated using a cross EMA strategy, which means the perturbed decoder parameters of the student network were updated with the non-perturbed decoder parameters of the student network and viceversa. By comparing with several state-of-the-art (SOTA) semi-supervised segmentation methods on four publicly available datasets, we validated that the proposed method outperforms existing models. The Dice similarity coefficient (DSC) and volume similarity (VS) were improved by at least 2.33% and 1.86%, respectively, compared to the corresponding sub-optimal methods. Through multiple ablation experiments, we verified that the proposed dual swap strategy can reduce the distributional differences between unlabeled data and labeled+mixed data. In addition, the cross EMA strategy can avoid early convergence of the student and teachernetworks. The proposed strategies can alleviate the confirmation bias caused by the distribution discrepancy between labeled and unlabeled data in semi-supervised learning, as well as the issue of parameter coupling between the student and teacher networks in the MT architecture, providing therefore a promising approach to semi-supervised medical image segmentation.

Hybrid Data-Free Knowledge Distillation

Data-free knowledge distillation aims to learn a compact student network from a pre-trained large teacher network without using the original training data of the teacher network. Existing collection-based and generation-based methods train student networks by collecting massive real examples and generating synthetic examples, respectively. However, they inevitably become weak in practical scenarios due to the difficulties in gathering or emulating sufficient real-world data. To solve this problem, we propose a novel method called Hybrid Data-Free Distillation (HiDFD), which leverages only a small amount of collected data as well as generates sufficient examples for training student networks. Our HiDFD comprises two primary modules, i.e., the teacher-guided generation and student distillation. The teacher-guided generation module guides a Generative Adversarial Network (GAN) by the teacher network to produce high-quality synthetic examples from very few real-world collected examples. Specifically, we design a feature integration mechanism to prevent the GAN from overfitting and facilitate the reliable representation learning from the teacher network. Meanwhile, we drive a category frequency smoothing technique via the teacher network to balance the generative training of each category. In the student distillation module, we explore a data inflation strategy to properly utilize a blend of real and synthetic data to train the student network via a classifier-sharing-based feature alignment technique. Intensive experiments across multiple benchmarks demonstrate that our HiDFD can achieve state-of-the-art performance using 120 times less collected data than existing methods.

Lightweight Contrastive Distilled Hashing for Online Cross-modal Retrieval

Deep online cross-modal hashing has gained much attention from researchers recently, as its promising applications with low storage requirement, fast retrieval efficiency and cross modality adaptive, etc. However, there still exists some technical hurdles that hinder its applications, e.g., 1) how to extract the coexistent semantic relevance of cross-modal data, 2) how to achieve competitive performance when handling the real time data streams, 3) how to transfer the knowledge learned from offline to online training in a lightweight manner. To address these problems, this paper proposes a lightweight contrastive distilled hashing (LCDH) for cross-modal retrieval, by innovatively bridging the offline and online cross-modal hashing by similarity matrix approximation in a knowledge distillation framework. Specifically, in the teacher network, LCDH first extracts the cross-modal features by CLIP, which are further fed into an attention module for representation enhancement after feature fusion. Then, the output of the attention module is fed into a FC layer to obtain hash codes for aligning the sizes of similarity matrices for online and offline training. In the student network, LCDH extracts the visual and textual features by lightweight models, and then the features are fed into a FC layer to generate binary codes. Finally, by approximating the similarity matrices, the performance of online hashing in the lightweight student network can be enhanced by the supervision of coexistent semantic relevance that is distilled from the teacher network. Experimental results on three widely used datasets demonstrate that LCDH outperforms some state-of-the-art methods.

SCKD: Semi-Supervised Cross-Modality Knowledge Distillation for 4D Radar Object Detection

3D object detection is one of the fundamental perception tasks for autonomous vehicles. Fulfilling such a task with a 4D millimeter-wave radar is very attractive since the sensor is able to acquire 3D point clouds similar to Lidar while maintaining robust measurements under adverse weather. However, due to the high sparsity and noise associated with the radar point clouds, the performance of the existing methods is still much lower than expected. In this paper, we propose a novel Semi-supervised Cross-modality Knowledge Distillation (SCKD) method for 4D radar-based 3D object detection. It characterizes the capability of learning the feature from a Lidar-radar-fused teacher network with semi-supervised distillation. We first propose an adaptive fusion module in the teacher network to boost its performance. Then, two feature distillation modules are designed to facilitate the cross-modality knowledge transfer. Finally, a semi-supervised output distillation is proposed to increase the effectiveness and flexibility of the distillation framework. With the same network structure, our radar-only student trained by SCKD boosts the mAP by 10.38% over the baseline and outperforms the state-of-the-art works on the VoD dataset. The experiment on ZJUODset also shows 5.12% mAP improvements on the moderate difficulty level over the baseline when extra unlabeled data are available.

DDML: Multi-Student Knowledge Distillation for Hate Speech.

Recent studies have shown that hate speech on social media negatively impacts users' mental health and is a contributing factor to suicide attempts. On a broader scale, online hate speech can undermine social stability. With the continuous growth of the internet, the prevalence of online hate speech is rising, making its detection an urgent issue. Recent advances in natural language processing, particularly with transformer-based models, have shown significant promise in hate speech detection. However, these models come with a large number of parameters, leading to high computational requirements and making them difficult to deploy on personal computers. To address these challenges, knowledge distillation offers a solution by training smaller student networks using larger teacher networks. Recognizing that learning also occurs through peer interactions, we propose a knowledge distillation method called Deep Distill-Mutual Learning (DDML). DDML employs one teacher network and two or more student networks. While the student networks benefit from the teacher's knowledge, they also engage in mutual learning with each other. We trained numerous deep neural networks for hate speech detection based on DDML and demonstrated that these networks perform well across various datasets. We tested our method across ten languages and nine datasets. The results demonstrate that DDML enhances the performance of deep neural networks, achieving an average F1 score increase of 4.87% over the baseline.

Unlocking the Potential of Reverse Distillation for Anomaly Detection

Knowledge Distillation (KD) is a promising approach for unsupervised Anomaly Detection (AD). However, the student network's over-generalization often diminishes the crucial representation differences between teacher and student in anomalous regions, leading to detection failures. To address this problem, the widely accepted Reverse Distillation (RD) paradigm designs the asymmetry teacher and student network, using an encoder as teacher and a decoder as student. Yet, the design of RD does not ensure that the teacher encoder effectively distinguishes between normal and abnormal features or that the student decoder generates anomaly-free features. Additionally, the absence of skip connections results in a loss of fine details during feature reconstruction. To address these issues, we propose RD with Expert, which introduces a novel Expert-Teacher-Student network for simultaneous distillation of both the teacher encoder and student decoder. The added expert network enhances the student's ability to generate normal features and optimizes the teacher's differentiation between normal and abnormal features, reducing missed detections. Additionally, Guided Information Injection is designed to filter and transfer features from teacher to student, improving detail reconstruction and minimizing false positives. Experiments on several benchmarks prove that our method outperforms existing unsupervised AD methods under RD paradigm, fully unlocking RD’s potential.

Filter or Compensate: Towards Invariant Representation from Distribution Shift for Anomaly Detection

Recent Anomaly Detection (AD) methods have achieved great success with In-Distribution (ID) data. However, real-world data often exhibits distribution shift, causing huge performance decay on traditional AD methods. From this perspective, few previous work has explored AD with distribution shift, and the distribution-invariant normality learning has been proposed based on the Reverse Distillation (RD) framework. However, we observe the misalignment issue between the teacher and the student network that causes detection failure, thereby propose FiCo, Filter or Compensate, to address the distribution shift issue in AD. FiCo firstly compensates the distribution-specific information to reduce the misalignment between the teacher and student network via the Distribution-Specific Compensation (DiSCo) module, and secondly filters all abnormal information to capture distribution-invariant normality with the Distribution-Invariant Filter (DiIFi) module. Extensive experiments on three different AD benchmarks demonstrate the effectiveness of FiCo, which outperforms all existing state-of-the-art (SOTA) methods, and even achieves better results on the ID scenario compared with RD-based methods.