Reconstruction-based Approaches Research Articles

Out-of-distribution detection in digital pathology: Do foundation models bring the end to reconstruction-based approaches?

Artificial intelligence (AI) has shown promising results for computational pathology tasks. However, one of the limitations in clinical practice is that these algorithms are optimised for the distribution represented by the training data. For out-of-distribution (OOD) data, they often deliver predictions with equal confidence, even though these often are incorrect. In the pursuit of OOD detection in digital pathology, this study evaluates the state-of-the-art (SOTA) in computational pathology OOD detection, based on diffusion probabilistic models, specifically by adapting the latent diffusion model (LDM) for this purpose (AnoLDM). We compare this against post-hoc methods based on the latent space of foundation models, which are SOTA in general computer vision research. The approaches are not only evaluated on data from the same medical centres as the training set, but also on several datasets with data distribution shifts. The results show that AnoLDM performs similarly well or better than diffusion model based approaches published in previous studies in computational pathology but with reduced computational costs. However, our optimal configuration of an approach based on foundation models (kang_residual) outperforms AnoLDM on OOD detection on data not experiencing any covariate shifts, with an AUROC of 96.17 versus 91.86. Interestingly, AnoLDM is more successful at handling the data distribution shifts investigated in this study. However, both AnoLDM and kang_residual suffer substantial loss in the performance under the data distribution shifts, hence future work should focus on improving the generalisation of OOD detection for computational pathology applications.

An Iterative Method for Unsupervised Robust Anomaly Detection Under Data Contamination.

Most deep anomaly detection models are based on learning normality from datasets due to the difficulty of defining abnormality by its diverse and inconsistent nature. Therefore, it has been a common practice to learn normality under the assumption that anomalous data are absent in a training dataset, which we call normality assumption. However, in practice, the normality assumption is often violated due to the nature of real data distributions that includes anomalous tails, i.e., a contaminated dataset. Thereby, the gap between the assumption and actual training data affects detrimentally in learning of an anomaly detection model. In this work, we propose a learning framework to reduce this gap and achieve better normality representation. Our key idea is to identify sample-wise normality and utilize it as an importance weight, which is updated iteratively during the training. Our framework is designed to be model-agnostic and hyperparameter insensitive so that it applies to a wide range of existing methods without careful parameter tuning. We apply our framework to three different representative approaches of deep anomaly detection that are classified into one-class classification-, probabilistic model-, and reconstruction-based approaches. In addition, we address the importance of a termination condition for iterative methods and propose a termination criterion inspired by the anomaly detection objective. We validate that our framework improves the robustness of the anomaly detection models under different levels of contamination ratios on five anomaly detection benchmark datasets and two image datasets. On various contaminated datasets, our framework improves the performance of three representative anomaly detection methods, measured by area under the ROC curve.

A Diffusion-Based Framework for Multi-Class Anomaly Detection

Reconstruction-based approaches have achieved remarkable outcomes in anomaly detection. The exceptional image reconstruction capabilities of recently popular diffusion models have sparked research efforts to utilize them for enhanced reconstruction of anomalous images. Nonetheless, these methods might face challenges related to the preservation of image categories and pixel-wise structural integrity in the more practical multi-class setting. To solve the above problems, we propose a Difusion-based Anomaly Detection (DiAD) framework for multi-class anomaly detection, which consists of a pixel-space autoencoder, a latent-space Semantic-Guided (SG) network with a connection to the stable diffusion’s denoising network, and a feature-space pre-trained feature extractor. Firstly, The SG network is proposed for reconstructing anomalous regions while preserving the original image’s semantic information. Secondly, we introduce Spatial-aware Feature Fusion (SFF) block to maximize reconstruction accuracy when dealing with extensively reconstructed areas. Thirdly, the input and reconstructed images are processed by a pre-trained feature extractor to generate anomaly maps based on features extracted at different scales. Experiments on MVTec-AD and VisA datasets demonstrate the effectiveness of our approach which surpasses the state-of-the-art methods, e.g., achieving 96.8/52.6 and 97.2/99.0 (AUROC/AP) for localization and detection respectively on multi-class MVTec-AD dataset. Code will be available at https://lewandofskee.github.io/projects/diad.

Industrial Product Surface Anomaly Detection with Realistic Synthetic Anomalies Based on Defect Map Prediction.

The occurrence of anomalies on the surface of industrial products can lead to issues such as decreased product quality, reduced production efficiency, and safety hazards. Early detection and resolution of these problems are crucial for ensuring the quality and efficiency of production. The key challenge in applying deep learning to surface defect detection of industrial products is the scarcity of defect samples, which will make supervised learning methods unsuitable for surface defect detection problems. Therefore, it is a reasonable solution to use anomaly detection methods to deal with surface defect detection. Among image-based anomaly detection, reconstruction-based methods are the most commonly used. However, reconstruction-based approaches lack the involvement of defect samples in the training process, posing the risk of a perfect reconstruction of defects by the reconstruction network. In this paper, we propose a reconstruction-based defect detection algorithm that addresses these challenges by utilizing more realistic synthetic anomalies for training. Our model focuses on creating authentic synthetic defects and introduces an auto-encoder image reconstruction network with deep feature consistency constraints, as well as a defect separation network with a large receptive field. We conducted experiments on the challenging MVTec anomaly detection dataset and our trained model achieved an AUROC score of 99.70% and an average precision (AP) score of 99.87%. Our method surpasses recently proposed defect detection algorithms, thereby enhancing the accuracy of surface defect detection in industrial products.

Deep temporal clustering features for speech emotion recognition

Deep clustering is a popular unsupervised technique for feature representation learning. We recently proposed the chunk-based DeepEmoCluster framework for speech emotion recognition (SER) to adopt the concept of deep clustering as a novel semi-supervised learning (SSL) framework, which achieved improved recognition performances over conventional reconstruction-based approaches. However, the vanilla DeepEmoCluster lacks critical sentence-level temporal information that is useful for SER tasks. This study builds upon the DeepEmoCluster framework, creating a powerful SSL approach that leverages temporal information within a sentence. We propose two sentence-level temporal modeling alternatives using either the temporal-net or the triplet loss function, resulting in a novel temporal-enhanced DeepEmoCluster framework to capture essential temporal information. The key contribution to achieving this goal is the proposed sentence-level uniform sampling strategy, which preserves the original temporal order of the data for the clustering process. An extra network module (e.g., gated recurrent unit) is utilized for the temporal-net option to encode temporal information across the data chunks. Alternatively, we can impose additional temporal constraints by using the triplet loss function while training the DeepEmoCluster framework, which does not increase model complexity. Our experimental results based on the MSP-Podcast corpus demonstrate that the proposed temporal-enhanced framework significantly outperforms the vanilla DeepEmoCluster framework and other existing SSL approaches in regression tasks for the emotional attributes arousal, dominance, and valence. The improvements are observed in fully-supervised learning or SSL implementations. Further analyses validate the effectiveness of the proposed temporal modeling, showing (1) high temporal consistency in the cluster assignment, and (2) well-separated emotional patterns in the generated clusters.

CL-TAD: A Contrastive-Learning-Based Method for Time Series Anomaly Detection

Anomaly detection has gained increasing attention in recent years, but detecting anomalies in time series data remains challenging due to temporal dynamics, label scarcity, and data diversity in real-world applications. To address these challenges, we introduce a novel method for anomaly detection in time series data, called CL-TAD (Contrastive-Learning-based method for Times series Anomaly Detection), which employs a contrastive-learning-based representation learning technique. Inspired by the successes of reconstruction-based approaches and contrastive learning approaches, the proposed method seeks to leverage these approaches for time series anomaly detection. The CL-TAD method is comprised of two main components: positive sample generation and contrastive-learning-based representation learning. The former component generates positive samples by trying to reconstruct the original data from masked samples. These positive samples, in conjunction with the original data, serve as input for the contrastive-learning-based representation learning component. The representations of input original data and their masked data are used to detect anomalies later on. Experimental results have demonstrated that the CL-TAD method achieved the best performance on five datasets out of nine benchmark datasets over 10 other recent methods. By leveraging the reconstruction learning and contrastive learning techniques, our method offers a promising solution for effectively detecting anomalies in time series data by handling the issues raised by label scarcity and data diversity, delivering high performance.

Unsupervised Domain Adaptation for Mitigating Sensor Variability and Interspecies Heterogeneity in Animal Activity Recognition.

Animal activity recognition (AAR) using wearable sensor data has gained significant attention due to its applications in monitoring and understanding animal behavior. However, two major challenges hinder the development of robust AAR models: domain variability and the difficulty of obtaining labeled datasets. To address this issue, this study intensively investigates the impact of unsupervised domain adaptation (UDA) for AAR. We compared three distinct types of UDA techniques: minimizing divergence-based, adversarial-based, and reconstruction-based approaches. By leveraging UDA, AAR classifiers enable the model to learn domain-invariant features, allowing classifiers trained on the source domain to perform well on the target domain without labels. We evaluated the effectiveness of UDA techniques using dog movement sensor data and additional data from horses. The application of UDA across sensor positions (neck and back), sizes (middle-sized and large-sized), and gender (female and male) within the dog data, as well as across species (dog and horses), exhibits significant improvements in the classification performance and reduced the domain discrepancy. The results highlight the potential of UDA to mitigate the domain shift and enhance AAR in various settings and for different animal species, providing valuable insights for practical applications in real-world scenarios where labeled data is scarce.

Self-supervised cross-modal visual retrieval from brain activities

We study the problem of restoring visual stimuli from visually-evoked electroencephalography (EEG) signals. Using a supervised classification-then-generation framework, the reconstruction-based approaches learn the mapping between distributions of two modalities but fail to reproduce the exact visual stimulus. Instead, we propose a self-supervised cross-modal retrieval paradigm that seeks instance-level alignment by maximizing the mutual information between the EEG encoding and associated visual stimulus. We demonstrate the threefold advantages of the self-supervised retrieval over supervised reconstruction on the largest visual-evoked EEG dataset with two evaluation protocols. First, it restores the exact visual stimulus without accessing the image class information, which was not possible with previous approaches. Second, it produces more recognizable results than generated ones and bypasses the challenge of training an image generator. Finally, it illustrates the benefits of self-supervision over supervised models in handling open-set data.

Nonlinear process monitoring based on generic reconstruction-based auto-associative neural network

A significant concern with statistical fault diagnosis is the large number of false alarms caused by the smearing effect. Although the reconstruction-based approach effectively solves this problem, most of them only focus on linear rather than nonlinear systems. In the present work, a generic reconstruction-based auto-associative neural network (GRBAANN) is proposed that uses the reconstruction-based approach to isolate simple and complex faults for nonlinear systems. Nevertheless, in GRBAANN, it is challenging to acquire a trivial solution for the reconstruction-based index, which is equivalent to a complex vector fixed-point problem. In this regard, the Steffensen method is employed to deal with this problem with an accelerated iterative process, which is appropriate for both single and multiple variable faults. The variable selection procedure is time-consuming but imperative for reconstruction-based approaches, with no exception to the proposed method. In order to ensure the real-time diagnosis for large-scale systems, the Sequential floating forward selection method with memory is proposed to minimize the computation time of the variable selection procedure. The effectiveness of the proposed GRBAANN scheme is illustrated through a validation example and an industrial example. Comparisons with the state-of-art methods are also presented.

PseudoBound: Limiting the anomaly reconstruction capability of one-class classifiers using pseudo anomalies

Due to the rarity of anomalous events, video anomaly detection is typically approached as one-class classification (OCC) problem. Typically in OCC, an autoencoder (AE) is trained to reconstruct the normal only training data with the expectation that, in test time, it can poorly reconstruct the anomalous data. However, previous studies have shown that, even trained with only normal data, AEs can often reconstruct anomalous data as well, resulting in a decreased performance. To mitigate this problem, we propose to limit the anomaly reconstruction capability of AEs by incorporating pseudo anomalies during the training of an AE. Extensive experiments using five types of pseudo anomalies show the robustness of our training mechanism towards any kind of pseudo anomaly. Moreover, we demonstrate the effectiveness of our proposed pseudo anomaly based training approach against several existing state-of-the-art (SOTA) methods on three benchmark video anomaly datasets, outperforming all the other reconstruction-based approaches in two datasets and showing the second best performance in the other dataset.

Double Prior Network for Multidegradation Remote Sensing Image Super-Resolution

Image super-resolution (SR) is widely used in remote sensing because it can effectively increase image details. Neural networks have shown remarkable performance in recent years, benefitting from their end-to-end training. However, remote sensing images contain a variety of degradation factors. Neural networks lack flexibility in dealing with these complex issues compared with reconstruction-based approaches. Traditional neural network methods can not take advantage of prior knowledge and lack interpretability. To develop a flexible, accurate, and interpretable algorithm for remote sensing SR, we proposed an effective SR network called YSRNet. It is performed by unfolding a traditional optimization process into a learnable network. Combining conventional reconstruction-based methods and neural networks can significantly improve the algorithm's performance. Since the gradient features of remote sensing images contain valuable information, the total variation (TV) constraints and the deep prior constraints are introduced into the objective function for image SR. Furthermore, we propose an enhanced version called YSRNet+, which can apply attention weights to different prior terms and channels. Compared with the YSRNet, the YSRNet+ enables networks to focus more on useful prior information and improve the interpretability of networks. Experiments on three remote sensing datasets are performed to evaluate the algorithm's effectiveness. The experimental results demonstrate that the proposed algorithm performs better than some state-of-the-art neural network algorithms, especially in the scenario of the multi-degradation factors.

Distributional Gaussian Processes Layers for Out-of-Distribution Detection

Machine learning models deployed on medical imaging tasks must be equipped with out-of-distribution detection capabilities in order to avoid erroneous predictions. It is unsure whether out-of-distribution detection models reliant on deep neural networks are suitable for detecting domain shifts in medical imaging. Gaussian Processes can reliably separate in-distribution data points from out-of-distribution data points via their mathematical construction. Hence, we propose a parameter efficient Bayesian layer for hierarchical convolutional Gaussian Processes that incorporates Gaussian Processes operating in Wasserstein-2 space to reliably propagate uncertainty. This directly replaces convolving Gaussian Processes with a distance-preserving affine operator on distributions. Our experiments on brain tissue-segmentation show that the resulting architecture approaches the performance of well-established deterministic segmentation algorithms (U-Net), which has not been achieved with previous hierarchical Gaussian Processes. Moreover, by applying the same segmentation model to out-of-distribution data (i.e., images with pathology such as brain tumors), we show that our uncertainty estimates result in out-of-distribution detection that outperforms the capabilities of previous Bayesian networks and reconstruction-based approaches that learn normative distributions.

Improved Anomaly Detection Based on Image Reconstruction and Global Template Features for Industrial Products

Anomaly detection in industry applications is a challenging problem when negative (defective) samples are unavailable, especially in the case where there are missing parts or foreign objects occupied a relatively large region. Conventional reconstruction-based approaches cannot guarantee the restored image being a normal one, leading to poor segmentation results. In this work, we propose an unsupervised anomaly detection approach to tackle the problem of large-area anomaly detection by incorporating global template features into an Auto-Encoder like reconstruction model. In particular, our model infers the value of each pixel based on both the surrounding local-neighborhood information and the global information encoded at the same pixel position. During the reconstruction phase, any abnormal features are then replaced with normal ones, avoiding over-reconstruction of large-area abnormalities. The experimental results in comparison with other methods demonstrate its effectiveness for industrial anomaly detection.

Latent-Insensitive Autoencoders for Anomaly Detection

Reconstruction-based approaches to anomaly detection tend to fall short when applied to complex datasets with target classes that possess high inter-class variance. Similar to the idea of self-taught learning used in transfer learning, many domains are rich with similar unlabeled datasets that could be leveraged as a proxy for out-of-distribution samples. In this paper we introduce the latent-insensitive autoencoder (LIS-AE) where unlabeled data from a similar domain are utilized as negative examples to shape the latent layer (bottleneck) of a regular autoencoder such that it is only capable of reconstructing one task. We provide theoretical justification for the proposed training process and loss functions along with an extensive ablation study highlighting important aspects of our model. We test our model in multiple anomaly detection settings presenting quantitative and qualitative analysis showcasing the significant performance improvement of our model for anomaly detection tasks.

Body Size Measurement Using a Smartphone

Measuring body sizes accurately and rapidly for optimal garment fit detection has been a challenge for fashion retailers. Especially for apparel e-commerce, there is an increasing need for digital and convenient ways to obtain body measurements to provide their customers with correct-fitting products. However, the currently available methods depend on cumbersome and complex 3D reconstruction-based approaches. In this paper, we propose a novel smartphone-based body size measurement method that does not require any additional objects of a known size as a reference when acquiring a subject’s body image using a smartphone. The novelty of our proposed method is that it acquires measurement positions using body proportions and machine learning techniques, and it performs 3D reconstruction of the body using measurements obtained from two silhouette images. We applied our proposed method to measure body sizes (i.e., waist, lower hip, and thigh circumferences) of males and females for selecting well-fitted pants. The experimental results show that our proposed method gives an accuracy of 95.59% on average when estimating the size of the waist, lower hip, and thigh circumferences. Our proposed method is expected to solve issues with digital body measurements and provide a convenient garment fit detection solution for online shopping.

Robust Graph Structure Learning for Multimedia Data Analysis

With the rapid development of computer network technology, we can acquire a large amount of multimedia data, and it becomes a very important task to analyze these data. Since graph construction or graph learning is a powerful tool for multimedia data analysis, many graph‐based subspace learning and clustering approaches have been proposed. Among the existing graph learning algorithms, the sample reconstruction‐based approaches have gone the mainstream. Nevertheless, these approaches not only ignore the local and global structure information but also are sensitive to noise. To address these limitations, this paper proposes a graph learning framework, termed Robust Graph Structure Learning (RGSL). Different from the existing graph learning approaches, our approach adopts the self‐expressiveness of samples to capture the global structure, meanwhile utilizing data locality to depict the local structure. Specially, in order to improve the robustness of our approach against noise, we introduce l2,1‐norm regularization criterion and nonnegative constraint into the graph construction process. Furthermore, an iterative updating optimization algorithm is designed to solve the objective function. A large number of subspace learning and clustering experiments are carried out to verify the effectiveness of the proposed approach.

Detecting anomalies in business process event logs using statistical leverage

The presence of anomalous information in a business process event log, such as missing, duplicated or swapped events, hampers the possibility of extracting useful insights from event log analysis. A number of approaches exist in the literature to detect anomalous cases in event logs based on different paradigms, such as probabilistic, distance-based or reconstruction-based anomaly detection. This paper proposes a novel method for anomaly detection in event logs based on the information-theoretic paradigm, which has not been considered before in event log anomaly detection. In particular, we propose an anomaly score for cases of a process based on statistical leverage and three different methods to set the anomaly detection threshold. The proposed approach does not require large data sets to train machine learning models, which are necessary for instance in reconstruction-based approaches. The proposed approach shows remarkable anomaly detection capability in experiments conducted using publicly available event logs in respect of existing methods in the literature. One of the proposed anomaly detection thresholds also shows to handle variable case anomaly ratios more effectively than other methods in the literature.

A Frequency-Domain SPICE Approach to High-Resolution Time Delay Estimation

We propose a frequency-domain sparse iterative covariance-based estimation (FD-SPICE) approach for the high-resolution time delay estimation of spread spectrum signals. The conventional correlation method fails to resolve the signals in the case where they are spaced within a Rayleigh resolution limit and further processing should be applied to the correlation outputs in order to improve performance. SPICE is one of the most prominent candidates for further processing since it has various advantages over subspace-based approaches. SPICE is implemented normally in the time domain like other sparse signal reconstruction-based approaches. However, the time-domain SPICE (TD-SPICE) approach suffers from a severe performance degradation in coherent signal cases, as in direction-of-arrival estimation. We develop an FD-SPICE approach which makes use of the Fourier transform bins of the correlation output samples to form a sample covariance matrix. Numerical examples demonstrate the superiority of FD-SPICE over TD-SPICE, especially in coherent signal cases.

SGCRSR: Sequential gradient constrained regression for single image super-resolution

Single image super-resolution (SISR), which aims to produce an image with higher resolution and better visual quality from the given single low-resolution (LR) image, has attracted extensive attention in recent years. In particular, the regression-based SISR approaches, which learn the mapping between LR and high-resolution (HR) patch pairs, are efficient and effective as a whole. However, the super-resolved images produced by this kind of method often suffer from visual artifacts as no extra constraints or priors are enforced. To alleviate these shortcomings, we propose a Sequential Gradient Constrained Regression-based single image Super-Resolution (SGCRSR) framework, which provides an effective way to combine the conventional learning-based and reconstruction-based approaches. Firstly, we improve the performance of the well-known super-resolution (SR) method A+ by addressing its deficiencies in both training and testing stages and propose the enhanced A+ (EA+). Then, the EA+ model is applied in dual intensity–gradient domain to construct the Gradient Constrained Regression (GCR)-based SR unit. Finally, a GCR-based sequential SR framework, namely the SGCRSR, is established to improve the quality of super-resolved images gradually. Extensive experiments show that the proposed SGCRSR achieves better performance than several state-of-the-art SR algorithms.

Super-resolution for biometrics: A comprehensive survey

The lack of resolution of imaging systems has critically adverse impacts on the recognition and performance of biometric systems, especially in the case of long range biometrics and surveillance such as face recognition at a distance, iris recognition and gait recognition. Super-resolution, as one of the core innovations in computer vision, has been an attractive but challenging solution to address this problem in both general imaging systems and biometric systems. However, a fundamental difference exists between conventional super-resolution motivations and those required for biometrics. The former aims to enhance the visual clarity of the scene while the latter, more significantly, aims to improve the recognition accuracy of classifiers by exploiting specific characteristics of the observed biometric traits. This paper comprehensively surveys the state-of-the-art super-resolution approaches proposed for four major biometric modalities: face (2D+3D), iris, fingerprint and gait. We approach the super-resolution problem in biometrics from several different perspectives, including from the spatial and frequency domains, single and multiple input images, learning-based and reconstruction-based approaches. Especially, we highlight two special categories: feature-domain super-resolution which performs super-resolution directly on the feature space to purposely improve the recognition performance, and deep-learning super-resolution which discusses the most recent advances in deep learning for the super-resolution task. Finally, we discuss the current and open research challenges and provide recommendations into the future for the improved use of super-resolution with biometrics.