Corrupted Input Data Research Articles

Domain Generalization with Vital Phase Augmentation

Deep neural networks have shown remarkable performance in image classification. However, their performance significantly deteriorates with corrupted input data. Domain generalization methods have been proposed to train robust models against out-of-distribution data. Data augmentation in the frequency domain is one of such approaches that enable a model to learn phase features to establish domain-invariant representations. This approach changes the amplitudes of the input data while preserving the phases. However, using fixed phases leads to susceptibility to phase fluctuations because amplitudes and phase fluctuations commonly occur in out-of-distribution. In this study, to address this problem, we introduce an approach using finite variation of the phases of input data rather than maintaining fixed phases. Based on the assumption that the degree of domain-invariant features varies for each phase, we propose a method to distinguish phases based on this degree. In addition, we propose a method called vital phase augmentation (VIPAug) that applies the variation to the phases differently according to the degree of domain-invariant features of given phases. The model depends more on the vital phases that contain more domain-invariant features for attaining robustness to amplitude and phase fluctuations. We present experimental evaluations of our proposed approach, which exhibited improved performance for both clean and corrupted data. VIPAug achieved SOTA performance on the benchmark CIFAR-10 and CIFAR-100 datasets, as well as near-SOTA performance on the ImageNet-100 and ImageNet datasets. Our code is available at https://github.com/excitedkid/vipaug.

Deep feature fusion‐based stacked denoising autoencoder for tag recommendation systems

AbstractWith the rapid development of artificial intelligence technology, commercial robots have gradually entered our daily lives. In order to promote product dissemination, shopping guide robots are a new service options of commerce platforms that use tag recommendation systems to identify users' intentions. A large number of applications combine user historical tagging information with the multi‐round dialogue ability of shopping guide robots to help users efficiently search for and retrieve products of interest. Recently, tensor decomposition methods have become a common approach for modelling entity interaction relationships in tag recommendation systems. However, due to the sparsity of data, these methods only consider low‐order information of entities, making it difficult to capture the higher‐order collaborative signals among entities. Recommendation methods by autoencoders can effectively extract abstract feature representations while they only focus on the two‐dimensional relationship between users and items, ignoring the interaction relationship among users, items and tags in real complex recommendation scenarios. The authors focus on modelling the similarity relationship among entities and propose a method called deep feature fusion tag (DFFT) based on the deep feature fusion of stacked denoising autoencoders. This method can extract high‐order information with different embedding dimensions and fuse them in a unified framework. To extract robust feature representations, the authors inject random noise (mask‐out/drop‐out noise) into the tag information corresponding to users and items to generate corrupted input data, and then utilise autoencoders to encode the interaction relationship among entities. To further obtain the interaction relationship with different dimensions, different encoding layers are stacked and combined to produce a better expanded model which can reinforce each other. Finally, a decoding component is used to reconstruct the original input data. According to the experimental results on two common datasets, the proposed DFFT method outperforms other baselines in terms of the F1@N, NDCG@N and Recall@N evaluation metrics.

Sparse Principal Component Analysis via Variable Projection

Sparse principal component analysis (SPCA) has emerged as a powerful technique for modern data analysis, providing improved interpretation of low-rank structures by identifying localized spatial structures in the data and disambiguating between distinct time scales. We demonstrate a robust and scalable SPCA algorithm by formulating it as a value-function optimization problem. This viewpoint leads to a flexible and computationally efficient algorithm. The approach can further leverage randomized methods from linear algebra to extend SPCA to the large-scale (big data) setting. Our proposed innovation also allows for a robust SPCA formulation which obtains meaningful sparse principal components in spite of grossly corrupted input data. The proposed algorithms are demonstrated using both synthetic and real world data, and show exceptional computational efficiency and diagnostic performance.

Unsupervised learning in probabilistic neural networks with multi-state metal-oxide memristive synapses.

In an increasingly data-rich world the need for developing computing systems that cannot only process, but ideally also interpret big data is becoming continuously more pressing. Brain-inspired concepts have shown great promise towards addressing this need. Here we demonstrate unsupervised learning in a probabilistic neural network that utilizes metal-oxide memristive devices as multi-state synapses. Our approach can be exploited for processing unlabelled data and can adapt to time-varying clusters that underlie incoming data by supporting the capability of reversible unsupervised learning. The potential of this work is showcased through the demonstration of successful learning in the presence of corrupted input data and probabilistic neurons, thus paving the way towards robust big-data processors.

Image Transformation Based on Learning Dictionaries across Image Spaces

In this paper, we propose a framework of transforming images from a source image space to a target image space, based on learning coupled dictionaries from a training set of paired images. The framework can be used for applications such as image super-resolution and estimation of image intrinsic components (shading and albedo). It is based on a local parametric regression approach, using sparse feature representations over learned coupled dictionaries across the source and target image spaces. After coupled dictionary learning, sparse coefficient vectors of training image patch pairs are partitioned into easily retrievable local clusters. For any test image patch, we can fast index into its closest local cluster and perform a local parametric regression between the learned sparse feature spaces. The obtained sparse representation (together with the learned target space dictionary) provides multiple constraints for each pixel of the target image to be estimated. The final target image is reconstructed based on these constraints. The contributions of our proposed framework are three-fold. 1) We propose a concept of coupled dictionary learning based on coupled sparse coding which requires the sparse coefficient vectors of a pair of corresponding source and target image patches to have the same support, i.e., the same indices of nonzero elements. 2) We devise a space partitioning scheme to divide the high-dimensional but sparse feature space into local clusters. The partitioning facilitates extremely fast retrieval of closest local clusters for query patches. 3) Benefiting from sparse feature-based image transformation, our method is more robust to corrupted input data, and can be considered as a simultaneous image restoration and transformation process. Experiments on intrinsic image estimation and super-resolution demonstrate the effectiveness and efficiency of our proposed method.

Feature-Preserving MRI Denoising: A Nonparametric Empirical Bayes Approach

This paper presents a novel method for Bayesian denoising of magnetic resonance (MR) images that bootstraps itself by inferring the prior, i.e., the uncorrupted-image statistics, from the corrupted input data and the knowledge of the Rician noise model. The proposed method relies on principles from empirical Bayes (EB) estimation. It models the prior in a nonparametric Markov random field (MRF) framework and estimates this prior by optimizing an information-theoretic metric using the expectation-maximization algorithm. The generality and power of nonparametric modeling, coupled with the EB approach for prior estimation, avoids imposing ill-fitting prior models for denoising. The results demonstrate that, unlike typical denoising methods, the proposed method preserves most of the important features in brain MR images. Furthermore, this paper presents a novel Bayesian-inference algorithm on MRFs, namely iterated conditional entropy reduction (ICER). This paper also extends the application of the proposed method for denoising diffusion-weighted MR images. Validation results and quantitative comparisons with the state of the art in MR-image denoising clearly depict the advantages of the proposed method.

Neural network architectures for content-addressable memory

The paper investigates whether neural network content-addressable memories (CAMs) can compete with the non-neural alternatives which are currently available. The storage and retrieval of 64-bit patterns is used as a test problem which reflects the requirements of today's computer technology. The two main strategies available for implementing a CAM with a neural network architecture, feedback networks and twostage CAMs, and in particular their ability to retrieve patterns from corrupted input data, are investigated in detail. The storage capacity of the Hopfield network is very poor although it can be improved with the use of an iterative algorithm, such as the threshold algorithm which is described in this paper. However, the possibility of generating spurious patterns always remains with feedback networks. Two-stage CAMs are much more efficient, provided that an appropriate algorithm is used for the input classification stage. The wellknown perceptron and least-mean squares algorithms need to be modified if they are to cope with corrupted input patterns, but the optimal classifier for the type of problem under consideration is the minimum-distance classifier (or Hamming network for binary patterns). The implementation of the latter in analogue VLSI is discussed in the last section of the paper as an alternative to conventional technology.