Minimum Reconstruction Error Research Articles

Cosine 2DPCA With Weighted Projection Maximization.

Two-dimensional principal component analysis (2DPCA), known to be very sensitive to outliers, employs the square F -norm as the distance metric and only satisfies the optimal objective of maximizing projection variance. However, the objective of minimizing reconstruction errors for all samples is not optimized as much as possible. To handle the problem, a novel cosine objective function is first presented for maximizing weighted projection, in which the 2-norm of vectors with an adjustable power parameter is employed as the distance metric. Not only the objective with the maximum projection distance is accomplished in the cosine objective function, but also the objective with the minimum sum of reconstruction errors is also optimized indirectly. Then, the cosine 2DPCA (Cos-2DPCA) method is proposed, and the greedy iterative algorithm to solve Cos-2DPCA is also developed. The convergence and correlation of solutions are proved theoretically and discussed in detail. Finally, the series of experiments are carried out on the artificial dataset and eight standard datasets, respectively. The results demonstrate that the performances of Cos-2DPCA are significantly improved on the reconstruction, correlation, complexity, and classification, and it outperforms most of the existing robust 2DPCA methods.

A Sparse Sensor Placement Strategy Based on Information Entropy and Data Reconstruction for Ocean Monitoring

Sparse sensor placement strategies are applied to reconstruct a region’s full-state data conditioned to a limited number of sensors, particularly crucial to ocean monitoring systems. In maritime systems, existing sparse sensor placement methods consider the reconstruction error of data or rely on specific requirements. Considering how sensors acquire essential information for monitoring systems, the utilization of entropy from information theory becomes quite interesting. In this article, we show that entropy measurements on different quantities of information are sensitive to indicate the border areas, thus requiring a balance between the number of sensors needed and the amount of information collected by them in coastal areas. Due to such, we propose (i) a novel sparse sensor placement strategy based on entropy, where the entropy measurements in temporal dimension are utilized for sample selection, so portions of samples selected are utilized for training data, significantly improving the training efficiency without sacrificing accuracy of subsequent data reconstruction. In the proposed strategy, (ii) we use orthogonal triangle decomposition from linear algebra, where a low-cost sensor is employed as a pivot. In terms of spatial dimension, the entropy of each location is adopted as entropy weight to reconstruct full-state data. Additionally, (iii) the strategy employs a greedy algorithm of weighted column pivoting for the orthogonal triangle decomposition, which is designed to suit yet effectively seek additional information and minimal reconstruction error in each iteration processing step. Experimental results using Sea Surface Temperature (SST) data show that the proposed strategy outperforms existing methods, acquiring more information, ensuring higher efficiency, and reducing costs while minimizing reconstruction errors.

DMVSVDD: Multi-View Data Novelty Detection with Deep Autoencoding Support Vector Data Description

Novelty detection is usually defined as the identification of new or abnormal objects (outliers) from the normal ones (inliers), which has wide potential applications in the real world. Recently, an effective algorithm called Deep Support Vector Data Description (Deep SVDD) has been proposed for novelty detection, which jointly trains a deep neural network while optimizing a data-enclosing hypersphere in output space. However, some constraints such as hypersphere collapse, limit the adaptability of the model and may affect the model performance. Moreover, most of the existing studies concerning Deep SVDD focus on the novelty detection for single-view data, which may fail to provide an accurate and reliable decision because the single-view data sometimes cannot fully reflect the actual condition of the problem. In this study, we developed an end-to-end deep learning method of novelty detection for multi-view data, i.e., the Deep Multi-View SVDD (DMVSVDD). To fully preserve the correlative and complementary information of multi-view data, we jointly trained multiple deep autoencoding neural networks for multiple views while adaptively optimizing the data-enclosing hypersphere of each view in latent space. A global objective function was proposed, which takes both of the sample reconstruction error minimization and the hypersphere volume minimization into consideration simultaneously to prevent hypersphere collapse in the model. In the global objective function, the hypersphere centers and view weights of different views were designed to adaptively select the better representative features after each epoch during the training process by embedding multi-view target samples into multiple data-enclosing hyperspheres with minimum volumes. The experimental results on the MNIST-USPS and NUS-WIDE-OBJECT datasets reveal that our proposed method learns the target class effectively and is superior to some state-of-the-art methods.

Proximity-based density description with regularized reconstruction algorithm for anomaly detection

This study addresses unsupervised anomaly detection using one-class classification, which constructs a decision boundary to determine if a new instance belongs to the target class. Existing one-class classification methods often fail in real-world scenarios due to their sensitivity to noise and inability to handle complex structures. We propose a proximity-based density description with a regularized reconstruction algorithm to overcome these limitations. Our method defines density-descriptive coefficients to reconstruct initial density and derives optimal coefficients by minimizing reconstruction error subject to sparsity and smoothness constraints. The sparsity constraint reduces noise effects, while the smoothness constraint encourages a flexible decision boundary. We evaluate our algorithm on benchmark datasets and compare it to existing methods, demonstrating superior performance.

TED-Face: Texture-Enhanced Deep Face Reconstruction in the Wild.

We present TED-Face, a new method for recovering high-fidelity 3D facial geometry and appearance with enhanced textures from single-view images. While vision-based face reconstruction has received intensive research in the past decades due to its broad applications, it remains a challenging problem because human eyes are particularly sensitive to numerically minute yet perceptually significant details. Previous methods that seek to minimize reconstruction errors within a low-dimensional face space can suffer from this issue and generate close yet low-fidelity approximations. The loss of high-frequency texture details is a key factor in their process, which we propose to address by learning to recover both dense radiance residuals and sparse facial texture features from a single image, in addition to the variables solved by previous work-shape, appearance, illumination, and camera. We integrate the estimation of all these factors in a single unified deep neural network and train it on several popular face reconstruction datasets. We also introduce two new metrics, visual fidelity (VIF) and structural similarity (SSIM), to compensate for the fact that reconstruction error is not a consistent perceptual metric of quality. On the popular FaceWarehouse facial reconstruction benchmark, our proposed system achieves a VIF score of 0.4802 and an SSIM score of 0.9622, improving over the state-of-the-art Deep3D method by 6.69% and 0.86%, respectively. On the widely used LS3D-300W dataset, we obtain a VIF score of 0.3922 and an SSIM score of 0.9079 for indoor images, and the scores for outdoor images are 0.4100 and 0.9160, respectively, which also represent an improvement over those of Deep3D. These results show that our method is able to recover visually more realistic facial appearance details compared with previous methods.

Data-driven templates with dictionary learning and sparse representations for photometric redshift estimation

The quality of photometric redshift estimates is fundamental for cosmological applications, as the constraints on the cosmological parameters obtained by the photometric surveys strongly rely on their precision and accuracy. In order to obtain reliable redshift estimates, a large number of studies have proposed different algorithms based on SED template fitting and machine learning methodologies. While template fitting strategies do not need spectroscopic data for training, they also provide consistent estimates in most scenarios. On the other hand, machine learning-based approaches succeed at building a mapping function between the input space (composed by magnitudes and other physical parameters) and the target photometric redshift space. These empirical methods lead to more accurate results as long as the spectroscopic ground truth properly represents the actual galaxy population distribution. Thus, combining template fitting and machine learning techniques would allow to leverage the advantages of both methodologies yielding hybrid methods. The goal is to provide a good estimation accuracy while maximizing the photometric range coverage. This is particularly important in the high-z regime, where the spectroscopic training sample is rarely available. We propose in this article a new method to derive data-driven templates from simulated spectra using dictionary learning. As these representations are obtained directly from the data, they allow to capture the physical properties of the galaxies better than theoretical templates. Inspired by hybrid algorithms, the dictionary is built hierarchically and the features of the different types of galaxies are separated. Once the dictionary has been created, the data-driven templates are artificially redshifted and the observed spectra are sparsely decomposed on this dictionary. The value providing the minimum reconstruction error is selected as the photometric redshift estimate. This new technique, astride template fitting and machine learning, builds representations for the galaxy spectra through unsupervised learning and computes the photometric redshifts by χ2 minimization. The performance of the algorithm has been evaluated on realistic galaxy photometric simulations as well as on real data from the Canada–France–Hawaii Telescope Lensing Survey.

Multi-viewpoint optical positioning algorithm based on minimizing reconstruction error

The obvious drawback of optical positioning based on binocular vision is the light occlusion. A multi-viewpoint optical positioning algorithm based on minimizing reconstruction error is proposed to realize mutual compensation optical tracking from the multi-viewpoint. Firstly, coordinate conversion model is used to obtain the conversion matrix between the spatial coordinate system of all possible binocular viewpoints and the reference coordinate system, so as to realize the unification of multi-viewpoint coordinate. Then, the parallel stereo vision measurement accuracy analysis model is established. Finally, the overall error of the reconstruction of different viewpoints is calculated in the multi-viewpoint measurement, and the binocular viewpoint with the smallest reconstruction error and no occlusion is selected to reconstruct the three-dimensional coordinates of markers. Two experiments are performed to verify the proposed algorithm, which show that the proposed algorithm is feasible to select the optimal reconstruction view and track surgical instruments continuously when occlusion exists.

RETRACTED: Optimization using Internet of Agent based Stacked Sparse Autoencoder Model for Heart Disease Prediction

AbstractRecently, machine learning methods have been successfully used for the prediction of cardiovascular disease. Early diagnosis and prediction is necessary for giving effective treatment to avoid higher mortality rates. Several classification algorithms have been developed recently which satisfy the need, but show limited accuracy while predicting the heart disease. Hence, the focus of this study is on early prediction of heart disease and to improve the accuracy of prediction using benchmark heart disease datasets such as UCI Cleveland dataset and Heart disease clinical dataset by implementing effective classification and optimization algorithms. Optimization algorithms generally exhibit the benefit of dealing with complex non‐linear issues with better adaptability and flexibility. The Emperor penguin optimization algorithm, which can select the best features for classification has been utilized in this study to improve the efficiency, minimize reconstruction errors, and increase the quality of heart disease classification. Further, the newly developed stacked sparse convolutional neural network based auto encoder (SSC‐AE) classification algorithm has been employed for significant feature classification with higher robustness and efficacy. Accuracy, Area Under Curve (AUC), and F1 score are some of the measures used to compare the outcomes of several machine learning algorithms to those of the proposed model in this study. The results show that the proposed model, SSC‐AE, is superior to other classification models.

High-spatial-resolution φ-OFDR shape sensor based on multicore optical fiber with femtosecond-laser-induced permanent scatter arrays.

An optical fiber φ-OFDR shape sensor with a submillimeter spatial resolution of 200 µm was demonstrated by using femtosecond-laser-induced permanent scatter array (PS array) multicore fiber (MCF). A PS array was successfully inscribed in each slightly twisted core of the 400-mm-long MCF. The two-dimensional (2D) and three-dimensional (3D) shapes of the PS-array-inscribed MCF were successfully reconstructed by using PS-assisted φ-OFDR, vector projections, and the Bishop frame based on the PS-array-inscribed MCF. The minimum reconstruction error per unit length of the 2D and 3D shape sensor was 2.21% and 1.45%, respectively.

A joint-norm distance metric 2DPCA for robust dimensionality reduction

Two-dimensional principal component analysis (2DPCA) is one of the most representative dimensionality reduction methods and the robustness degrades in the presence of noise. To improve the robustness, several 2DPCA methods using similar distance metrics were proposed to achieve the objective of the maximum projection distance or the minimum reconstruction error. However, these two objectives cannot be accomplished simultaneously or are constrained by the same projection matrix. To handle these problems, we propose a generalized robust 2DPCA method called 2DPCA-2-Lp, which joins 2-norm and lp-norm metrics in the objective function. It aims to maximize the ratio of projected vectors to image row vectors and to fulfill the dual objectives of directly maximizing projection distances and indirectly minimizing reconstruction errors. 2DPCA-2-Lp measures the distances between row vectors of matrices rather than the distances between matrices, thus remarkably enhancing the robustness. Furthermore, the relationship between reconstruction errors and projection distances for different parameter values is analyzed theoretically under the joint-norm metric. Then, the closed greedy iterative algorithm is developed for obtaining optimal solutions. Finally, extensive experimental results on four datasets show that 2DPCA-2-Lp outperforms most existing 2DPCA methods in terms of reconstruction and classification performances and has better robustness against noise.

Spectral Correlation-Based Diverse Band Selection for Hyperspectral Image Classification

Band selection which can reduce the spectral dimensionality effectively, has become one of the most popular topics in hyperspectral image (HSI) analysis. Recently, sparse representation based band selection (BS) has emerged as a popular tool. The existing sparse models mainly focus on minimizing reconstruction error and sparsity, while do not fully exploit the unique correlations among hundreds of continuous bands, which may cause representative bands missed and highly-correlated bands selected. Therefore, this paper proposes the spectral correlation based diverse band selection (SCDBS) for HSIs to improve representativeness and diversity of the selected bands. Specifically, a correlation derived weight is used to perform weighted sparse reconstruction to select the bands that are more correlated to the whole HSI, and a correlation minimization term is designed to remove the highly-correlated bands simultaneously. In addition, the proposed method imposes an adjustable sparse constraint by using an ℓ <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2,0&lt;p≤1</sub> norm, which extends and unifies the commonly used ℓ <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2,1</sub> norm to provide more flexible sparsity level. To optimize the proposed BS model, an iteration algorithm with relatively low computational cost is designed, of which the convergence is theoretically presented. Experimental results on three benchmark datasets have demonstrated that the proposed SCDBS outperforms state-of-the-art methods in HSI classification.

A deep generative model based on CNN-CVAE for wind turbine condition monitoring

Auto-encoder (AE)-based condition monitoring (CM) methods for fault detection of wind turbines have received considerable attention due to their powerful feature extraction ability. However, traditional AE-based monitoring methods can only learn point-to-point features by minimizing reconstruction errors, which leads to a low sensitivity to anomaly data and weak robustness to noise data. To this end, we introduce a novel deep generative method based on the convolutional neural network (CNN)-conditional variational auto-encoder (CVAE). The key idea of CNN-CVAE is to unify the representation learning capacity of the CVAE and CNN. Specifically, CVAE can learn a probability distribution model by being trained on an anomaly-free supervisory control and data acquisition systems (SCADA) dataset; CNN and deconvolution operations are adopted for better time-series feature extraction and reconstruction performance. A statistical process control chart is applied to determine the alarm threshold. The effectiveness of the CNN-CVAE-based method is validated by datasets collected by SCADA installed in a commercial wind farm in China for impending blade breakage and gearbox failure. Abundant experiments with state-of-the-art deep learning-based CM methods are conducted, which indicate that our proposed method outperforms other methods in robustness, fault detection data sensitivity, fault warning time, and model parameters.

Self-supervised Discriminative Representation Learning by Fuzzy Autoencoder

Representation learning based on autoencoders has received great concern for its potential ability to capture valuable latent information. Conventional autoencoders pursue minimal reconstruction error, but in most machine learning tasks such as classification and clustering, the discrimination of feature representation is also important. To address this limitation, an enhanced self-supervised discriminative fuzzy autoencoder (FAE) is innovatively proposed, which focuses on exploring information within data to guide the unsupervised training process and enhancing feature discrimination in a self-supervised manner. In FAE, fuzzy membership is applied to provide a means of self-supervised, which allows FAE can not only utilize AE’s outstanding representation learning capabilities but can also transform the original data into another space with improved discrimination. First, the objective function corresponding to FAE is proposed by reconstruction loss and clustering oriented loss simultaneously. Subsequently, Mini-Batch Gradient Descent is applied to infer the objective function and the detailed process is illustrated step by step. Finally, empirical studies on clustering tasks have demonstrated the superiority of FAE over the state of the art.

Gumbel-Softmax based Neural Architecture Search for Hierarchical Brain Networks Decomposition.

Understanding the brain's functional architecture has been an important topic in the neuroimaging field. A variety of brain network modeling methods have been proposed. Recently, deep neural network-based methods have shown a great advantage in modeling the hierarchical and complex functional brain networks (FBNs). However, most of these deep neural networks were handcrafted, making it time-consuming to find the relatively optimal architecture. To address this problem, we propose a novel unsupervised differentiable neural architecture search (NAS) algorithm, named Gumbel-Softmax based Neural Architecture Search (GS-NAS), to automate the architecture design of deep belief network (DBN) for hierarchical FBN decomposition. Specifically, we introduce the Gumbel-Softmax scheme to reframe the discrete architecture sampling procedure during NAS to be continuous. Guided by the reconstruction error minimization procedure, the architecture search can be driven by the intrinsic functional architecture of the brain, thereby revealing the possible hierarchical functional brain organization via DBN structure. The proposed GS-NAS algorithm can simultaneously optimize the number of hidden units for each layer and the network depth. Extensive experiment results on both task and resting-state functional magnetic resonance imaging data have demonstrated the effectiveness and efficiency of the proposed GS-NAS model. The identified hierarchically organized FBNs provide novel insight into understanding human brain function.

A Decoder-Free Variational Deep Embedding for Unsupervised Clustering.

In deep clustering frameworks, autoencoder (AE)- or variational AE-based clustering approaches are the most popular and competitive ones that encourage the model to obtain suitable representations and avoid the tendency for degenerate solutions simultaneously. However, for the clustering task, the decoder for reconstructing the original input is usually useless when the model is finished training. The encoder-decoder architecture limits the depth of the encoder so that the learning capacity is reduced severely. In this article, we propose a decoder-free variational deep embedding for unsupervised clustering (DFVC). It is well known that minimizing reconstruction error amounts to maximizing a lower bound on the mutual information (MI) between the input and its representation. That provides a theoretical guarantee for us to discard the bloated decoder. Inspired by contrastive self-supervised learning, we can directly calculate or estimate the MI of the continuous variables. Specifically, we investigate unsupervised representation learning by simultaneously considering the MI estimation of continuous representations and the MI computation of categorical representations. By introducing the data augmentation technique, we incorporate the original input, the augmented input, and their high-level representations into the MI estimation framework to learn more discriminative representations. Instead of matching to a simple standard normal distribution adversarially, we use end-to-end learning to constrain the latent space to be cluster-friendly by applying the Gaussian mixture distribution as the prior. Extensive experiments on challenging data sets show that our model achieves higher performance over a wide range of state-of-the-art clustering approaches.

Unsupervised and Supervised Feature Selection for Incomplete Data via L2,1-Norm and Reconstruction Error Minimization

Feature selection has been widely used in machine learning and data mining since it can alleviate the burden of the so-called curse of dimensionality of high-dimensional data. However, in previous works, researchers have designed feature selection methods with the assumption that all the information from a data set can be observed. In this paper, we propose unsupervised and supervised feature selection methods for use with incomplete data, further introducing an L2,1 norm and a reconstruction error minimization method. Specifically, the proposed feature selection objective functions take advantage of an indicator matrix reflecting unobserved information in incomplete data sets, and we present pairwise constraints, minimizing the L2,1-norm-robust loss functionand performing error reconstruction simultaneously. Furthermore, we derive two alternative iterative algorithms to effectively optimize the proposed objective functions and the convergence of the proposed algorithms is proven theoretically. Extensive experimental studies were performed on both real and synthetic incomplete data sets to demonstrate the performance of the proposed methods.

Undercover Boolean Matrix Factorization with MaxSAT

The k-undercover Boolean matrix factorization problem aims to approximate a m×n Boolean matrix X as the Boolean product of an m×k and a k×n matrices A◦B such that X is a cover of A◦B, i.e., no representation error is allowed on the 0’s entries of the matrix X. To infer an optimal and “block-optimal” k-undercover, we propose two exact methods based on MaxSAT encodings. From a theoretical standpoint, we prove that our method of inferring “block-optimal” k-undercover is a (1 - 1/e) ≈ 0.632 approximation for the optimal k-undercover problem. From a practical standpoint, experimental results indicate that our “block-optimal” k-undercover algorithm outperforms the state-of-the-art even when compared with algorithms for the more general k-undercover Boolean Matrix Factorization problem for which only minimizing reconstruction error is required.

Enhanced dictionary learning based sparse classification approach with applications to planetary bearing fault diagnosis

To date, planetary bearings remain challenging for machinery fault diagnosis because of their intricate kinematics, time-variant modulations, and strong interferences. To address this challenge, this study presents an enhanced dictionary learning based sparse classification (EDL-SC) approach to diagnose planetary bearings. Our main novelty lies in that the data augmentation and dictionary learning strategies are incorporated into our proposed EDL-SC approach, which can significantly enhance the representation ability and recognition ability for the sparse classification-based intelligent diagnosis criterion. Firstly, vibration data augmentation is implemented with an overlapping segmentation strategy to enhance the quality of training samples. Secondly, data-driven dictionary design is achieved by means of dictionary learning, which learns sub-dictionaries and adaptively designs the whole dictionary through considering both the inter-class and intra-class features. Thirdly, a sparse classification strategy is established for intelligent diagnostics by the aid of a discrimination criterion of minimal reconstruction errors. The feasibility and advantage of EDL-SC have been thoroughly evaluated with a challenging planetary bearing dataset. Experiment verification results of planetary bearing fault diagnosis indicate that EDL-SC obtains a superior diagnosis accuracy of 99.63%, strong robustness to noises, and competitive computation efficiency over advanced deep learning and sparse representation classification methods. This work can bring new insights for the application of sparse representation theory from the perspective of pattern recognition, and shows great potentials of EDL-SC for data-driven machinery fault diagnosis.

Flexible capped principal component analysis with applications in image recognition

Robust principal component analysis (PCA) for feature extraction, which takes various robust norms as the distance metric, has been shown to be valid in data reconstruction and recognition tasks. However, some existing approaches do not achieve satisfactory performance and robustness owing to the imperfection of norms used for measuring the distances in models. To solve this problem, we propose a new formulation of the robust PCA feature extraction method, called flexible capped PCA (FCPCA), in which we use capped L2,p-norm distance metric to minimize reconstruction errors of data. In addition to having rotational invariance, capped L2,p-norm distance metric can better weaken the role of outliers, even if the outliers’ effect is magnified when p is a large value. This characteristic makes our model more powerful than existing approaches. Also, the proposed model is more flexible than many other robust models, which can be derived by specifying the value of p. However, solving the resulting objective problem is a challenge. One of the key contributions of this study is the design of a new iterative algorithm for the derivation of an optimal solution, in which each iteration computes an eigenvalue problem associated with a weighted covariance matrix. Experimental results on several public datasets prove the algorithm’s effectiveness.

Interpretable Deep Probabilistic Model for HRR Radar Signal and its Application to Target Recognition

With the advent of neural network, unprecedented advancements have been achieved in many tasks, including radar signal processing. However, a key disadvantage of the current methods is the black-box structure, which makes it hard to interpret or assess the hidden representations of data. In this paper, by combining the physical generative mechanism of the high range resolution (HRR) radar signal with neural network, we develop an interpretable deep probabilistic model to learn the latent features from HRR radar signals that can characterize the physical structure of targets. In detail, based on the radar target&#x0027;s scattering center model which describes the HRR radar signal as the summation of echoes from the scattering centers, a deep probabilistic model is constructed to depict the generative process from the scattering centers to observations, where the latent features comprise the locations and amplitudes of scattering centers. Considering that the locations of scattering centers are nearly invariable and the amplitudes of them are fluctuant for the signals within a small angular range, our model defines the shared locations and independent amplitudes of scattering centers for the signals in a small angular range, aiming to improve the robustness of interpretable feature extraction model. In addition, our model performs probabilistic inference on the posterior distribution of latent features with high preciseness. With the proposed model, we further design a recognition scheme based on the minimum reconstruction error criterion. Experiments on the measured HRR radar dataset validate the effectiveness of our model on learning interpretable features and superior recognition performance.