Points In Feature Space Research Articles

Verification of the person’s dynamic signature on a limited number of samples

Objectives. The goal of the research is to develop a new person-dependent method for verification of a signature of one person made on a tablet with a stylus in the presence of a limited number of signature samples of this person.Methods. The paper shows how to construct an individual pattern of the dynamic signatures of any person, which is described by points in a multidimensional feature space and is intended for subsequent verification of the authenticity of the signatures of a given person. It is constructed using 5<N<20 samples of genuine human signatures. The pattern forms a convex object in a multidimensional feature space. It describes the peculiar properties of a signature performed by a specific person.Results . The dynamics of signature execution is represented by three discrete parametric functions: coordinates of the stylus X, Y and its pressure on the tablet P, recorded at fixed time intervals. In the process of research, a number of secondary functions-features were selected and calculated from them. Since these data sets have different lengths, the dynamic time warping algorithm is used to compare them. The results of this transformation are distances between the dynamic features of two signatures, which serve as coordinates of a point in the feature space that describes the similarity of these signatures. The set of such points describes similarity of all pairs of genuine human signatures presented for verification in a multidimensional feature space. The convex hull of the cloud of these points is used as a pattern of a particular person's signature. The genuine signatures of any person are always different from each other; significant differences between them can distort the verification result.Conclusion. Experimental studies performed on genuine and fake signatures of 498 people from the largest available database of dynamic signatures, DeepSignDB, showed a verification accuracy of about 98 % when analyzing 24,900 signatures. Half of them are genuine, half are fake

Identifying Plausible Labels from Noisy Training Data for a Land Use and Land Cover Classification Application in Amazônia Legal

Most studies in the field of land use and land cover (LULC) classification in remote sensing rely on supervised classification, which requires a substantial amount of accurate label data. However, reliable data are often not immediately available, and are obtained through time-consuming manual labor. One potential solution to this problem is the use of already available classification maps, which may not be the true ground truth and may contain noise from multiple possible sources. This is also true for the classification maps of the MapBiomas project, which provides land use and land cover (LULC) maps on a yearly basis, classifying the Amazon basin into more than 24 classes based on the Landsat data. In this study, we utilize the Sentinel-2 data with a higher spatial resolution in conjunction with the MapBiomas maps to evaluate a proposed noise removal method and to improve classification results. We introduce a novel noise detection method that relies on identifying anchor points in feature space through clustering with self-organizing maps (SOM). The pixel label is relabeled using nearest neighbor rules, or can be removed if it is unknown. A challenge in this approach is the quantification of noise in such a real-world dataset. To overcome this problem, highly reliable validation sets were manually created for quantitative performance assessment. The results demonstrate a significant increase in overall accuracy compared to MapBiomas labels, from 79.85% to 89.65%. Additionally, we trained the L2HNet using both MapBiomas labels and the filtered labels from our approach. The overall accuracy for this model reached 93.75% with the filtered labels, compared to the baseline of 74.31%. This highlights the significance of noise detection and filtering in remote sensing, and emphasizes the need for further research in this area.

MeshCL: Towards robust 3D mesh analysis via contrastive learning

Recent years have witnessed the success of deep learning in 3D mesh analysis, including classification and segmentation. Previous works mainly rely on supervised learning, which trains models on the labeled 3D mesh data with the cross-entropy loss. However, in the supervised training, the very limited amount of 3D mesh data restricts the feature’s discriminative power and the model’s robustness. To address or at least alleviate this issue, we propose to regularize the learned representations by associating the data points in feature space via contrastive learning. To achieve this, we first explore mesh samples with diverse geometric structures, and then apply contrastive regularization into the positive and negative pairs sampled from the feature space. In this way, our method, MeshCL, can learn robust feature representations invariant to the geometric deformation for 3D manifold mesh. In detail, to augment the geometric diversity, we examine different data augmentation techniques for mesh data. Furthermore, considering the geometric characteristics of 3D mesh, we develop more efficient positive/negative pair sampling strategies for the dense prediction task. To verify the effectiveness of MeshCL, we study two typical mesh analysis tasks, classification and segmentation. Experimental results demonstrate that MeshCL not only achieves state-of-the-art performance on the two tasks mentioned above but also significantly improves the model’s robustness.

No silver bullet: interpretable ML models must be explained.

Recent years witnessed a number of proposals for the use of the so-called interpretable models in specific application domains. These include high-risk, but also safety-critical domains. In contrast, other works reported some pitfalls of machine learning model interpretability, in part justified by the lack of a rigorous definition of what an interpretable model should represent. This study proposes to relate interpretability with the ability of a model to offer explanations of why a prediction is made given some point in feature space. Under this general goal of offering explanations to predictions, this study reveals additional limitations of interpretable models. Concretely, this study considers application domains where the purpose is to help human decision makers to understand why some prediction was made or why was not some other prediction made, and where irreducible (and so minimal) information is sought. In such domains, this study argues that answers to such why (or why not) questions can exhibit arbitrary redundancy, i.e., the answers can be simplified, as long as these answers are obtained by human inspection of the interpretable ML model representation.

DEEP PERONA–MALIK DIFFUSIVE MEAN SHIFT IMAGE CLASSIFICATION FOR EARLY GLAUCOMA AND STARGARDT DISEASE DETECTION

Glaucoma and Stargardt’s, an inherited disease predominantly affect the retinal portion of the eye. The diagnosis of Glaucoma in a fundus image is an arduous, time consuming process. There were many research works carried out to detect early stages of Glaucoma and Stargardt’s disease. However, the accuracy, diagnostic time and performance were not improved. To resolve the above said problems, a computational method called Deep Neural Perona–Malik Diffusive Mean Shift Mode Seeking Segmented Image Classification (DNP-MDMSMSIC) is introduced for the early detection of Glaucoma and Stargardt’s disease with retinal fundus images. The DNP-MDMSMSIC method comprises diverse types of layers that support to identify early detection of disease with improved accuracy and less time. Process as explained; initially, numerous qualified retinal images are given as input to the input layer. These input images are transmitted further to the hidden layer 1 to perform image pre-processing. In DNP-MDMSMSIC, Space-Variant Perona–Malik Diffusive Image Preprocessing is carried out to decrease the noise from input image without removing contents like edges, lines, etc., for image interpretation with a higher peak signal-to-noise ratio. This preprocessed image is further processed in the hidden layer 2 where the feature extraction process is performed to extract features like color, texture, and intensity with a higher degree of accuracy. Based on the extracted features, an input feature image gets segmented in hidden layer 3. Mean Shift Mode Seeking Segmentation algorithm is employed to segment the pixels in image space with corresponding feature space points. Then the segmented images are given to the output layer to perform retinal fundus image classification using Bregman Divergence Function. During the image classification, the distance between two segmented regions (i.e., testing image region of particular class and training image region) with convex is measured. In this way, the retinal fundus images get classified with higher accuracy. Experimental evaluation is performed by considering the metrics such as peak signal-to-noise, disease detection accuracy, disease detection time, and error rate corresponding to the number of retina fundus images and image size. DNP-MDMSMSIC method is designed to detect Glaucoma and Stargardt’s disease at an earlier stage with higher accuracy by 8% and less time by 20% with aid of ACRIMA database.

ArithFusion: An Arithmetic Deep Model for Temporal Remote Sensing Image Fusion

Different satellite images may consist of variable numbers of channels which have different resolutions, and each satellite has a unique revisit period. For example, the Landsat-8 satellite images have 30 m resolution in their multispectral channels, the Sentinel-2 satellite images have 10 m resolution in the pan-sharp channel, and the National Agriculture Imagery Program (NAIP) aerial images have 1 m resolution. In this study, we propose a simple yet effective arithmetic deep model for multimodal temporal remote sensing image fusion. The proposed model takes both low- and high-resolution remote sensing images at t1 together with low-resolution images at a future time t2 from the same location as inputs and fuses them to generate high-resolution images for the same location at t2. We propose an arithmetic operation applied to the low-resolution images at the two time points in feature space to take care of temporal changes. We evaluated the proposed model on three modality pairs for multimodal temporal image fusion, including downsampled WorldView-2/original WorldView-2, Landsat-8/Sentinel-2, and Sentinel-2/NAIP. Experimental results show that our model outperforms traditional algorithms and recent deep learning-based models by large margins in most scenarios, achieving sharp fused images while appropriately addressing temporal changes.

Feature Transformation Framework for Enhancing Compactness and Separability of Data Points in Feature Space for Small Datasets

Compactness and separability of data points are two important properties that contribute to the accuracy of machine learning tasks such as classification and clustering. We propose a framework that enhances the goodness criteria of the two properties by transforming the data points to a subspace in the same feature space, where data points of the same class are most similar to each other. Most related research about feature engineering in the input data points space relies on manually specified transformation functions. In contrast, our work utilizes a fully automated pipeline, in which the transformation function is learnt via an autoencoder for extraction of latent representation and multi-layer perceptron (MLP) regressors for the feature mapping. We tested our framework on both standard small datasets and benchmark-simulated small datasets by taking small fractions of their samples for training. Our framework consistently produced the best results in all semi-supervised clustering experiments based on K-means and different seeding techniques, with regards to clustering metrics and execution time. In addition, it enhances the performance of linear support vector machine (LSVM) and artificial neural network (ANN) classifier, when embedded as a preprocessing step before applying the classifiers.

GeometryMotion-Net: A Strong Two-Stream Baseline for 3D Action Recognition

In this work, we propose a strong two-stream baseline method referred to as GeometryMotion-Net for 3D action recognition. For efficient 3D action recognition, we first represent each point cloud sequence as a limited number of randomly sampled frames with each frame consisting of a sparse set of points. After that, we propose a new two-stream framework for effective 3D action recognition. For the geometry stream, we propose a new module to produce a virtual overall geometry point cloud by first merging all 3D points from these selected frames, and then we exploit local neighborhood information of each point in the feature space. In the motion stream, for any two neighboring point cloud frames, we also propose a new module to generate one virtual forward motion point cloud and one virtual backward motion point cloud. Specifically, for each point in the current frame, we first produce a set of 3D offset features relative to the neighboring points in the reference frame ( <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i.e.</i> , the previous/subsequent frame) and then exploit local neighborhood information of this point in the offset feature space. Based on the newly generated virtual overall geometry point cloud and multiple virtual forward/backward motion point clouds, any existing point cloud analysis methods ( <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">e.g.</i> , PointNet) can be readily adopted for extracting discriminant geometry and bidirectional motion features in the geometry and motion streams, respectively, which are further aggregated to make our two-stream network trainable in an end-to-end fashion. Comprehensive experiments on both large-scale datasets ( <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i.e.</i> NTU RGB+D 60 and NTU RGB+D 120) and small-scale datasets ( <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i.e.</i> , N-UCLA and UWA3DII) demonstrate the effectiveness and efficiency of our two-stream network for 3D action recognition.

Comparative assessment of drought monitoring index susceptibility using geospatial techniques.

There are two main categories of dryness monitoring indices based on spectral feature space. One category uses the vertical distance from any point to a line passing through the coordinate origin, which is perpendicular to a soil line, to monitor the dryness conditions. The most popular indices are the Perpendicular Dryness Index (PDI) and the modified perpendicular dryness index (MPDI). The other category uses the distance from any point in feature space to the coordinate origin to represent the dryness status, for instance, the soil moisture (SM) monitoring index (SMMI) and the modified soil moisture monitoring index (MSMMI). In this study, the performances and differences of these four indicators were evaluated using field-measured SM (FSM) data based on Gaofen-1 (GF-1) wide field of view (WFV), Landsat-8 Operational Land Imager (OLI), and Sentinel-2 Multi-Spectral Instrument (MSI) sensors. Performance evaluations were conducted in two study areas, namely an arid and semi-arid region of northwest China and a humid agricultural region of southwest Canada. We employed gradient-based structural similarity (GSSIM) to quantitatively assess the similarity of the structural information and structural characteristics among these four indicators. Monitoring SM in bare soil or low vegetation-covered areas in the semi-arid region, the SMMI, PDI, MSMMI, and MPDI from Near-Infrared (NIR)-Red had significantly negative linear correlations with the FSM at 0-5-cm depth (P < 0.01). However, SMMI was better than PDI in estimating SM in bare soil, which was better than MSMMI and MPDI for GF-1. Moreover, the PDI and SMMI had similar SM evaluation abilities, which were better than those of MPDI and MSMMI for Landsat-8. The GSSIM map of the SMMI/PDI and the MSMMI/MPDI showed that the low change areas accounted for 99.89% and 98.89% for GF-1, respectively, and 95.78% and 94.45% for Landsat-8, respectively. This result indicated that the SMMI, PDI, MSMMI, and MPDI values from NIR-Red in low vegetation cover were similar. In monitoring SM in agricultural vegetation areas, the accuracy of the four indices from Short-Wave Infrared (SWIR) feature space was higher than that from NIR-Red feature space for Sentinel-2. The SM monitoring effect of MSMMI and MPDI was better than that of SMMI and PDI. Due to the lack of SWIR band, GF-1 was limited in monitoring SM in vegetation-covered areas. The SMMI and MSMMI, which do not rely on the soil line, were more suitable than PDI and MPDI for retrieving SM in the complex surface environment depending on the soil line and the number of parameters. GF-1 with 16-m resolution had higher accuracy in SM assessment than Landsat-8 with 30-m resolution and had almost the same accuracy as Sentinel-2 with 20 m.

Noniterative Sparse LS-SVM Based on Globally Representative Point Selection.

A least squares support vector machine (LS-SVM) offers performance comparable to that of SVMs for classification and regression. The main limitation of LS-SVM is that it lacks sparsity compared with SVMs, making LS-SVM unsuitable for handling large-scale data due to computation and memory costs. To obtain sparse LS-SVM, several pruning methods based on an iterative strategy were recently proposed but did not consider the quantity constraint on the number of reserved support vectors, as widely used in real-life applications. In this article, a noniterative algorithm is proposed based on the selection of globally representative points (global-representation-based sparse least squares support vector machine, GRS-LSSVM) to improve the performance of sparse LS-SVM. For the first time, we present a model of sparse LS-SVM with a quantity constraint. In solving the optimal solution of the model, we find that using globally representative points to construct the reserved support vector set produces a better solution than other methods. We design an indicator based on point density and point dispersion to evaluate the global representation of points in feature space. Using the indicator, the top globally representative points are selected in one step from all points to construct the reserved support vector set of sparse LS-SVM. After obtaining the set, the decision hyperplane of sparse LS-SVM is directly computed using an algebraic formula. This algorithm only consumes O(N2) in computational complexity and O(N) in memory cost which makes it suitable for large-scale data sets. The experimental results show that the proposed algorithm has higher sparsity, greater stability, and lower computational complexity than the traditional iterative algorithms.

The Utilization of the Radon Transform for the Extraction of the Orientation of Linear Features in Binary Images

This paper considers the idea of using the Radon transform to extract the orientation features of lines in binary images. The Radon transform sends a line at a particular orientation in image space to a point in feature space. The ensuing set of points in feature space is called a sinogram. This computation is usually performed for a large group of angles (over the interval [0,179] degrees taken in integer increments in this paper). Therefore, linear features at specific orientations will be mapped to points having maximum value at particular angles. For angular spacing of at least 5 degrees, the peaks of the sinogram at the angles corresponding to the orientations of the lines will be clearly visible (in a bar plot of sinogram peaks) above sinogram values at other angles. The mapping from image space to feature space accomplished by the Radon transform which maps rectangular coordinates (x,y) to coordinates (range, angle) provides for the garnering of the orientation of the linear features in binary images. In particular, the coordinates (range, angle) in the sinogram allow for distinguishing between lines oriented at one angle versus lines oriented at another angle or angles. This particular property of the sinogram allows for the extraction of the orientation features of lines in an image.

Attention-based relation and context modeling for point cloud semantic segmentation

Semantic segmentation of point cloud is a fundamental problem in scene-level understanding. Despite advancement in recent years by leveraging capabilities of Neural Networks and massive labeling datasets available, providing fine-grained semantic segmentation for point cloud is still challenging, given the fact that point cloud is usually unstructured, unordered and sparse. In this paper, we achieve semantic point cloud labeling by adaptively exploring semantic relation and aggregating contextual information between points. Specifically, we first introduce an attention-based local relation learning module for collecting local features, which can capture semantic relation in a manner of anisotropy. And we then design a novel context aggregation module guided by multi-scale supervision to obtain long-range dependencies between semantically-correlated points and enhance the distinctive ability of points in feature space. In addition, a gated propagation strategy is adopted instead of skip links to conditionally concatenate local point features in different layers. We empirically evaluate our method on public benchmarks (S3DIS and ShapeNetPart), and demonstrate our performance is on par or better than state-of-the-art methods.

On-orbit space camera self-calibration based on the orthogonal vanishing points obtained from solar panels

As one of the key steps of machine visual investigation, camera calibration is indispensable for obtaining an accurate 3D profile of measured objects and positioning the target. However, it is almost impossible to calibrate a camera on-orbit in real time, for the feature points in space are irregular. To self-calibrate camera parameters in orbit, we propose a method to calibrate the camera using the orthogonal vanishing points obtained from a solar panel, which is a common component of most man-made space satellites. Using two sets of images of orthogonal parallel ribs of a solar panel under any two positions, four vanishing points are achieved. Based on the geometrical property of the orthogonal vanishing point and the infinite homography relationship between the corresponding vanishing points under different camera positions, a camera self-calibration method is proposed, and constraint equations are established to calibrate the intrinsic parameters of the camera linearly. Aiming at restraining the influence of the noise on the calibration result, an objective function based on the inverse point characteristics of rectangular imaging is proposed, and the Levenberg–Marquardt optimization algorithm is used to nonlinearly optimize the intrinsic parameters. Simulated and experimental results show that the 2D reprojection error is 0.86 pixels. This method, therefore, has better applicability in some significant space and national defense missions, such as space rendezvous and docking, attack-defense, and on-orbit servicing. The optimization algorithm shows good anti-noise interference performance, therefore the robustness of the calibration algorithm is improved.

The Weight-Shape decomposition of density estimates: A framework for clustering and image analysis algorithms

We propose an analysis scheme which addresses the Parzen-window and mixture model methods for estimating the probability density function of data points in feature space. Both methods construct the estimate as a sum of kernel functions (usually Gaussians). By adding an entropy-like function we prove that the probability distribution is a product of a weight function and a shape distribution. This Weight-Shape decomposition leads to new interpretations of established clustering algorithms. Furthermore, it suggests the construction of three different clustering schemes, which are based on gradient-ascent flow of replica points in feature space. Two of these are Quantum Clustering and the Mean-Shift algorithm. The third algorithm is based on maximal-entropy. In our terminology they become Maximal Shape Clustering, Maximal Probability Clustering and Maximal Weight Clustering, correspondingly. We demonstrate the different methods and compare them to each other on one artificial example and two natural data sets. We also apply the Weight-Shape decomposition to image analysis. The shape distribution acts as an edge detector. It serves to generate contours, as demonstrated on face images. Furthermore, it allows for defining a convolutional Shape Filter.

A density invariant approach to clustering

Organizing data into sensible groups is called as ‘data clustering.’ It is an open research problem in various scientific fields. Neither a universal solution nor an absolute strategy for its evaluation exists in the literature. In this context, through this paper, we make following three contributions: (1) A new method for finding ‘natural groupings’ or clusters in the data set is presented. For this, a new term ‘vicinity’ is coined. Vicinity captures the idea of density together with spatial distribution of data points in feature space. This new notion has a potential to separate various type of clusters. In summary, the approach presented here is non-convex admissive (i.e., convex hulls of the clusters found can intersect which is desirable for non-convex clusters), cluster proportion and omission admissive (i.e., duplicating a cluster arbitrary number of times or deleting a cluster does not alter other cluster’s boundaries), scale covariant, consistent (shrinking within cluster distances and enlarging inter-cluster distances does not affect the clustering results) but not rich (does not generates exhaustive partitions of the data) and density invariant. (2) Strategy for automatic detection of various tunable parameters in the proposed ‘Vicinity Based Cluster Detection’ (VBCD) algorithm is presented. (3) New internal evaluation index called ‘Space-Density Index’ (SDI) for the clustered results (by any method) is also presented. Experimental results reveal that VBCD captures the idea of ‘natural groupings’ better than the existing approaches. Also, SDI evaluation scheme provides a better judgment as compared to earlier internal cluster validity indices.

Denoising by semi-supervised kernel PCA preimaging

Kernel Principal Component Analysis (PCA) has proven a powerful tool for nonlinear feature extraction, and is often applied as a pre-processing step for classification algorithms. In denoising applications Kernel PCA provides the basis for dimensionality reduction, prior to the so-called pre-image problem where denoised feature space points are mapped back into input space. This problem is inherently ill-posed due to the non-bijective feature space mapping. We present a semi-supervised denoising scheme based on kernel PCA and the pre-image problem, where class labels on a subset of the data points are used to improve the denoising. Moreover, by warping the Reproducing Kernel Hilbert Space (RKHS) we also account for the intrinsic manifold structure yielding a Kernel PCA basis that also benefit from unlabeled data points. Our two main contributions are; (1) a generalization of Kernel PCA by incorporating a loss term, leading to an iterative algorithm for finding orthonormal components biased by the class labels, and (2) a fixed-point iteration for solving the pre-image problem based on a manifold warped RKHS. We prove viability of the proposed methods on both synthetic data and images from The Amsterdam Library of Object Images (Geusebroek et al., 2005) [7].

Mammographic images segmentation based on chaotic map clustering algorithm

BackgroundThis work investigates the applicability of a novel clustering approach to the segmentation of mammographic digital images. The chaotic map clustering algorithm is used to group together similar subsets of image pixels resulting in a medically meaningful partition of the mammography.MethodsThe image is divided into pixels subsets characterized by a set of conveniently chosen features and each of the corresponding points in the feature space is associated to a map. A mutual coupling strength between the maps depending on the associated distance between feature space points is subsequently introduced. On the system of maps, the simulated evolution through chaotic dynamics leads to its natural partitioning, which corresponds to a particular segmentation scheme of the initial mammographic image.ResultsThe system provides a high recognition rate for small mass lesions (about 94% correctly segmented inside the breast) and the reproduction of the shape of regions with denser micro-calcifications in about 2/3 of the cases, while being less effective on identification of larger mass lesions.ConclusionsWe can summarize our analysis by asserting that due to the particularities of the mammographic images, the chaotic map clustering algorithm should not be used as the sole method of segmentation. It is rather the joint use of this method along with other segmentation techniques that could be successfully used for increasing the segmentation performance and for providing extra information for the subsequent analysis stages such as the classification of the segmented ROI.

Novel method of mining classification information for SVM training

Support vector machine (SVM) is an important classification tool in the pattern recognition and machine learning community, but its training is a time-consuming process. To deal with this problem, we propose a novel method to mine the useful information about classification hidden in the training sample for improving the training algorithm, and every training point is assigned to a value that represents the classification information, respectively, where training points with the higher values are chosen as candidate support vectors for SVM training. The classification information value for a training point is computed based on the classification accuracy of an appropriate hyperplane for the training sample, where the hyperplane goes through the mapped target of the training point in feature space defined by a kernel function. Experimental results on various benchmark datasets show the effectiveness of our algorithm.

Sparse non-linear denoising: Generalization performance and pattern reproducibility in functional MRI

We investigate sparse non-linear denoising of functional brain images by kernel principal component analysis (kernel PCA). The main challenge is the mapping of denoised feature space points back into input space, also referred to as “the pre-image problem”. Since the feature space mapping is typically not bijective, pre-image estimation is inherently illposed. In many applications, including functional magnetic resonance imaging (fMRI) data which is the application used for illustration in the present work, it is of interest to denoise a sparse signal. To meet this objective we investigate sparse pre-image reconstruction by Lasso regularization. We find that sparse estimation provides better brain state decoding accuracy and a more reproducible pre-image. These two important metrics are combined in an evaluation framework which allow us to optimize both the degree of sparsity and the non-linearity of the kernel embedding. The latter result provides evidence of signal manifold non-linearity in the specific fMRI case study.

Regularized Pre-image Estimation for Kernel PCA De-noising

The main challenge in de-noising by kernel Principal Component Analysis (PCA) is the mapping of de-noised feature space points back into input space, also referred to as "the pre-image problem". Since the feature space mapping is typically not bijective, pre-image estimation is inherently illposed. As a consequence the most widely used estimation schemes lack stability. A common way to stabilize such estimates is by augmenting the cost function by a suitable constraint on the solution values. For de-noising applications we here propose Tikhonov input space distance regularization as a stabilizer for pre-image estimation, or sparse reconstruction by Lasso regularization in cases where the main objective is to improve the visual simplicity. We perform extensive experiments on the USPS digit modeling problem to evaluate the stability of three widely used pre-image estimators. We show that the previous methods lack stability in the is non-linear regime, however, by applying our proposed input space distance regularizer the estimates are stabilized with a limited sacrifice in terms of de-noising efficiency. Furthermore, we show how sparse reconstruction can lead to improved visual quality of the estimated pre-image.