L1-norm Distance Research Articles

Efficient Time Series Clustering by Minimizing Dynamic Time Warping Utilization

Dynamic Time Warping (DTW) is a widely used distance measurement in time series clustering. DTW distance is invariant to time series phase perturbations but has a quadratic complexity. An effective acceleration method must reduce the DTW utilization ratio during time series clustering; for example, TADPole uses both upper and lower bounds to prune off a large ratio of expensive DTW calculations. To further reduce the DTW utilization ratio, we find that the linear-complexity L1-norm distance (Manhattan distance) is effective enough when the time series only comprise small phase perturbations. Therefore, we propose a novel time series clustering by Minimizing Dynamic Time Warping Utilization (MiniDTW) algorithm to accelerate time series clustering. In MiniDTW, the dataset is first greedily summarized into seed clusters, which comprise time series of small phase perturbations, by L1-norm distance. Then, we develop a new Sparse Symmetric Non-negative Matrix Factorization (SSNMF) algorithm, which factorizes the DTW distance matrix of seed cluster centers, to merge the seed clusters into the final clusters. The experiments on UCR time series datasets demonstrate that MiniDTW, pruning 98.52% of the DTW utilization, is better than the counterpart method, TADPole, which only prunes 75.56% of the DTW utilization; and thus MiniDTW is 10 times faster than TADPole.

Robust supervised and semi-supervised twin extreme learning machines for pattern classification

In this paper, we first propose a novel robust loss function called adaptive capped Lθε-loss. The Lθε-loss has some interesting properties, such as robustness, non-convexity, and boundedness. During the learning process, for different problems, we can choose different loss functions through adaptive parameter θ. Then, a new robust twin extreme learning machine (RTELM) framework is presented by applying Lθε-loss and capped L1-norm distance metric. Compared with the twin extreme learning machine (TELM), RTELM overcomes the disadvantages of L2-norm distance metric and hinge loss, especially for the problem with outliers, while inherits the advantages of TELM. Further, we present a new Laplacian RTELM (Lap-RTELM for short) by introducing manifold regularization terms into RTELM. Intuitively, the Lap-RTELM can effectively utilize geometric information embedded in unlabeled samples and merge them as manifold regularization terms to learn a more reasonable classifier for semi-supervised classification (SSC) problems. Finally, two effective iterative algorithms are designed to solve the challenges brought by the non-convex optimization problems RTELM and Lap-RTELM, and theoretically guarantee the convergence, local optimality, and computational complexity of algorithms. Experiments on multiple datasets show that the proposed RTELM and Lap-RTELM are competitive with existing methods.

A presentation and retrieval hash scheme of images based on principal component analysis

Image representation and approximate query is always a research challenge and is affected greatly by the dimension and size of images. Since hash-based methods and binary encodings in combination with other techniques, such as kernel tricks, a longer binary code and mapping vectors rotation, can maintain a linear query time and query accuracy, they have been used in this area broadly. This paper develops principal component analysis hashing (PCAH) and unequal length of binary coding to divide images into more categories, denoted as PCA-MD, to improve accuracy of the representation and lookup of images. This paper firstly proves that the eigenvector mapping is locality sensitive, which is the basis for more classes division. For the anisotropy of the eigenvectors, PCA-MD utilizes an unequal length of binary coding and fewer eigenvectors, rather than an equal code, to divide the images mapped on every eigenvector to more categories. Moreover, L1-norm distance is applied to measure the distances of images to avoid the enormous computation of Euclidean distance. Theoretical analysis and extensive experimental results demonstrate that the PCA-MD has a higher query performance and a slight longer run time than the state-of-the-art approaches based on the Hamming distance. This in turn verifies that PCAH is a locality sensitive hash and that partitioning into more categories rather than only two categories is reasonable.

Capped L1-norm distance metric-based fast robust twin extreme learning machine

In this paper, we propose a new 0.00,0.00,1.00 fast robust twin extreme learning machine (FRTELM) based on the least squares sense and capped L1-norm distance metric. FRTELM first replaced the inequality constraints in TELM with equality constraints, and then introduced the capped L1-norm distance metric to replace the L2-norm distance metric in TELM. FRTELM not only retains the advantages of TELM, but also overcomes the shortcomings of TELM exaggeration of outliers based on squared L2-norm distance metrics. This improvement improves the robustness and learning efficiency of TELM in solving outlier problems. An efficient optimization algorithm is exploited to solve the nonconvex and nonsmooth challenging problem. In theory, we analyze and discuss the complexity of the algorithm in detail, and prove the convergence and local optimality of the algorithm. Extensive experiments conducted across multiple datasets demonstrates that the proposed method is competitive with state-of-the-art methods in terms of robustness and feasibility.

Robust discriminant feature selection via joint [formula omitted]-norm distance minimization and maximization

Discriminative Feature Selection (DFS) is an algorithm, proposed recently for effective feature selection by considering both joint linear discriminant analysis and row sparsity regularization. However, this method is not robust enough to protect the data from outliers, because it utilizes the squared L2-norm distance metric. To overcome this problem, we present in this paper, a novel discriminative feature selection algorithm, which uses the robust L2,1-norm for measuring the distances in DFS. Although the algorithm is apparently simple, it should not be considered trivial because of its non-convexity. Also, we present an analysis of the convergence, both theoretical and empirical. More importantly, we proposed an iterative algorithm to achieve optimal results. Experimental results, using various data sets, demonstrate the effectiveness of the proposed method.

Generalized two-dimensional PCA based on $$\ell _{2,p}$$-norm minimization

To exploit the information from two-dimensional structured data, two-dimensional principal component analysis (2-DPCA) has been widely used for dimensionality reduction and feature extraction. However, 2-DPCA is sensitive to outliers which are common in real applications. Therefore, many robust 2-DPCA methods have been proposed to improve the robustness of 2-DPCA. But existing robust 2-DPCAs have several weaknesses. First, these methods cannot be robust enough to outliers. Second, to center a sample set mixed with outliers using the L2-norm distance is usually biased. Third, most methods do not preserve the nice property of 2-DPCA (rotational invariance), which is important for learning algorithm. To alleviate these issues, we present a generalized robust 2-DPCA, which is named as 2-DPCA with $$\ell _{2,p}$$ -norm minimization ( $$\ell _{2,p}$$ -2-DPCA), for image representation and recognition. In $$\ell _{2,p}$$ -2-DPCA, $$\ell _{2,p}$$ -norm is employed as the distance metric to measure the reconstruction error, which can alleviate the effect of outliers. Therefore, the proposed method is robust to outliers and preserves the desirable property of 2-DPCA which is invariant to rotational and well characterizes the geometric structure of samples. Moreover, most existing robust PCA methods estimate sample mean from database with outliers by averaging, which is usually biased. Sample mean are treated as an unknown variable to remedy the bias of computing sample mean in $$\ell _{2,p}$$ -2-DPCA. To solve $$\ell _{2,p}$$ -2-DPCA, we propose an iterative algorithm, which has a closed-form solution in each iteration. Experimental results on several benchmark databases demonstrate the effectiveness and advantages of our method.

Nonpeaked Discriminant Analysis for Data Representation.

Of late, there are many studies on the robust discriminant analysis, which adopt L1-norm as the distance metric, but their results are not robust enough to gain universal acceptance. To overcome this problem, the authors of this article present a nonpeaked discriminant analysis (NPDA) technique, in which cutting L1-norm is adopted as the distance metric. As this kind of norm can better eliminate heavy outliers in learning models, the proposed algorithm is expected to be stronger in performing feature extraction tasks for data representation than the existing robust discriminant analysis techniques, which are based on the L1-norm distance metric. The authors also present a comprehensive analysis to show that cutting L1-norm distance can be computed equally well, using the difference between two special convex functions. Against this background, an efficient iterative algorithm is designed for the optimization of the proposed objective. Theoretical proofs on the convergence of the algorithm are also presented. Theoretical insights and effectiveness of the proposed method are validated by experimental tests on several real data sets.

Deep stacking l1-norm center configuration convex hull and its application in fault diagnosis of rolling bearing

Maximum margin classifier with flexible convex hulls (MMC-FCH) is an adaptive pattern recognition method based on convex hull vector and shrinkage factor, which can effectively identify different fault states. However, MMC-FCH is a shallow learning algorithm that cannot effectively diagnose complex signals. Meanwhile, MMC-FCH is essentially a binary classifier. For multi-classification, MMC-FCH can only perform multiple binary classifications. To overcome the shortcomings of MMC-FCH, we propose a deep stacking center configuration convex hull (DSC3H), which combines the convex hull with the idea of stacking-based representation learning (SRL). In DSC3H, the output of all previous modules combine with the original signal as the input of the next module to fully learn the information in the original signal. At the same time, the concept of center configuration is used to construct the multi-classification objective function of center configuration convex hull (C3H). However, redundant information and noise information may still exist in the proposed DSC3H method. Therefore, we further propose a deep stacking l1-norm center configuration convex hull (DSl1C3H) method, which makes the prediction model more robust and sparser under the constraint of l1-norm distance. The experiments of rolling bearings show that the proposed DSl1C3H method has better classification performance.

Flexible non-greedy discriminant subspace feature extraction

Recently, L1-norm-based non-greedy linear discriminant analysis (NLDA-L1) for feature extraction has been shown to be effective for dimensionality reduction, which obtains projection vectors by a non-greedy algorithm. However, it usually acquires unsatisfactory performances due to the utilization of L1-norm distance measurement. Therefore, in this brief paper, we propose a flexible non-greedy discriminant subspace feature extraction method, which is an extension of NLDA-L1 by maximizing the ratio of Lp-norm inter-class dispersion to intra-class dispersion. Besides, we put forward a powerful iterative algorithm to solve the resulted objective function and also conduct theoretical analysis on the algorithm. Finally, experimental results on image databases show the effectiveness of our method

Robust capped L1-norm twin support vector machine

Twin support vector machine (TWSVM) is a classical and effective classifier for binary classification. However, its robustness cannot be guaranteed due to the utilization of squared L2-norm distance that can usually exaggerate the influence of outliers. In this paper, we propose a new robust capped L1-norm twin support vector machine (CTWSVM), which sustains the advantages of TWSVM and promotes the robustness in solving a binary classification problem with outliers. The solution of the proposed method can be achieved by optimizing a pair of capped L1-norm related problems using a newly-designed effective iterative algorithm. Also, we present some theoretical analysis on existence of local optimum and convergence of the algorithm. Extensive experiments on an artificial dataset and several UCI datasets demonstrate the robustness and feasibility of our proposed CTWSVM.

Low-light image enhancement using Gaussian Process for features retrieval

Low-light is a challenging environment for image processing and computer vision tasks, either in contrast enhancement for better visibility and quality, or application oriented tasks such as detection. We found that the current trend of low-light enhancement research is heavily on quality improvement. In this work, we aim to shift the focus towards a more functional direction, that is enhancement that prioritizes feature retrieval. For this reason, we first propose to model low-light enhancement as a set of localized functions using Gaussian Process (GP) that is trained at runtime using data from a simple Convolutional Neural Network (CNN) to provide the necessary feature information as reference. The CNN is in turn trained using large amount of synthetic data, based upon the luminance distribution of real world low-light images to learn the relationship between features and pixels. Secondly, we also proposed two new evaluation metrics to better assess enhancement algorithms to support high level computer vision tasks, namely, local features matching and intensity histogram similarity. In our experiments, our proposed low-light enhancement framework outperforms the state-of-the-arts with significant improvement in Recall, F1, and F2-score of SIFT features matching, and achieve comparable results for l1-norm distance of histograms as well as the PSNR. Moreover, our analysis of the performance showed that the PSNR quality metric is not only unable to assess the practicality of the results, but also inappropriately gives high assessment to low visual quality images.

A Trace Lasso Regularized Robust Nonparallel Proximal Support Vector Machine for Noisy Classification

Generalized eigenvalue proximal support vector machine (GEPSVM) and its improvement IGEPSVM are excellent nonparallel classification methods due to their excellent generalization. However, all of them adopt the square L 2 -norm metric to implement their empirical risk or penalty, which is sensitive to noise and outliers. Moreover, in many real-world learning tasks, it is a significant challenge for GEPSVMs when the data appears highly correlated. To alleviate the above issues, in this paper, we propose a novel trace lasso regularized robust nonparallel proximal support vector machine (RNPSVM) for noisy classification. Compared with GEPSVMs, our RNPSVM enjoys the following advantages. First, the empirical risk of RNPSVM is implemented by the robust L 1 -norm metric with a maximum margin criterion. Namely, it aims to maximize the L 1 -norm inter-class distance dispersion while minimizing the L 1 -norm intra-class distance dispersion simultaneously. Second, to capture the sparsity and the underlying correlation of data, a trace lasso (adaptive norm-based training data) is further introduced to regularize RNPSVM. Third, an iterative algorithm is designed to solve the maximization optimization problem of RNPSVM, whose convergence is guaranteed theoretically. The extensive experimental results on both synthetic and real-world noisy datasets demonstrate the effectiveness of RNPSVM.

IFLEX: A Fully Open-Source, High-Density Field-Programmable Gate Array (FPGA)-Based Hardware Co-Processor for Vector Similarity Searching

Vector similarity searching consists of comparing a query vector against a high volume of entries in a reference data set, according to a chosen similarity metric such as the L1-norm, L2-norm, or Hamming distance. Large-scale research and commercial applications of these computations are developing rapidly across artificial intelligence fields as diverse as semantic text querying, retrieval of multimedia materials, and prediction of the properties of pharmacological molecules and engineered materials. While vector similarity searching is, at present, predominantly implemented on standard central processing unit (CPU) hardware running optimized indexing algorithms, the interest in massively-parallel computing architectures is increasing. Accordingly, a range of systems based on graphics processing units (GPU) and field-programmable gate arrays (FPGA) have been proposed; however, the availability of the design materials for these systems remains largely confined to a small number of corporations and research institutions. Here, we introduce a fully open-source hardware accelerator for vector similarity searching, based on an array of 21 FPGAs densely intertwined with 42 GB of high-speed dynamic memory and installed on a custom-designed compute node board, which yields an aggregate bandwidth of 33.6 GB/s and can be seamlessly reconfigured to implement nearly arbitrary distance calculations. A novel logic and software architecture, based on a lane-wise organization of independent engines implementing distributed distance calculation and sorting, allows attaining noteworthy query latency and power consumption performance on both single- and multi-node system configurations. The entire circuit board hardware, FPGA logic, and host software design is herein presented and freely provided for unlimited use, supporting open innovation and research in this area.

Multi-View Learning With Robust Generalized Eigenvalue Proximal SVM

Multi-view learning mechanism, which enhances learning performance by training multi-model data sets, is a popular filed in recent years. Multi-view generalized eigenvalue proximal support vector machine (MvGSVM), as a most recently proposed classifier, has been shown to be successful in multi-model classification, which incorporates multi-view learning into classical GEPSVM. However, this method is still based on squared L2-norm distance measure, thus its robustness is not guaranteed in the presence of outliers. To address this problem, we propose a robust multi-view GEPSVM based on Lp-norm minimization and Ls-norm maximization. But, the introduction of the Lp-norm and Ls-norm makes the problem different from the generalized eigenvalue problem. So an efficient iterative algorithm is designed to solve this problem, and we also give the proof of convergence of the algorithm. The performances in extensive experiments demonstrate the effectiveness and robustness of the algorithm.

Efficient and robust TWSVM classification via a minimum L1-norm distance metric criterion

A twin support vector machine (TWSVM) is a classic distance metric learning method for classification problems. The TWSVM criterion is formulated based on the squared L2-norm distance, making it prone to being influenced by the presence of outliers. In this paper, to develop a robust distance metric learning method, we propose a new objective function, called L1-TWSVM, for the TWSVM classifier using the robust L1-norm distance metric. The optimization strategy is to maximize the ratio of the inter-class distance dispersion to the intra-class distance dispersion by using the robust L1-norm distance rather than the traditional L2-norm distance. The resulting objective function is much more challenging to optimize because it involves a non-smooth L1-norm term. As an important contribution of this paper, we design a simple but valid iterative algorithm for solving L1-norm optimal problems. This algorithm is easy to implement, and its convergence to an optimum is theoretically guaranteed. The efficiency and robustness of L1-TWSVM have been validated by extensive experiments on both UCI datasets as well as synthetic datasets. The promising experimental results indicate that our proposal approaches outperform relevant state-of-the-art methods in all kinds of experimental settings.

Unsupervised Deep Feature Learning for Remote Sensing Image Retrieval

Due to the specific characteristics and complicated contents of remote sensing (RS) images, remote sensing image retrieval (RSIR) is always an open and tough research topic in the RS community. There are two basic blocks in RSIR, including feature learning and similarity matching. In this paper, we focus on developing an effective feature learning method for RSIR. With the help of the deep learning technique, the proposed feature learning method is designed under the bag-of-words (BOW) paradigm. Thus, we name the obtained feature deep BOW (DBOW). The learning process consists of two parts, including image descriptor learning and feature construction. First, to explore the complex contents within the RS image, we extract the image descriptor in the image patch level rather than the whole image. In addition, instead of using the handcrafted feature to describe the patches, we propose the deep convolutional auto-encoder (DCAE) model to deeply learn the discriminative descriptor for the RS image. Second, the k-means algorithm is selected to generate the codebook using the obtained deep descriptors. Then, the final histogrammic DBOW features are acquired by counting the frequency of the single code word. When we get the DBOW features from the RS images, the similarities between RS images are measured using L1-norm distance. Then, the retrieval results can be acquired according to the similarity order. The encouraging experimental results counted on four public RS image archives demonstrate that our DBOW feature is effective for the RSIR task. Compared with the existing RS image features, our DBOW can achieve improved behavior on RSIR.

Lp- and Ls-Norm Distance Based Robust Linear Discriminant Analysis

Recently, L1-norm distance measure based Linear Discriminant Analysis (LDA) techniques have been shown to be robust against outliers. However, these methods have no guarantee of obtaining a satisfactory-enough performance due to the insufficient robustness of L1-norm measure. To mitigate this problem, inspired by recent works on Lp-norm based learning, this paper proposes a new discriminant method, called Lp- and Ls-Norm Distance Based Robust Linear Discriminant Analysis (FLDA-Lsp). The proposed method achieves robustness by replacing the L2-norm within- and between-class distances in conventional LDA with Lp- and Ls-norm ones. By specifying the values of p and s, many of previous efforts can be naturally expressed by our objective. The requirement of simultaneously maximizing and minimizing a number of Lp- and Ls-norm terms results in a difficulty to the optimization of the formulated objective. As one of the important contributions of this paper, we design an efficient iterative algorithm to address this problem, and also conduct some insightful analysis on the existence of local minimum and the convergence of the proposed algorithm. Theoretical insights of our method are further supported by promising experimental results on several images databases.

Thermal quantum coherence and correlation in the extended XY spin chain

Quantum coherence and correlation of thermal states in the extended XY spin chain are studied in terms of the recently proposed l1 norm, skew information, and Bures distance of geometry discord (BGD), respectively. The entanglement measured via concurrence is calculated for reference. A two-dimensional susceptibility is introduced to explore their capability in highlighting the critical lines associated with quantum phase transitions in the model. It is shown that the susceptibility of the skew information and BGD is a genuine indicator of quantum phase transitions, and characterizes the factorization. However, the l1 norm is trivial for the factorization. An explicit scaling law of BGD is captured at low temperature in the XY model. In contrast to the entanglement, quantum coherence reveals a kind of long-range nonclassical correlation. Moreover, the obvious relation among model parameters is extracted for the factorized line in the extended model. Those are instructive for the understanding of quantum coherence and correlation in the theory of quantum information, and quantum phase transitions and factorization in condensed-matter physics.

Fast Algorithms for Computing Path-Difference Distances.

Tree comparison metrics are an important tool for the study of phylogenetic trees. Path-difference distances measure the dissimilarity between two phylogenetic trees (on the same set of taxa) by comparing their path-length vectors. Various norms can be applied to this distance. Three important examples are the $l_{1}\text{-},\;l_{2}\text{-}$l1-,l2-, and $l_{{\infty }}$l∞-norms. The previous best algorithms for computing path-difference distances all have $O(n^{2})$O(n2) running time. In this paper, we show how to compute the $l_{1}$l1-norm path-difference distance in $O(n\;{\log}^{2}\;n)$O(nlog2n) time and how to compute the $l_{2}$l2- and $l_{{\infty }}$l∞-norm path-difference distances in $O(n\;{\log}\;n)$O(nlogn) time. By extending the presented algorithms, we also show that the $l_{p}$lp-norm path-difference distance can be computed in $O(pn\;{\log}^{2}\;n)$O(pnlog2n) time for any positive integer $p$p. In addition, when the integer $p$p is even, we show that the distance can be computed in $O(p^{2}n\;{\log}\;n)$O(p2nlogn) time as well.

L1-Norm Distance Linear Discriminant Analysis Based on an Effective Iterative Algorithm

Recent works have proposed two L1-norm distance measure-based linear discriminant analysis (LDA) methods, L1-LD and LDA-L1, which aim to promote the robustness of the conventional LDA against outliers. In LDA-L1, a gradient ascending iterative algorithm is applied, which, however, suffers from the choice of stepwise. In L1-LDA, an alternating optimization strategy is proposed to overcome this problem. In this paper, however, we show that due to the use of this strategy, L1-LDA is accompanied with some serious problems that hinder the derivation of the optimal discrimination for data. Then, we propose an effective iterative framework to solve a general L1-norm minimization–maximization ( minmax ) problem. Based on the framework, we further develop a effective L1-norm distance-based LDA (called L1-ELDA) method. Theoretical insights into the convergence and effectiveness of our algorithm are provided and further verified by extensive experimental results on image databases.