Low-dimensional Transformation Research Articles

GenURL: A General Framework for Unsupervised Representation Learning.

Unsupervised representation learning (URL) that learns compact embeddings of high-dimensional data without supervision has achieved remarkable progress recently. However, the development of URLs for different requirements is independent, which limits the generalization of the algorithms, especially prohibitive as the number of tasks grows. For example, dimension reduction (DR) methods, t-SNE and UMAP, optimize pairwise data relationships by preserving the global geometric structure, while self-supervised learning, SimCLR and BYOL, focuses on mining the local statistics of instances under specific augmentations. To address this dilemma, we summarize and propose a unified similarity-based URL framework, GenURL, which can adapt to various URL tasks smoothly. In this article, we regard URL tasks as different implicit constraints on the data geometric structure that help to seek optimal low-dimensional representations that boil down to data structural modeling (DSM) and low-dimensional transformation (LDT). Specifically, DSM provides a structure-based submodule to describe the global structures, and LDT learns compact low-dimensional embeddings with given pretext tasks. Moreover, an objective function, general Kullback-Leibler (GKL) divergence, is proposed to connect DSM and LDT naturally. Comprehensive experiments demonstrate that GenURL achieves consistent state-of-the-art performance in self-supervised visual learning, unsupervised knowledge distillation (KD), graph embeddings (GEs), and DR.

Dual-ISM: Duality-Based Image Sequence Matching for Similar Image Search

In this paper, we propose the duality-based image sequence matching method, which is called Dual-ISM, a subsequence matching method for searching for similar images. We first extract feature points from the given image data and configure the feature vectors as one data sequence. Next, the feature vectors are configured in the form of a disjoint window, and a low-dimensional transformation is carried out. Subsequently, the query image that is entered to construct the candidate set is similarly subjected to a low-dimensional transformation, and the low-dimensional transformed window of the data sequence and window that are less than the allowable value, ε, is regarded as the candidate set using a distance calculation. Finally, similar images are searched in the candidate set using the distance calculation that are based on the original feature vector.

Analysis of Denoising Methods of Underwater Acoustic Pulse Signal Based on Wavelet and Wavelet Packet

Underwater acoustic countermeasures include active sonar countermeasures and passive sonar countermeasures. In order to achieve the best operational efficiency of jamming equipment, the generation of jamming signals must adapt to the development trend of underwater acoustic detection technology-nonlinear time-varying and broadband characteristics. Feature extraction is to map the high-dimensional original data to the low-dimensional transformation space through a certain mapping relationship, which can suppress a large amount of redundant information in the data and highlight the category information of the data. One of the applications of wavelet analysis in signal analysis and processing is to remove noise components from signals. In practical engineering applications, sampled signals are inevitably polluted by various noises and interferences. Through the analysis of noise characteristics, it can be seen that the wavelet denoising method is very effective in removing signal noise. In this paper, aiming at the signals with different spectrum distribution, we use various methods to de-noise in order to find the de-noising methods suitable for underwater acoustic pulse signals with different frequency characteristics, and thus find an effective means to detect signals.

A review on the generalization of sufficient dimension reduction methods with the additional information

Sufficient dimension reduction (SDR) has been shown to be a powerful statistical method that is able to reduce the dimension of covariates without losing information with respect to the response. Subsequent analysis can then be based on a lower dimensional transformations of covariates, which has the potential to assist model building and to increase the estimation efficiency. In some situations, the additional information could be also available during the data collection process. Although one can proceed with the conventional method, properly utilizing the additional information can greatly improve making statistical inference. It is thus of interest to incorporate the additional information into the practice of SDR methods. In this article, we review the generalizations of SDR methods that are able to utilize different types of the additional information. One will see that, depending on the sources of the additional information, different techniques are required to modify conventional SDR methods to improve estimating the target of interest. WIREs Comput Stat 2017, 9:e1401. doi: 10.1002/wics.1401This article is categorized under: Applications of Computational Statistics > Computational Mathematics Applications of Computational Statistics > Computational and Molecular Biology Statistical and Graphical Methods of Data Analysis > Dimension Reduction

Spatial Evidential Clustering With Adaptive Distance Metric for Tumor Segmentation in FDG-PET Images.

While the accurate delineation of tumor volumes in FDG-positron emission tomography (PET) is a vital task for diverse objectives in clinical oncology, noise and blur due to the imaging system make it a challenging work. In this paper, we propose to address the imprecision and noise inherent in PET using Dempster-Shafer theory, a powerful tool for modeling and reasoning with uncertain and/or imprecise information. Based on Dempster-Shafer theory, a novel evidential clustering algorithm is proposed and tailored for the tumor segmentation task in three-dimensional. For accurate clustering of PET voxels, each voxel is described not only by the single intensity value but also complementarily by textural features extracted from a patch surrounding the voxel. Considering that there are a large amount of textures without consensus regarding the most informative ones, and some of the extracted features are even unreliable due to the low-quality PET images, a specific procedure is included in the proposed clustering algorithm to adapt distance metric for properly representing the clustering distortions and the similarities between neighboring voxels. This integrated metric adaptation procedure will realize a low-dimensional transformation from the original space, and will limit the influence of unreliable inputs via feature selection. A Dempster-Shafer-theory-based spatial regularization is also proposed and included in the clustering algorithm, so as to effectively quantify the local homogeneity. The proposed method has been compared with other methods on the real-patient FDG-PET images, showing good performance.

Efficient Two-Step Protocol and Its Discriminative Feature Selections in Secure Similar Document Detection

Recently, the risk of information disclosure is increasing significantly. Accordingly, privacy-preserving data mining (PPDM) is being actively studied to obtain accurate mining results while preserving the data privacy. We here focus on secure similar document detection (SSDD), which identifies similar documents of two parties when each party does not disclose its own sensitive documents to the another party. In this paper, we propose an efficient two-step protocol that exploits a feature selection as a lower-dimensional transformation, and we present discriminative feature selections to maximize the performance of the protocol. The proposed protocol consists of two steps: thefilteringstep and thepostprocessingstep. For the feature selection, we first consider the simplest one, random projection (RP), and propose its two-step solution,SSDD-RP. We then present two discriminative feature selections and their solutions:SSDD-LFwhich selects a few dimensions locally frequent in the current querying vector andSSDD-GFwhich selects ones globally frequent in the set of all document vectors. We finally propose a hybrid one,SSDD-HF, which takes advantage of bothSSDD-LFandSSDD-GF. We empirically show that the proposed two-step protocol significantly outperforms the previous one-step protocol by three or four orders of magnitude.

Dissimilarity Metric Learning in the Belief Function Framework

The evidential K-nearest-neighbor (EK-NN) method provided a global treatment of imperfect knowledge regarding the class membership of training patterns. It has outperformed traditional K-NN rules in many applications, but still shares some of their basic limitations, e.g., 1) classification accuracy depends heavily on how to quantify the dissimilarity between different patterns and 2) no guarantee for satisfactory performance when training patterns contain unreliable (imprecise and/or uncertain) input features. In this paper, we propose to address these issues by learning a suitable metric, using a low-dimensional transformation of the input space, so as to maximize both the accuracy and efficiency of the EK-NN classification. To this end, a novel loss function to learn the dissimilarity metric is constructed. It consists of two terms: the first one quantifies the imprecision regarding the class membership of each training pattern, while, by means of feature selection, the second one controls the influence of unreliable input features on the output linear transformation. The proposed method has been compared with some other metric learning methods on several synthetic and real datasets. It consistently led to comparable performance with regard to testing accuracy and class structure visualization.

대용량 데이터베이스에서 다차원 인덱스를 사용한 효율적인 다단계 k-NN 검색

In this paper, we address the problem of improving the performance of multi-step k-NN search using multi-dimensional indexes. Due to information loss by lower-dimensional transformations, existing multi-step k-NN search solutions produce a large tolerance (i.e., a large search range), and thus, incur a large number of candidates, which are retrieved by a range query. Those many candidates lead to overwhelming I/O and CPU overheads in the postprocessing step. To overcome this problem, we propose two efficient solutions that improve the search performance by reducing the tolerance of a range query, and accordingly, reducing the number of candidates. First, we propose a tolerance reduction-based (approximate) solution that forcibly decreases the tolerance, which is determined by a k-NN query on the index, by the average ratio of high- and low-dimensional distances. Second, we propose a coefficient control-based (exact) solution that uses c k instead of k in a k-NN query to obtain a tigher tolerance and performs a range query using this tigher tolerance. Experimental results show that the proposed solutions significantly reduce the number of candidates, and accordingly, improve the search performance in comparison with the existing multi-step k-NN solution.

Safe MBR-transformation in similar sequence matching

When a multidimensional index is used for similar sequence matching, the traditional approach is to transform high-dimensional window sequences to low-dimensional sequences and bounding them into a low-dimensional minimum bounding rectangle (MBR). In this paper, we propose a new approach which constructs a low-dimensional MBR by directly transforming a high-dimensional MBR (called MBR-transformation) bounding the high-dimensional sequences. This approach significantly reduces the number of lower-dimensional transformations needed in similar sequence matching. However, it poses a risk that some transformed sequences may fall outside the transformed low-dimensional MBR. We thus propose safe MBR-transformation which has the property that every possible transformed sequence is inside a safe MBR-transformed MBR. Then, considering the discrete Fourier transform (DFT) and the discrete Cosine transform (DCT), we prove that they are not safe as MBR-transformations, and modify them to become safe MBR-transformations (called mbrDFT if DFT-based and mbrDCT if DCT-based). Then, we prove the safeness and optimality of mbrDFT and mbrDCT. Analyses and experiments show that the mbrDFT and mbrDCT reduce the execution time by several orders of magnitude due to the reduction in the number of lower-dimensional transformations. The proposed safe MBR-transformation provides a useful framework for a variety of applications that require a direct transformation of a high-dimensional MBR to a low-dimensional MBR.

Multiple-instance discriminant analysis

Multiple-instance discriminant analysis (MIDA) is proposed to cope with the feature extraction problem in multiple-instance learning. Similar to MidLABS, MIDA is also derived from linear discriminant analysis (LDA), and both algorithms can be treated as multiple-instance extensions of LDA. Different from MidLABS which learns from the bag level, MIDA is designed from the instance level. MIDA consists of two versions, i.e., binary-class MIDA (B-MIDA) and multi-class MIDA (M-MIDA), which are utilized to cope with binary-class (standard) and multi-class multiple-instance learning tasks, respectively. The block coordinate ascent approach, by which we seek positive prototypes (the most positive instance in a positive bag is termed as the positive prototype of this bag) and projection vectors alternatively and iteratively, is proposed to optimize B-MIDA and M-MIDA to obtain lower dimensional transformation subspaces. Extensive experiments empirically demonstrate the effectiveness of B-MIDA and M-MIDA in extracting discriminative components and weakening class-label ambiguities for instances in positive bags.

Hybrid Lower-Dimensional Transformation for Similar Sequence Matching

ABSTRACT We generally use lower-dimensional transformations to convert high-dimensional sequences into low-dimensional points in similar sequence matching. These traditional transformations, however, show different characteristics in indexing performance by the type of time-series data. It means that the selection of lower-dimensional transformations makes a significant influence on the indexing performance in similar sequence matching. To solve this problem, in this paper we propose a hybrid approach that integrates multiple transformations and uses them in a single multidimensional index. We first propose a new notion of hybrid lower-dimensional transformation that exploits different lower-dimensional transformations for a sequence. We next define the hybrid distance to compute the distance between the transformed sequences. We then formally prove that the hybrid approach performs the similar sequence matching correctly. We also present the index building and the similar sequence matching algorithms that use the hybrid approach. Experimental results for various time-series data sets show that our hybrid approach outperforms the single transformation-based approach. These results indicate that the hybrid approach can be widely used for various time-series data with different characteristics.Key Words:Data Mining, Time-Series Databases, Hybrid Lower-Dimensional Transformation, Similar Sequence Matching

유사 시퀀스 매칭을 위한 하이브리드 저차원 변환

We generally use lower-dimensional transformations to convert high-dimensional sequences into low-dimensional points in similar sequence matching. These traditional transformations, however, show different characteristics in indexing performance by the type of time-series data. It means that the selection of lower-dimensional transformations makes a significant influence on the indexing performance in similar sequence matching. To solve this problem, in this paper we propose a hybrid approach that integrates multiple transformations and uses them in a single multidimensional index. We first propose a new notion of hybrid lower-dimensional transformation that exploits different lower-dimensional transformations for a sequence. We next define the hybrid distance to compute the distance between the transformed sequences. We then formally prove that the hybrid approach performs the similar sequence matching correctly. We also present the index building and the similar sequence matching algorithms that use the hybrid approach. Experimental results for various time-series data sets show that our hybrid approach outperforms the single transformation-based approach. These results indicate that the hybrid approach can be widely used for various time-series data with different characteristics.