Real-valued Data Research Articles

Robust Semiparametric Efficient Estimators in Complex Elliptically Symmetric Distributions

Covariance matrices play a major role in statistics, signal processing and machine learning applications. This paper focuses on the semi-parametric covariance/scatter matrix estimation problem in elliptical distributions. The class of elliptical distributions can be seen as a semiparametric model where the finite-dimensional vector of interest is given by the location vector and by the (vectorized) covariance/scatter matrix, while the density generator represents an infinite-dimensional nuisance function. The main aim of this work is then to provide possible estimators of the finite-dimensional parameter vector able to reconcile the two dichotomic concepts of robustness and (semiparametric) efficiency. An R-estimator satisfying these requirements has been recently proposed by Hallin, Oja and Paindaveine for real-valued elliptical data by exploiting the Le Cam's theory of one-step efficient estimators and the rank-based statistics. In this paper, we firstly recall the building blocks underlying the derivation of such real-valued R-estimator, then its extension to complex-valued data is proposed. Moreover, through numerical simulations, its estimation performance and robustness to outliers are investigated in a finite-sample regime.

Robust Optimality and Duality in Multiobjective Optimization Problems under Data Uncertainty

In this paper, we employ advanced techniques of variational analysis and generalized differentiation to examine robust optimality conditions and robust duality for an uncertain nonsmooth multiobjective optimization problem under arbitrary uncertainty nonempty sets. We establish necessary and sufficient optimality conditions for (local) robust (weakly) efficient solutions of the considered problem. Our problem involves nonsmooth real-valued functions and data uncertainty in both the objective and constraint functions, and its necessary and sufficient optimality conditions are exhibited in terms of multipliers and the Mordukhovich or Clarke subdifferentials of the related functions. Moreover, we formulate a dual multiobjective problem to the underlying program and examine robust weak, strong, and converse duality relations between the primal problem and its dual under assumptions of (strictly) generalized convexity.

Machine Fault Detection for Intelligent Self-Driving Networks

To build the mechanism of a SelfDN is becoming an emerging direction for future IoT. The detection of device status, such as fault or normal, is a very fundamental module in SelfDN. In this article, we first review recent studies devoted to applying fault detection techniques in IoT networks. Taking the challenge of processing the real-valued IoT data into account, we propose a novel fault detection architecture for SelfDN. Under this architecture, we present an algorithm, named GBRBM-based deep neural network with auto-encoder (i.e., GBRBM-DAE) to transform the fault detection problem into a classification problem. The real-world trace-driven experimental results show that the proposed algorithm outperforms other popular machine learning algorithms, including linear discriminant analysis, support vector machine, pure deep neural network, and so on. Finally, we summarize some open issues of this study. We expect that this article will inspire successive studies on the related topics of SelfDN.

Three-way decision for incomplete real-valued data

An information system (IS) is a database that expresses relationships between objects and attributes. An IS with decision attributes is said to be a decision information system (DIS). An incomplete real-valued decision information system (IRVDIS) is a DIS based on incomplete real-valued data. This paper studies three-way decision (3WD) for incomplete real-valued data and its application. In the first place, the distance between two objects on the basis of the conditional attribute set in an IRVDIS is constructed. In the next place, the fuzzy Tcos-equivalence relation on the object set of an IRVDIS is received by means of Gaussian kernel. After that, the decision-theoretic rough set model for an IRVDIS is presented. Furthermore, the 3WD method is proposed based on this model. Lastly, to illustrate the feasibility of the proposed method, an application of the proposed method is given. It is worth mentioning that levels of risk may be determined by thresholds that can be directly acquired according to risk preference of different decision-makers, as well as the decision rule for each decision class under different levels of risk is showed in tabular forms.

Mahalanobis-Taguchi System for Symbolic Interval Data Based on Kernel Mahalanobis Distance

Mahalanobis-Taguchi System (MTS), as a pattern recognition method by constructing a continuous measurement scale, has a very good performance on classification and feature selection for real-valued data. However, the record of symbolic interval data has become a common practice with the recent advances in database technologies. Kernel methods not only are powerful statistical nonlinear learning methods, but also can be defined over objects as diverse as graphs, sets, strings, and text documents. In this paper, we derive kernel Mahalanobis distance (KMD) to extend MTS to symbolic interval data. To evaluate the proposed method, four experiments with synthetic symbolic interval data sets and seven experiments with real symbolic interval data sets are performed and we have compared our method with MTS based on interval Mahalanobis distance (IMD). The experimental results show our method has a better classification performance than MTS based on IMD on Accuracy, Specificity, Sensitivity, and G-means. However, MTS based on IMD has a stronger dimension reduction rate than our method.

Active Incremental Feature Selection Using a Fuzzy-Rough-Set-Based Information Entropy

Feature selection is a popular technique of preprocessing data. In order to deal with dynamic or large data, incremental feature selection has been developed, in which the features selected from existing data are integrated with those mined from both existing and dynamic data in the manner of incremental computation. Fuzzy rough set theory is powerful in handling uncertainty in real-valued data or even mixed data, and one of its most important applications is feature selection. Nevertheless, not much work has been found on fuzzy-rough-set-based incremental feature selection. Therefore, in this article, we investigate the incremental feature selection using a fuzzy-rough-set-based information entropy with incoming instances. Specifically, the representative instances from the incoming ones are first selected according to the information coverage of fuzzy granules generated by fuzzy rough sets. Then, the incremental mechanism of the fuzzy-rough-set-based information entropy is formulated by adding newcome instances. Finally, an incremental feature selection procedure, which we call active incremental feature selection, is proposed. Furthermore, some numerical experiments are conducted to assess the performance of the proposed feature selection algorithm, and the results show that our algorithm is of a prominent advantage in terms of computational time, especially for a dataset with large number of instances.

Real to H-Space Autoencoders for Theme Identification in Telephone Conversations

Machine learning (ML) and deep learning with deep neural networks (DNN), have drastically improved the performances of modern systems on numerous spoken language understanding (SLU) related tasks. Since most of current researches focus on new neural architectures to enhance the performances in realistic conditions, few recent works investigated the use of different algebras with neural networks (NN), to better represent the nature of the data being processed. To this extent, quaternion-valued neural networks (QNN) have shown better performances, and an important reduction of the number of neural parameters compared to traditional real-valued neural networks, when dealing with multidimensional signal. Nonetheless, the use of QNNs is strictly limited to quaternion input or output features. This article introduces a new unsupervised method based on a hybrid autoencoder (AE) called real-to-quaternion autoencoder (R2H), to extract a quaternion-valued input signal from any real-valued data, to be processed by QNNs. The experiments performed to identify the most related theme of a given telephone conversation from a customer care service (CCS), demonstrate that the R2H approach outperforms all the previously established models, either real- or quaternion-valued ones, in term of accuracy and with up to four times fewer neural parameters.

A Hardware-Deployable Neuromorphic Solution for Encoding and Classification of Electronic Nose Data.

In several application domains, electronic nose systems employing conventional data processing approaches incur substantial power and computational costs and limitations, such as significant latency and poor accuracy for classification. Recent developments in spike-based bio-inspired approaches have delivered solutions for the highly accurate classification of multivariate sensor data with minimized computational and power requirements. Although these methods have addressed issues related to efficient data processing and classification accuracy, other areas, such as reducing the processing latency to support real-time application and deploying spike-based solutions on supported hardware, have yet to be studied in detail. Through this investigation, we proposed a spiking neural network (SNN)-based classifier, implemented in a chip-emulation-based development environment, that can be seamlessly deployed on a neuromorphic system-on-a-chip (NSoC). Under three different scenarios of increasing complexity, the SNN was determined to be able to classify real-valued sensor data with greater than 90% accuracy and with a maximum latency of 3 s on the software-based platform. Highlights of this work included the design and implementation of a novel encoder for artificial olfactory systems, implementation of unsupervised spike-timing-dependent plasticity (STDP) for learning, and a foundational study on early classification capability using the SNN-based classifier.

Efficiently utilizing complex-valued PolSAR image data via a multi-task deep learning framework

Convolutional neural networks (CNNs) have been widely used to improve the accuracy of polarimetric synthetic aperture radar (PolSAR) image classification. However, in most studies, the difference between PolSAR images and optical images is rarely considered. Most of the existing CNNs are not tailored for the task of PolSAR image classification, in which complex-valued PolSAR data have been simply equated to real-valued data to fit the optical image processing architectures and avoid complex-valued operations. This is one of the reasons CNNs unable to perform their full capabilities in PolSAR classification. To solve the above problem, the objective of this paper is to develop a tailored CNN framework for PolSAR image classification, which can be implemented from two aspects: Seeking a better form of PolSAR data as the input of CNNs and building matched CNN architectures based on the proposed input form. In this paper, considering the properties of complex-valued numbers, amplitude and phase of complex-valued PolSAR data are extracted as the input for the first time to maintain the integrity of original information while avoiding immature complex-valued operations. Then, a multi-task CNN (MCNN) architecture is proposed to match the improved input form and achieve better classification results. Furthermore, depthwise separable convolution is introduced to the proposed architecture in order to better extract information from the phase information. Experiments on three PolSAR benchmark datasets not only prove that using amplitude and phase as the input do contribute to the improvement of PolSAR classification, but also verify the adaptability between the improved input form and the well-designed architectures.

Estimation of mutual information for real-valued data with error bars and controlled bias.

Estimation of mutual information between (multidimensional) real-valued variables is used in analysis of complex systems, biological systems, and recently also quantum systems. This estimation is a hard problem, and universally good estimators provably do not exist. We focus on the estimator introduced by Kraskov etal. [Phys. Rev. E 69, 066138 (2004)PLEEE81539-375510.1103/PhysRevE.69.066138] based on the statistics of distances between neighboring data points, which empirically works for a wide class of underlying probability distributions. First, we illustrate pitfalls of naively applying bootstrapping to estimate the variance of the mutual information estimate. Then we improve this estimator by (1) expanding its range of applicability and by providing (2) a self-consistent way of verifying the absence of bias, (3) a method for estimation of its variance, and (4) guidelines for choosing the values of the free parameter of the estimator. We demonstrate the performance of our estimator on synthetic data sets, as well as on neurophysiological and systems biology data sets.

Improved Gaussian–Bernoulli restricted Boltzmann machine for learning discriminative representations

Restricted Boltzmann machines (RBMs) have received considerable research interest in recent years because of their capability to discover latent representations in an unsupervised manner. The standard RBM is only suitable for processing binary-valued data. To address this limitation, the Gaussian–Bernoulli RBM (GRBM) has been designed to model real-valued data, particularly images. A GRBM that seeks to map real-valued data nonlinearly into a latent representation space is typically trained by contrastive divergence learning. However, most existing GRBM-based models neglect the inherent interpoint affinity information from the original data that can be used to enhance the expression ability of the model. In this study, a novel interpoint-affinity-based GRBM (abGRBM) is proposed to learn discriminative representations in the hidden layer. By incorporating the interpoint affinity information into the training process of the GRBM, the proposed model can not only utilize the GRBM’s powerful latent representation learning capabilities for real-valued data, it can also transform the original data into another space with improved separability. We prove the availability of our model using several image datasets of Microsoft Research Asia Multimedia for unsupervised clustering and supervised classification tasks. The experimental results show the superior performance of the GRBM in discovering discriminative representations and demonstrate the effectiveness of the affinity information.

Fuzzy-Rough Set Bireducts for Data Reduction

Data reduction is an important step that helps ease the computational intractability for learning techniques when data are large. This is particularly true for the huge datasets that have become commonplace in recent times. The main problem facing both data preprocessors and learning techniques is that data are expanding both in terms of dimensionality and also in terms of the number of data instances. Approaches based on fuzzy-rough sets offer many advantages for both feature selection and classification, particularly for real-valued and noisy data; however, the majority of recent approaches tend to address the task of data reduction in terms of either dimensionality or training data size in isolation. This paper demonstrates how the notion of fuzzy-rough bireducts can be used for the simultaneous reduction of data size and dimensionality. It also shows how bireducts and, therefore, reduced subtables of data can be used not only as a preprocessing tool but also for the learning of compact and robust classifiers. Furthermore, the ideas can also be extended to the unsupervised domain when dealing with unlabeled data. Experimental evaluation of various techniques demonstrate that high levels of simultaneous reduction of both dimensionality and data size can be achieved whilst maintaining robust performance.

Non-unique decision differential entropy-based feature selection

Feature selection plays an important role in reducing irrelevant and redundant features, while retaining the underlying semantics of selected ones. An effective feature selection method is expected to result in a significantly reduced subset of the original features without sacrificing the quality of problem-solving (e.g., classification). In this paper, a non-unique decision measure is proposed that captures the degree of a given feature subset being relevant to different categories. This helps to represent the uncertainty information in the boundary region of a granular model, such as rough sets or fuzzy-rough sets in an efficient manner. Based on this measure, the paper further introduce a differentiation entropy as an evaluator of feature subsets to implement a novel feature selection algorithm. The resulting feature selection method is capable of dealing with either nominal or real-valued data. Experimental results on both benchmark data sets and a real application problem demonstrate that the features selected by the proposed approach outperform those attained by state-of-the-art feature selection techniques, in terms of both the size of feature reduction and the classification accuracy.

The Market Generator

We propose to use a special type of generative neural networks - a Restricted Boltzmann Machine (RBM) - to build a powerful generator of synthetic market data that can replicate the probability distribution of the original market data. An RBM constructed with stochastic binary activation units in both the hidden and the visible layers (Bernoulli RBM) can learn complex dependence structures while avoiding overfitting. In this paper we consider an efficient data transformation and sampling approach that allows us to operate Bernoulli RBM on real-valued data sets and control the degree of autocorrelation and non-stationarity in the generated time series.

Double-Bit Quantization and Index Hashing for Nearest Neighbor Search

As binary code is storage efficient and fast to compute, it has become a trend to compact real-valued data to binary codes for the nearest neighbors (NN) search in a large-scale database. However, the use of binary code for the NN search leads to low retrieval accuracy. To increase the discriminability of the binary codes of existing hash functions, in this paper, we propose a framework of double-bit quantization and index hashing for an effective NN search. The main contributions of our framework are: first, a novel double-bit quantization (DBQ) is designed to assign more bits to each dimension for higher retrieval accuracy; second, a double-bit index hashing (DBIH) is presented to efficiently index binary codes generated by DBQ; and third, a weighted distance measurement for DBQ binary codes is put forward to re-rank the search results from DBIH. The empirical results on three benchmark databases demonstrate the superiority of our framework over existing approaches in terms of both retrieval accuracy and query efficiency. Specifically, we observe an absolute improvement on precision of 10%-25% in most cases and the query speed increases over 30 times compared to traditional binary embedding methods and linear scan, respectively.

A New Complex-Valued Polynomial Model

This paper presents a novel complex-valued polynomial model (CPM) for real-valued prediction and classification problems. In a CPM, function, independent variables and dependent variables are complex-valued. Before CPM optimization, real-valued data need to be converted into complex values. As the linear version of additive tree model, additive expression tree is proposed to optimize the complex-valued structure of CPM. Real parts and imaginary parts of the complex-valued coefficients are encoded into a chromosome and brain storm optimization is utilized to evolve the complex-valued coefficients of CPM. CPM is utilized to predict three financial datasets and classify n-class problems. The prediction results show that CPM presents higher forecasting accuracy than real-valued polynomial model, other real-valued neural networks and ordinary differential equation. The classification performance of CPM is compared with existing methods on IRIS, Liver and Ionosphere datasets. And the results reveal that CPM performs better than well-established and newly proposed real-valued classifiers.

Bivariate Weibull Distribution: Properties and Different Methods of Estimation

The bivariate Weibull distribution is an important lifetime distribution in survival analysis. In this paper, Farlie–Gumbel–Morgenstern (FGM) copula and Weibull marginal distribution are used for creating bivariate distribution which is called FGM bivariate Weibull (FGMBW) distribution. FGMBW distribution is used for describing bivariate data that have weak correlation between variables in lifetime data. It is a good alternative to bivariate several lifetime distributions for modeling real-valued data in application. Some properties of the FGMBW distribution are obtained such as product moment, skewness, kurtosis, moment generation function, reliability function and hazard function. Three different estimation methods for parameters estimation are discussed for FGMBW distribution namely; maximum likelihood estimation, inference function for margins method and semi-parametric method. To evaluate the performance of the estimators, a Monte Carlo simulations study is conducted to compare the preferences between estimation methods. Also, a real data set is introduced, analyzed to investigate the model and useful results are obtained for illustrative purposes.

A new class of metrics for learning on real-valued and structured data

We propose a new class of metrics on sets, vectors, and functions that can be used in various stages of data mining, including exploratory data analysis, learning, and result interpretation. These new distance functions unify and generalize some of the popular metrics, such as the Jaccard and bag distances on sets, Manhattan distance on vector spaces, and Marczewski-Steinhaus distance on integrable functions. We prove that the new metrics are complete and show useful relationships with f-divergences for probability distributions. To further extend our approach to structured objects such as ontologies, we introduce information-theoretic metrics on directed acyclic graphs drawn according to a fixed probability distribution. We conduct empirical investigation to demonstrate the effectiveness on real-valued, high-dimensional, and structured data. Overall, the new metrics compare favorably to multiple similarity and dissimilarity functions traditionally used in data mining, including the Minkowski ( $$L^p$$ ) family, the fractional $$L^p$$ family, two f-divergences, cosine distance, and two correlation coefficients. We provide evidence that they are particularly appropriate for rapid processing of high-dimensional and structured data in distance-based learning.

Overview Feature Selection using Fish Swarm Algorithm

Feature selection is a process of representing wanted features based on the requirement needed by selecting the best subset of a dataset without changing the originality of the dataset. The aim of feature selection is to obtain most optimal feature subset to represent the data and for that purpose feature selection offered a few methods. This paper gives an easy understanding of the feature selection concept and the available methods in feature selection. As nowadays metaheuristics is catching attention researchers in many fields and feature selection is one of them, this paper intentionally brief feature selection using metaheuristics that implement Fish Swarm Algorithm (FSA) in the feature selection process. FSA classified as one of the Swarm Intelligence (SI) techniques have several advantages mainly to solve optimization problems. A number of previous works are reviewed. Based on the reviewed and the outcome results that has been tested using high dimensional, real-valued benchmark data sets, FSA reflect good performance among others SI.

Data Discovery and Anomaly Detection Using Atypicality for Real-Valued Data.

The aim of using atypicality is to extract small, rare, unusual and interesting pieces out of big data. This complements statistics about typical data to give insight into data. In order to find such “interesting” parts of data, universal approaches are required, since it is not known in advance what we are looking for. We therefore base the atypicality criterion on codelength. In a prior paper we developed the methodology for discrete-valued data, and the current paper extends this to real-valued data. This is done by using minimum description length (MDL). We develop the information-theoretic methodology for a number of “universal” signal processing models, and finally apply them to recorded hydrophone data and heart rate variability (HRV) signal.