Nonlinear Data Structure Research Articles

Nonnegative matrix factorization with local similarity learning

Existing nonnegative matrix factorization methods usually focus on learning global structure of the data to construct basis and coefficient matrices, which ignores the local structure that commonly exists among data. To overcome this drawback, in this paper, we propose a new type of nonnegative matrix factorization method, which learns local similarity and clustering in a mutually enhanced way. The learned new representation is more representative in that it better reveals inherent geometric property of the data. Moreover, the new representation is performed in the kernel space, which enhances the capability of the proposed model in discovering nonlinear structures of data. Multiplicative updating rules are developed with theoretical convergence guarantees. Extensive experimental results have confirmed the effectiveness of the proposed model.

Testing for Non-Linear Structures in Artificial and Real-World Financial Data with Recurrence Quantification Analysis

Early applications of empirical methods from chaos theory suggested the existence of low dimensional chaotic motion in empirical financial data. However, such results were questioned, and it is then believed that the search for low dimensional chaos in financial data was not successful. On the other hand, at the same time that the hypotheses that raw returns are independent and identically distributed (IID) is often rejected, they indeed present a quite small degree of autocorrelation. These facts suggest that prices in financial markets do not behave completely at random, although their hidden structures seem more complex than those observed in low dimensional chaotic systems. Previous work tested for non-linearity or the presence of low dimensional chaos in artificial financial data generated from the Lux-Marchesi model by means of the BDS and Kaplan tests. Addressing the same model, researchers extended those results by applying Hinich’s bi-spectral and White’s tests and introducing the application of Recurrence Quantification Analysis (RQA) on artificial financial data based on Recurrence Rate, Determinism, Entropy, and Maximal Diagonal Length. Contributing to this research, the present paper has two main goals: (i) to contrast previous findings with an RQA application on data generated by a more evolved of microscopic model of financial markets – the Structural Stochastic Volatility (SSV ) model; and (ii) to extend the RQA investigation above with additional recurrence measures (namely, Divergence, Laminarity, and Maximal Vertical Length) being applied to distinct real-world financial data. The objective is to assess if RQA results could help to distinguish between artificial and real-world data, even if linearity is rejected in both cases. It is shown evidence, in agreement previous findings, to support the rejection of linearity or low dimensional chaotic motion in an artificial financial time series generated from the SSV microscopic model. In addition, it is also shown that that RQA measures can help to discriminate artificial from real-world financial data, at least when specific RQA measures are considered.

Robust Fuzzy K-Means Clustering With Shrunk Patterns Learning

Fuzzy K-Means (FKM) clustering regards each cluster as a fuzzy set and assigns each sample to multiple clusters with a certain degree of membership. However, conventional FKM methods perform clustering on original data directly where the intrinsic structure of data may be corrupted by the noise. According, the performance of these methods would be challenged. In this paper, we present a novel fuzzy K-Means clustering model to conduct clustering tasks on the flexible manifold. Technically, we perform fuzzy clustering based on the shrunk patterns which have desired manifold structure. The shrunk patterns can be viewed as an approximation to the original data; and a penalty term is employed to measure the mismatch between them. Moreover, we integrate the learning of shrunk patterns and the learning of membership degree between shrunk patterns and clusters into a unified framework. Furthermore, we extend the proposed model for projected FKM clustering to find a suitable subspace to fit the non-linear manifold structure of data, reduce the interference of the noise and redundant features and gather homogeneous samples together simultaneously. Two alternating iterative algorithms are derived to solve these two models, respectively. Extensive experimental results demonstrate the feasibility and effectiveness of our proposed clustering algorithms.

On Order and Regime Determination of SETAR Model in Modelling Nonlinear Stationary Time Series Data Structure: Application to Lafia Rainfall Data, Nasarawa State, Nigeria

The linear time series model refers to the class of models for which fixed correlation parameters can fully explain the dependency between two random variables, but many real-life circumstances, such as monthly unemployment results, supplies and demands, interest rate, exchange rate, share prices, rainfall, etc., violate the assumption of linearity. For fitting and forecasting of nonlinear time series data, the self-exiting threshold autoregressive (SETAR) model was suggested. Using R to generate random nonlinear autoregressive data, a Monte Carlo simulation was performed, the SETAR model was fitted to the simulated data and Lafia rainfall data, Nasarawa State, Nigeria to determine the best regime orders and/or scheme number to make future forecast. Using Mean Square Error (MSE) and Akaike Information Criteria (AIC), the relative performance of models was examined. At a specific autoregressive order, regime order, sample size and step ahead, the model with minimum criteria was considered as the best. The results show that the best autoregressive and regime orders to be chosen are 3rd and 2nd [SETAR (3, 2)] respectively for fitting and forecasting nonlinear autoregressive time series data with small and moderate sample sizes. As the sample size increases, the output of the four models increases. Finally, it is shown that when sample size and number of steps forward are increased, the efficiency and forecasting capacity of the four models improves.

Autoencoder Constrained Clustering With Adaptive Neighbors.

The conventional subspace clustering method obtains explicit data representation that captures the global structure of data and clusters via the associated subspace. However, due to the limitation of intrinsic linearity and fixed structure, the advantages of prior structure are limited. To address this problem, in this brief, we embed the structured graph learning with adaptive neighbors into the deep autoencoder networks such that an adaptive deep clustering approach, namely, autoencoder constrained clustering with adaptive neighbors (ACC_AN), is developed. The proposed method not only can adaptively investigate the nonlinear structure of data via a parameter-free graph built upon deep features but also can iteratively strengthen the correlations among the deep representations in the learning process. In addition, the local structure of raw data is preserved by minimizing the reconstruction error. Compared to the state-of-the-art works, ACC_AN is the first deep clustering method embedded with the adaptive structured graph learning to update the latent representation of data and structured deep graph simultaneously.

A novel dimension reduction and dictionary learning framework for high-dimensional data classification

High-dimensional problem poses significant challenges for dictionary learning based classification architecture. Joint Dimension Reduction and Dictionary Learning (JDRDL) framework shows great potential for overcoming the challenges caused by high dimensionality. However, most of the existing JDRDL approaches do not consider the complex nonlinear relationships within high-dimensional data, which limits their classification performance. To overcome this problem, a novel joint dimension reduction and dictionary learning framework is proposed in this paper for high-dimensional data classification. Firstly, at dimension reduction stage, an autoencoder is employed to learn a nonlinear mapping that reduces dimensionality and preserves nonlinear structure of the high-dimensional data. Then, at dictionary learning stage, the locality constraint with label embedding, which takes the locality and label information into account together, is incorporated into the learning process to preserve desirable nonlinear local structure and enhance class discrimination. Moreover, the mapping function and dictionary are optimized simultaneously to enhance the performance. Encouraging experimental results on multiple benchmark datasets confirm that the proposed framework is effective and efficient for high-dimensional data classification.

Free-Breathing and Ungated Dynamic MRI Using Navigator-Less Spiral SToRM.

We introduce a kernel low-rank algorithm to recover free-breathing and ungated dynamic MRI from spiral acquisitions without explicit k-space navigators. It is often challenging for low-rank methods to recover free-breathing and ungated images from undersampled measurements; extensive cardiac and respiratory motion often results in the Casorati matrix not being sufficiently low-rank. Therefore, we exploit the non-linear structure of the dynamic data, which gives the low-rank kernel matrix. Unlike prior work that rely on navigators to estimate the manifold structure, we propose a kernel low-rank matrix completion method to directly fill in the missing k-space data from variable density spiral acquisitions. We validate the proposed scheme using simulated data and in-vivo data. Our results show that the proposed scheme provides improved reconstructions compared to the classical methods such as low-rank and XD-GRASP. The comparison with breath-held cine data shows that the quantitative metrics agree, whereas the image quality is marginally lower.

Multi-nonlinear multi-view locality-preserving projection with similarity learning for random cross-view gait recognition

View variation is one of the greatest challenges in the field of gait recognition. Subspace learning approaches are designed to solve this issue by projecting cross-view features into a common subspace before recognition. However, similarity measures are data-dependent, which results in low accuracy when cross-view gait samples are randomly arranged. Inspired by the recent developments of data-driven similarity learning and multi-nonlinear projection, we propose a new unsupervised projection approach, called multi-nonlinear multi-view locality-preserving projections with similarity learning (M2LPP-SL). The similarity information among cross-view samples can be learned adaptively in our M2LPP-SL. Besides, the complex nonlinear structure of original data can be well preserved through multiple explicit nonlinear projection functions. Nevertheless, its performance is largely affected by the choice of nonlinear projection functions. Considering the excellent ability of kernel trick for capturing nonlinear structure information, we further extend M2LPP-SL into kernel space, and propose its multiple kernel version MKMLPP-SL. As a result, our approaches can capture linear and nonlinear structure more precisely, and also learn similarity information hidden in the multi-view gait dataset. The proposed models can be solved efficiently by alternating direction optimization method. Extensive experimental results over various view combinations on the multi-view gait database CASIA-B have demonstrated the superiority of the proposed algorithms.

Multiscale Supervised Kernel Dictionary Learning for SAR Target Recognition

In this article, a supervised nonlinear dictionary learning (DL) method, called multiscale supervised kernel DL (MSK-DL), is proposed for target recognition in synthetic aperture radar (SAR) images. We use Frost filters with different parameters to extract an SAR image’s multiscale features for data augmentation and noise suppression. In order to reduce the computation cost, the dimension of each scale feature is reduced by principal component analysis (PCA). Instead of the widely used linear DL, we learn multiple nonlinear dictionaries to capture the nonlinear structure of data by introducing the dimension-reduced features into the nonlinear reconstruction error terms. A classification model, which is defined as a discriminative classification error term, is learned simultaneously. Hence, the objective function contains the nonlinear reconstruction error terms and a classification error term. Two optimization algorithms, called multiscale supervised kernel K-singular value decomposition (MSK-KSVD) and multiscale supervised incremental kernel DL (MSIK-DL), are proposed to compute the multidictionary and the classifier. Experiments on the moving and stationary target automatic recognition (MSTAR) data set are performed to evaluate the effectiveness of the two proposed algorithms. And the experimental results demonstrate that the proposed scheme outperforms some representative common machine learning strategies, state-of-the-art convolutional neural network (CNN) models and some representative DL methods, especially in terms of its robustness against training set size and noise.

A comparative study of estimating solar radiation using machine learning approaches: DL, SMGRT, and ANFIS

ABSTRACT Solar energy has a key role in producing clean and emissions-free power compare to conventional methods. However, sustainable development also requires a reliable and predictable energy source. It also needs methods to measure and predict predictable supply. The main aim of the study is to improve reliable and precise solar radiation prediction models on monthly mean daily basis using various machine learning techniques. Simple Membership Function and Fuzzy Rule Generating Technique (SMGRT), which does not require error and trial for model adjustment, is the first-choice model in this study. Experience and observations about the model will greatly reduce the volume of processing for the fuzzy SMGRT model. On the other hand, Deep Learning (DL) and Adaptive Neuro-Fuzzy Inference System (ANFIS) have become increasingly popular in understanding nonlinear data structures and solving complex problems. Therefore, DL and ANFIS were also applied to estimate solar radiation. The data set used in the study were created using sunshine duration (s), extra-terrestrial solar radiation (H0 ), relative humidity (RH), cloudiness (C), air temperature (T) and soil temperature (ST) parameters. Estimation performance of models was evaluated by using several statistical indicators which are Mean Bias Error (MBE), Mean Square Error (MSE), Root Mean Square Error (RMSE), and Correlation Coefficient (R2). When the performances of the models were compared, it was seen that all three models obtained remarkable results. In addition, it was shown that the models performed well based on the metrics in the testing phase. The SMGRT model has slightly better performance than DL and ANFIS for different input combinations. SMGRT Model 1 (with inputs H0, s, and T) shows the best statistical performance (MBE = 0.156, MSE = 1.878, RMSE = 1.371, and R2 = 0.960) not only in SMGRT models but also in others.

Non-Linear Data Structure for Data Coding for Size Compression

Data compression is a special case of communication system which is a basic problem in engineering and computer science. In this work, the new scheme has been introduced for data coding to compress the size. Compression by using one type of Non-linear Data-Structure which is Prefix Binary Tree data compression is essential and beneficial in the diversity of application which is embraced in general into two branches: the storage and transmission.the storage of a file in a compressed form takes fewer bit than in uncompressed form, therefore make the most of storage resources (containing main memory itself), and by transmission means sending the files over a channel with reduced number of bits by compression process. This leads to more efficacious transmission process. The compression process of stored data or that to be transmitted leads to significantly reducing both of the storage and transmission. We use text type as a data in the experiments. The results show the compression rate is reduce the actual size above 50%.

Energy analysis of Internet of things data mining algorithm for smart green communication networks

With the continuous development of Internet technology and electronic information technology, big data technology and cloud computing technology also rise and develop, and have a positive impact on people’s lives. Data mining system can deeply mine the value information contained in big data, so as to assist users to solve practical problems and help users to make correct decisions and judgments. This paper presents an energy analysis of data mining algorithm based on cloud platform for Internet of things (IoT). First of all, an improved Apriori algorithm is proposed, which is based on Boolean matrix and sorting index rules. Then Boolean matrix is obtained after scanning the data set and the Boolean matrix is preprocessed to delete the useless transactions and the item set, which are combined with sorting index to produce other item sets, effectively improving the efficiency of frequent item mining, which effectively reduce the memory usage. Secondly, the density-based spatial clustering of applications with noise (DBSCAN) clustering algorithm needs human intervention in the global parameter selection, and the process of regional query is complex and the query is easy to lose objects. An improved parameter adaptive and regional query density clustering algorithm is proposed, which can effectively delete the redundant data in the high-level complex data space on the premise of retaining the internal nonlinear structure of the IoT data. The efficiency of clustering is also improved accordingly Finally, the simulation based on cloud platform verifies the effectiveness and superiority of the algorithm.

Neighbouring Prediction for Mortality

We propose new neighbouring prediction models for mortality forecasting. For each mortality rate at age x in year t, denoted as mx,t, we construct images of neighbourhood mortality data around mx,t, i.e., ℇmx,t (x1, x2, s), which includes mortality information for ages in [x − x1, x + x2], lagging k years (1 ≤ k ≤ s). Combined with the deep learning model - convolutional neural networks (CNN), this framework is able to capture the intricate nonlinear structure in the mortality data: the neighbourhood effect, which can go beyond the directions of period, age, and cohort as in classic mortality models. By performing an extensive empirical analysis on all the 41 countries and regions in the Human Mortality Database (HMD), we find that the proposed model achieves superior forecasting performance. This model can be further enhanced to capture the patterns and interactions between multiple populations.

Air-Conditioning Load Forecasting for Prosumer Based on Meta Ensemble Learning

Accurate and reliable prediction of air-conditioning load plays a significant role in prosumer energy management system (EMS), because air-conditioning load accounts for a large proportion of the building's total energy consumption. This paper proposes a new meta ensemble learning method to realize short-term prediction of air-conditioning load for prosumers. This method is a hybrid of meta ensemble learning and stacked auto-encoder (SAE). First, we design multiple different forecasting structures based on SAE to achieve point prediction of air-conditioning loads. SAE is used to learn the deep features in air-conditioning load data. Second, a new meta ensemble learning prediction model is proposed. Meta ensemble learning is used to learn the nonlinear features and invariant structures in data, and determine the coefficients of each SAE-based point forecaster. Finally, the prediction results of each point forecaster are aggregated and integrated to estimate the final air-conditioning load prediction result. Air-conditioning load data from a commercial building in Singapore are used to validate the feasibility and effectiveness of the proposed method, demonstrating that the proposed meta ensemble learning method is attractive in prosumer energy management.

Robust Kernel Principal Component Analysis With ℓ2,1-Regularized Loss Minimization

Principal component analysis (PCA) is a widely used unsupervised method for dimensionality reduction. The kernelized version is called kernel principal component analysis (KPCA), which can capture the nonlinear data structure. KPCA is derived from the Gram matrix, which is not robust when outliers exist in the data. This may yield the principal axis in the feature space deviated by outliers, leading to misinterpretation of the principal components. In this paper, we propose a robust method for KPCA with a reformulation in Euclidean space to construct a robust KPCA method, where an error measurement is introduced into the loss function, and l 2,1 -regularization is added to the loss function. The idea of l 2,1 -regularization of the proposed method is motivated by sparse PCA via variable projection. However, because orthogonality is not satisfied in the proposed method, orthonormal bases are obtained by using the Gram-Schmidt orthonormalization process. In the experiments, a toy example and real data are used for outlier detection to verify the method's performance and effectiveness. In the toy example, the proposed method reduces the influence of outliers and detects more outliers than KPCA. For the real data, the proposed method improves detection in comparison to other existing methods.

Jointly Learning Kernel Representation Tensor and Affinity Matrix for Multi-View Clustering

Multi-view clustering refers to the task of partitioning numerous unlabeled multimedia data into several distinct clusters using multiple features. In this paper, we propose a novel nonlinear method called joint learning multi-view clustering (JLMVC) to jointly learn kernel representation tensor and affinity matrix. The proposed JLMVC has three advantages: (1) unlike existing low-rank representation-based multi-view clustering methods that learn the representation tensor and affinity matrix in two separate steps, JLMVC jointly learns them both; (2) using the “kernel trick,” JLMVC can handle nonlinear data structures for various real applications; and (3) different from most existing methods that treat representations of all views equally, JLMVC automatically learns a reasonable weight for each view. Based on the alternating direction method of multipliers, an effective algorithm is designed to solve the proposed model. Extensive experiments on eight multimedia datasets demonstrate the superiority of the proposed JLMVC over state-of-the-art methods.

Distributed and parallel processing for real-time and dynamic spatio-temporal graph

As a non-linear data structure consisting of nodes and edges, the graph data span many different domains. In the real world, applications based on such data structures are always time-sensitive, that is, the value of graph data tends to decrease with time. Furthermore, the application based on spatio-temporal graph is one of the typical representatives of time-sensitive, since the time dimension is an inherent feature of spatio-temporal data. The Distributed Stream Processing Engine (DSPE) seems an excellent choice for the above requirement, which is commonly partitioned and concurrently processed by a number of threads to maximize the throughput. However, it is not feasible to do such mission directly using the traditional DSPE. In this paper, we propose a computational model suitable for handling the spatio-temporal graph in DSPE, by reconstructing the DSPE’s parallel processing slots. Specifically, our proposal includes a general processing framework to deal with the data structure of the spatio-temporal graph, a state information compensation mechanism to ensure the correctness of processing such stateful operation in DSPE, a lightweight summary information calculation method to ensure the performance of the system. Empirical studies on real-world stream applications validate the usefulness of our proposals and prove the considerable advantage of our approaches over state-of-the-art solutions in the literature.

Robust subspace clustering based on non-convex low-rank approximation and adaptive kernel

As a relatively advanced method, the low-rank kernel space clustering method shows good performance in dealing with nonlinear structure of high-dimensional data. Unfortunately, this method is sensitive to large corruptions and doesn’t balance the contribution of all singular values. To solve the above problems, the low-rank kernel method is modified, and a robust subspace clustering method (LAKRSC) based on non-convex low-rank approximation and adaptive kernel is proposed. In our model, the weighted Schatten p-norm is introduced to balance the importance of different singular values, which can more accurately approximate the rank function and be more flexible in practical applications. Therefore, applying weighted Schatten p-norm to adaptive kernel can approximate the original low rank hypothesis better when the data is mapped into the feature space. In addition, our model uses correntropy to handle complex noise which enhances the robustness of the model. A new algorithm HQ&ADMM, combined by Half-Quadratic technique (HQ) and ADMM, is studied to solve our model. Experiments on four real-world datasets show that the clustering performance of LAKRSC is significantly better than that of several more advanced methods.

Label consistent locally linear embedding based cross-modal hashing

Hashing methods have gained widespread attention in cross-modal retrieval applications due to their efficiency and effectiveness. Many works have been done but they fail to capture the feature based similarity consistency or the discriminative semantics of label consistency. In addition, most of them suffer from large quantization loss, resulting in low retrieval performance. To address these issues, we propose a novel cross-modal hashing method named Label Consistent Locally Linear Embedding based Cross-modal Hashing (LCLCH). LCLCH preserves the non-linear manifold structure of different modality data by Locally Linear Embedding, and transforms heterogeneous data into a latent common semantic space to reduce the semantic gap and support cross-modal retrieval tasks. Therefore, it not only discovers the potential correlation of heterogeneous cross-modal data but also maintains label consistency. To further ensure the effectiveness of hash code learning, we utilize an iterative quantization method to handle the discrete optimization task and obtain the hash codes directly. We compare LCLCH with some advanced supervised and unsupervised methods on three datasets to evaluate its effectiveness.

Hyperspectral Band Selection Using Weighted Kernel Regularization

A band selection method named weighted kernel regularization (WKR) is proposed for hyperspectral imagery (HSI) classification. The WKR aims to select dissimilar and class-separable bands to better model the relationship between labeled samples. First, the WKR considers nonlinear structure of hyperspectral data and models nonlinear relations between HSI samples and their class labels using a weighted kernel ridge regression (WKRR) program with respect to sample coefficients. Second, it combines the L1 penalty term of weights on all bands with the above WKRR program into the unified framework of WKR. The L1 penalty term considers divergent contributions from different bands in describing nonlinear relations and guarantees the sparsity of band weights. Third, the WKR algorithm implements the KerNel Iterative-based Feature Extraction (KNIFE) algorithm to estimate the proper band weights. The KNIFE linearizes the nonlinear kernels to avoid high computational cost, and iteratively minimizes two convex subproblems with respect to the sample coefficients and band weights. Finally, the first k bands with larger weights and larger dissimilarity with other bands are automatically chosen to form the band subset. Experimental results show that the WKR outperforms the state-of-the-art methods in classification accuracies with a lower computational cost.