Non-uniform Data Research Articles

Sensitivity analysis of solutions of the harmonic inversion problem: Are all data points created equal?

We consider the harmonic inversion problem, and the associated spectral estimation problem, both of which are key numerical problems in NMR data analysis. Under certain conditions (in particular, in exact arithmetic) these problems have unique solutions. Therefore, these solutions must not depend on the inversion algorithm, as long as it is exact in principle. In this paper, we are not concerned with the algorithmic aspects of harmonic inversion, but rather with the sensitivity of the solutions of the problem to perturbations of the time-domain data. A sensitivity analysis was performed and the counterintuitive results call into question the common assumption made in “super-resolution” methods using non-uniform data sampling, namely, that the data should be sampled more often where the time signal has the largest signal-to-noise ratio. The numerical analysis herein demonstrates that the spectral parameters (such as the peak positions and amplitudes) resulting from the solution of the harmonic inversion problem are least susceptible to the perturbations in the values of data points at the edges of the time interval and most susceptible to the perturbations in the values at intermediate times.

A labeled-tree approach to semantic and structural data interoperability applied in hydrology domain

The issues of data integration and interoperability pose significant challenges in scientific hydrological and environmental studies, due largely to the inherent semantic and structural heterogeneities of massive datasets and non-uniform autonomous data sources. To address these data integration challenges, we propose a unified data integration framework, called Hydrological Integrated Data Environment (HIDE). HIDE is based on a labeled-tree data integration model referred to as DataNode tree. Using this framework, characteristics of datasets gathered from diverse data sources – with different logical and access organizations – can be extracted and classified as Time–Space–Attribute (TSA) labels and are subsequently arranged in a DataNode tree. The uniqueness of our approach is that it effectively combines the semantic aspects of the scientific domain with diverse datasets having different logical organizations to form a unified view. Further, we also adopt a metadata-based approach for specifying the TSA-DataNode tree in order to achieve flexibility and extensibility. The search engine of our HIDE prototype system evaluates a simple user query systematically on the TSA-DataNode tree, presenting integrated results in a standardized format that facilitates both effective and efficient data integration.

Efficient k-nearest neighbor searching in nonordered discrete data spaces

Numerous techniques have been proposed in the past for supporting efficient k -nearest neighbor ( k -NN) queries in continuous data spaces. Limited work has been reported in the literature for k -NN queries in a nonordered discrete data space (NDDS). Performing k -NN queries in an NDDS raises new challenges. The Hamming distance is usually used to measure the distance between two vectors (objects) in an NDDS. Due to the coarse granularity of the Hamming distance, a k -NN query in an NDDS may lead to a high degree of nondeterminism for the query result. We propose a new distance measure, called Granularity-Enhanced Hamming (GEH) distance, which effectively reduces the number of candidate solutions for a query. We have also implemented k -NN queries using multidimensional database indexing in NDDSs. Further, we use the properties of our multidimensional NDDS index to derive the probability of encountering valid neighbors within specific regions of the index. This probability is used to develop a new search ordering heuristic. Our experiments on synthetic and genomic data sets demonstrate that our index-based k -NN algorithm is efficient in finding k -NNs in both uniform and nonuniform data sets in NDDSs and that our heuristics are effective in improving the performance of such queries.

On Reconstruction from Non-uniform Spectral Data

This paper addresses the reconstruction of compactly supported functions from non-uniform samples of their Fourier transform. We briefly investigate the consequences of acquiring non-uniform spectral data. We summarize two often applied reconstruction methods, convolutional gridding and uniform re-sampling, and investigate the reconstruction accuracy as it relates to sampling density. Finally, we provide preliminary results from employing spectral re-projection methods in the reconstruction.

A NOTE ON OPTIMAL RECONSTRUCTION OF MAGNETIC RESONANCE IMAGES FROM NON-UNIFORM SAMPLES IN k-SPACE

A goal of Magnetic Resonance Imaging is reproducing a spatial map of the effective spin density from the measured Fourier coefficients of a specimen. The imaging procedure can be done by inverse Fourier transformation or backward fast Fourier transformation if the data are sampled on a regular grid in frequency space; however, it is still a challenging question how to reconstruct an image from a finite set of Fourier data on irregular points in k-space. In this paper, we describe some mathematical and numerical properties of imaging techniques from non-uniform MR data using the pseudo-inverse or the diagonal-inverse weight matrix. This note is written as an easy guide to readers interested in the non-uniform MRI techniques and it basically follows the ideas given in the paper by Greengard-Lee-Inati [10, 11].

Efficient Load Balancing Techniques for Self-organizing Content Addressable Networks

Balancing the load in a decentralized P2P system is a challenging problem due to the dynamic nature of such environment and the absence of global knowledge about the actual composition of the system.In this paper, we address the problem of load balancing in large scale and selforganizing P2P systems managing multidimensional data. We propose simple and efficient decentralized mechanisms to evenly distribute the data load among the participating nodes in Content Addressable Networks. The basic idea is to enable a new node that joins the system to share the load with a heavily loaded node which is already in the system, such that the load is still evenly distributed among all the participating nodes. In the multiple random choices method, the new node probes the load of some existing nodes selected uniformly at random, then chooses the heaviest node among them to share the load with. In this paper, we extend this method in three ways. First, the new node probes a pool of nodes proportional to the network size and composition. Specifically, the number of probed nodes is logarithmic to the network size. This property enables to achieve a constant load imbalance factor which is very small without the need to estimate the network size. Second, the probed nodes are not selected at random, but they are well spread over the key space; which enables a good estimation of the actual data distribution and network composition, which enables to cope well with large-scale data imbalance. Third, the selection of nodes to probe is restricted to the immediate and distant neighbors of a randomly chosen node. The cost incurred by our join-based load balancing method is very small, since all load information is piggybacked to periodic maintenance messages exchanged between nodes and their neighbors. Unlike other methods, we do not make use of external index nor assume any global knowledge. We also generalize the first method to enable locating a heavily loaded node through a sequential walk starting from a randomly selected node. This new method incurs additional overhead, however it achieves much smaller load imbalance. We also study the robustness of our join-based load balancing method against adversarial attacks. Using simulation, we analyze the impact of the number of entry points on load balancing. To the best of our knowledge we are the first to address this problem. We conduct an experimental study using uniform and nonuniform data distributions to demonstrate the effectiveness and the scalability of our proposals.

Spectral analysis of nonuniformly sampled data – a review

In this paper, we present a comprehensive review of methods for spectral analysis of nonuniformly sampled data. For a given finite set of nonuniformly sampled data, a reasonable way to choose the Nyquist frequency and the resampling time are discussed. The various existing methods for spectral analysis of nonuniform data are grouped and described under four broad categories: methods based on least squares; methods based on interpolation techniques; methods based on slotted resampling; methods based on continuous time models. The performance of the methods under each category is evaluated on simulated data sets. The methods are then classified according to their capabilities to handle different types of spectrum, signal models and sampling patterns. Finally the performance of the different methods is evaluated on two real life nonuniform data sets. Apart from the spectral analysis methods, methods for exact signal reconstruction from nonuniform data are also reviewed.

Iterative algorithm of wavelet network learning from nonuniform data

The learning algorithm based on multiresolution analysis (LAMA) is a powerful tool for wavelet networks. It has many advantages over other algorithms, but it seldom does well in the learning of nonuniform data. A new algorithm is proposed to solve this problem, which develops from the learning algorithm based on sampling theory (LAST). From the good concentration of wavelet energy, we discuss the approximation capacity of wavelet network in the local domain when the training data are not dense enough. From this discussion, the new algorithm is realized by the iterative application of LAST. The corresponding theorems based on the sampling theory are also proposed to prove the rationality of new algorithm. In the simulation, we compare the performance of new algorithm with that of LAMA and LAST. The results show that our new algorithm has as many advantages as LAMA and LAST, does better in the learning of nonuniform data and has high approximation accuracy.

A CONNECTION BETWEEN APPARENT VLBA JET SPEEDS AND INITIAL ACTIVE GALACTIC NUCLEUS DETECTIONS MADE BY THE FERMI GAMMA-RAY OBSERVATORY

In its first three months of operations, the Fermi Gamma-Ray Observatory has detected approximately one quarter of the radio-flux-limited MOJAVE sample of bright flat-spectrum active galactic nuclei (AGNs) at energies above 100 MeV. We have investigated the apparent parsec-scale jet speeds of 26 MOJAVE AGNs measured by the Very Long Baseline Array (VLBA) that are in the LAT bright AGN sample (LBAS). We find that the gamma-ray bright quasars have faster jets on average than the non-LBAS quasars, with a median of 15 c, and values ranging up to 34 c. The LBAS AGNs in which the LAT has detected significant gamma-ray flux variability generally have faster jets than the nonvariable ones. These findings are in overall agreement with earlier results based on nonuniform EGRET data which suggested that gamma-ray bright AGNs have preferentially higher Doppler boosting factors than other blazar jets. However, the relatively low LAT detection rates for the full MOJAVE sample (24%) and previously known MOJAVE EGRET-detected blazars (43%) imply that Doppler boosting is not the sole factor that determines whether a particular AGN is bright at gamma-ray energies. The slower apparent jet speeds of LBAS BL Lac objects and their higher overall LAT detection rate as compared to quasars suggest that the former are being detected by Fermi because of their higher intrinsic (unbeamed) gamma-ray to radio luminosity ratios.

Improving Performance of Dynamic Programming via Parallelism and Locality on Multicore Architectures

Dynamic programming (DP) is a popular technique which is used to solve combinatorial search and optimization problems. This paper focuses on one type of DP, which is called nonserial polyadic dynamic programming (NPDP). Owing to the nonuniform data dependencies of NPDP, it is difficult to exploit either parallelism or locality. Worse still, the emerging multi/many-core architectures with small on-chip memory make these issues more challenging. In this paper, we address the challenges of exploiting the fine grain parallelism and locality of NPDP on multicore architectures. We describe a latency-tolerant model and a percolation technique for programming on multicore architectures. On an algorithmic level, both parallelism and locality do benefit from a specific data dependence transformation of NPDP. Next, we propose a parallel pipelining algorithm by decomposing computation operators and percolating data through a memory hierarchy to create just-in-time locality. In order to predict the execution time, we formulate an analytical performance model of the parallel algorithm. The parallel pipelining algorithm achieves not only high scalability on the 160-core IBM Cyclops64, but portable performance as well, across the 8-core Sun Niagara and quad-cores Intel Clovertown.

Efficient Routing in Multi-ring Content Addressable Network

RCAN [1] is a novel multi-ring content addressable peer-to-peer system. RCAN was proposed in the aim of improving the routing performance of CAN [4] overlays while minimizing the maintenance overhead during nodes churn in large networks. The key idea of RCAN is to equip each node with few long-links towards some distant nodes. Long-links are clockwise directed and wrap around to form small rings along each dimension. The number of rings and their sizes self-adjust as nodes join and leave the systems. RCAN is a pure P2P design, where all nodes assume the same responsibility. Unlike some existing P2P overlays, RCAN is self-organizing and does not assume any a-priori fixed limits for the network size or the routing state per node. Each node auto-adapts its routing state to cope with network changes. We present in this paper an extensive study of the routing performance of RCAN under uniform and non-uniform data distributions. Experimental results show that in an overlay of n nodes, a node maintains a routing state of O(log n) long-links in average, and is able to reach any other nodes within O(log n) routing hops even in the presence of non-uniform space partitioning. Using simulation we demonstrate the full scalability and efficiency of our design and its advantages over other existing methods.

Scheduling non-uniform data with expected-time constraint in wireless multi-channel environments

This paper investigates the time-constrained data scheduling problem for broadcasting data of non-uniform length under multiple broadcast channel environments. We first define and analyze the problem of time-constrained services and then propose an effective scheduling algorithm that constructs a broadcast program to satisfy as many clients’ requirements within the expected times as possible. The performance study reveals the superiority of the proposed method.

Four-point curve subdivision based on iterated chordal and centripetal parameterizations

Dubuc's interpolatory four-point scheme inserts a new point by fitting a cubic polynomial to neighbouring points over uniformly spaced parameter values. In this paper we replace uniform parameter values by chordal and centripetal ones. Since we update the parameterization at each refinement level, both schemes are non-linear. Because of this data-dependent parameterization, the schemes are only invariant under solid body and isotropic scaling transformations, but not under general affine transformations. We prove convergence of the two schemes and bound the distance between the limit curve and the initial control polygon. Numerical examples indicate that the limit curves are smooth and that the centripetal one is tighter, as suggested by the distance bounds. Similar to cubic spline interpolation, the use of centripetal parameter values for highly non-uniform initial data yields better results than the use of uniform or chordal ones.

Parallelizing query optimization

Many commercial RDBMSs employ cost-based query optimization exploiting dynamic programming (DP) to efficiently generate the optimal query execution plan. However, optimization time increases rapidly for queries joining more than 10 tables. Randomized or heuristic search algorithms reduce query optimization time for large join queries by considering fewer plans, sacrificing plan optimality. Though commercial systems executing query plans in parallel have existed for over a decade, the optimization of such plans still occurs serially. While modern microprocessors employ multiple cores to accelerate computations, parallelizing query optimization to exploit multi-core parallelism is not as straightforward as it may seem. The DP used in join enumeration belongs to the challenging nonserial polyadic DP class because of its non-uniform data dependencies. In this paper, we propose a comprehensive and practical solution for parallelizing query optimization in the multi-core processor architecture, including a parallel join enumeration algorithm and several alternative ways to allocate work to threads to balance their load. We also introduce a novel data structure called skip vector array to significantly reduce the generation of join partitions that are infeasible. This solution has been prototyped in PostgreSQL. Extensive experiments using various query graph topologies confirm that our algorithms allocate the work evenly, thereby achieving almost linear speed-up. Our parallel join enumeration algorithm enhanced with our skip vector array outperforms the conventional generate-and-filter DP algorithm by up to two orders of magnitude for star queries-linear speedup due to parallelism and an order of magnitude performance improvement due to the skip vector array.

ILP-Based energy minimization techniques for banked memories

Main memories can consume a significant portion of overall energy in many data-intensive embedded applications. One way of reducing this energy consumption is banking, that is, dividing available memory space into multiple banks and placing unused (idle) memory banks into low-power operating modes. Prior work investigated code-restructuring- and data-layout-reorganization-based approaches for increasing the energy benefits that could be obtained from a banked memory architecture. This article explores different techniques that can potentially coexist within the same optimization framework for maximizing benefits of low-power operating modes. These techniques include employing nonuniform bank sizes, data migration, data compression, and data replication. By using these techniques, we try to increase the chances for utilizing low-power operating modes in a more effective manner, and achieve further energy savings over what could be achieved by exploiting low-power modes alone. Specifically, nonuniform banking tries to match bank sizes with application-data access patterns. The goal of data migration is to cluster data with similar access patterns in the same set of banks. Data compression reduces the size of the data used by an application, and thus helps reduce the number of memory banks occupied by data. Finally, data replication increases bank idleness by duplicating select read-only data blocks across banks. We formulate each of these techniques as an ILP (integer linear programming) problem, and solve them using a commercial solver. Our experimental analysis using several benchmarks indicates that all the techniques presented in this framework are successful in reducing memory energy consumption. Based on our experience with these techniques, we recommend to compiler writers for banked memories to consider data compression, replication, and migration.

Hierarchical probabilistic modeling of short-crack growth behavior in AISI 4340 steel

Probabilistic analyses of non-uniform crack growth data sets require a flexible statistical framework to determine the influence of each crack on the resulting inference. Hierarchical generalized linear models provide a rigorous method to analyze such data sets properly. Bayesian techniques are well-suited to analyze these models, especially when the inference, or portions thereof, are ill-posed. A hierarchical generalized linear crack growth model is developed using a semi-conjugate formulation that enables Gibbs sampling simulation. The model is applied to create a probabilistic crack growth model from short-crack data generated from AISI 4340 steel single-edge-notch tension (SENT) specimens. Simulation of the model is performed using a Gibbs sampling procedure, and key results are discussed. Stress ratio effects on experimental scatter and crack growth rates are quantified and discussed.

Prediction of far-field from uniform and non-uniform under-sampled near-field data in planar near-field antenna measurements

In near-field antenna measurements various forms of uniform and non-uniform sampling techniques have been widely deployed. Considering the fact that the near-field pattern of any antenna is a spatially quasi-band-width-limited function of space coordinates, Shannon's theorem simply defines the sampling frequency. Based on the sampling theorem, in order to precisely reconstruct a band-limited signal from its samples, the sampling frequency must be at least twice as much as the signal's bandwidth. Through the simulations and theoretical evaluations this research shows that if the near-field pattern is either uniformly or non-uniformly under-sampled due to any practical reasons, yet a good estimation of far-field pattern can be obtained especially if the antenna under test (AUT) is a directive high-gain or super high-gain antenna. Also the time efficiency of far-field prediction from under-sampled near-field data is discussed and the advantages and disadvantages are highlighted.

The distribution of the domination number of class cover catch digraphs for non-uniform one-dimensional data

For two or more classes of points in R d with d ≥ 1 , the class cover catch digraphs (CCCDs) can be constructed using the relative positions of the points from one class with respect to the points from one or all of the other classes. The CCCDs were introduced by Priebe et al. [C.E. Priebe, J.G. DeVinney, D.J. Marchette, On the distribution of the domination number of random class catch cover digraphs. Statistics and Probability Letters 55 (2001) 239–246] who investigated the case of two classes, X and Y . They calculated the exact (i.e., finite sample) distribution of the domination number of the CCCDs based on X points relative to Y points both of which were uniformly distributed on a bounded interval. We investigate the distribution of the domination number of the CCCDs based on data from non-uniform X points on an interval with end points from Y . Then we extend these calculations for multiple Y points on bounded intervals.

On Bandwidth Selection in Local Polynomial Regression Analysis and Its Application to Multi-resolution Analysis of Non-uniform Data

This paper studies adaptive bandwidth selection method for local polynomial regression (LPR) and its application to multi-resolution analysis (MRA) of non-uniformly sampled data. In LPR, the observations are modeled locally by a polynomial using least-squares criterion with a kernel having a certain support or bandwidth so that a better bias-variance tradeoff can be achieved. In this paper, two bandwidth selection methods, namely the Fan and Gijbels's bandwidth selection (FGBS) method (Fan and Gijbels, Local Polynomial Modelling and Its Applications, Chapman and Hall, London, 1996; Fan and Gijbels, Stat Sin 57:371---394, 1995) in the statistical community and the intersection of confidence intervals (ICI) method commonly used in the signal and image processing communities, are reviewed and compared in terms of their performance and implementation complexity using standard testing data sets. Furthermore, using the result of Stankovi (IEEE Trans Signal Proc 52:1228---1234, 2004), a new refined ICI-based adaptive bandwidth selection method for LPR and its associated reliability analysis are proposed. In addition, recursive implementations of LPR with the two classes of bandwidth selection methods are considered for online applications. Simulation results show that the performances of the FGBS method and the refined ICI method are comparable for the data sets tested. Since LPR with adaptive bandwidths can be naturally applied to non-uniformly sampled noisy observations, we propose to use it as a pre-processing step to a conventional MRA so that a MRA of non-uniformly sampled data can be realized. Simulation results show that the proposed LPR-based MRA gives better results than conventional linear interpolation of the data.

The epidemiology of bladder cancer in Russia

This study assessed the epidemiology of bladder cancer in Russia. The available publications in Russian were analysed, as well as information from the database of N. N. Blokhin Cancer Research Center, accumulating non-uniform data from different cities and regions of Russia. In 2006 there were 68 129 patients with bladder cancer in Russia, accounting for 2.8% of all cancer cases, with unknown proportions of males and females. In the same year 11 973 new bladder cancer cases were diagnosed, with morphological confirmation in 82.3% of cases. From 1999 to 2004 the incidence of bladder carcinoma increased by 5.3% in males and by 12.5% in females. In 2006 57.4% of patients with newly diagnosed disease were staged as T1 and T2, 26.8% as T3 and 11.4% as T4 bladder cancer cases. The mortality rate of patients with bladder carcinoma increased by 10.9% in males from 1999 to 2004 and did not change in females. In conclusion, over the past 10 years both the prevalence and incidence of bladder carcinoma have increased in Russia. There was a trend towards detecting less advanced cases of the disease, and stages T3 and T4 are diagnosed less frequently today than in 1996. As a result the mortality rate of bladder cancer patients within the first year from diagnosis has decreased, although the overall mortality rate in males has risen.