Time Series Databases Research Articles

Pattern mining and fault detection via $$\textit{COP}_{\textit{therm}}$$ COP therm -based profiling with correlation analysis of circuit variables in chiller systems

In this paper, we propose methods of handling, analyzing, and profiling monitoring data of energy systems using their thermal coefficient of performance seen in uneven segmentations in their time series databases. Aside from assessing the performance of chillers using this parameter, we dealt with pinpointing different trends that this parameter undergoes through while the systems operate. From these results, we identified and cross-validated with domain experts outlier behavior which were ultimately identified as faulty operation of the chiller. Finally, we establish correlations of the parameter with the other independent variables across the different circuits of the machine with or without the observed faulty behavior.

Fuzzy clustering of time series data using dynamic time warping distance

Clustering is a powerful vehicle to reveal and visualize structure of data. When dealing with time series, selecting a suitable measure to evaluate the similarities/dissimilarities within the data becomes necessary and subsequently it exhibits a significant impact on the results of clustering. This selection should be based upon the nature of time series and the application itself. When grouping time series based on their shape information is of interest (shape-based clustering), using a Dynamic Time Warping (DTW) distance is a desirable choice. Using stretching or compressing segments of temporal data, DTW determines an optimal match between any two time series. In this way, time series exhibiting similar patterns occurring at different time periods, are considered as being similar. Although DTW is a suitable choice for comparing data with respect to their shape information, calculating the average of a collection of time series (which is required in clustering methods) based on this distance becomes a challenging problem. As the result, employing clustering techniques like K-Means and Fuzzy C-Means (where the cluster centers – prototypes are calculated through averaging the data) along with the DTW distance is a challenging task and may produce unsatisfactory results. In this study, three alternatives for fuzzy clustering of time series using DTW distance are proposed. In the first method, a DTW-based averaging technique proposed in the literature, has been applied to the Fuzzy C-Means clustering. The second method considers a Fuzzy C-Medoids clustering, while the third alternative comes as a hybrid technique, which exploits the advantages of both the Fuzzy C-Means and Fuzzy C-Medoids when clustering time series. Experimental studies are reported over a set of time series coming from the UCR time series database.

A Novel Gaussian Based Similarity Measure for Clustering Customer Transactions Using Transaction Sequence Vector

Abstract Clustering transactions in sequence databases, temporal databases, and time series databases is achieving an important attention from the database researchers. There is a significant research being carried towards defining and validating the suitability of new similarity measures for sequence databases, temporal databases, time series databases which can accurately and efficiently find the similarity between any two given user transactions in the database of transactions to predict the user behavior. The distribution of items present in the transactions contributes to a great extent in finding the degree of similarity between them. This forms the key idea for the design of the proposed similarity measure. The main objective of this research is to design similarity function to find similarity between two user transactions by defining two terms called transaction sequence vector and transaction vector and use them for defining the proposed measure. We then carry out the analysis for worst case, average case and best case situations. The Similarity measure designed is Gaussian based and preserves the properties of Gaussian function.

Time Series Prediction Using Restricted Boltzmann Machines and Backpropagation

Abstract Time series prediction appear in many real-world problems, e.g., financial market, signal processing, weather forecasting among others. The underlying models and time series data of those problems are generally complex in a way that reasonable accurate estimation cannot be easily achieved, thus requiring more advanced techniques. Statistical models are the classical approaches for tackling this problem. Many works extended different architectures of Artificial neural networks to work with time series prediction, such as Feedforward, Boltzmann Machines and Deep Belief Network. A Deep Belief Network based on hybridization between Gaussian-Bernoulli Restricted Boltzmann Machine and the Backpropagation algorithm is presented. The hybrid algorithm is tested on three time series databases showing promising results.

Efficient Top-k Similarity Join of Massive Time Series Using MapReduce

Top-k similarity join of time series, designed to find top-k most similar pairs of time series records, is a primitive operation widely adopted by many time series data analysis applications. However, computing such top-k similarity join is a challenging problem today, as many modern applications are creating massive amounts of time series data. Obviously, a centralized machine is difficult to perform top-k similarity join in a large time series database efficiently. In this paper, we investigate how to perform the top-k similarity join of massive time series in parallel using MapReduce over a large cluster of commodity machines. Our proposed MapReduce-based algorithm consists of four steps, which takes as input a set of time series records and output an ordered list of top k closest pairs. To improve the efficiency in computing top-k similarity join, we proposed several solutions. We first introduce an efficient distance function based on LSH (Locality Sensitive Hash) for time series to improve the efficiency in pairwise similarity comparison. We next propose all pair partitioning methods to minimize the amount of data transfers between map and reduce functions. Moreover, we make use of serial computation strategy for parallelizing the computation of local top-k closest pairs in each partition. Our performance study confirms the effectiveness and scalability of our MapReduce algorithms.

Determinants of demand for wood products in the US construction sector: an econometric analysis of a system of demand equations

A system of conditional demand equations for wood products in the US construction sector is derived from the relevant end-use cost functions by taking advantage of the duality between the production function and the cost functions. Seven different inputs, including five wood products, labor, and capital, and three different end uses (new housing construction, repair and remodeling, and nonresidential construction) are analyzed. A seemingly unrelated regression estimation procedure is employed using a detailed annual time series database that spans the period from 1950 to 2009. Results suggest large variability in own-price long-term demand elasticity among wood inputs and across end uses. Softwood lumber, for example, is highly inelastic for new housing construction, inelastic for repair and remodeling, and nearly unitary elastic for nonresidential construction. Softwood plywood, on the other hand, is elastic for new housing construction and highly inelastic for nonresidential construction. In addition, we find that dynamic adjustments to long-term conditional intensity factor demand are prevalent across all inputs and among all three end-use categories.

The Study on Evolutive Model of Heavy Metal Pollution in Urban Surface Soil and Design of Time-Series Database

In order to study how the heavy metal pollution disseminates in urban surface soil and to recognize its evolutive pattern, this paper provides a model with time variable based on Euler method which could avoid the shortcoming of the traditional constant time assumption. We establish a set of spatial-temporal equations and derive their analytical solutions by 3-order Fourier transform.

Time series forecasting by neural networks: A knee point-based multiobjective evolutionary algorithm approach

In this paper, we investigate the problem of time series forecasting using single hidden layer feedforward neural networks (SLFNs), which is optimized via multiobjective evolutionary algorithms. By utilizing the adaptive differential evolution (JADE) and the knee point strategy, a nondominated sorting adaptive differential evolution (NSJADE) and its improved version knee point-based NSJADE (KP-NSJADE) are developed for optimizing SLFNs. JADE aiming at refining the search area is introduced in nondominated sorting genetic algorithm II (NSGA-II). The presented NSJADE shows superiority on multimodal problems when compared with NSGA-II. Then NSJADE is applied to train SLFNs for time series forecasting. It is revealed that individuals with better forecasting performance in the whole population gather around the knee point. Therefore, KP-NSJADE is proposed to explore the neighborhood of the knee point in the objective space. And the simulation results of eight popular time series databases illustrate the effectiveness of our proposed algorithm in comparison with several popular algorithms.

Long-Term Series of Data and Development Economics

This article deals with the many long time series databases that have been developed over the last two decades. These series cover a span of more than 100 years and allow us to compare the situation of current developed countries at a time when they were developing countries with that of present-day developing countries. Longitudinal econometric analyses are now feasible instead of cross-sectional analyses of recent data concerning developing and already developed countries. The field of development economics is thus significantly enlarged. This opens up new perspectives and allows us to test models and to understand the role of exogenous factors such as technology or institutions, which contribute to assessing the validity of models in two different periods. This article offers examples that show how useful such long time series, such as education data, can be to explaining variations in, for example, fertility rates.

Are BIDs Good for Business? The Impact of BIDs on Neighborhood Retailers in New York City

The business improvement district (BID) model has been widely adopted as a place-based strategy for commercial renewal and sustainability. This study uses the National Establishment Time-Series (NETS) Database to estimate the impact of New York City BIDs on change in retail sales and change employment between 2000 and 2008. Since the self-financing mechanism of BIDs becomes a compulsory tax on retail tenants, this study examines whether the model improves retail performance relative to comparable areas of the city that never adopted BIDs. Although effects vary by BID size, retail sales and employment are depressed by small community BIDs.

Measuring and Comparing Immigration, Asylum and Naturalization Policies Across Countries: Challenges and Solutions

Academics and policy makers require a better understanding of the variation of policies that regulate global migration, asylum and immigrant naturalization. At present, however, there is no comprehensive cross-national, time-series database of such policies, rendering the analysis of policy trends across and within these areas difficult at best. Several new immigration databases and indices have been developed in recent years. However, there is no consensus on how best to conceptualize, measure and aggregate migration policy indicators to allow for meaningful comparisons through time and across space. This article discusses these methodological challenges and introduces practical solutions that involve historical, multi-dimensional, dis-aggregated and transparent conceptualizing, measuring and compiling of cross-national immigration policies. Such an approach informs the International Migration Policy and Law Analysis (IMPALA) database.

Exploring the Effects of Water Sector Investment in Economic Development in Iran

The aim of this study is to examine the effects of water sector investment on Iran economic development. In this study, in order to see the effects of water sector investment from production function [view] of Solo growth model (assuming constant return to scale), we use [data from] Economic Time Series Database of National Accounts in[sources as] Central Bank, Vice-Presidency for Strategic Planning and Supervision, and Iran Water Resources Management Co., which were gathered during 1980-2010. The amount of this effect is exploited from the model in question on four economic sectors in Iran; and its econometric method is Panel Data. The results show that continuing trend in population growth, indiscriminate use of resources and low efficiency intensify water shortages. Solo Growth Model (primary neo-classic one) for different sectors (agriculture, industry and mine, services, and petroleum sectors) in Iranian economy implies that the elasticity of water investment in agricultural sector is significant, and positive, with the amount of 1.3% and for the rest of the other sectors is non-significant. The investment effect of water sector for groundwater discharge is about 2.4 per cent which is significant; this maintains the important issue that by increasing investment, the amounts of water harvesting is not decreased. Conversely was happened for surface waters with the amount of about -2.7%. The elasticity of investment in water sector based on Solo Growth Model, justified for economic sectors of Iran with confidence coefficient of 66% is less than 1, about 0.02% in agricultural sector, and for the rest of the other sectors is negative and non-significant.

Analysis of Various Periodicity Detection Algorithms in Time Series Data with Design of New Algorithm

Time series datasets consist of sequence of numeric values obtained over repeated measurements of time. They are Popular in many applications such as stock market analysis, power consumption, economic and sells forecasting, temperature etc. Periodic pattern mining or periodicity detection is process of finding periodic patterns in time series database. It has a number of applications, such as prediction, forecasting, detection of unusual activities, etc. Periodicity mining needs to give more attention as its increased need in real life applications. The types of periodicities are symbol periodicity, sequence periodicity and segment periodicity and they should be identified even in the presence of noise in the time series database. There are number of algorithms exists for periodic pattern mining. Those algorithms have some advantages and disadvantages. In this paper, we have compared different periodicity mining algorithms and given plan for developing efficient periodicity mining algorithm which detect symbol periodicity, sequence periodicity and segment periodicity and noise-resilient .

Result Analysis of Mining Fast Frequent Itemset Using Compacted Data

Data mining and knowledge discovery of database is magnetizing wide array of non-trivial research arena, making easy to industrial decision support systems and continues to expand even beyond imagination in one such promising field like Artificial Intelligence and facing the real world challenges. Association rules forms an important paradigm in the field of data mining for various databases like transactional database, time-series database, spatial, object-oriented databases etc. The burgeoning amount of data in multiple heterogeneous sources coalesces with the impediment in building and preserving central vital repositories compels the need for effectual distributive mining techniques. The majority of the previous studies rely on an Apriori-like candidate set generation-and-test approach. For these applications, these forms of aged techniques are found to be quite expensive, sluggish and highly subjective in case there exists long length patterns.

3-D Geovisualization of satellite images on smart devices by the integration of spatial DBMS, RESTful API and WebGL

Recent technological advancements in web-based geographic information systems have enabled access to satellite images on smart devices. The Representational State Transfer (REST) architecture overcomes difficulties that are associated with conventional data communications on the web, and the Web Graphics Library (WebGL) can be used as an alternative to web-based three-dimensional geographic visualization (3-D geovisualization) due to its efficient image processing capabilities. This paper describes a 3-D geovisualization system that was developed for satellite images on smart devices by integrating a spatial database management system (DBMS), a RESTful application programming interface (API), and WebGL. Spatiotemporal objects were constructed for time-series satellite images within a DBMS and a RESTful API was built for spatiotemporal queries to the time-series database so that the requested satellite data could be represented in 3-D on smart devices using WebGL. Satellite images that are represented in WebGL give a more realistic 3-D experience when they are combined with terrain data and provide for intuitive observations of the relationships between pixel values and associated geospatial conditions. This paper shows that a creative combination of existing technologies can be used to enhance and display satellite images on smart devices for 3-D geovisualization.

Trends in the daily and extreme temperatures in the Qiantang River basin, China

ABSTRACTTemporal and spatial changes in the annual and seasonal temperatures in a typical basin of the Qiantang River were analysed based on the time series databases of daily temperatures from 14 meteorological stations. Both the Mann–Kendall (MK) trend test and simple linear regression analyses were employed to detect trends in the mean and extreme temperatures. The temperature changes during the three periods of 1960–1990, 1960–2000 and 1960–2006 were investigated at each meteorological station and over the entire basin as an average. The spatial and temporal changes were characterized by significant warming throughout the region, with the minimum temperature (Tn) increasing the most, particularly after 1990. Various percentiles of extreme temperatures, as well as their corresponding frequencies, were chosen to explore the trends of extreme climate change in this region. Linear regression analyses showed a significant warming trend in cold events both on an annual and seasonal basis, especially in the winter. Conversely, the hot events were dominated by an insignificant warming trend (p > 0.05). When comparing the time series before and after 1990, trend shifts were apparent in both the mean and extreme temperatures, particularly in the spring and winter. In addition to the large‐scale circulation, regional factors may have influenced the observed climate change in the studied region. Climate change has already influenced human society by, for example, increasing the frequency of haze in the study region. We conclude that the warming here is mainly attributed to changes in the minimum temperature.

Durable Queries over Historical Time Series

This paper studies the problem of finding objects with durable quality over time in historical time series databases. For example, a sociologist may be interested in the top 10 web search terms during the period of some historical events; the police may seek for vehicles that move close to a suspect 70 percent of the time during a certain time period and so on. Durable top-k (DTop-k) and nearest neighbor (DkNN) queries can be viewed as natural extensions of the standard snapshot top-k and NN queries to timestamped sequences of values or locations. Although their snapshot counterparts have been studied extensively, to our knowledge, there is little prior work that addresses this new class of durable queries. Existing methods for DTop-k processing either apply trivial solutions, or rely on domain-specific properties. Motivated by this, we propose efficient and scalable algorithms for the DTop-k and DkNN queries, based on novel indexing and query evaluation techniques. Our experiments show that the proposed algorithms outperform previous and baseline solutions by a wide margin.

Statistical estimation of quadratic Rényi entropy for a stationary m-dependent sequence

The Rényi entropy is a generalisation of the Shannon entropy and is widely used in mathematical statistics and applied sciences for quantifying the uncertainty in a probability distribution. We consider estimation of the quadratic Rényi entropy and related functionals for the marginal distribution of a stationary m-dependent sequence. The U-statistic estimators under study are based on the number of ε-close vector observations in the corresponding sample. A variety of asymptotic properties for these estimators are obtained (e.g. consistency, asymptotic normality, and Poisson convergence). The results can be used in diverse statistical and computer science problems whenever the conventional independence assumption is too strong (e.g. ε-keys in time series databases and distribution identification problems for dependent samples).

Extensions and relationships of some existing lower-bound functions for dynamic time warping

Dynamic time warping (DTW) is a state-of-the-art time series similarity measure method, which warps time axes to match the same shape between two time series with different lengths. However, its quadratic time and space complexity is an obstacle to its applications in the large time series data mining. To address this problem, some lower-bound functions for DTW, fast methods to approximately measure the distance between time series, are used to prune the dissimilar objects from time series database so as to retain the candidates for further measuring their similarity with DTW. So far, the existing lower-bound functions for DTW have been widely accepted for time series similarity search and indexing. In this paper, we propose the extensions of two existing lower-bound functions and discuss the relationships among them. The extensions are improved with high tightness and without much time cost. At the same time, we theoretically prove that these extensions satisfy lower-bound requirement and are better than their old versions respectively. The experimental results demonstrate that in most cases the quality of the proposed extensions of lower-bound functions for DTW outperforms the original versions except for a slightly higher time cost.

Survey On: Mining High Utility Itemsets from Transactional Database

Utility mining emerges as an important topic in data mining field. Mining high utility itemsets from databases refers to finding the itemsets with high profits. The meaning of itemset utility is interestingness, importance, or profitability of an item to users. Mining high utility itemsets from a transactional database refers to the discovery of itemsets with high utility like profits. Although several relevant algorithm has been proposed in recent years, they incur the problem of producing a large number of candidate itemsets for high utility itemsets. Such a large number of candidate itemsets degrades the mining performance in terms of execution time and space requirement. The situation may become worse when the database contains lots of long transactions or long high utility itemsets. This research work focuses on efficient method for high utility itemset mining, to reduce number of overestimated itemsets and to reduce the search space in order to improve the performance of high utility itemset mining. Key Terms: Candidate itemset, frequent itemset, high utility itemset, utility mining, data mining. I. Introduction Now a days computers are used widely in different areas. Fast reliable and unlimited secondary storage provides a perfect environment for the users to collect and store large amount of the data. Computers are also used to extract the useful information from the mass of data. This is called as the knowledge discovery or Data mining. Data mining is the process of revealing previously unknown and potentially useful information from large databases. The primary goal is to discover the hidden patterns in the data and to extract previously unknown interesting patterns. Discovering useful patterns hidden in a database plays an essential role in several data mining tasks, like frequent pattern mining and high utility pattern mining. Among them the fundamental research topic is frequent pattern mining which can be applied to different kinds of databases such as transactional database, streaming database and time series database and many applications domains like bioinformatics, web click-stream analysis and mobile environment.