Time Series Data Research Articles

Comparative analysis of ARIMA and LSTM for predicting fluctuating time series data

The investigation of time series data forecasting is a critical topic within the realms of economics and business. The autoregressive integrated moving average (ARIMA) model has been prevalently utilized, notwithstanding its limitations, which include the necessity for a substantial quantity of data points and the presumption of data linearity. However, with recent developments, the long short-term memory (LSTM) network has emerged as a promising alternative, potentially overcoming these limitations. The objective of this study is to determine an effective approach for managing time series data characterized by volatility and missing values. Evaluation was conducted using RMSE for accuracy assessment, and the execution time measured using the Python Timeit library. The findings indicates that in a dataset comprising 60 data points, the LSTM model (RMSE 0.037618) surpasses the ARIMA model (RMSE 0.062667) in terms of accuracy. However, this trend reverses in a larger dataset of 228 data points, where the ARIMA model demonstrates superior accuracy (RMSE 0.006949) compared to the LSTM model (RMSE 0.036025). In scenarios with missing data, the LSTM model consistently outperforms the ARIMA model, although the accuracy of both models diminishes with an increase in the number of missing values. The ARIMA model significantly outpaces the LSTM model.

Mix-method toolbox for monitoring greenhouse gas production and microbiome responses to soil amendments

In this study, we adopt an interdisciplinary approach, integrating agronomic field experiments with soil chemistry, molecular biology techniques, and statistics to investigate the impact of organic residue amendments, such as vinasse (a by-product of sugarcane ethanol production), on soil microbiome and greenhouse gas (GHG) production. The research investigates the effects of distinct disturbances, including organic residue application alone or combined with inorganic N fertilizer on the environment. The methods assess soil microbiome dynamics (composition and function), GHG emissions, and plant productivity. Detailed steps for field experimental setup, soil sampling, soil chemical analyses, determination of bacterial and fungal community diversity, quantification of genes related to nitrification and denitrification pathways, measurement and analysis of gas fluxes (N2O, CH4, and CO2), and determination of plant productivity are provided. The outcomes of the methods are detailed in our publications (Lourenço et al., 2018a; Lourenço et al., 2018b; Lourenço et al., 2019; Lourenço et al., 2020). Additionally, the statistical methods and scripts used for analyzing large datasets are outlined. The aim is to assist researchers by addressing common challenges in large-scale field experiments, offering practical recommendations to avoid common pitfalls, and proposing potential analyses, thereby encouraging collaboration among diverse research groups.•Interdisciplinary methods and scientific questions allow for exploring broader interconnected environmental problems.•The proposed method can serve as a model and protocol for evaluating the impact of soil amendments on soil microbiome, GHG emissions, and plant productivity, promoting more sustainable management practices.•Time-series data can offer detailed insights into specific ecosystems, particularly concerning soil microbiota (taxonomy and functions).

Utility-Aware Time Series Data Release With Anomalies Under TLDP

Utility-Aware Time Series Data Release With Anomalies Under TLDP

Path to clean and sustainable energy from nuclear and renewable sources: Evidence from France

France is one of the countries that have made commendable progress towards accomplishing sustainable development goals but still struggles with combating environmental challenges. Decarbonizing the energy sector by deploying clean energy and reducing fossil fuels in the energy mix are essential steps. The study uses annual time series data from 1980 to 2021 and employs the ridge regression estimation technique to estimate a twice-differentiable transcendental production function, which approximates the second-order Taylor series to explore the potential for such decarbonization for France. The study finds that substitution possibilities exist between clean energy and fossil fuels, with substantial substitution possibilities coming from nuclear energy. Nuclear energy and renewables are found to be complements that can be deployed concurrently, but since nuclear energy already constitutes a significant proportion of clean energy for France, the study recommends increasing the proportion of renewable energy in the overall energy mix as the most plausible path to decarbonization. Other complementary sustainable energy policies are recommended in the body of the paper.

Study the impact of renewable and non-renewable energy sources on micro-grid using time series data based information transfer

This paper addresses the challenges arising from the integration of renewable energy sources into microgrids, focusing on the impact on system dynamics and stability. Employing an advanced time series-based information transfer approach, the study identifies critical parameters influencing system instability under small perturbations. Unlike traditional methods, this technique effectively analyzes information flow and interactions within the complex system, providing insights into the influence of states on different operating modes. The study specifically investigates the disturbance caused by increased load demand and employs a three-layered small signal stability index to pinpoint crucial system-affecting parameters. Subsequently, Distributed Generation (DG) units are strategically placed to restore stability, guided by the identified critical parameters. The proposed approach is validated on IEEE 3 Bus and IEEE 39 Bus systems, demonstrating its effectiveness and outperforming existing methodologies in predicting and addressing system uncertainties. These systems experienced instability as a result of an introduced disturbance in the form of increased load, affecting the dynamics of the generator at the immediate bus. To enhance the generator dynamics and address the impact of the disturbance, a new DG unit was introduced to provide support. The DG unit was placed at the bus which was deduced to be most critical from the three-layered small signal stability.

Assessing the Impact of Seasonal Decomposition on the Time Series Analysis Accuracy: A Comprehensive Study

Abstract: Time series forecasting plays a pivotal role in decision-making across various domains, ranging from finance to healthcare and weather prediction. The accurate prediction of future values in a time series is vital for informed planning and resource allocation. The aim of this study is to explore whether the utilization of seasonal decomposition techniques, such as classical decomposition, X-12-ARIMA, and seasonal decomposition of time series (STL), can improve the effectiveness of time series forecasting models by separating the data into its distinct components, including trend and seasonality. We conduct a comprehensive analysis using real-world time series data, employing popular forecasting models like ARIMA, exponential smoothing, and machine learning-based approaches. By comparing the forecasting accuracy of these models with and without the application of seasonal decomposition techniques, we provide empirical evidence to support the hypothesis. Our research results provide valuable insights into the tangible ad- vantages of integrating seasonal decomposition techniques in time series forecasting, potentially contributing to enhanced decision support systems across diverse application domains.

ARIMA Model Time Series Forecasting

Abstract: Time series forecasting is a critical component in various fields such as finance, economics, meteorology, and engineering. Among the multitude of methods available for time series forecasting, the Autoregressive Integrated Moving Average (ARIMA) model stands out for its simplicity and effectiveness. This paper provides a comprehensive review of ARIMA models, focusing on their application in forecasting time series data. We begin with an overview of time series analysis and the theoretical foundations of ARIMA models. Subsequently, we delve into the process of building and fitting ARIMA models, discussing the steps involved and the considerations for model selection. Furthermore, we explore advanced topics such as seasonal ARIMA (SARIMA) models and discuss their relevance in handling seasonal data patterns. Additionally, we review recent advancements and extensions of ARIMA models, including hybrid models and machine learning-based approaches. Finally, we discuss the challenges and limitations associated with ARIMA modeling and provide recommendations for future research directions.

Bayesian quantile inference and order shrinkage for hysteretic quantile autoregressive models

Hysteretic quantile autoregressive model combines the hysteretic patterns and quantile autoregression, which can capture the dynamic relationship and nonlinear characteristics at different quantiles in time series data. In this paper, the Bayesian quantile inference and order shrinkage are studied for a class of hysteretic quantile autoregressive time series models. By using Markov Chain Monte Carlo (MCMC) techniques, the proposed Bayesian quantile method can handle the sparse hysteretic quantile autoregressive model well. It can accurately determine order of the model and estimate non-zero coefficients. Both simulation studies and a data example show that the proposed methods are feasible, reliable and appropriate for analysing the US Gross National Product data set.

Climate Dynamics over Kerala, India: Insight from a Century-long Temperature and Rainfall Data Analysis

Historical climate data analysis is of great significance in climate change adaptation and mitigation planning at global as well as regional levels. This article attempted to study the long-term trends of temperature and rainfall across the districts in Kerala, India. CRU monthly time series data of rainfall and temperature data spanning from 1901-2022 were used for the analysis. Mann-Kendall test and Sen’s Slop estimator were applied to detect the presence and magnitude of the trend, and Pettitt’s homogeneity test was used to find the climate change point in temperature time series data. The analysis found a significant and positive temperature trend across all districts in Kerala, with temperature increases ranging from 0.0086°C/year to 0.0102°C/year. In terms of rainfall trends, June and January experienced a significant decrease, while July and September saw a significant increase over the years. The year 1976 was identified as the point of climate change. It was observed that there was an increase in the southwest and Northeast monsoons in the post-period, with higher variability in the latter. Winter rainfall notably decreased during the post-period. The variability in climate parameters identified in this study could impact crop cycles and agricultural productivity, requiring further investigation at a micro-level for effective adaptation and mitigation strategies for the state.

Pattern transitions in diary data of MDD patients: a mixed-methods multiple case study of psychotherapy dynamics

AimMixed-methods approaches promise a deep understanding of psychotherapeutic processes. This study uses qualitative and quantitative data from daily diary entries and daily self-assessments during inpatient treatment. The aim of the study is to get an insight into the similarities and differences between both types of data and how they represent self-organized pattern transitions in psychotherapy. While a complete correlation of results is not expected, we anticipate observing amplifying and subsidiary patterns from both perspectives.Materials and methodsDaily, five MDD patients wrote diaries and completed self-assessments using the Therapy Process Questionnaire, a questionnaire for monitoring the change dynamics of psychotherapy. The data were collected using the Synergetic Navigation System, an online tool for real-time monitoring. Diary entries of the patients described their experiences in everyday life. The qualitative text analysis was conducted using Mixed Grounded Theory, which provided categories representing the patients’ ongoing experiences of transformation and stagnation. The time series data was analyzed using the dynamic complexity algorithm and the pattern transition detection algorithm. Results from qualitative and quantitative analyses were combined and compared. Following the process of data triangulation, the leading perspective came from the theory of self-organization. In addition to presenting the overall results for all five patients, we delve into two specific case examples in greater detail.ResultsSpecific and highly diversified diary entries of 5 patients were classified into the categories of perceived pattern stability, noticing improvement, broadening the perspective, critical instability, and experiencing moments of Kairos. Patients reported problems not only related to their disorder (e.g., lack of energy and hopelessness) but also to phases and steps of change, which could be related to the theory of self-organization (e.g., problem attractors, critical fluctuations, pattern transitions, and Kairos). Qualitative and quantitative analysis provide important supplementary results without being redundant or identical.ConclusionData triangulation allows for a comprehensive and multi-perspective understanding of therapeutic change dynamics. The different topics expressed in the diary entries especially help to follow micro-psychological processes, which are far from being a simple reaction to interventions. The way patients experience themselves being in stability or instability and stagnation or transformation is surprisingly close to the general features of self-organizing processes in complex systems.

Modeling Economic Growth in Papua Province Using Panel Data

This study aims to determine and analyze simultaneously the effect of Average Years of Schooling, and School Participation Rate on GRDPHB in Papua Province, the type of research used is the panel data regression model. This study uses panel data regression methods, the data used involves a combination of time series and cross-section data. The results showed a positive and significant effect of RLS and TPAK on GRDPHB in Papua Province with an adjusted R Square value of 0.998016 or 99.80%. This finding confirms the importance of focusing on improving the quality of education along with maintaining a high labor force participation rate as a strategy to strengthen Papua's economic base. Thus, human development and strengthening human resources will be the key to sustainable economic growth in Papua.

Effects of the COVID-19 pandemic on the rates of adverse birth outcomes and fetal mortality in Japan: an analysis of national data from 2010 to 2022

BackgroundAlthough the coronavirus disease 2019 (COVID-19) pandemic affected trends of multiple health outcomes in Japan, there is a paucity of studies investigating the effect of the pandemic on adverse birth outcomes and fetal mortality. This study aimed to investigate the effect of the onset of the pandemic on the trends in adverse birth outcomes and fetal mortality using national data in Japan.MethodsWe used the 2010–2022 birth and fetal mortality data from the Vital Statistics in Japan. We defined the starting time of the effect of the pandemic as April 2020, and the period from January 2010 to March 2020 and that from April 2020 to December 2022 were defined as the pre- and post- pandemic period, respectively. The rates of preterm birth, term low birth weight (TLBW), small-for-gestational-age (SGA), large-for-gestational-age (LGA), spontaneous fetal mortality, and artificial fetal mortality were used as outcomes. An interrupted time series analysis was conducted using monthly time series data of the outcomes to evaluate the effects of the pandemic. In addition, a modified Poisson regression model was used to evaluate the effects of the pandemic on these outcomes using individual-level data, and the adjusted risk ratio of the effect was calculated.ResultsThe adverse birth and fetal mortality outcomes showed a decreasing trend over the years, except for preterm birth and LGA birth rates, and SGA birth rates tended to reach their lowest values after the onset of the pandemic. The interrupted time series analysis revealed that the pandemic decreased preterm birth, TLBW, and SGA birth rates. In addition, the regression analysis revealed that the pandemic decreased the TLBW, SGA, and artificial fetal mortality rates.ConclusionsAnalyses performed using national data suggested that the pandemic decreased the TLBW and SGA rates in Japan.

A Stock Index Futures Price Prediction Approach Based on the MULTI-GARCH-LSTM Mixed Model

As a type of financial derivative, the price fluctuation of futures is influenced by a multitude of factors, including macroeconomic conditions, policy changes, and market sentiment. The interaction of these factors makes the future trend become complex and difficult to predict. However, for investors, the ability to accurately predict the future trend of stock index futures price is directly related to the correctness of investment decisions and investment returns. Therefore, predicting the stock index futures market remains a leading and critical issue in the field of finance. To improve the accuracy of predicting stock index futures price, this paper introduces an innovative forecasting method by combining the strengths of Long Short-Term Memory (LSTM) networks and various Generalized Autoregressive Conditional Heteroskedasticity (GARCH)-family models namely, MULTI-GARCH-LSTM. This integrated approach is specifically designed to tackle the challenges posed by the nonstationary and nonlinear characteristics of stock index futures price series. This synergy not only enhances the model’s ability to capture a wide range of market behaviors but also significantly improves the precision of future price predictions, catering to the intricate nature of financial time series data. Initially, we extract insights into the volatility characteristics, such as the aggregation of volatility in futures closing prices, by formulating a model from the GARCH family. Subsequently, the LSTM model decodes the complex nonlinear relationships inherent in the futures price series and incorporates assimilated volatility characteristics to predict future prices. The efficacy of this model is validated by applying it to an authentic dataset of gold futures. The empirical findings demonstrate that the performance of our proposed MULTI-GARCH-LSTM hybrid model consistently surpasses that of the individual models, thereby confirming the model’s effectiveness and superior predictive capability.

Clustering time series by extremal dependence

AbstractThe goal of this paper is to characterize the temporal dependence structure on the extremes of time series and use such dependency to group them. In particular, three similarity measures to capture extremal dependence are proposed, being their performance assessed in different scenarios. This will involve the use of classical time series clustering algorithms, as well as rigorous evaluation of their performance in both simulated scenarios and real-world time series data sets. The focus will be on comparing the performance of these similarity measures with different clustering methods, and illustrate the efficacy of extremal dependence-based clustering in meteorological data. To achieve this, we will consider a dataset consisting of daily maximum temperatures recorded at 500 stations across Europe.

More is better? The role of strategic data management in a lean manufacturing process

Purpose Lean manufacturing has been pivotal in emphasizing the reduction of cycle times, minimizing manufacturing costs and diminishing inventories. This research endeavors to formulate a lean data management paradigm, through the design and execution of a strategic edge-cloud data governance approach. This study aims to discern anomalous or unforeseen patterns within data sets, enabling an efficacious examination of product shortcomings within manufacturing processes, while concurrently minimizing the redundancy associated with the storage, access and processing of nonvalue-adding data. Design/methodology/approach Adopting a lean data management framework within both edge and cloud computing contexts, this study ensures the preservation of significant time series sequences, while ascertaining the optimal quantity of normal time series data to retain. The efficacy of detected anomalous patterns, both at the edge and in the cloud, is assessed. A comparative analysis between traditional data management practices and the strategic edge-cloud data governance approach facilitates an exploration into the equilibrium between anomaly detection and space conservation in cloud environments for aggregated data analysis. Findings Evaluation of the proposed framework through a real-world inspection case study revealed its capability to navigate alternative strategies for harmonizing anomaly detection with data storage efficiency in cloud-based analysis. Contrary to the conventional belief that retaining comprehensive data in the cloud maximizes anomaly detection rates, our findings suggest that a strategic edge-cloud data governance model, which retains a specific subset of normal data, can achieve comparable or superior accuracy with less normal data relative to traditional methods. This approach further demonstrates enhanced space efficiency and mitigates various forms of waste, including temporal delays, storage of noncontributory normal data, costs associated with the analysis of such data and excess data transmission. Originality/value By treating inspected normal data as nonvalue-added, this study probes the intricacies of maintaining an optimal balance of such data in light of anomaly detection performance from aggregated data sets. Our proposed methodology augments existing research by integrating a strategic edge-cloud data governance model within a lean data analytical framework, thereby ensuring the retention of adequate data for effective anomaly detection.

Dynamics of cold-water dinoflagellates in the northern Baltic Sea based on 18S rRNA gene metabarcoding

Cold-water dinoflagellates contribute significantly to spring blooms in temperate coastal waters. In the northern Baltic Sea, Peridiniella catenata, Apocalathium malmogiense, Gymnodinium corollarium, and Biecheleria baltica predominate the dinoflagellate spring blooms. Spring dynamics of cold-water dinoflagellates have been previously studied using cyst sedimentation data. Here, we utilize time-series data based on amplicons of the V4 region of the 18S rRNA gene spanning from autumn 2012 to spring 2013 and describe dynamics and habitat preferences of the relatively most abundant dinoflagellates in the northern Baltic Sea. Our results indicate that these species preferred either sea ice, under-ice water, or deeper water columns during the ice-covered season and that they shifted in temporal dominance during our sampling period.

Bioinspired In-Sensor Multimodal Fusion for Enhanced Spatial and Spatiotemporal Association.

Multimodal perception can capture more precise and comprehensive information compared with unimodal approaches. However, current sensory systems typically merge multimodal signals at computing terminals following parallel processing and transmission, which results in the potential loss of spatial association information and requires time stamps to maintain temporal coherence for time-series data. Here we demonstrate bioinspired in-sensor multimodal fusion, which effectively enhances comprehensive perception and reduces the level of data transfer between sensory terminal and computation units. By adopting floating gate phototransistors with reconfigurable photoresponse plasticity, we realize the agile spatial and spatiotemporal fusion under nonvolatile and volatile photoresponse modes. To realize an optimal spatial estimation, we integrate spatial information from visual-tactile signals. For dynamic events, we capture and fuse in real time spatiotemporal information from visual-audio signals, realizing a dance-music synchronization recognition task without a time-stamping process. This in-sensor multimodal fusion approach provides the potential to simplify the multimodal integration system, extending the in-sensor computing paradigm.

EDA-Graph: Graph Signal Processing of Electrodermal Activity for Emotional States Detection.

The continuous detection of emotional states has many applications in mental health, marketing, human-computer interaction, and assistive robotics. Electrodermal activity (EDA), a signal modulated by sympathetic nervous system activity, provides continuous insight into emotional states. However, EDA possesses intricate nonstationary and nonlinear characteristics, making the extraction of emotion-relevant information challenging. We propose a novel graph signal processing (GSP) approach to model EDA signals as graphical networks, termed EDA-graph. The GSP leverages graph theory concepts to capture complex relationships in time-series data. To test the usefulness of EDA-graphs to detect emotions, we processed EDA recordings from the CASE emotion dataset using GSP by quantizing and linking values based on the Euclidean distance between the nearest neighbors. From these EDA-graphs, we computed the features of graph analysis, including total load centrality (TLC), total harmonic centrality (THC), number of cliques (GNC), diameter, and graph radius, and compared those features with features obtained using traditional EDA processing techniques. EDA-graph features encompassing TLC, THC, GNC, diameter, and radius demonstrated significant differences (p<0.05) between five emotional states (Neutral, Amused, Bored, Relaxed, and Scared). Using machine learning models for classifying emotional states evaluated using leave-one-subject-out cross-validation, we achieved a five-class F1 score of up to 0.68.

Effects of ambient temperature and humidity on COPD mortality in Ganzhou city, China.

This study used the time series data of Ganzhou city to explore the individual and interaction effects of temperature and humidity on COPD death, and identify vulnerable subgroups of the population. We collected daily COPD mortality and meteorological data in Ganzhou from 2016 to 2019. The nonlinear distribution lag model was used to examine the associations and interaction between daily mean temperature and humidity and COPD mortality. For the total population, male and 65 years old or above, the relative risk (RR) for COPD mortality could be significant at extremely low temperature (3.3 ℃), reaching 1.799 (95% confidence interval [CI]: 1.216, 2.662), 1.894 (95% CI: 1.164, 3.084) and 1.779 (95% CI:1.185, 2.670). Also, at extremely low humidity (47.8%), the risk reached 1.888 (95% CI: 1.217, 2.930), 1.837 (95% CI: 1.066, 3.165) and 2.166 (95% CI: 1.375, 3.414). The cumulative COPD death risk for females was 3.524 (95% CI: 1.340, 9.267) at high temperature (30.7 ℃), 1.953(95% CI: 1.036, 3.683) at low humidity (47.8%) and 1.726 (95% CI: 1.048, 2.845) at high humidity (96.7%). For the total COPD deaths and subgroups, the interaction effects between daily temperature and humidity were not significant (p > 0.05). Both extremely low temperature and low humidity increased the risk of COPD death in Ganzhou city, especially for males and people over 65 years old. Females were more sensitive to extremely high temperature and humidity. Patients with COPD should pay attention to self-protection under extreme temperature and humidity weather conditions.

BATSCCD: a new change detection method for mapping building age in rapidly changing urban areas using Landsat time series data

ABSTRACT The building age is an important feature in the vulnerability assessment of disasters and building energy consumption modeling. It provides indispensable information for the study of built environment. However, obtaining high-precision maps of building ages is challenging, which impedes the development of relevant research and applications. In this study, a new building age mapping method based on Landsat time series classification and change detection (BATSCCD) was proposed, which integrates Landsat time series data with random forest and LandTrendr algorithm to reflect non-building to building changes, building reconstruction, and corresponding change characteristics. The proposed method achieved overall building age mapping accuracies of 80.45% within a 2-year tolerance and 87.66% within a 3-year tolerance in Guangzhou and accuracies of 78.08% and 87.52%, respectively, in Beijing. The improvement was notable, especially compared with the continuous change detection and classification (CCDC) method, with corresponding accuracy values of 36.22% and 46.31%, respectively, in Guangzhou and 49.7% and 60.19%, respectively, in Beijing. The results suggest that the BATSCCD method can be used to accurately estimate the building age and provide reliable mapping results. Thanks to the accessibility of required data and its high accuracy of building age estimation, the proposed method exhibits great application potential.