Underlying Data Generation Process Research Articles

Multi-stage deep residual collaboration learning framework for complex spatial–temporal traffic data imputation

Performing accurate and efficient traffic data repair has become an essential task before proceeding with other applications of intelligent transportation systems. However, existing repair models often impose assumptions on the underlying data generation process or fail to sufficiently capture the complex spatial–temporal dependencies between several nodes at different intervals. To address these issues, we propose a novel multi-stage deep residual collaboration learning framework (named Multi-DRCF) to tackle the complex task of repairing missing traffic data. In particular, a spatiotemporal fusion network (i.e., ST-imputator) incorporating the graph-based convolutions with recurrent structures is introduced for modeling the spatial topology and dynamic temporal coherences of the traffic graph network in turn. Moreover, to further achieve both higher accuracy and efficiency, we exploit a stackable bi-directional residual optimization (i.e., Bi-RO) structure to enhance the role of ST-imputator in the utilization of the spatiotemporal properties of the residuals. Indeed, this combination of ST-imputator and Bi-RO structure can serve as an iterable unit for Multi-DRCF, and the whole computation process is highly modular and flexible in operation. We evaluate Multi-DRCF using two real-world large-scale traffic speed datasets, i.e., Seattle-Loop and METR-LA. Experimental results demonstrate that Multi-DRCF can effectively extract the spatial and temporal properties from traffic data, resulting in an excellent repair performance with an acceptable computational cost. Compared with the state-of-the-art baseline models, Multi-DRCF performs more competitively for three types of missing patterns, and also provides steadier repair results with missing rates ranging from 10% to 90%. Finally, the ablation experiments and visualization analysis also present well insights for better understanding the superiority of the Multi-DRCF for complex spatial–temporal traffic data imputation.

Diagnostic spatio-temporal transformer with faithful encoding

This paper addresses the task of anomaly diagnosis when the underlying data generation process has a complex spatio-temporal (ST) dependency. The key technical challenge is to extract actionable insights from the dependency tensor characterizing high-order interactions among temporal and spatial indices. We formalize the problem as supervised dependency discovery, where the ST dependency is learned as a side product of multivariate time-series classification. We show that temporal positional encoding used in existing ST transformer works has a serious limitation in capturing higher frequencies (short time scales). We propose a new positional encoding with a theoretical guarantee, based on discrete Fourier transform. We also propose a new ST dependency discovery framework, which can provide readily consumable diagnostic information in both spatial and temporal directions. Finally, we demonstrate the utility of the proposed model, DFStrans (Diagnostic Fourier-based Spatio-temporal Transformer), in a real industrial application of building elevator control.

Beyond addressing multicollinearity: Robust quantitative analysis and machine learning in international business research

We reconcile the recommendations made by Kalnins (J Int Bus Stud, 2022) on the one hand and by Lindner, Puck and Verbeke (J Int Bus Stud 51(3):283–298, 2020) on the other, on how international business (IB) quantitative researchers should treat multicollinearity. We explain that, in principle, treatment depends on the underlying data generation process, but note that datasets based on any single generation process are rare. In doing so, we broaden the discussion to include how research methods should be selected and robust statistical models built. In addition, we highlight the importance of a comprehensive literature review in selecting appropriate control variables. We also make suggestions on addressing cross-level dependencies and selecting robustness checks to avoid bias in statistical results. Finally, we go beyond regression and include a broader palette of research methodologies building on machine-learning approaches.

Bayesian and frequentist approaches on estimation and testing for a zero-inflated binomial distribution

To analyze discrete count data with excessive zeros, different zero-inflated statistical models that allow for frequent zero-valued observations have been developed. When the underlying data generation process of non-zero values is based on the number of successes in a sequence of independent Bernoulli trials, the zero-inflated binomial distribution is perhaps adequate for modeling purposes. In this paper, we discuss statistical inference for a zero-inflated binomial distribution using the objective Bayesian and frequentist approaches. Point and interval estimation of the model parameters and hypothesis testing for excessive zeros in a zero-inflated binomial distribution are developed. A Monte Carlo simulation study is used to assess the performance of estimation and hypothesis testing procedures. A comparative study of the objective Bayesian approach and the frequentist approach is provided. The proposed statistical inferential methods are applied to analyze an earthquake dataset and a baseball dataset for illustration.

Fast wireless sensor for anomaly detection based on data stream in an edge-computing-enabled smart greenhouse

Edge-computing-enabled smart greenhouses are a representative application of the Internet of Things (IoT) technology, which can monitor the environmental information in real-time and employ the information to contribute to intelligent decision-making. In the process, anomaly detection for wireless sensor data plays an important role. However, the traditional anomaly detection algorithms originally designed for anomaly detection in static data do not properly consider the inherent characteristics of the data stream produced by wireless sensors such as infiniteness, correlations, and concept drift, which may pose a considerable challenge to anomaly detection based on data stream and lead to low detection accuracy and efficiency. First, the data stream is usually generated quickly, which means that the data stream is infinite and enormous. Hence, any traditional off-line anomaly detection algorithm that attempts to store the whole dataset or to scan the dataset multiple times for anomaly detection will run out of memory space. Second, there exist correlations among different data streams, and traditional algorithms hardly consider these correlations. Third, the underlying data generation process or distribution may change over time. Thus, traditional anomaly detection algorithms with no model update will lose their effects. Considering these issues, a novel method (called DLSHiForest) based on Locality-Sensitive Hashing and the time window technique is proposed to solve these problems while achieving accurate and efficient detection. Comprehensive experiments are executed using a real-world agricultural greenhouse dataset to demonstrate the feasibility of our approach. Experimental results show that our proposal is practical for addressing the challenges of traditional anomaly detection while ensuring accuracy and efficiency.

A review of irregular time series data handling with gated recurrent neural networks

Irregular time series data is becoming increasingly prevalent with the growth of multi-sensor systems as well as the continued use of unstructured manual data recording mechanisms. Irregular data and the resulting missing values severely limit the data's ability to be analysed and modelled for classification and forecasting tasks. Often, conventional methods used for handling time series data introduce bias and make strong assumptions on the underlying data generation process, which can lead to poor model predictions. Traditional machine learning and deep learning methods, although at the forefront of data modelling, are at best compromised by irregular time series data sets and fail to model the temporal irregularity of incomplete time series. Gated recurrent neural networks (RNN), such as LSTM and GRU, have had outstanding success in sequential modelling, and have been applied in many application fields, including natural language processing. These models have become an obvious choice for time series modelling and a promising tool for handling irregular time series data. RNNs have a unique ability to be adapted to make effective use of missing value patterns, time intervals and complex temporal dependencies in irregular univariate and multivariate time series data. In this paper, we provide a systematic review of recent studies in which gated recurrent neural networks have been successfully applied to irregular time series data for prediction tasks within several fields, including medical, human activity recognition, traffic monitoring and environmental monitoring. The review highlights the two common approaches for handling irregular time series data: missing value imputation at the data pre-processing stage and modification of algorithms to directly handle missing values in the learning process. Reviewed models are confined to those that can address issues with irregular time series data and does not cover the broader range of models that deal more generally with sequences and regular time series. This paper aims to present the most effective techniques emerging within this branch of research as well as to identify remaining challenges, so that researchers may build upon this platform of work towards further novel techniques for handling irregular time series data.

TSF-DBSCAN: A Novel Fuzzy Density-Based Approach for Clustering Unbounded Data Streams

In recent years, several clustering algorithms have been proposed with the aim of mining knowledge from streams of data generated at a high speed by a variety of hardware platforms and software applications. Among these algorithms, density-based approaches have proved to be particularly attractive, thanks to their capability of handling outliers and capturing clusters with arbitrary shapes. The streaming setting poses additional challenges that need to be addressed as well: data streams are potentially unbounded and affected by concept drift, i.e., a modification over time in the underlying data generation process. In this article, we propose temporal streaming fuzzy density-based spatial clustering of applications with noise (TSF-DBSCAN), a novel fuzzy clustering algorithm for streaming data. TSF-DBSCAN is an extension of the well-known DBSCAN algorithm, one of the most popular density-based clustering approaches. Fuzziness is introduced in TSF-DBSCAN to model the uncertainty about the distance threshold that defines the neighborhood of an object. As a consequence, TSF-DBSCAN identifies clusters with fuzzy overlapping borders. A fading model, which makes objects less relevant as they become more remote in time, endows TSF-DBSCAN with the capability of adapting to evolving data streams. The integration of the model in a two-stage approach ensures computational and memory efficiency: during the online stage, continuously arriving objects are organized in proper data structures that are later exploited in the offline stage to determine a fine-grained partition. An extensive experimental analysis on synthetic and real-world datasets shows that TSF-DBSCAN yields competitive performance when compared to other clustering algorithms recently proposed for streaming data.

Context-Aware Time Series Imputation for Multi-Analyte Clinical Data.

Clinical time series imputation is recognized as an essential task in clinical data analytics. Most models rely either on strong assumptions regarding the underlying data-generation process or on preservation of only local properties without effective consideration of global dependencies. To advance the state of the art in clinical time series imputation, we participated in the 2019 ICHI Data Analytics Challenge on Missing Data Imputation (DACMI). In this paper, we present our proposed model: Context-Aware Time Series Imputation (CATSI), a novel framework based on a bidirectional LSTM in which patients' health states are explicitly captured by learning a "global context vector" from the entire clinical time series. The imputations are then produced with reference to the global context vector. We also incorporate a cross-feature imputation component to explore the complex feature correlations. Empirical evaluations demonstrate that CATSI obtains a normalized root mean square deviation (nRMSD) of 0.1998, which is 10.6% better than that of state-of-the-art models. Further experiments on consecutive missing datasets also illustrate the effectiveness of incorporating the global context in the generation of accurate imputations.

Impact of discretization of the timeline for longitudinal causal inference methods.

In longitudinal settings, causal inference methods usually rely on a discretization of the patient timeline that may not reflect the underlying data generation process. This article investigates the estimation of causal parameters under discretized data. It presents the implicit assumptions practitioners make but do not acknowledge when discretizing data to assess longitudinal causal parameters. We illustrate that differences in point estimates under different discretizations are due to the data coarsening resulting in both a modified definition of the parameter of interest and loss of information about time-dependent confounders. We further investigate several tools to advise analysts in selecting a timeline discretization for use with pooled longitudinal targeted maximum likelihood estimation for the estimation of the parameters of a marginal structural model. We use a simulation study to empirically evaluate bias at different discretizations and assess the use of the cross-validated variance as a measure of data support to select a discretization under a chosen data coarsening mechanism. We then apply our approach to a study on the relative effect of alternative asthma treatments during pregnancy on pregnancy duration. The results of the simulation study illustrate how coarsening changes the target parameter of interest as well as how it may create bias due to a lack of appropriate control for time-dependent confounders. We also observe evidence that the cross-validated variance acts well as a measure of support in the data, by being minimized at finer discretizations as the sample size increases.

Detecting the Direction of Information Flow in Instantaneous Relations Between Variables

Data-based causality analysis tries to detect the true structural relations between measurements of complex multivariate systems. The detected relations should correspond to the true structure of the underlying data generation process. Even though there are many methodologies developed to extract causal relations from data, existence of instantaneous correlation between some variables in the data set, requires special care in order to correctly do the analysis. It is required to detect the instantaneous relations between variables as a prerequisite for subsequent causality analysis. Not only is detection of instantaneous relations important, but it is also necessary to discover the direction of information flow in the instantaneous relations. This piece of information plays a vital role in selection of correct modeling structure to achieve a reliable result about causal relations between variables. Using prior knowledge about the process or blind mathematical transformations are usual solutions for this problem in the literature. However, there is a lack of reliable mathematical methodologies to address this issue completely based on data analysis. This brief proposes a method to detect the direction of instantaneous causal relations between variables and supports it through simulation and case studies. The proposed algorithm uses a third variable as an instrument to detect the direction of information flow between any two instantaneously correlated variables. The instrument variable is required to meet some conditions for the algorithm to work; however, the application of the algorithm does not require any prior information about the process.

Convolutional generative models for underwater acoustic event classification

Despite the success of discriminative (DISC) classifiers, there remain serious flaws in the DISC approaches. Using adversarial sampling, a DISC network can be fooled into producing any desired classifier output by subtle changes in the data sample. The problem lies in the goal of DISC methods: assigning class identity without further considerations. In contrast, generative (GEN) methods model the underlying data generation process, can be interrogated and groomed by researchers, so have the potential to operate well with little training data. But, the GEN task is more difficult, so performance lags behind. But, given time and effort, GEN performance can even surpass performance of DISC methods as shown by deep belief network of Hinton in 2006 which performed better than comparable fully-connected (non-CONV) DISC networks. As CONV DISC networks (CNNs) have seen a quantum leap in performance, GEN methods have once again fallen behind. This talk details unexplored avenues to greatly improve CONV GEN models (CGMs). One avenue, “max-pooling for CGMs” is backed up by promising experiments. Max-pooling is a dimension-reduction step that is partly responsible for the success of CNNs, but presents a severe obstacle for CGMs by discarding the pooling positioning information (PPI). We show that encoding PPI info the features greatly improves the quality of GEN auto-encoders. When PPI together with added GEN neurons are “appended” to existing DISC CNNs, a hybrid GEN/DISC CNN is created with the best qualities of both approaches.

OM Forum—Empirical Research in Operations Management: From Field Studies to Analyzing Digital Exhaust

The amount of empirical research in operations management that has been published in this journal and in Management Science has recently witnessed a significant increase. Beyond this increase in numbers, research questions and empirical methodologies have also changed substantially. Whereas empirical research in the 1990s was primarily carried out at the firm, plant, or project level, recent research focuses much more on microlevel phenomena. This shift in the unit of analysis has been made possible by advances in data generation and collection technologies. This trend is likely to continue as the digitization of modern work creates massive data trails, which archive the state of an operation at a very fine level of granularity. I will refer to such data, which result as a by-product of the digital planning and control of operations, as digital exhaust. As such microlevel data become available abundantly, new opportunities for research arise. However, such data also pose new challenges, including the need for careful theory development and a deep contextual understanding of the underlying data generation process.

Separate cointegration in a VAR system subject to structural breaks

This note demonstrates that the analysis of separate cointegration can be facilitated by properly allowing for regime shifts in the underlying data generation process. A vector autoregressive (VAR) model is reformulated in such a way that it is subject to deterministic breaks and also contains the structure of separate cointegration. A comparative empirical study shows the advantage of the proposed VAR model over the standard model in revealing the structure.

Implementing the template method pattern in genetic programming for improved time series prediction

Modularity is an ongoing focus in genetic programming research. Enhanced modularity can accelerate solution convergence and increase human understanding and knowledge gained from evolved programs. Prior advances in modularity research have addressed programming language elements such as functions, modules, and recursion. This paper proposes improving modularity by considering non-language elements, specifically software design patterns. A new genetic programming technique implementing the template method pattern is described. This technique was tested and compared to existing genetic programming approaches in the prediction of nonlinear time series subject to abrupt changes in the underlying data generation process. Such series are often seen in areas such as finance and meteorology and have proved challenging for genetic programming to model and predict. Experimental results demonstrate the potential for incorporating additional software design patterns into genetic programming and applying these techniques to additional problem domains.

Accuracy of latent-variable estimation in Bayesian semi-supervised learning

Hierarchical probabilistic models, such as Gaussian mixture models, are widely used for unsupervised learning tasks. These models consist of observable and latent variables, which represent the observable data and the underlying data-generation process, respectively. Unsupervised learning tasks, such as cluster analysis, are regarded as estimations of latent variables based on the observable ones. The estimation of latent variables in semi-supervised learning, where some labels are observed, will be more precise than that in unsupervised, and one of the concerns is to clarify the effect of the labeled data. However, there has not been sufficient theoretical analysis of the accuracy of the estimation of latent variables. In a previous study, a distribution-based error function was formulated, and its asymptotic form was calculated for unsupervised learning with generative models. It has been shown that, for the estimation of latent variables, the Bayes method is more accurate than the maximum-likelihood method. The present paper reveals the asymptotic forms of the error function in Bayesian semi-supervised learning for both discriminative and generative models. The results show that the generative model, which uses all of the given data, performs better when the model is well specified.

Correcting for non‐ignorable missingness in smoking trends

Data missing not at random (MNAR) are a major challenge in survey sampling. We propose an approach based on registry data to deal with non‐ignorable missingness in health examination surveys. The approach relies on follow‐up data available from administrative registers several years after the survey. For illustration, we use data on smoking prevalence in Finnish National FINRISK study conducted in 1972–97. The data consist of measured survey information including missingness indicators, register‐based background information and register‐based time‐to‐disease survival data. The parameters of missingness mechanism are estimable with these data although the original survey data are MNAR. The underlying data generation process is modelled by a Bayesian model. The results indicate that the estimated smoking prevalence rates in Finland may be significantly affected by missing data. Copyright © 2015 John Wiley & Sons, Ltd.

An alternative test of the trade-off theory of capital structure

The purpose of this paper is to investigate the stochastic behavior of corporate debt ratios utilizing a balanced panel of 2,556 publicly traded U.S. firms during the period 1997-2010. We partition the panel into ten economic sectors and perform panel unit root tests on each sector employing book value and market value measures of debt ratio. First-generation panel unit root tests provide consistent evidence that debt ratios are mean reverting, which supports the trade-off theory. However, these tests rely on the assumption that the debt ratios are cross-sectionally independent, but tests of cross-sectional independence fail to uphold this assumption. Thus, utilizing a second-generation panel unit root test that controls for cross-sectional dependence, we uncover evidence showing that debt ratios are not mean reverting, which contradicts the trade-off hypothesis. We find that the recent macroeconomic developments triggered by the financial crisis and the Great Recession have considerable explanatory power over the dynamics of the debt ratios. In fact, when we exclude the years of the recent global financial crisis, the unit root hypothesis is rejected in one half of the sectors. We interpret these results as indicative that the recent global events may have produced in these sectors a structural change in the underlying data generation process (DGP). Overall, then, we find mixed evidence on the stationarity of debt ratios.

Forecasting Aggregate Demand: Analytical Comparison of Top-Down and Bottom-Up Approaches in a Multivariate Exponential Smoothing Framework

Forecasting aggregate demand is a crucial matter in all industrial sectors. In this paper, we provide the analytical prediction properties of top-down (TD) and bottom-up (BU) approaches when forecasting aggregate demand, using multivariate exponential smoothing as demand planning framework. We extend and generalize the results obtained by Widiarta, Viswanathan and Piplani (2009) by employing an unrestricted multivariate framework allowing for interdependency between the variables. Moreover, we establish the necessary and sufficient condition for the equality of mean squared errors (MSEs) of the two approaches. We show that the condition for the equality of MSEs also holds even when the moving average parameters of the individual components are not identical. In addition, we show that the relative forecasting accuracy of TD and BU depends on the parametric structure of the underlying framework. Simulation results confirm our theoretical findings. Indeed, the ranking of TD and BU forecasts is led by the parametric structure of the underlying data generation process, regardless of possible misspecification issues.

Forecasting aggregate demand: Analytical comparison of top-down and bottom-up approaches in a multivariate exponential smoothing framework

Forecasting aggregate demand represents a crucial aspect in all industrial sectors. In this paper, we provide the analytical prediction properties of top-down (TD) and bottom-up (BU) approaches when forecasting the aggregate demand using a multivariate exponential smoothing as demand planning framework. We extend and generalize the results achieved by Widiarta et al. (2009) by employing an unrestricted multivariate framework allowing for interdependency between its variables. Moreover, we establish the necessary and sufficient condition for the equality of mean squared errors (MSEs) of the two approaches. We show that the condition for the equality of MSEs holds even when the moving average parameters of the individual components are not identical. In addition, we show that the relative forecasting accuracy of TD and BU depends on the parametric structure of the underlying framework. Simulation results confirm our theoretical findings. Indeed, the ranking of TD and BU forecasts is led by the parametric structure of the underlying data generation process, regardless of possible misspecification issues.

Firm profitability: Mean-reverting or random-walk behavior?

We analyze the stochastic properties of three measures of profitability, return on assets (ROA), return on equity (ROE), and return on investment (ROI), using a balanced panel of US firms during the period 2001–2010. We employ a panel unit-root approach, which assists in identifying competitive outcomes versus situations that require regulatory intervention to achieve more competitive outcomes. Based upon conventional panel unit-root tests, we find substantial evidence supporting mean-reversion, which, in turn, lends support to the long-standing “competitive environment” hypothesis originally set forward by Mueller (1977). These results, however, prove contaminated by the assumption of cross-section independence. After controlling for cross-section dependence, we find that profitability evolves as a non-stationary process in some sectors in the US economy. Our findings, especially taken as a whole, remain fairly robust to various assumptions regarding the underlying data generation process.