First-order Markov Research Articles

Data-driven definitions of gazelle companies that rule out chance: application for Russia and Spain

AbstractThe phenomenon of fast-growing companies exhibiting sustained growth and creating disproportionally many new jobs, so-called “gazelles”, has been widely analyzed in the literature. The criteria defining “gazelles”, however, lack a consensus, while it cannot be ruled out that superior performance of these companies is just good luck. We use large firm-level datasets for Russia and Spain and conduct a Monte Carlo experiment with first-order Markov chains to derive a definition of “gazelle” companies and ensure that their existence cannot be explained by chance only. Our results demonstrate that the definitions of “gazelle” companies differ between the two countries warning against using same definition for different countries. We find that the “gazelles” account for about 1–2% of the companies in our datasets and are responsible for approximately 14% of employment growth in Russia and 9% in Spain. These companies are concentrated in economic sectors like retail trade, real estate and construction.

Markov chain Monte Carlo for seismic facies classification

Seismic facies classification aims to predict a facies model, or a set of facies models, from measured seismic data. We focus on stochastic classification methods to estimate the probability distribution of facies conditioned on seismic data. Bayesian classification methods based on analytical solutions are generally applied to seismically inverted elastic attributes or petrophysical properties rather than measured seismic data, and they do not include spatial correlation models in the prior distribution. Instead, iterative stochastic methods, such as Monte Carlo acceptance-rejection sampling and approximate Bayesian computation, can be directly applied to seismic data and can be implemented with complex prior models; however, their convergence is generally slow because it requires the simulation of a large number of facies models from the prior distribution. We develop an efficient implementation of a Markov chain Monte Carlo (MCMC) approach that adopts complex prior models, such as multiple-point statistics simulations based on a training image, to generate geologically realistic facies realizations. The novelty of the approach is that the proposal distribution of the proposed MCMC method is based on a gradual deformation algorithm, in which the proposal distribution is expressed as a linear combination of the indicator variables of the previously accepted model and a random prior simulation, according to a uniform random weight. The synthetic examples indicate accurate facies predictions with a success rate of approximately 80% for well-log facies classification based on elastic attributes and approximately 60% for seismic facies classification based on seismic data. The inversion is compared to an MCMC method with prior models sampled from a first-order Markov chain and Bayesian facies classification.

A Stochastic Weather Model for Drought Derivatives in Arid Regions: A Case Study in Qatar

In this paper, we propose a stochastic weather model consisting of temperature, humidity, and precipitation, which is used to calculate a reconnaissance drought index (RDI) in Qatar. The temperature and humidity models include stochastic differential equations and utilize an adjusted Ornstein–Uhlenbeck (O–U) process. For the precipitation model, a first-order Markov chain is used to differentiate between wet and dry days and the precipitation amount on wet days is determined by a probability distribution. Five different probability distributions were statistically tested to obtain an appropriate precipitation amount. The evapotranspiration used in the RDI calculation incorporates crop coefficient values, depends on the growth stages of the crops, and provides a crop-specific and more realistic representation of the drought conditions. Five different evapotranspiration formulations were investigated in order to obtain the most accurate RDI values. The calculated RDI was used to assess the intensity of drought in Doha, Qatar, and could be used for the pricing of financial drought derivatives, a form of weather derivative. These derivatives could be used by agricultural producers to hedge against the economic effects of droughts.

M3F: A novel multi-session and multi-protocol based malware traffic fingerprinting

In recent years, cyber attacks have become increasingly frequent, which has had a tremendous negative impact on public life and social order. Accurately and quickly finding malware traffic from massive network traffic is one of the keys to defending against network attacks. Traditional detection methods, whether signature-based or machine learning-based, use a packet or a session as the smallest detection unit. Usually, malware creates more than one session while executing malicious functions. Detecting these sessions alone without contextual information is prone to false negatives and false positives.To address these problems, we propose M3F, a Multi-session and Multi-protocol based Malware traffic Fingerprinting that uses multiple related sessions with different protocols as the smallest classification unit. We associate multiple sessions of different protocols together to form several session sequences. For each session in a session sequence, we represent it with a state with three features so that a session sequence can be transformed into a state sequence. For a malware family, we learn a first-order homogeneous Markov chain using its state sequences as its traffic fingerprint (aka M3F). M3Fs reflect the dynamics of malware communication traffic. Meanwhile, the approximate matching technique is used to deal with the evolution of malware, which can improve the recall rate. Armed with M3F, we can locate malware traffic in massive network traffic. We use a large amount of malicious traffic to verify M3F. The experimental results show that M3F has 99.41% precision and 99.54% recall, both outperforming the baseline methods. It does not mark normal traffic as malicious traffic. Additionally, M3F is well interpretable and is the network-level representation of the malware’s behavior.

An Order Reduction Design Framework for Higher-Order Binary Markov Random Fields.

The order reduction method is an important approach to optimize higher-order binary Markov random fields (HoMRFs), which are widely used in information theory, machine learning and image analysis. It transforms an HoMRF into an equivalent and easier reduced first-order binary Markov random field (RMRF) by elaborately setting the coefficients and auxiliary variables of RMRF. However, designing order reduction methods is difficult, and no previous study has investigated this design issue. In this paper, we propose an order reduction design framework to study this problem for the first time. Through study, we find that the design difficulty mainly lies in that the coefficients and variables of RMRF must be set simultaneously. Therefore, the proposed framework decomposes the design difficulty into two processes, and each process mainly considers the coefficients or auxiliary variables of RMRF. Some valuable properties are also proven. Based on our framework, a new family of 14 order reduction methods is provided. Experiments, such as synthetic data and image denoising, demonstrate the superiority of our method.

PCNN Model Guided by Saliency Mechanism for Image Fusion in Transform Domain.

In heterogeneous image fusion problems, different imaging mechanisms have always existed between time-of-flight and visible light heterogeneous images which are collected by binocular acquisition systems in orchard environments. Determining how to enhance the fusion quality is key to the solution. A shortcoming of the pulse coupled neural network model is that parameters are limited by manual experience settings and cannot be terminated adaptively. The limitations are obvious during the ignition process, and include ignoring the impact of image changes and fluctuations on the results, pixel artifacts, area blurring, and the occurrence of unclear edges. Aiming at these problems, an image fusion method in a pulse coupled neural network transform domain guided by a saliency mechanism is proposed. A non-subsampled shearlet transform is used to decompose the accurately registered image; the time-of-flight low-frequency component, after multiple lighting segmentation using a pulse coupled neural network, is simplified to a first-order Markov situation. The significance function is defined as first-order Markov mutual information to measure the termination condition. A new momentum-driven multi-objective artificial bee colony algorithm is used to optimize the parameters of the link channel feedback term, link strength, and dynamic threshold attenuation factor. The low-frequency components of time-of-flight and color images, after multiple lighting segmentation using a pulse coupled neural network, are fused using the weighted average rule. The high-frequency components are fused using improved bilateral filters. The results show that the proposed algorithm has the best fusion effect on the time-of-flight confidence image and the corresponding visible light image collected in the natural scene, according to nine objective image evaluation indicators. It is suitable for the heterogeneous image fusion of complex orchard environments in natural landscapes.

Creation and Destruction of Jobs in Urban Labour Market: Role of Gender, Caste and Religion in India.

This study analyses labour market transitions from job to jobless or jobless to job by using Periodic Labour Force Survey (PLFS)of India in urban labour market by applying first-order Markov process in constructing transition matrices and panel logit regression model using personal level information from the PLFS in India for 2019-20. While education is assumed to be the most important factor influencing the process of transitions in the labour market, the circumstance variables like gender, caste and religion are expected to have significant effects on it in a country like India. The empirical results show a significant difference in labour market transitions between men and women. The movement of workers between job status was very low and the major part of the transition occurred in the form of job losses for each type of workers both among men and women and also among all working age group and young age group.

A high-order hidden Markov model for dynamic decision analysis of multi-homing ride-sourcing drivers

Ride-sourcing drivers enjoy flexibility in scheduling work hours and choosing platforms, which generates multi-homing behavior on multiple platforms. It is challenging to observe the labor supply of multi-homing ride-sourcing drivers due to data limitations, which motivates our research on modeling drivers' dynamic decisions on labor supply in the competitive ride-sourcing market with multiple platforms. We propose a dynamic discrete choice framework by modeling drivers' high-frequency decision sequences on platform switching as high-order hidden Markov processes and considering time-varying factors. The high-order hidden Markov framework relaxes the first-order Markovian assumption and builds interdependencies of unobserved states across multiple time periods. The estimation of the proposed model takes advantage of city-scale multiple platforms datasets in Hangzhou, China, including more than 16 million records of the order payment information and more than 46 thousand records of active ride-sourcing drivers' information. The case study results indicate that the high-order hidden Markov model (HO-HMM) has superior explanatory power in fitting multiple platforms datasets than the multinomial logit model and the first-order hidden Markov model. HO-HMM performs an advantage in modeling the extended historical dependency of drivers' decisions and individual driver behavior modeling with high interpretability. The results uncover the variations of drivers' attitudes in different hidden (unobservable) states and towards different platforms. In general, ride-sourcing drivers respond actively and positively to income and working time. The findings support the platforms' decision-making on pricing, reward, and personalized management of ride-sourcing drivers, and provide beneficial suggestions for improving income in the competitive ride-sourcing market.

Spatially Weighted Bayesian Classification of Spatio-Temporal Areal Data Based on Gaussian-Hidden Markov Models

This article is concerned with an original approach to generative classification of spatiotemporal areal (or lattice) data based on implementation of spatial weighting to Hidden Markov Models (HMMs). In the framework of this approach data model at each areal unit is specified by conditionally independent Gaussian observations and first-order Markov chain for labels and call it local HMM. The proposed classification is based on modification of conventional HMM by the implementation of spatially weighted estimators of local HMMs parameters. We focus on classification rules based on Bayes discriminant function (BDF) with plugged in the spatially weighted parameter estimators obtained from the labeled training sample. For each local HMM, the estimators of regression coefficients and variances and two types of transition probabilities are used in two levels (higher and lower) of spatial weighting. The average accuracy rate (ACC) and balanced accuracy rate (BAC), computed from confusion matrices evaluated from a test sample, are used as performance measures of classifiers. The proposed methodology is illustrated for simulated data and for real dataset, i.e., annual death rate data collected by the Institute of Hygiene of the Republic of Lithuania from the 60 municipalities in the period from 2001 to 2019. Critical comparison of proposed classifiers is done. The experimental results showed that classifiers based on HMM with higher level of spatial weighting in majority cases have advantage in spatial–temporal consistency and classification accuracy over one with lower level of spatial weighting.

Novel Approaches for Resource Management Across Edge Servers

Edge computing aims at reducing computation and storage across the cloud and provides service with reduced latency. Edge devices can be mobile devices, routers, cameras, printers or any Internet of Things (IoT) devices that generate vast amounts of data. The processing of these data is done by virtual machines (VMs) present in the edge servers that are located within close proximity of the edge devices. This work proposes two models which predict resource contention at the edge servers, namely, a Dynamic Markov model for Resource Contention Prediction in Edge Cloud (DMRCP) and a Hybrid Cascade of Regression and Markov model for Resource Contention Prediction (CRMRCP). In DMRCP, a history matrix is updated based on the CPU utilization of a Virtual Machine (VM). This history matrix is used to update a transition probability matrix. This matrix is used to predict the future state of the VM. In the CRMRCP approach, the past CPU utilization values of the virtual machines in the edge servers are used for predicting a set of future CPU utilization values using linear regression, polynomial regression, lasso regression and ridge regression. Then, the predicted future CPU utilization values are used by the dynamic and the second-order Markov models to classify the state of the edge servers as overloaded, underloaded or normally loaded. In both the approaches, the VMs that may cause resource contention are predicted and are migrated to other edge servers such that the destination edge server does not get overloaded after the migration. The DMRCP method is compared with the first-order and the second-order Markov models and the number of VM migrations is analysed to evaluate the performance. The number of VM migrations in the CRMRCP method is compared with that in the second-order Markov model. The overall results prove that the number of VM migrations for the DMRCP is 52.9% less compared to the first-order Markov model and 21.1% less when compared to the second-order Markov model. The number of VM migrations in CRMRCP is reduced by 81.8% when ridge regression cascaded with the second-order Markov model is used.

Data-Driven Probabilistic System for Batsman Performance Prediction in a Cricket Match

Batsmen are the backbone of any cricket team and their selection is very critical to the team’s success. A good batsman not only scores run but also provides stability to the team’s innings. The most important factor in selecting a batsman is their ability to score runs. It is a generally accepted notion that the future performance of a batsman can be predicted by observing and analyzing their past record. This hypothesis is based on the fact that a player’s batting average is generally considered to be a good indicator of their future performance. We proposed a data-driven probabilistic system for batsman performance prediction in the game of cricket. It captures the dependencies between the runs scored by a batsman in consecutive balls. The system is evaluated using a dataset extracted from the Cricinfo website. The system is based on a Hidden Markov model (HMM). HMM is used to generate the prediction model to foresee players’ upcoming performances. The first-order Markov chain assumes that the probability of a batsman scoring runs in the next ball is only dependent on how many runs he scored in the current ball. We use a data-driven approach to learn the parameters of the HMM from data. A probabilistic matrix is made that predicts what scores the batter can do on the upcoming balls. The results show that the system can accurately predict the runs scored by a batsman in a ball.

Enhanced Dynamic Dual-Latent Variable Model for Multirate Process Monitoring and Its Industrial Application

Quality-related process monitoring is an important tool to ensure process safety and product quality. However, the existence of process dynamics and multirate sampling makes it difficult to construct an efficient monitoring model. In order to handle process dynamics and multirate sampling, a multirate process monitoring method based on a dynamic dual-latent variable model is proposed. The model involves two sets of latent variables modeled as first-order Markov chains, which are used to capture both quality-related and quality-unrelated dynamic information. In addition, to deal with multiple sampling rates in the process data, the proposed model is combined with a multirate Kalman filtering technique. An expectation-maximization (EM) algorithm is used to estimate the unknown parameters, and a fault detection strategy is developed. The higher fault detection rate of the proposed method is verified by two application studies including a real industrial experiment and the Tennessee Eastman (TE) process.

Adaptive Modified Unbiased Minimum-Variance Estimation for Highly Maneuvering Target Tracking With Model Mismatch

Due to the complexity, time variability, and mutability of the noncooperative maneuvering target, the dynamic tracking model of the target does not match with the actual state, which reduces the tracking accuracy or even completely tracking loss. In this article, an expectation maximization-based adaptive modified unbiased minimum-variance estimation (EM-AMUMVE) algorithm is proposed for highly maneuvering target tracking with model mismatch. First, the virtual maneuvering noise and the first-order Markov process model are integrated to quantitatively describe the maneuvering acceleration according to the boundness of the maneuvering acceleration. Then, the maneuvering acceleration and state estimation are derived based on the unbiased minimum-variance criterion, and the expectation maximization (EM) is introduced in the updating procedure of the quadratic Bayesian filter to adaptively estimate the mean and covariance of virtual maneuvering noise and thus to improve the accuracy of maneuvering acceleration estimation. Finally, the optimality of the adaptive modified maneuvering acceleration estimation is theoretically proved, and the tracking scenarios of time-varying maneuvering target with model mismatch and reentry gliding trajectory of typical hypersonic vehicle hypersonic technology vehicle 2 (HTV-2) are built to demonstrate the validness, convergence, sensitivity, and engineering practicability of the proposed algorithm.

Variational Latent-State GPT for Semi-Supervised Task-Oriented Dialog Systems

Recently, two approaches, fine-tuning large pre-trained language models and variational training, have attracted significant interests, separately, for semi-supervised end-to-end task-oriented dialog (TOD) systems. In this paper, we propose Variational Latent-State GPT model (VLS-GPT), which is the first to combine the strengths of the two approaches. Among many options of models, we propose the generative model and the inference model for variational learning of the end-to-end TOD system, both as auto-regressive language models based on GPT-2, which can be further trained over a mix of labeled and unlabeled dialog data in a semi-supervised manner. Variational training of VLS-GPT is both statistically and computationally more challenging than previous variational learning works for sequential latent variable models, which use turn-level first-order Markovian. The inference model in VLS-GPT is non-Markovian due to the use of the Transformer architecture. In this work, we establish Recursive Monte Carlo Approximation (RMCA) to the variational objective with non-Markovian inference model and prove its unbiasedness. Further, we develop the computational strategy of sampling-then-forward-computation to realize RMCA, which successfully overcomes the memory explosion issue of using GPT in variational learning and speeds up training. Semi-supervised TOD experiments are conducted on two benchmark multi-domain datasets of different languages - MultiWOZ2.1 and CrossWOZ. VLS-GPT is shown to significantly outperform both supervised-only and semi-supervised self-training baselines.

Bayesian Analysis of First-Order Markov Models for Autocorrelated Binary Responses.

In many clinical trials, patient outcomes are often binary-valued which are measured asynchronously over time across various dose levels. To account for autocorrelation among such longitudinally observed outcomes, a first-order Markov model for binary data is developed. Moreover, to account for the asynchronously observed time points, nonhomogeneous models for the transition probabilities are proposed. The transition probabilities are modeled using B-spline basis functions after suitable transformations. Additionally, if the underlying dose-response curve is assumed to be non-decreasing, our model allows for the estimation of any underlying non-decreasing curve based on suitably constructed prior distributions. We also extended our model to the mixed effect model to incorporate individual-specific random effects. Numerical comparisons with traditional models are provided based on simulated data sets, and also practical applications are illustrated using real data sets.

Workflow analysis of data science code in public GitHub repositories

Despite the ubiquity of data science, we are far from rigorously understanding how coding in data science is performed. Even though the scientific literature has hinted at the iterative and explorative nature of data science coding, we need further empirical evidence to understand this practice and its workflows in detail. Such understanding is critical to recognise the needs of data scientists and, for instance, inform tooling support. To obtain a deeper understanding of the iterative and explorative nature of data science coding, we analysed 470 Jupyter notebooks publicly available in GitHub repositories. We focused on the extent to which data scientists transition between different types of data science activities, or steps (such as data preprocessing and modelling), as well as the frequency and co-occurrence of such transitions. For our analysis, we developed a dataset with the help of five data science experts, who manually annotated the data science steps for each code cell within the aforementioned 470 notebooks. Using the first-order Markov chain model, we extracted the transitions and analysed the transition probabilities between the different steps. In addition to providing deeper insights into the implementation practices of data science coding, our results provide evidence that the steps in a data science workflow are indeed iterative and reveal specific patterns. We also evaluated the use of the annotated dataset to train machine-learning classifiers to predict the data science step(s) of a given code cell. We investigate the representativeness of the classification by comparing the workflow analysis applied to (a) the predicted data set and (b) the data set labelled by experts, finding an F1-score of about 71% for the 10-class data science step prediction problem.

Microphase Separation of Statistical Multiblock Copolymers

Random multiblock copolymers are characterized by a number of parameters, and their self-assembly is controlled, in particular, by correlations in monomer unit sequences. We investigate the microphase separation of random multiblock copolymers composed of A and B homo-oligomers (macromonomers) in melts. The copolymer synthesis and self-assembly are studied by computer simulations using the dissipative particle dynamics technique. The copolymers are formed by irreversible step-growth polymerization in an equimolar melt of end-functionalized macromonomers of length M up to a high conversion degree. Correlations between macromonomers of different types along the chains are controlled by the reaction probabilities. The molar mass and block mass distributions are described for linear macromolecules and for cycles. Macromonomer sequences in linear chains are well described by the first-order Markov statistics. With an increase in the Flory–Huggins parameter χ, microphase separation into a lamellar structure, for close-to-alternating block copolymers, and into irregularly arranged domains, for Bernoullian sequences of macromonomers, occurs. The transition χM values are approximately twice those obtained with weak segregation theory [Fredrickson, G. H. Macromolecules1992, 25, 6341−6354] and approximately half the values for which microphases were previously detected in the simulations of random copolymer melts [Gavrilov, A. A. J. Chem. Phys.2013, 139, 224901]. Microphase separation is described by the static structure factor and local composition, besides the block positions are monitored visually. The lamellar thickness H is always between the unperturbed sizes of one- and two-macromonomer blocks, and we characterize the separation as a moderate segregation regime. For the close-to-alternating block copolymers at a fixed χ, the thickness scales with the average block length approximately as H ∼ N̅bl2/3 that is in agreement with the theoretical description (Supporting Information) considering the elastic response of the shortest blocks only.

A Longitudinal Analysis of Respiratory Illness and Tobacco Use Transitions

Among individuals with chronic respiratory conditions, transitions between patterns of tobacco product use are not well understood. This study examines how transitions, including quitting altogether, differ over time between those who do and do not have chronic respiratory conditions. Data from youth and adult participants of the longitudinal Population Assessment of Tobacco and Health Study (2013-2018) were analyzed. Youth aged 12-17 years were included if they had aged into the adult sample by Wave 4. Stratified polytomous regression models built under a first-order Markov assumption modeled the probability of transitioning between different states/patterns of tobacco product use (exclusive current E-cigarette use, exclusive current combustible tobacco product use, current dual use of combustible products and E-cigarettes, and no current tobacco product use) at each wave. Marginal transition probabilities were computed as a function of ever or past-year diagnosis of a respiratory condition (separately for asthma and a composite variable representing chronic bronchitis, emphysema, and/or chronic obstructive pulmonary disease). Analyses were conducted in 2020-2021. Most individuals, regardless of respiratory condition, maintained the same pattern of tobacco use between waves. Exclusive combustible tobacco product users, including those with or without a respiratory condition, were not likely to become exclusive E-cigarette users or to quit using tobacco entirely. Although combustible tobacco use negatively impacts the management and prognosis of respiratory illnesses, combustible tobacco users who were recently diagnosed with a chronic respiratory condition were not likely to quit using tobacco. Efforts to encourage and support cessation in this medically vulnerable population should be increased.

Probabilistic Prediction of Collisions between Cyclists and Vehicles Based on Uncertainty of Cyclists’ Movements

The uncertainty of cyclists’ movements has a significant impact on predicting the risk of collisions between cyclists and vehicles. The purpose of this study was to provide a method for assessing collision risk using probability, taking into account the uncertainty of cyclists’ movements. A cyclist model was first developed using a first-order Markov model. Then, based on Monte Carlo sampling, the distribution characteristics of the minimum distance and the time-to-collision (TTC) between the vehicle and the cyclist were extracted. By fitting these features, the probability density functions of the collision distance and TTC were estimated to derive the collision probabilities. The effectiveness of the collision probability prediction model was benchmarked against a deterministic crash risk prediction model (autonomous emergency braking [AEB] system) applied to three real-world cases previously reconstructed in an in-depth crash database. The results show that the collision probability prediction model can effectively predict the risk of collisions between cyclists and vehicles with better accuracy than AEB systems using a fixed trigger threshold. This study is a valuable reference for the development of advanced vehicle collision avoidance systems to protect cyclists and other vulnerable road users.

Development of a time series imaging approach for fault classification of marine systems

As part of any Prognostics and Health Management (PHM) system for the shipping industry, the determination of the current health of marine systems is fundamental. As such, diagnostic analytics is performed; a process that is typically constituted by fault detection, fault isolation, and fault identification. Although some efforts have been made to distinguish the faults and malfunctions (fault detection) that can occur in marine systems, the implementation of fault identification to provide a description of any considered fault type and its nature is still an unexplored area due to the lack of fault data. To overcome this, a methodology for the identification of anomalies in marine systems is presented in this paper. The proposed approach aims to analyse the implementation of time series imaging through the application of the first-order Markov chain in tandem with an analysis of both ResNet50V2 and Convolutional Neural Networks (CNNs) as part of the image classification task. To highlight the performance of this methodology, anomalies have been simulated considering the power parameter of a diesel generator. Results demonstrated the potential of time series imaging and image classification approaches, as the Markov-CNN achieved an accuracy of 95% when performing the fault classification task.